How do I proxy requests to NTLM-protected websites, like TeamFoundation and SharePoint? I keep getting 401 authentication errors.
According to this Microsoft TechNet article, you can't.
Microsoft NTLM uses stateful HTTP, which is a violation of the HTTP/1.1 RFC. It relies on authentication (an affair which involves a handshake with a couple of initial 401 errors) and subsequent connections to be done through the exact same connection from client to server. This makes HTTP proxying nearly impossible, since each request would usually go through either a new or a random connection picked from a pool of open connections. It can be done though.
NGiNX apparently supports this through the "ntlm" option, but this is part of their commercial offering. Apache HTTPD seems to have a couple of experimental patches for this, but this requires rebuilding Apache. TinyProxy doesn't support this either. HAProxy to the rescue!
Here is an example of a running configuration which works - it's a fairly simple setup with a single backend server:
backend backend_tfs
server static teamfoundation.mycompany.com:8080 check maxconn 3
mode http
balance roundrobin
option http-keep-alive
option prefer-last-server
timeout server 30s
timeout connect 4s
frontend frontend_tfs
# You probably want something other than 127.0.0.1 here:
bind 127.0.0.1:8080 name frontend_tfs
mode http
option http-keep-alive
timeout client 30s
default_backend backend_tfs
The important options here are http-keep-alive and prefer-last-server.
One more thing for my scenerio;
If you are using ssl both sides(the iis servers and haproxy), the ssl must be same for iis and haproxy server. Otherwise ntlm doesn't work when you want to go iis from haproxy.
Maybe can help someone who has the same problem.
Related
I have been working towards switching the communication protocol for our application from HTTP/1.1 to HTTP/2.
The communication flow is some thing like this:
Client talks to an Amazon Application load balancer over HTTP/2
Application load balancer talks to a reverse proxy (HAProxy) over HTTP/1.1
Reverse proxy then talks to the webserver over HTTP/1.1
I wanted all of this to be HTTP/2 but due to a limitation of the load balancer (https://forums.aws.amazon.com/thread.jspa?threadID=332847) the communication between it and the reverse proxy can either be HTTP/2 or HTTP/1.1 but not both. I need to support both because there is a WebSocket connection that is opened over HTTP/1.1.
I have an option to make the communication between HAProxy and the Webserver to be HTTP/2 as our Webserver support it.
So the flow becomes:
Client -> ALB (HTTP/2)
ALB -> HAProxy (HTTP/1.1)
HAProxy -> Webserver (HTTP/2)
I wanted to understand two things
If this is possible with HAProxy?
If this is a good move? will this give me any performance improvements?
Thanks in advance!
Cheers
Technically Ha Proxy can do HTTP / 2.0 end to end (version like 2.0 or newer https://www.haproxy.com/fr/blog/haproxy-2-0-and-beyond/#end-to-end-http-2)
And in 2.4 you can do HTTP/2 WebSockets (https://www.haproxy.com/fr/blog/announcing-haproxy-2-4/)
My first thought about HTTP/2.0 for multiplexing is interesting to reduce latency. Latency which is usually between the client and your fisrt instance (here ALB).
I don't know if you have latencies between HaProxy -> Webserver but if you think you have it. This brings a nice performance increase between HAProxy and your backend servers, since it condenses multiple connections down to a single connection. Otherwise do not expect a big improvement.
Moreover it could depend if you use TLS between Haproxy and webservers.
But options like headers compression and persistent TCP Connections are interesting so this makes sense to use it.
I need your help.
I have implemented a haproxy configuration which correctly manages both http and websocket backends, except in one specific scenario.
Here below a summary about how this stuff works:
When I connect to :2703/webapp, haproxy correctly redirects to one of the two http configured backends (webapp-lb1 or webapp-lb2).
When I connect to :2703/webapp/events, haproxy correctly redirects to one of the two websocket configured backends (websocket-lb1 or websocket-lb2)
Webapp is a servlet running in apache tomcat.
When I stop one of the two backend tomcats, haproxy correctly switches to the other one (for both the http and the websocket).
On the contrary, when I try to simulate an outage of one of the http backends by stopping the webapp via the tomcat manager, haproxy reports a HTTP Status 404 error but does not switch to the other backend.
Being that I explicitly configured the http-check expect status 302 directive, I would expect that - in case of a 404 status - haproxy switches to the other backend.
I had a look at the haproxy official documentation and I also tested the http-check disable-on-404 configuration but this is not what I need, as the haproxy behavior remains exactly the same of the above one.
For info, activating the http-check disable-on-404, haproxy detects the DOWN of the backend I stopped but does nothing (which as far as I understand, is exactly what we have to expect from the http-check disable-on-404 configuration in case of 404 status); here below the haproxy log when this option is enabled:
Jul 23 14:19:23 localhost haproxy[4037]: Server webapp-lb/webapp-lb2 is stopping, reason: Layer7 check conditionally passed, code: 404, info: "Not Found", check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
Here below an extract of my haproxy configuration:
frontend haproxy-webapp
bind *:2703
monitor-uri /haproxy_check
stats enable
stats uri /haproxy_stats
stats realm Strictly Private
stats auth admin:zqxwcevr
acl is_websocket url_beg /webapp/events
use_backend websocket-lb if is_websocket
default_backend webapp-lb
log global
backend webapp-lb
server webapp-lb1 192.168.136.129:8888 maxconn 400 check cookie webapp-lb1
server webapp-lb2 192.168.136.130:8888 maxconn 400 check cookie webapp-lb2
balance roundrobin
cookie JSESSIONID prefix nocache
log global
#http-check disable-on-404
option httpchk GET /webapp
http-check expect status 302
backend websocket-lb
server websocket-lb1 192.168.136.129:8888 maxconn 400 check
server websocket-lb2 192.168.136.130:8888 maxconn 400 check
balance roundrobin
log global
Please give me a hint as I am spending ages in reading documentation and forums with no success.
Thanks!
Is my reasoning about HTTP Keep-Alive correct? Here are my assumptions:
In modern browsers, persistent connections are the default, so the Connection: Keep-Alive response header isn't likely to be sent by HAProxy for those clients. Not seeing it does not necessarily mean that the connection is not persistent.
A Keep-Alive is meant to be a short duration, for example a few hundred milliseconds, during which the client can reuse the same connection to the same server (for uses such as downloading images, CSS and JavaScript that are on the same page)
Longer persistence can be accomplished by cookies or stick tables
The reason I ask is because when I set a long keep-alive timeout (via timeout http-keep-alive), typing F5 rapidly loads a different backend server just as fast, rotating through all three without any effect from the keep-alive. I would have thought that if I hit F5 during the keep-alive timeout period of five seconds, I'd still get the same backend server.
Am I thinking about this wrong?
Here is my HAproxy config where I've set the keep-alive timeout to 5 seconds:
defaults
timeout connect 5s
timeout client 120s
timeout server 120s
timeout http-request 5s
timeout http-keep-alive 5s
option http-keep-alive
frontend http
mode http
bind *:80
default_backend servers1
backend servers1
mode http
balance roundrobin
server web1 web1:80 check
server web2 web2:80 check
server web3 web3:80 check
UPDATE
Using Wireshark, it looks like, from the client perspective, the client is reusing the same socket connection to HAProxy's frontend until the timeout expires. So the keep-alive appears to be working between client and frontend. However, I added an image to the Web page to see which backend server would serve it, and if it would be a different server than the one that served the HTML...and it was different. In roundrobin mode, two different backend servers served up content to the same client, even though I had keep-alive enabled.
This all seems to me like keep-alive is not working between frontend and backend. This is the behavior I would expect if I had set option http-server-close, but not when using option http-keep-alive which should keep the connection to the server open.
you should ensure your server does not close the connection.
you should enable the "option prefer-last-server", in your HAProxy's defaults section.
you should re-read the "option http-keep-alive" documentation of HAProxy: http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#option%20http-keep-alive.
It clearly states that:
If the client request has to go to another backend or another server due to content switching or the load balancing algorithm, the idle connection will immediately be closed and a new one re-opened. Option "prefer-last-server" is available to try optimize server selection so that if the server currently attached to an idle connection is usable, it will be used.
(hence point 2)
So your observations looks normal.
I have a running HTTP web application and I am facing problems to make it run over HTTPS.
I am thinking of bringing some HTTPS Proxy that accepts user requests and forward it to the HTTP web app.
What do you think of that? and How can I accomplish that?
Setting up stunnel is a no-brainer - and its available for Unix/Linux/Posix/MSWindows (you might have mentioned what OS you are using).
(Also you can run the program to encrypt or decrpyt, at the server or at the client side)
It's possible to run Apache Httpd (for example) using HTTPS and use mod_proxy_http as a reverse proxy to forward the requests to your existing HTTP server. Of course, for this to be of any use, you'd need the reverse proxy and the target server to be connected in such a way that connections cannot be sniffed or altered.
You may find that the existing server needs certain extra settings for it to be aware it's using HTTPS (for example, special Valves in Apache Tomcat to set the HTTPS flag to true).
Apache httpd reverse-proxy?
I'm working on Comet support for CppCMS framework via long XMLHttpRequest polls. In many cases, such request is closed by client before any response from server was given -- for example the page is closed, user moves to other page or it is just refeshed.
At the server side I expect that I would recieve the notification that connection is dropped. I tested the application via 3 connectors: FastCGI, SCGI and simple HTTP Proxy.
From 3 major UNIX web servers, Apache2, lighttpd and Nginx, only the last one had closed
connection as expected allowing my application to remove the request from wait queue -- this worked for both FastCGI and HTTP Proxy connectors. (Nginx does not have scgi module by default).
Others, Apache and Lighttpd do not close connection or inform the backend about disconnected
clients, the proceed as if the client is still on line. This happens for all 3 supported APIs: FastCGI, SCGI and HTTP Proxy.
I had opened an issue for Lighttpd, but what
more conserns me is the fact that Apache -- mature and well supported web server as lighttpd
and does not discloses the server backend that client had gone.
Questions:
Is this a bug or this is a feature? Is there any reason not to close the connection between web server and application backend?
Are there real life Comet application working behind these servers via FastCGI/SCGI/HTTP-Proxy backends?
If the above true, how do they deal with this issue? I understand that I can timeout all connections every 10 seconds, but I would like to keep them idle as far as client listens -- because this allows easier scale up -- each connection is very cheep -- the cost is only the opended socket.
Thanks!
(1) Feature. Or, more specifically, fallout from an implementation detail.
A TCP/IP connection does not involve a constant flow of traffic back and forth. Thus, there is no way to know that a client is gone without (a) the client telling you it is closing the connection or (b) a timeout.
(2) I'm not specifically familiar with Comet or CppCMS. But, yes, there are all kinds of CMS servers running behind the mentioned web servers and they all have to deal with this issue (and, yes, it is a pain).
(3) Timeouts are the only way, but you can mitigate the pain, so to speak. Have the client ping the server across the connection every N seconds when there is otherwise no activity. Doesn't have to do anything and you can tack stuff on the reply; notifications of concurrent edits or whatever you need.
You are correct in that it is surprising that mod_fastcgi doesn't support telling the backend that Apache has detected the disconnect or the connection timed out. And you aren't the first to be dismayed.
The second patch on this page should fix that particular issue:
http://osdir.com/ml/web.fastcgi.devel/2006-02/msg00015.html
http://ncannasse.fr/blog/tora_comet
I don't have any concrete information for you, but this article does mention that they can detect when the client has disconnected from Apache. See tora.Queue. And it sounds like the source is available in the neko CVS, so you might be able to find some clues there. Good luck.