What are the advantages and disadvantages of setting it low (10s) or high (60s) and how does it different from keepalive_timeout?
From the docs:
Syntax: keepalive_timeout timeout [header_timeout];
Default: keepalive_timeout 75s;
Context: http, server, location
The first parameter sets a timeout during which a keep-alive client
connection will stay open on the server side. The zero value disables
keep-alive client connections.
The optional second parameter sets a
value in the “Keep-Alive: timeout=time” response header field. Two
parameters may differ.
The “Keep-Alive: timeout=time” header field is recognized by Mozilla
and Konqueror. MSIE closes keep-alive connections by itself in about
60 seconds.
Syntax: send_timeout time;
Default: send_timeout 60s;
Context: http, server, location
Sets a timeout for transmitting a response to the client. The timeout
is set only between two successive write operations, not for the
transmission of the whole response. If the client does not receive
anything within this time, the connection is closed.
So when using keepalive_timeout, the browser does not have to make multiple connections, but uses the already established connection. This controls how long that stays active / open.
I would suggest that if you set the send_timeout small then your web server, it will close connections quickly, which will give more overall connections available to connecting hosts.
These parameters are most likely only relevant in a high traffic webserver, both supporting the same goal: less connections and more efficient handling of requests, either putting all requests into one connection (keep alive) or closing connections quickly to handle more requests (send timeout).
Related
I have an application that accepts TCP traffic (not HTTP) and I'd like the ability to have the traffic load balanced to it. However, one requirement is that when a client makes a connection, we do not close that connection under any circumstances (ideally) since we are dealing with some clients with older technology.
I've set up the kubernetes nginx ingress controller, but it isn't behaving how I'm hoping/expecting. What I would like is: If the connection to one of the upstream servers closes, then the client connection remains open for some amount of time while nginx picks a new upstream server and starts sending data to it. I am not concerned about the stream's data being split across different upstream servers, I just need the connection to stay open from the client's perspective during something like a redeploy.
What is actually happening is that from my client's perspective, currently when the upstream server app closes the connection, my connection is closed and I have to reconnect.
The ingress controller has this configuration, which I thought would accomplish what I want, but it doesn't seem to be working as expected:
server {
preread_by_lua_block {
ngx.var.proxy_upstream_name="tcp-my-namespace-my-service-7550";
}
listen 7550;
proxy_timeout 600s;
proxy_next_upstream on;
proxy_next_upstream_timeout 600s;
proxy_next_upstream_tries 3;
proxy_pass upstream_balancer;
}
Any help at all is greatly appreciated and I'm happy to provide more info.
What you describe is how nginx works out of the box with http. However
Nginx has a detailed understanding of http
HTTP is a message based protocol i.e. uses requests and replies
Since nginx knows nothing about the protocol you are using, even if it uses a request/reply mechanism with no implied state, nginx does not know whether it has received a request not to to replay it elsewhere.
You need to implement a protol-aware mitm.
Unfortunately I haven't been able to get this functionality working with nginx. What I've ended up doing is writing my own basic TCP reverse-proxy that does what I need - if a connection to a backend instance is lost, it attempts to get a new one without interrupting the frontend connection. The traffic that we receive is fairly predictable in that I don't expect that moving the connection will interrupt any of the "logical" messages on the stream 99% of the time.
I'd still love to hear if anyone knows of an existing tool that has this functionality, but at the moment I'm convinced that there isn't one readily available.
I think you need to configure your Nginx Ingress to enable the keepalive options as listed in the documentation here. For instance in your nginx configuration as:
...
keepalive 32;
...
This will activate the keepalive functionality with a cache of upto 32 connections active at a time.
I have client (10.1.30.29), that sends HTTP requests to server (port 6500, 10.1.30.11-127.0.0.1) behind NGINX reverse proxy (port 80, 10.1.30.11-127.0.0.1). Most of the times (~5 of 6), server is not receiving requests.
Digged into with wireshark, I found that client sends TCP FIN ACK packet after it sends data and before NGINX responded with TCP ACK for data:
Data transmission from NGINX to server starts, but ends before any data was transferred.
In other (correct) case data is fully transmitted:
Key difference from first case is that NGINX managed to send ACK on data before client have sent FIN ACK.
In both cases access log in NIGNX contains records about requests; error log is empty.
Unfortunately, I barely can influence the client's behavior, but I know, that other HTTP server implementations can work with request data even if client incorrect closes TCP transmission. The question is if there any way to force NGINX to ignore such incorrect client's behavior and always proxy request data?
P. S. already tried postpone_output NGINX option - no luck.
Found two solutions, that seems pretty similar and valid (in my case):
proxy_ignore_client_abort on;
2. proxy_http_version 1.1;
proxy_request_buffering off;
From nginx.org, the default value of keepalive config is —, however I don't quite understand what this means.
Syntax: keepalive connections;
Default: —
Context: upstream
This directive appeared in version 1.1.4.
In order Nginx to keep TCP connection alive both upstream section and origin server should be configured to not finalise the connection. Upstream section keepalive default value means no keepalive, hence connection won't be reused, each time you can see TCP stream number increases per every request to origin server, opposite to what happens with keepalive. You can check it with using tcpdump.
10 Tips for 10x Application Performance blog post describes it very well:
Client keepalives – Keepalive connections reduce overhead, especially
when SSL/TLS is in use. For NGINX, you can increase the maximum number
of keepalive_requests a client can make over a given connection from
the default of 100, and you can increase the keepalive_timeout to
allow the keepalive connection to stay open longer, resulting in
faster subsequent requests.
Upstream keepalives – Upstream connections – connections to
application servers, database servers, and so on – benefit from
keepalive connections as well. For upstream connections, you can
increase keepalive, the number of idle keepalive connections that
remain open for each worker process. This allows for increased
connection reuse, cutting down on the need to open brand new
connections. For more information, refer to our blog post, HTTP
Keepalive Connections and Web Performance.
See also RFC-793 Section 3.5:
A TCP connection may terminate in two ways: (1) the normal TCP close
sequence using a FIN handshake, and (2) an "abort" in which one or
more RST segments are sent and the connection state is immediately
discarded. If a TCP connection is closed by the remote site, the local
application MUST be informed whether it closed normally or was
aborted.
Two examples, take a look on Application Data below.
Without keepalive:
With keepalive:
Is my reasoning about HTTP Keep-Alive correct? Here are my assumptions:
In modern browsers, persistent connections are the default, so the Connection: Keep-Alive response header isn't likely to be sent by HAProxy for those clients. Not seeing it does not necessarily mean that the connection is not persistent.
A Keep-Alive is meant to be a short duration, for example a few hundred milliseconds, during which the client can reuse the same connection to the same server (for uses such as downloading images, CSS and JavaScript that are on the same page)
Longer persistence can be accomplished by cookies or stick tables
The reason I ask is because when I set a long keep-alive timeout (via timeout http-keep-alive), typing F5 rapidly loads a different backend server just as fast, rotating through all three without any effect from the keep-alive. I would have thought that if I hit F5 during the keep-alive timeout period of five seconds, I'd still get the same backend server.
Am I thinking about this wrong?
Here is my HAproxy config where I've set the keep-alive timeout to 5 seconds:
defaults
timeout connect 5s
timeout client 120s
timeout server 120s
timeout http-request 5s
timeout http-keep-alive 5s
option http-keep-alive
frontend http
mode http
bind *:80
default_backend servers1
backend servers1
mode http
balance roundrobin
server web1 web1:80 check
server web2 web2:80 check
server web3 web3:80 check
UPDATE
Using Wireshark, it looks like, from the client perspective, the client is reusing the same socket connection to HAProxy's frontend until the timeout expires. So the keep-alive appears to be working between client and frontend. However, I added an image to the Web page to see which backend server would serve it, and if it would be a different server than the one that served the HTML...and it was different. In roundrobin mode, two different backend servers served up content to the same client, even though I had keep-alive enabled.
This all seems to me like keep-alive is not working between frontend and backend. This is the behavior I would expect if I had set option http-server-close, but not when using option http-keep-alive which should keep the connection to the server open.
you should ensure your server does not close the connection.
you should enable the "option prefer-last-server", in your HAProxy's defaults section.
you should re-read the "option http-keep-alive" documentation of HAProxy: http://cbonte.github.io/haproxy-dconv/configuration-1.6.html#option%20http-keep-alive.
It clearly states that:
If the client request has to go to another backend or another server due to content switching or the load balancing algorithm, the idle connection will immediately be closed and a new one re-opened. Option "prefer-last-server" is available to try optimize server selection so that if the server currently attached to an idle connection is usable, it will be used.
(hence point 2)
So your observations looks normal.
We use nginx with an application server as a backend.
We need to limit number of simultaneous connections per IP to backend. We used limit_conn nginx directive for this purpose. But it doesn't work well in all cases.
If user generates a lot of connections from one IP and quickly closes them, then nginx passes this request to a backend, but because client connection is already closed, this connection is not count in limit_conn.
Is it possible to limit number of simultaneous connections per IP to backend server with nginx?
You may want to set
proxy_ignore_client_abort off;
Determines should the connection with a proxied server be closed if a
client closes a connection without waiting for a response.
from the documentation
Another suggestion is to use limit_req to limit the request rate.
I'm afraid this facility is not yet available for nginx out of the box. According to the Nginx FAQ
Many users have requested that Nginx implement a feature in the load
balancer to limit the number of requests per backend (usually to one).
While support for this is planned, it's worth mentioning that demand
for this feature is rooted in misbehaviour on the part of the
application being proxied
I've seen some 3rd parties module for that nginx-limit-upstream but I've never tried.