I'm trying to make nginx delay connections so that at most 1 is processes per IP at a time.
I'm able to achieve dropping excess requests with error 503 like this:
limit_zone addr $binary_remote_addr 10m;
limit_conn addr 1;
But I want to make excess processes wait until the previous one is completed instead. Is it possible?
Related
What are the advantages and disadvantages of setting it low (10s) or high (60s) and how does it different from keepalive_timeout?
From the docs:
Syntax: keepalive_timeout timeout [header_timeout];
Default: keepalive_timeout 75s;
Context: http, server, location
The first parameter sets a timeout during which a keep-alive client
connection will stay open on the server side. The zero value disables
keep-alive client connections.
The optional second parameter sets a
value in the “Keep-Alive: timeout=time” response header field. Two
parameters may differ.
The “Keep-Alive: timeout=time” header field is recognized by Mozilla
and Konqueror. MSIE closes keep-alive connections by itself in about
60 seconds.
Syntax: send_timeout time;
Default: send_timeout 60s;
Context: http, server, location
Sets a timeout for transmitting a response to the client. The timeout
is set only between two successive write operations, not for the
transmission of the whole response. If the client does not receive
anything within this time, the connection is closed.
So when using keepalive_timeout, the browser does not have to make multiple connections, but uses the already established connection. This controls how long that stays active / open.
I would suggest that if you set the send_timeout small then your web server, it will close connections quickly, which will give more overall connections available to connecting hosts.
These parameters are most likely only relevant in a high traffic webserver, both supporting the same goal: less connections and more efficient handling of requests, either putting all requests into one connection (keep alive) or closing connections quickly to handle more requests (send timeout).
From nginx.org, the default value of keepalive config is —, however I don't quite understand what this means.
Syntax: keepalive connections;
Default: —
Context: upstream
This directive appeared in version 1.1.4.
In order Nginx to keep TCP connection alive both upstream section and origin server should be configured to not finalise the connection. Upstream section keepalive default value means no keepalive, hence connection won't be reused, each time you can see TCP stream number increases per every request to origin server, opposite to what happens with keepalive. You can check it with using tcpdump.
10 Tips for 10x Application Performance blog post describes it very well:
Client keepalives – Keepalive connections reduce overhead, especially
when SSL/TLS is in use. For NGINX, you can increase the maximum number
of keepalive_requests a client can make over a given connection from
the default of 100, and you can increase the keepalive_timeout to
allow the keepalive connection to stay open longer, resulting in
faster subsequent requests.
Upstream keepalives – Upstream connections – connections to
application servers, database servers, and so on – benefit from
keepalive connections as well. For upstream connections, you can
increase keepalive, the number of idle keepalive connections that
remain open for each worker process. This allows for increased
connection reuse, cutting down on the need to open brand new
connections. For more information, refer to our blog post, HTTP
Keepalive Connections and Web Performance.
See also RFC-793 Section 3.5:
A TCP connection may terminate in two ways: (1) the normal TCP close
sequence using a FIN handshake, and (2) an "abort" in which one or
more RST segments are sent and the connection state is immediately
discarded. If a TCP connection is closed by the remote site, the local
application MUST be informed whether it closed normally or was
aborted.
Two examples, take a look on Application Data below.
Without keepalive:
With keepalive:
I'd like to throttle incoming requests into an nginx route.
The current config is similar to this:
upstream up0 {
server x.x.x.x:1111;
keepalive 1024;
}
server {
location /auc {
limit_req zone=one burst=2100;
proxy_pass http://up0/auc;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}
I'd like to control the number of requests I see at the upstream server. For all other requests I'd like nginx to respond with a 204 response.
Controlling by percentage of incoming requests would also work.
Thanks.
Nginx is very effective at limiting requests using limit_req_zone and limit_req.
First create a zone which has defined limits. For a global limit, the key of the zone can be static, it's also possible to use variables such as the source ip address as a key for the zone which is useful for limiting by specific ip's or just slower pages on your site. The rate can be defined in requests per second or minute.
limit_req_zone key zone=name:size rate=rate;
Next, create a rule to apply that zone to incoming requests. The location directive can be used first to apply the rule only to specific requests or it can be server wide. The burst option will queue a specified number requests that exceed the rate limit and is useful to throttle short bursts of traffic rather than return errors.
limit_req zone=name [burst=number] [nodelay];
The default response code for traffic exceeding the rate limit and not held in a burst queue is 503 (Service Unvailable). Alternate codes like 204 (No content) can be set.
limit_req_status code;
Putting all that together a valid config to limit all requests in the location block to 10 per second with a buffer to queue up to 50 requests before returning errors and return the specified 204 response could would look like:
http {
....
limit_req_zone $hostname zone=limit:20m rate=10r/s;
limit_req_status 204;
server {
...
location / {
...
limit_req zone=limit burst=50;
}
}
}
In practice it's likely the server block will be in a different file included from within the http block. I've just condensed them for clarity.
To test, either use a flood tool or set the request rate=10r/m (10 per minute) and use a browser. It's useful to check the logs and monitor the amount of rejected requests so that you are aware of any impact on your users.
Multiple limit_req_zone rules can be combined to specify loose global limits and then stricter per source ip limits. This will enable targeting of the most persistent few users before the wider user base.
I am experiencing unusual behavior with rate limiting in NGinX. I have been tasked with supporting 10 requests per second and not to use the burst option. I am using the nodelay option to reject any requests over my set rate.
My config is:
..
http
{
..
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;
..
server
{
..
location /
{
limit_req zone=one nodelay;
limit_req_status 503;
..
}
}
}
The behavior I am seeing is if a request is sent before a response is received from a previous request NGinX will return a 503 error. I see this behavior with as little as 2 requests in a second.
Is there something missing from my configuration which is causing this behavior?
Is the burst option needed to service multiple requests at once?
Burst Works like a queue. No delay means the requests will not be delayed for next second. If you are not specifying a queue then you are not allowing any other simultaneous request to come in from that IP. The zone takes effect for per ip as your key is $binary_remote_addr.
You need a burst.
We use nginx with an application server as a backend.
We need to limit number of simultaneous connections per IP to backend. We used limit_conn nginx directive for this purpose. But it doesn't work well in all cases.
If user generates a lot of connections from one IP and quickly closes them, then nginx passes this request to a backend, but because client connection is already closed, this connection is not count in limit_conn.
Is it possible to limit number of simultaneous connections per IP to backend server with nginx?
You may want to set
proxy_ignore_client_abort off;
Determines should the connection with a proxied server be closed if a
client closes a connection without waiting for a response.
from the documentation
Another suggestion is to use limit_req to limit the request rate.
I'm afraid this facility is not yet available for nginx out of the box. According to the Nginx FAQ
Many users have requested that Nginx implement a feature in the load
balancer to limit the number of requests per backend (usually to one).
While support for this is planned, it's worth mentioning that demand
for this feature is rooted in misbehaviour on the part of the
application being proxied
I've seen some 3rd parties module for that nginx-limit-upstream but I've never tried.