Is there are a way to enable rate limiting only for successful requests (i.e. HTTP status code 200)?
For example, in the following snippet from my configuration...
http {
limit_req_zone $binary_remote_addr zone=test:10m rate=2r/m;
server {
location / {
limit_req zone=test;
proxy_pass http://localhost:3000/;
...
}
...
}
...
}
...requests are successfully rate limited (up to two requests per minute).
However, as this is for a contact form which sends me emails, I do not care about rate limiting if http://localhost:3000/ returns an error as no email will have been sent.
No, there isn't.
Nginx processes HTTP request in 11 phases from reading request to sending reponse: post-read, server-rewrite, find-config, rewrite, post-rewrite, pre-access, access, post-access, try-files, content, log.
proxy_pass is in content phase while limit_req is in pre-access phase (refer ngx_http_limit_req_module.c), pre-access phase handlers is executed before content phase handlers, so limit_req handler can not check if the response code is OK.
Related
Is there any way to tell nginx to ignore missing HTTP headers when proxying requests?
There is an existing proprietary HTTP Server sending requests without any header. The Server can not be configured. I need various endpoints from this server in a web application. Therefore I want to setup my nginx to proxy requests to this server. I have location configuration in my regular server.
location /api/ {
proxy_pass http://localhost:80/;
}
When calling corresponding URIs nginx complains:
upstream sent no valid HTTP/1.0 header while reading response header from upstream, client: 127.0.0.1, server: localhost, request: ....
Is there any way to tell nginx not to expect any headers, and just to forward the received payload?
Thanks for your help!
Kind regards,
Andreas
Edit: Ok, found server is using http 0.9, when calling curl directly to the server threw an error:
curl: (1) Received HTTP/0.9 when not allowed
Using the option --http0.9 got the desired result. (Which is received in a browser without further ado). Any chance to tell nginx to proxy to an http 0.9 server?
You can configure nginx to ignore missing headers by setting the proxy_ignore_headers directive to "X-Accel-Expires" and "Expires". This will tell nginx to ignore those specific headers and not expect them to be present when proxying requests.
You can add this to the location block in your nginx configuration:
proxy_ignore_headers X-Accel-Expires Expires;
Additionally, proxy_pass_request_headers off; could be used, this tells nginx to ignore all headers passed to the proxied server.
As for the payload, as long as the underlying protocol is HTTP, it should be fine, as the payload will be sent in the body of the HTTP request.
So your location block could look like this
location /api/ {
proxy_pass http://localhost:80/;
proxy_pass_request_headers off;
}
Please note that this is may not be recommended, as the missing headers may contain important information like authentication and should be handled correctly.
I am using a rate liming based on ip address and below example work's perfectly for that.
http {
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
...
server {
...
location /search/ {
limit_req zone=one burst=5;
}
Now we need to implement rate limiting based on apiid/apikey which will part of http request.Each api key will have restricted number of connections and when that goes beyond the restricted number i must give 503 or some thing like that.
How to get the apikey/apid from url into a variable and set a limit for each apikey we have ?
Let's say I have ip_hash; turned on for load balancing between 4 different servers. So, client's IP address is used as a hashing key to determine which server his requests get routed to.
However, for file upload, it's best to keep all files in a single server. So, I want all /upload requests get routed to server 1 for any client. This means all requests obey IP-hash, except POST /upload which must be sent to server 1.
Is there a way to create this exception in NGINX? Thanks!
Define two upstream containers, one with full load balancing and another with the POST specific service requirements:
upstream balancing { ... }
upstream uploading { ... }
Also, within the http container, define a map of the request method:
map $request_method $upstream {
default balancing;
POST uploading;
}
Finally, within the server container, define a specific proxy_pass for the /upload URI:
location / {
proxy_pass http://balancing;
}
location /upload {
proxy_pass http://$upstream;
}
The upstream specification is evaluated from the value of the REQUEST_METHOD.
I'd like to throttle incoming requests into an nginx route.
The current config is similar to this:
upstream up0 {
server x.x.x.x:1111;
keepalive 1024;
}
server {
location /auc {
limit_req zone=one burst=2100;
proxy_pass http://up0/auc;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}
I'd like to control the number of requests I see at the upstream server. For all other requests I'd like nginx to respond with a 204 response.
Controlling by percentage of incoming requests would also work.
Thanks.
Nginx is very effective at limiting requests using limit_req_zone and limit_req.
First create a zone which has defined limits. For a global limit, the key of the zone can be static, it's also possible to use variables such as the source ip address as a key for the zone which is useful for limiting by specific ip's or just slower pages on your site. The rate can be defined in requests per second or minute.
limit_req_zone key zone=name:size rate=rate;
Next, create a rule to apply that zone to incoming requests. The location directive can be used first to apply the rule only to specific requests or it can be server wide. The burst option will queue a specified number requests that exceed the rate limit and is useful to throttle short bursts of traffic rather than return errors.
limit_req zone=name [burst=number] [nodelay];
The default response code for traffic exceeding the rate limit and not held in a burst queue is 503 (Service Unvailable). Alternate codes like 204 (No content) can be set.
limit_req_status code;
Putting all that together a valid config to limit all requests in the location block to 10 per second with a buffer to queue up to 50 requests before returning errors and return the specified 204 response could would look like:
http {
....
limit_req_zone $hostname zone=limit:20m rate=10r/s;
limit_req_status 204;
server {
...
location / {
...
limit_req zone=limit burst=50;
}
}
}
In practice it's likely the server block will be in a different file included from within the http block. I've just condensed them for clarity.
To test, either use a flood tool or set the request rate=10r/m (10 per minute) and use a browser. It's useful to check the logs and monitor the amount of rejected requests so that you are aware of any impact on your users.
Multiple limit_req_zone rules can be combined to specify loose global limits and then stricter per source ip limits. This will enable targeting of the most persistent few users before the wider user base.
I am experiencing unusual behavior with rate limiting in NGinX. I have been tasked with supporting 10 requests per second and not to use the burst option. I am using the nodelay option to reject any requests over my set rate.
My config is:
..
http
{
..
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;
..
server
{
..
location /
{
limit_req zone=one nodelay;
limit_req_status 503;
..
}
}
}
The behavior I am seeing is if a request is sent before a response is received from a previous request NGinX will return a 503 error. I see this behavior with as little as 2 requests in a second.
Is there something missing from my configuration which is causing this behavior?
Is the burst option needed to service multiple requests at once?
Burst Works like a queue. No delay means the requests will not be delayed for next second. If you are not specifying a queue then you are not allowing any other simultaneous request to come in from that IP. The zone takes effect for per ip as your key is $binary_remote_addr.
You need a burst.