I want to limit requests to my service depending on the time of day.
It should be 4 r/s by night (21:00 - 06:59) and 1 r/s by day (07:00-20:59).
I could have 2 different req_limit_zone like
limit_req_zone $server_name zone=day:1m rate=1r/s;
limit_req_zone $server_name zone=night:1m rate=4r/s;
but how can I differ them in limit_req depending on the time of day?
The service behind nginx is an API wrapper for another service with limits described above. My service is written with Flask-RESTful, runs by uWSGI with a few processes, so it would be pretty painful to implement limiting logic and sync it between processes with something like redis.
Is it possible to configure nginx this way?
If it's not, are there any common workarounds? How do other services solve this task?
To be more specific, it should be a leaking bucket, so clients just wait for an answer without any errors like HTTP 429 Too Many Requests.
The answer
The key is to bind limit_req_zone and map.
There are two relevant notes about them in the docs. map:
default value
sets the resulting value if the source value matches none of the specified variants. When default is not specified, the default resulting value will be an empty string.
and limit_req_zone:
limit_req_zone key zone=name:size rate=rate;
Sets parameters for a shared memory zone that will keep states for various keys. In particular, the state stores the current number of excessive requests. The key can contain text, variables, and their combination. Requests with an empty key value are not accounted.
So one does simply use both day and night limit_req_zone with $server_name as key if it should work and with empty string if it shouldn't. And map will return either $server_name or empty string depending on the time of day.
map $date_gmt $day {
# 07:00-20:59 GMT
~(0[7-9]|1[0-9]|20):[0-5][0-9]:[0-5][0-9] $server_name;
}
map $date_gmt $night {
# 21:00-06:59 GMT
~(2[1-4]|0[0-6]):[0-5][0-9]:[0-5][0-9] $server_name;
}
limit_req_zone $day zone=day_zone:1m rate=1r/s;
limit_req_zone $night zone=night_zone:1m rate=4r/s;
...
limit_req zone=day_zone burst=100;
limit_req zone=night_zone burst=100;
Some notes about req_limit
At first attempt I tried to use mapped variable in limit_req, but nginx don't understand such syntax. Further more, at first glance nginx -s reload worked without any problem, but in fact there was no any reload and only service nginx restart showed an error (nginx version: 1.12.2). So I'd like to show how one should not to do:
limit_req_zone $server_name zone=day:1m rate=1r/s;
limit_req_zone $server_name zone=night:1m rate=4r/s;
map $date_gmt $time_of_day {
~(0[7-9]|1[0-9]|20):[0-5][0-9]:[0-5][0-9] day;
~(2[1-4]|0[0-6]):[0-5][0-9]:[0-5][0-9] night;
}
...
limit_req zone=$time_of_day burst=100;
Some notes about performance
It's not the best solution because of double time checking on every request. There is option to have two separate nginx configs and switch them with something like cron. It would be ugly and buggy because you should remember to edit two configs, but it would work faster; I think this option should be used as a last resort. If there is an ability for horizontal scaling, it's better to load balance multiple servers.
In my case it's not a big deal: service gets just about 8k requests per hour (2.2 r/s).
Related
I have a website that allows users to convert their zipcode to their street address, city, and some other info. This is a website that is intended for end users and not for companies. However, since last week, I am getting lots of requests (1000 a day or so) from a couple of IPs and after some debugging it seems that they are (ab)using my website for commercial purposes.
I want my NGINX to only allow 5 POST requests from a unique IP per day, unless the requests are from my own IPs (office and home IPs). I am okay with GET requests having no limit. Also, I only want this to be applicable on this specific vhost and not my entire nginx config. How should i proceed? Is this even possible?
Nginx doesn't allow rate limits slower than 1 request per minute. However, you could use this custom version of nginx that seems to have the feature you want. With it, you could do something like:
limit_req_zone $binary_remote_addr zone=zonepost:10m rate=1r/d; # limit to 1 request per day
Then in your locations you could use an if to check if the request method is get or post and apply the limit_req accordingly. For example, you could have a second less restrictive zone and configure it like so:
limit_req_zone $binary_remote_addr zone=zonepost:10m rate=5r/d; # limit 5 per day
limit_req_zone $binary_remote_addr zone=zoneget:10m rate=10r/s; # limit 10 per sec
server {
location / {
# Select the right zone (limit)
if( $request_method = get ) {
set $zone "zoneget";
}
if( $request_method = post ) {
set $zone "zonepost";
}
limit_req zone=$zone;
# Do other stuff...
}
}
Refer to this page on nginx.com for a manual.
For most of my website I have a normal rate limit of 1 request/second:
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
I now need a limit of 10 per day for a certain location. Is that possible?
limit_req_zone $binary_remote_addr zone=loginmin:10m rate=1tth;
I get: invalid rate "rate=1tth"
A quote from the article you linked:
After source code review I’ve found a way to decrease minimal rate down to 86 requests per day. Patch below adds ability to use next configuration directive:
There are two things to learn from this:
You will need to patch the source code of Nginx for the approach in the article to work.
Even with the patch in place, you will only be able to throttle it down to 86 requests/day.
Therefore we can conclude that what you are looking for is currently not possible with Nginx alone.
My suggestion is to use fail2ban for this purpose. You can create a custom jail:
# /usr/local/etc/fail2ban/jail.d/nginx-extreme-ratelimit.local
[nginx-extreme-ratelimit]
enabled = true
filter = nginx-ratelimit
action = pf <-- or ufw, ipfilter, ipfw, whatever firewall you use
logpath = /var/log/nginx-access.log
findtime = 86400
maxretry = 10
bantime = 86400
And a respective custom filter:
# /usr/local/etc/fail2ban/filter.d/nginx-ratelimit.conf
[Definition]
failregex = ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>,
ignoreregex =
datepattern = {^LN-BEG}
The failregex obviously has to be adjusted to match the access string.
Additionally, set up a cron job that runs at 12 pm every day to unban the IP address:
00 00 * * * /usr/local/bin/fail2ban-client set nginx-extreme-ratelimit unbanip 1.2.3.4
With nginx-mod, you can simply put:
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/d;
Of course, your traffic should be small enough (~less than 160K visitors to the rate limited resources) with this config.
Or you'll need to adjust zone's memory size to be large enough to accommodate all visitor IP addresses to the rate limited resources within 24 hrs.
is it possible to use dynamic connection limits in nginx?
lets say i have the following routes:
/route/1
/route/2
...
/route/*
I do not want to have a global rate limit for /route/* but a specific for each route. Is that possible in nginx?
So that each route have a connection limit of 2 connections in a minute.
What i think of: everything that comes after /route/ should be act as an id. And each id has its own connection limit.
Maybe i could be somehting like:
limit_conn_zone $request_uri zone=addr:10m;
server {
...
limit_conn addr 1;
}
But im not sure, if this works as i expect.
limit_conn can be used inside location block. But limit_conn restrict number of simultaneous connections. If you want to limit rate, you can use limit_req module http://nginx.org/en/docs/http/ngx_http_limit_req_module.html which can be used inside location too.
Also, if you want separate limits for each location - there is two ways. First - separate zones (limit_req_zone) for each location. Second - one zone, but using route as key. First case usually better due memory usage, but in you case (unlimited number of routes) second way is better. So, just extract your ID from route and use it as limit_req_zone key.
limit_req_zone $myid zone=one:50m rate=2r/m;
...
location ~ ^/route/(?<myid>\d+) {
limit_req zone=one;
}
If you need separate limit for each location for each client IP address, use limit_req_zone $binary_remote_addr$myid ... key.
I found that ngx_http_limit_req_module can be used to limit the maximum number of requests per time. But in my understanding, this applies to a whole virtual location. What I want is limit the rate per arbitrary URL.
Example:
I want requests to /api/list/1/votes to be blocked for a specific client for 30 seconds after the client has made one request. However, he can should still be able to call /api/list/2/votes (but after that call, also the /2 route should get blocked for several seconds).
My initial idea was to use a regex to define a location for every route that ends with /votes, but than (in the example) /1 AND /2 would be blocked, which is not what I want.
Any ideas?
You need to include the id in the key like this:
limit_req_zone $vote_limit_key zone=votes:10m rate=1r/s;
map $request_uri $vote_limit_key {
default ""; # ignore urls not matching, these will not be rate limited
"~^/api/list/([0-9]+)/votes$" ${binary_remote_addr}$1;
}
location ~* ^/api/list/([0-9])/votes$ {
# apply rate limiting
limit_req zone=votes burst=5;
}
The map will match the url, extract the id and combine it with $binary_remote_addr to provide a unique key per IP address and id. Of course you can modify the map to handle other urls too, or default to $binary_remote_addr to treat all other urls as one and only limit by IP address.
You can set the nginx config to something like this
limit_req_zone $binary_remote_addr zone=votes:10m rate=1r/s;
location ~* ^/api/list/([0-9])/votes$ {
# apply rate limiting
limit_req zone=votes burst=5;
}
The zone=votes:10m specifies creates a shared memory zone called "votes" to store a log of IP addresses that access the rate limited URL(s). We set rate=1r/s to specify that only one request will be allowed to be served from this IP per second. The burst argument tells Nginx to start dropping requests if more than 5 queue up from a specific IP. The regex that I have placed ([0-9]) can be replace with any regex that matches your route.
Refer this blog post for more on limiting request rate.
I'm developing a facebook canvas application and I want to load-test it. I'm aware of the facebook restriction on automated testing, so I simulated the graph api calls by creating a fake web application served under nginx and altering my /etc/hosts to point graph.facebook.com to 127.0.0.1.
I'm using jmeter to load-test the application and the simulation is working ok. Now I want to simulate slow graph api responses and see how they affect my application. How can I configure nginx so that it inserts a delay to each request sent to the simulated graph.facebook.com application?
You can slow the speed of localhost (network) by adding delay.
Use ifconfig command to see network device: on localhost it may be lo and on LAN its eth0.
to add delay use this command (adding 1000ms delay on lo network device)
tc qdisc add dev lo root netem delay 1000ms
to change delay use this one
tc qdisc change dev lo root netem delay 1ms
to see current delay
tc qdisc show dev lo
and to remove delay
tc qdisc del dev lo root netem delay 1000ms
My earlier answer works but it is more adapted to a case where all requests need to be slowed down. I've since had to come up with a solution that would allow me to turn on the rate limit only on a case-by-case basis, and came up with the following configuration. Make sure to read the entire answer before you use this, because there are important nuances to know.
location / {
if (-f somewhere/sensible/LIMIT) {
echo_sleep 1;
# Yes, we need this here too.
echo_exec /proxy$request_uri;
}
echo_exec /proxy$request_uri;
}
location /proxy/ {
internal;
# Ultimately, all this goes to a Django server.
proxy_pass http://django/;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
}
Important note: the presence or absence of forward slashes in the various paths makes a difference. For instance, proxy_pass http://django, without a trailing slash, does not do the same thing as the line in the code above.
The principle of operation is simple. If the file somewhere/sensible/LIMIT exists, then requests that match location / are paused for one second before moving on. So in my test suite, when I want a network slowdown, I create the file, and when I want to remove the slowdown, I remove it. (And I have cleanup code that removes it between each test.) In theory I'd much prefer using variables for this than a file, but the problem is that variables are reinitialized with each request. So we cannot have a location block that would set a variable to turn the limit, and another to turn it off. (That's the first thing I tried, and it failed due to the lifetime of variables). It would probably be possible to use the Perl module or Lua to persist variables or fiddle with cookies, but I've decided not to go down these routes.
Important notes:
It is not a good idea to mix directives from the echo module (like echo_sleep and echo_exec) with the stock directives of nginx that result in the production of a response. I initially had echo_sleep together with proxy_pass and got bad results. That's why we have the location /proxy/ block that segregates the stock directives from the echo stuff. (See this issue for a similar conflict that was resolved by splitting a block.)
The two echo_exec directives, inside and outside the if, are necessary due to how if works.
The internal directive prevents clients from directly requesting /proxy/... URLs.
I've modified a nginx config to use limit_req_zone and limit_req to introduce delays. The following reduces the rate of service to 20 requests per second (rate=20r/s). I've set burst=1000 so that my application would not get 503 responses.
http {
limit_req_zone $binary_remote_addr zone=one:10m rate=20r/s;
[...]
server {
[...]
location / {
limit_req zone=one burst=1000;
[...]
}
}
}
The documentation is here. I do not believe there is a way to specify a uniform delay using this method.