More specific, will this work ?
upstream backend {
hash $request_uri consistent;
server backend1.example.com weight=1;
server backend2.example.com weight=2;
}
will backend2.example.com receive twice as much traffic ?
Also, what happens if a weight is changed or another server is added to the mix. Will the "only few keys will be remapped" still hold ?
The optional consistent parameter of the hash directive enables ketama consistent hash load balancing. Requests will be evenly distributed across all upstream servers based on the user-defined hashed key value. If an upstream server is added to or removed from an upstream group, only few keys will be remapped which will minimize cache misses in case of load balancing cache servers and other applications that accumulate state.
from https://www.nginx.com/resources/admin-guide/load-balancer/
In this configuration, consistent hash is more important than weight.
In other words, if one upstream presents both weight and a consistent hash, then the main thing will be a consistent hash.
And hashes are distributed to the servers according to the weight.
upstream consistent_test {
server consistent_test.example.ru:80 weight=90;
server consistent_test2.example.ru:80 weight=10;
hash $arg_consistent consistent;
}
Experiment
1) Default state
upstream balancer_test {
hash $arg_someid consistent;
server server1.example.ru:8080;
server server2.example.ru:8080;
server server3.example.ru:8080 down;
}
Request hashes pined to hosts:
server1.example.ru ==> 535
server2.example.ru ==> 462
server3.example.ru ==> 0
2) First step: enable the node and set the weight
upstream balancer_test {
hash $api_sessionid consistent;
server server1.example.ru:8080 weight=250;
server server2.example.ru:8080 weight=500;
server server3.example.ru:8080 weight=250;
}
Request hashes pined to hosts:
server1.example.ru:8080 ==> 263
server2.example.ru:8080 ==> 473
server3.example.ru:8080 ==> 254
3) The second step: Finish the translation of traffic and disable the old node
upstream balancer_test {
hash $api_sessionid consistent;
server1.example.ru:8080 down;
server2.example.ru:8080;
server3.example.ru:8080;
}
Request hashes pined to hosts:
server1.example.ru:8080 ==> 0
server2.example.ru:8080 ==> 533
server3.example.ru:8080 ==> 464
server1.example.ru:
1) before = 463
2) on step_2 = 533
3) hash hits = 306
server2.example.ru:
1) before = 536
2) on step_1 = 263
3) hash hits = 148
server3.example.ru:
1) before = 255
2) on step 1 = 464
3) hash hits = 115
Related
I am using OpenResty to generate SSL certificates dynamically.
I am trying to find out the user-agent of request before running ssl_certificate_by_lua_block and decide If I want to continue with the request or not.
I found out that ssl_client_hello_by_lua_block directive runs before ssl_certificate_by_lua_block but if I try to execute ngx.req.get_headers()["user-agent"] inside ssl_client_hello_by_lua_block I get the following error
2022/06/13 09:20:58 [error] 31918#31918: *18 lua entry thread aborted: runtime error: ssl_client_hello_by_lua:6: API disabled in the current context
stack traceback:
coroutine 0:
[C]: in function 'error'
/usr/local/openresty/lualib/resty/core/request.lua:140: in function 'get_headers'
ssl_client_hello_by_lua:6: in main chunk, context: ssl_client_hello_by_lua*, client: 1.2.3.4, server: 0.0.0.0:443
I tried rewrite_by_lua_block but it runs after ssl_certificate_by_lua_block
Are there any directive that can let me access ngx.req.get_headers()["user-agent"] and run before ssl_certificate_by_lua_block as well?
My Nginx conf for reference.
nginx.conf
# HTTPS server
server {
listen 443 ssl;
rewrite_by_lua_block {
local user_agent = ngx.req.get_headers()["user-agent"]
ngx.log(ngx.ERR, "rewrite_by_lua_block user_agent -- > ", user_agent)
}
ssl_client_hello_by_lua_block {
ngx.log(ngx.ERR, "I am from ssl_client_hello_by_lua_block")
local ssl_clt = require "ngx.ssl.clienthello"
local host, err = ssl_clt.get_client_hello_server_name()
ngx.log(ngx.ERR, "hosts -- > ", host)
-- local user_agent = ngx.req.get_headers()["user-agent"]
-- ngx.log(ngx.ERR, "user_agent -- > ", user_agent)
}
ssl_certificate_by_lua_block {
auto_ssl:ssl_certificate()
}
ssl_certificate /etc/ssl/resty-auto-ssl-fallback.crt;
ssl_certificate_key /etc/ssl/resty-auto-ssl-fallback.key;
location / {
proxy_pass http://backend_proxy$request_uri;
}
}
If someone is facing the same issue.
Here is the email group of OpenResty that helped me.
I was not thinking correctly. The certificate negotiation happens before a client send user-agent data(that comes in after the SYNACK reaches the client). So you cant save issuing the certificate in the process. Hard luck.
Once the handshake and the Client/Server Hello happens then the server has the user-agent, you can do the blocking under access_by_lua_block.
We are trying to implement a request rate limit on our Django NGINX server. I went through some articles about how to do this, such as https://docs.nginx.com/nginx/admin-guide/security-controls/controlling-access-proxied-http/.
I can see that there's the ability to put a specific rate limit on an IP address, and also on a specific "location", that is an API endpoint. But is it also possible to group together some IP addresses and apply a cumulative rate limit.
For example, we have a partner that has 5 IP addresses and we want to apply a rate limit of let's say 10 rps on a specific API (/users/), but this 10 rps is a cumulative limit on all the 5 IP addresses, and NOT 10 rps for EACH IP address. I couldn't find on the official NGINX docs or anywhere on Google if this is possible or not.
For more clarity, let's consider these examples
123.123.123.123 - /users - 10 rps
123.123.123.123 - /products - 15 rps
45.45.45.45, 65.65.65.65 - /users - 20 rps
45.45.45.45, 65.65.65.65 - /products - 30 rps
So API requests to /users endpoint from IP 123.123.123.123 should be limited to 10 rps, and /products endpoint from the same IP should have a limit of 15 rps.
API requests to /users endpoint from both 45.45.45.45 and 65.65.65.65 COMBINED should be limited to 20 rps.
Similarly, for /products endpoint from both those IPs should be limited to 30 rps.
Any other IP will have let's say a default rate limit of 5 rps, no matter what API endpoint is called.
Hope this clears up any confusion! Thanks!
Your conditions are quite complicated. If, by example, requirements would be to apply a fixed 10 rps rate for the group of 5 IP addresses, and the same 10 rps rate for every other IP, you could do this with a single limit_req zone:
limit_req_zone $users_key zone=users:10m rate=10r/s;
geo $users_key {
<ip1> unique_customer_id;
<ip2> unique_customer_id;
<ip3> unique_customer_id;
<ip4> unique_customer_id;
<ip5> unique_customer_id;
default $binary_remote_addr;
}
server {
...
location /users/ {
limit_req zone=users;
... proxy request to the backend
}
}
That unique_customer_id can be any string that won't be equal to any possible $binary_remote_addr value (which is a binary-packed string of 4 or 16 bytes length, so using something as simple as 1 will be enough; don't use a long id string to minimize the shared zone memory footprint).
However since your requirements are more complicated, including different rates for different users, we need to declare many different zones to achieve the result you want. The solution is based upon a fact that an empty zone key won't be accounted to limit requests with a single limit_req rule.
Here is the possible solution.
# zones definitions
limit_req_zone $unknown_customer zone=users_5:10m rate=5r/s;
limit_req_zone $customer_limited_group_1 zone=users_10:1m rate=10r/s;
limit_req_zone $customer_limited_group_2 zone=users_20:1m rate=20r/s;
limit_req_zone $unknown_customer zone=products_5:10m rate=5r/s;
limit_req_zone $customer_limited_group_1 zone=products_15:1m rate=15r/s;
limit_req_zone $customer_limited_group_2 zone=products_30:1m rate=30r/s;
# calculating keys
map $remote_addr $unknown_customer {
123.123.123.123 '';
45.45.45.45 '';
65.65.65.65 '';
# exclude any other IP for customers with different rates here, e.g.
# 124.124.124.124 '';
# 85.85.85.85 '';
default $binary_remote_addr;
}
map $remote_addr $customer_limited_group_1 {
123.123.123.123 1;
# here you can add another customers limited to 10/15 rps with another id, e.g.
# 124.124.124.124 2;
# the default value will be an empty string
}
map $remote_addr $customer_limited_group_2 {
# this will be a key for a different zone;
# value '1' does not interfere with the '1' from the previous map block in any way
45.45.45.45 1;
65.65.65.65 1;
# another customers limited to 20/30 rps can be added here with different ids, e.g.
# 85.85.85.85 2;
}
server {
...
location /users/ {
limit_req zone=users_5;
limit_req zone=users_10;
limit_req zone=users_20;
... proxy request to the backend
}
location /products/ {
limit_req zone=products_5;
limit_req zone=products_15;
limit_req zone=products_30;
... proxy request to the backend
}
}
I have the following nginx.config file:
events {}
http {
# ...
# application version 1a
upstream version_1a {
server localhost:8090;
}
# application version 1b
upstream version_1b {
server localhost:8091;
}
split_clients "${arg_token}" $appversion {
50% version_1a;
50% version_1b;
}
server {
# ...
listen 7080;
location / {
proxy_set_header Host $host;
proxy_pass http://$appversion;
}
}
}
I have two nodejs servers listening on port 8090 and 8091 and I am hitting the URL http://localhost:7080, my expectation here is the Nginx will randomly split the traffic to version_1a and version_1b upstream, but, all the traffic is going to version_1a. Any insight into why this might be happening?
(I want to have this configuration for the canary traffic)
Validate the variable you are using to split the traffic is set correctly, and the variable's value should be uniformly distributed else the traffic will not be split evenly.
I'm trying to a/b (split) test two webpages on a single web application with a single hostname & instance.
Here's what I'm trying to achieve:
HTTP request for /
Request gets proxied to backend.app.com
Request is either proxied to backend.app.com/a or backend.app.com/b
Ideally that would be a sticky session that would maintain appropriate /a or /b during their session similar to what can be achieved with application pools.
Possible? Ideas?
You are looking for the split_clients directive.
Example from https://www.nginx.com/blog/performing-a-b-testing-nginx-plus/
http {
# ...
# application version 1a
upstream version_1a {
server 10.0.0.100:3001;
server 10.0.0.101:3001;
}
# application version 1b
upstream version_1b {
server 10.0.0.104:6002;
server 10.0.0.105:6002;
}
split_clients "${arg_token}" $appversion {
95% version_1a;
* version_1b;
}
server {
# ...
listen 80;
location / {
proxy_set_header Host $host;
proxy_pass http://$appversion;
}
}
}
The arg_token in this case can be pretty much any variable you want.
I have started a openresty with one tcp server and two backends. The tcp server dispatch the request to backends according to the content from tcp stream. Following is an example of openresty configuration:
stream {
# define a TCP server listening on the port 1234:
upstream backend1 {
server 172.17.0.1:8081;
}
upstream backend2 {
server 172.17.0.1:8082;
}
server {
listen 1234;
content_by_lua_block {
local sock = ngx.req.socket( true )
-- reveive first byte
local data, err = sock:receive( 1 )
--dispatch two backend1 if data is greater than 'a', otherwise dispatch to backend2
local a = string.byte(data, 1, 1 )
if a > 'a' then
--how to send to backend1
else
--how to send to backend2
end
}
}
}
I don't know how to make a bridge between the request and the backend according to the first byte in the request with lua script.
If anyone can help one this?
The question is pretty old, but I hope that my answer is still relevant for you.
stream {
lua_code_cache on;
init_by_lua_block {
-- cache package on startup
require('ngx.balancer')
-- share backend addresses via global table
-- (not recommended, only for demo purposes)
_G.BACKENDS = {
{'172.17.0.1', 8081},
{'172.17.0.1', 8082},
}
}
upstream lua_dispatcher {
# just an invalid address as a placeholder
server 0.0.0.1:1234;
balancer_by_lua_block {
local balancer = require('ngx.balancer')
local backend_index
if ngx.ctx.request_first_byte > 'a' then
backend_index = 1
else
backend_index = 2
end
local backend_table = _G.BACKENDS[backend_index]
local ok, err = balancer.set_current_peer(table.unpack(backend_table))
if not ok then
ngx.log(ngx.ERR, err)
ngx.exit(ngx.ERROR)
end
}
}
# proxy
server {
listen 9000;
proxy_pass lua_dispatcher;
# cosocket API not available in balancer_by_lua_block,
# so we read the first byte here and keep it in ngx.ctx table
preread_by_lua_block {
local sock = ngx.req.socket()
local data, err = sock:receive(1)
if not data then
ngx.log(ngx.ERR, err)
ngx.exit(ngx.ERROR)
end
ngx.ctx.request_first_byte = data:sub(1, 1)
}
}
# mock upstream 1
server {
listen 172.17.0.1:8081;
content_by_lua_block {
ngx.say('first')
}
}
# mock upstream 2
server {
listen 172.17.0.1:8082;
content_by_lua_block {
ngx.say('second')
}
}
}
$ nc -C localhost 9000 <<< '123'
second
$ nc -C localhost 9000 <<< '223'
second
$ nc -C localhost 9000 <<< 'a23'
second
$ nc -C localhost 9000 <<< 'b23'
first
$ nc -C localhost 9000 <<< 'c23'
first