nginx match request body without using lua module - nginx

is there a way in nginx to do something based on if the request body has a string or not?
I am sure i can do it with using Lua module..
I am try to find out if there is a way using nginx alone.
I am hoping something like below will work.
location /students-api {
if ($request_body ~* "(.*)specialstudent(.*)" ) {
set $student_status 'special';
}
// and use student_status for some logic
}

I think it should work, however it needs to be tested. In practice I used $request_body only for logging, not sure if it is available at the rewrite stage of request processing. Here is an official description which says:
The variable’s value is made available in locations processed by the proxy_pass, fastcgi_pass, uwsgi_pass, and scgi_pass directives when the request body was read to a memory buffer.
Additionally, you don't need those capture groups to check a variable for substring presence if you don't use them later (in fact you just wasting resources to keep them in memory), just if ($request_body ~* "specialstudent") { ... } should be enough.
Update
Here is another approach that has more chances to work since proxy_add_header directive is definitely executed later than the rewrite stage of request processing:
map $request_body $special {
~*"specialstudent" "special";
# otherwise '$special' variable value will be empty
}
server {
...
location /students-api {
...
proxy_set_header X-Student-Status $special;
...
}
}
Update 2
After testing all of this, I can confirm that the if approach does not work:
server {
...
location /students-api {
if ($request_body ~* "specialstudent") {
set $student_status "special";
}
proxy_set_header X-Student-Status $student_status;
...
}
}
As being expected, the $request_body variable doesn't get initialized at the rewrite stage of request processing. However, the map approach work as expected:
map $request_body $student_status {
~*"specialstudent" "special";
# otherwise '$special' variable value will be empty
}
server {
...
location /students-api {
proxy_set_header X-Student-Status $student_status;
...
}
}
What really surprises me is that the following example doesn't set any of two headers:
map $request_body $student_status {
~*"specialstudent" "special";
# otherwise '$special' variable value will be empty
}
server {
...
location /students-api {
if ($request_body ~* "specialstudent") {
set $student_special "special";
}
proxy_set_header X-Student-Status $student_status;
proxy_set_header X-Student-Special $student_special;
...
}
}
Somehow accessing the $request_body variable at the early rewrite stage of request processing leads the map translation to stop working too. I didn't have an explanation of this behavior for now and would be grateful if someone could explain what happened here.
Update 3
I think I'm finally found an explanation of what happened with the last example in the Nginx Tutorials written by Yichun Zhang, the author of famous lua-nginx-module and the OpenResty bundle:
Some Nginx variables choose to use their value containers as a data cache when the "get handler" is configured. In this setting, the "get handler" is run only once, i.e., at the first time the variable is read, which reduces overhead when the variable is read multiple times during its lifetime.
Looks like the $request_body variable behaves exactly this way, if being accessed at the early NGX_HTTP_REWRITE_PHASE (see the request processing phases description). Its value, if being read during that phase, gets cached as an empty value and became useless during the later request processing phases.

Related

nginx - Proxy to a different backend based on query param

I've got a case where I want to proxy a particular call down to a different backend based on the existence of a query param. The following is sort of what I start with
location ~ ^/abc/xyz/?$ {
proxy_pass $backend_url;
}
What I'd like to do is check for a query param foo (or even just the existence of that string anywhere). So I thought I could do this
location ~ ^/abc/xyz/?$ {
set $backend_url "somelocation"
if ($request_url ~ .*foo.*) {
set $backend_url "someotherlocation"
proxy_pass $backend_url
}
proxy_pass $backend_url;
}
But this doesn't seem to actually proxy to the new location. Am I doing something wrong with my code, or is the whole approach wrong?
I don't know why are you using two proxy_pass directives, this block should do it in a logic you described:
location ~ ^/abc/xyz/?$ {
set $backend_url "somelocation";
if ($request_url ~ \?(.*&)?foo(=|&|$)) {
set $backend_url "otherlocation";
}
proxy_pass $backend_url;
}
I slightly modified your regex to match only request URLs where foo is a query argument name and not a query argument value (or its part). However I'd rather use map directive for this purpose (map block should be placed outside the server block):
map $arg_foo $backend_url {
"" somelocation; # if the value is empty
default otherlocation; # otherwise
}
server {
...
location ~ ^/abc/xyz/?$ {
proxy_pass $backend_url;
}
}
Pay an attention you may need to define a resolver for this configuration to work (some additional info about this can be found here).

How to debug and fix nginx wrong location redirection problem?

I have an Nginx config file list below. I want to send the request to different server base on Refer.
When I send a request with URL "doamin.com/capi/a/b" and refer "a.com/a/1/test", everything is good, server "be" will get "be/a/b" request.
But if I send a request with URL "doamin.com/capi/a/b" and refer "a.com/a/0/test", server "be_demo" will get "be_demo/" request, the path "a/b" is missing.
I've tried to add "/" at the end of "be_demo", it doesn't work.
map $http_referer $be_pool {
default be;
"~a\.com\/.*\/0\/.*" be_demo;
}
server {
...
location ~ ^/capi/(.*)$ {
proxy_pass http://$be_pool/$1;
}
}
Thanks.
The numeric capture $1 is set by the last regular expression to be evaluated. In the second case, the regular expression in the map statement is evaluated after the regular expression in the location statement.
The solution is to use a named capture instead.
For example:
map $http_referer $be_pool {
default be;
"~a\.com\/.*\/0\/.*" be_demo;
}
server {
...
location ~ ^/capi/(?<myuri>.*)$ {
proxy_pass http://$be_pool/$myuri;
}
}

Calling external api in Nginx location section

I am trying to resolve proxy_pass value dynamically (through web api) in nginx.
I need something like below;
Example taken from: https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
location /proxy-pass-uri {
set $urlToProxy = CallWebAPI("http://localhost:8081/resolver?url=" + $url);
proxy_pass $urlToProxy;
}
So, my question is that, is it possible to make HTTP request or to write method such as CallWebAPI?
I know it might be a bad practice, but the website I am dealing with has thousands of web urls, which are mapped as key-value pairs, and 90% of them does not obey any specific regex rules. So I have content mapped database, and I need to fetch incoming url with content dynamically.
I am trying to use a very light web service to look up URLs from redis, and return proxy url.
Would this be a valid scenario, or is there any other built in solution in nginx like this?
I doubt this can be done with "pure" nginx, but this definitely can be done with openresty or ngx_http_lua_module with the help of ngx.location.capture method. For example:
resolver 8.8.8.8;
location ~/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("http://localhost:8081/resolver?url=" .. ngx.var.url)
ngx.var.proxy = res.body
}
proxy_pass $proxy$url;
}
There is also an ngx_http_js_module (documentation, GitHub) which have an ability to do subrequests (example), but I never used it and cannot tell if it can be used this way.
Important update
After almost a three years since this answer was written, it comes that I needed the similar functionality myself, and it turns out that the above answer is completely broken and unworkable. You can't do a subrequest via ngx.location.capture to anything else but to some other nginx location. So the correct (checked and confirmed to be workable) example for the above question is
resolver 8.8.8.8;
location /resolver {
internal;
proxy_pass http://localhost:8081;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/resolver?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
The above example assumes that the proxy resolution service is really expecting request in a /resolver?url=<uri> form. The location /resolver { ... } while being internal behaves like any other prefix location, so if the /resolver prefix for that location cannot be used for some reason, the same can be written as
resolver 8.8.8.8;
location /get_proxy {
internal;
proxy_pass http://localhost:8081/resolver;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/get_proxy?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}

Nginx rewrite: how to change URL from dynamic to static

I'm configuring nginx as reverse proxy.
I need to change (rewrite?) the URLs, example: when the request (to nginx Reverse Proxy) is "http://example.com/test/?username=test1;password=passwdtest1" it will must "modified" to the main server as "http://example.com/test/?username=production;password=passwdproduction1".
Consider that in the original request the fields "username=test1;password=passwdtest1" are not always the same (they changes), instead the "modified" to the main server are always the same.
Others example to be more clear:
"/test/?username=test1;password=passwdtest1" -> "/test/?username=production;password=passwdproduction1"
"/test/?username=test1876;password=somepasswd" -> "/test/?username=production;password=passwdproduction1"
"/test/?username=somevalues;password=somepasswdvalue" -> "/test/?username=production;password=passwdproduction1"
So, independently to what are the values of "?username=somevalues;password=somepasswdvalue" it should always become "?username=production;password=passwdproduction1".
Thanks for your help!
A little late on the answer but this should work for you:
location ~* /test/? {
if ($arg_username ~ "^$|\s+") { return 404; }
if ($arg_password ~ "^$|\s+") { return 404; }
rewrite ^ /test?username=production&password=passwdproduction1? permanent;
}
The code above checks if it is within the example.com/test path. If it is it will check if the user name or the password variable are present and not empty in the query string. In case if any isn't present or is empty it will return a 404 else it will redirect you to the preferred url.
By the way, instead of the semicolon in your example urls I would use an ampersand (&).

Custom routing in load balancer

Is it possible to define a custom routing in NGINX or other Load Balancer?
I.e. I have a cookie or a header and based on its value I decide which backend server to choose?
I need some very simple logic - values a1,a2,a3 - to server A, values b1,b2 to server B, all other to server C
In nginx you can do it simply by using if:
location / {
if ($http_cookie ~* "yourcookiename=a") {
proxy_pass http://upstream_a;
break;
}
if ($http_cookie ~* "yourcookiename=b") {
proxy_pass http://upstream_b;
break;
}
proxy_pass http://upstream_c;
}
This is simple regexp , so this way if "yourcookiename" has value a1,a2 etc. it will go to uprstream_a and so on.
Hope it helps...
If you need some sticky session, there are open source third party modules that can do that with nginx, while the native implementation is part of the commercial subscription. Also, tengine, an open source chinese fork of nginx developed by Alibaba can do that natively.
If you want to do it the custom way, use a map to avoid processing a chain of if blocks for all requests. This is also better for readability. For instance, using a cookie :
map $cookie_mycookie $node {
"~^a[1-3]$" "A";
"~^b[1-2]$" "B";
default "C";
}
server {
location / {
proxy_pass http://$node;
}
}

Resources