nginx reverse-proxy rewriting custom header (NOT URL) - nginx

I am trying to rewrite custom header information like "Author" (not part of the URL) using nginx reverse proxy.
The header information "Author:" should be rewritten from "test123" to e.g. "BASIC"
command:
admin1#nginx1:~$ curl -x 192.168.175.134:80 http://home1.MyWeb.eu:8081/home1/index.html?t=1 -H "Author: test123" -vk
TCPdump on apache:
--
GET /home1/index.html?t=1 HTTP/1.0
Host: home1.MyWeb.eu
Connection: close
User-Agent: curl/7.58.0
Accept: */*
Proxy-Connection: Keep-Alive
Author: test123
wanted result:
--
GET /home1/index.html?t=1 HTTP/1.0
Host: home1.MyWeb.eu
Connection: close
User-Agent: curl/7.58.0
Accept: */*
Proxy-Connection: Keep-Alive
Author: BASIC

You can use the proxy_set_header in your configuration. I.e.:
proxy_set_header Author "BASIC";

I made with setting variables. A little ugly but seems to work.
location / {
<...>
set $rewritten_header $http_myheader;
if ($http_myheader = "something") {
set $rewritten_header somethingelse;
}
proxy_set_header Myheader $rewritten_header;
}
The above will rewrite your header only if the condition match. Otherwise keep original value.
I think more elegant to use map in case you have a large mapping.

Hi I have solved the issue like this:
curl -x localhost:80 https://www.dummy.com -H "Authorization: test12" -vk
NGINX configuration:
server {
listen 127.0.0.2:443 ssl;
# the included file below contains ssl certificates
include snippets/www.dummy.com.conf;
root /var/www/html;
set $MyAuthorization 'Basic bGaa9zX25ljYhhWxlcl9=';
location / {
proxy_pass https://www;
proxy_set_header Host www.dummy.com;
proxy_set_header Authorization $MyAuthorization;
}
}

Related

Nginx: log the actual forwarded proxy_pass request URI to upstream

I've got the following nginx conf:
http {
log_format upstream_logging '[proxied request] '
'$server_name$request_uri -> $upstream_addr';
access_log /dev/stdout upstream_logging;
server {
listen 80;
server_name localhost;
location ~ /test/(.*)/foo {
proxy_pass http://127.0.0.1:3000/$1;
}
}
}
When I hit:
http://localhost/test/bar/foo
My actual output is:
[proxied request] localhost/test/bar/foo -> 127.0.0.1:3000
While my expected output is:
[proxied request] localhost/test/bar/foo -> 127.0.0.1:3000/bar
Is there a variable or a way to produce the actual proxied URI in the log?
If not production, you can test what is being sent by nginx after launching the simplest listening server on the desired local address and port (instead of a real one):
$ nc -l 127.0.0.1 3000
POST /some/uri HTTP/1.0
Host: 127.0.0.1
Connection: close
Content-Length: 14
some payload
Response can be simulated by manually entering HTTP/1.1 200 OK, followed with 2 new lines, while nc is running.

root context with preference over a regular expression

I have read the documentation and seems it is the same as explained here enter link description here, but during my tests the root context is having precedence over the regular expression one.
Does anyone know the reason?
See my nginx.conf file and a test curl I did to validate the configuration:
http {
server {
listen 80;
root /usr/share/nginx/html;
include /etc/nginx/mime.types;
location ~ /ui(/.*) {
try_files $1 $1/ #htmlext;
}
location #htmlext {
rewrite ^/ui(/\w*)(/.*)*$ $1.html last;
}
location / {
return 301 /ui$request_uri;
}
}
}
/ # curl -v http://localhost/ui/message?msg=error.forbidden
* Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /ui/message?msg=error.forbidden HTTP/1.1
> Host: localhost
> User-Agent: curl/7.67.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.19.0
< Date: Thu, 09 Jul 2020 22:34:49 GMT
< Content-Type: text/html
< Content-Length: 169
< Location: http://localhost/ui/ui/message?msg=error.forbidden
< Connection: keep-alive
<
<html>
<head><title>301 Moved Permanently</title></head>
<body>
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx/1.19.0</center>
</body>
</html>
* Connection #0 to host localhost left intact
As you can see in the curl it is redirecting my /ui/message page instead of rendering it. If I remove the location / from the configuration it seems working, if I remove the #htmlext location and add that logic in the location ~ /ui(/.*) it seems working too. Then I think the reason of my error is the #htmlext redirect.
Does anyone know why?
Thanks

Lua nginx extracting destination path from POST header

I am working on uploading files using nginx. I have installed openresty module to support ngx lua .While uploading files I need to change the upload_store variable to the path in the POST header received. So I am thinking of using lua to change upload_store directive value in nginx.conf. I am trying to get the Dst-Dir from header like below:
location /umtest {
set $upload_store /mnt/share_marvel/uploaded_files;
rewrite_by_lua '
local header = ngx.req.raw_header()
ngx.say("type header",header)
dst_path_dir = #need to extract from header
ngx.var.upload_store = dst_path_dir
ngx.say("upload store path" ,ngx.var.upload_store)
';
Header received from ngx.req.raw_header() is
POST /umtest HTTP/1.1
Host: X.X.X.X:8888
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache
Dst-Dir : "/path/to/upload"
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9
How can I extract the value of Dst-Dir, so that i can set that value to upload_store? I am pretty new to lua.
Give a try to ngx.req.get_headers(), for example:
dst_path_dir = ngx.req.get_headers()["Dst-Dir"]

Hot change "Location" in the reply header 302 code from proxy-host.

I have the kubernetes-cluster and a few pods/containers with web-app. Pods connecting to each other by pod's name with listen port 9999 (like security-rest-api:9999, common-rest-api:9999, etc).
To outside listen nginx-pod with outside address http://e.net:30200/.
((app-pods:9999)-(nginx-pod:80)-(nginx-service:30200))-Network
Nginx use follow configuration for interactive with app-pods.
server {
listen 80;
server_name e.net;
location / {
proxy_pass http://web-console:9999/;
proxy_redirect http://web-console:9999/ http://e.net:30200/;
}
location /common {
proxy_pass http://common-rest-api:9999/common;
proxy_redirect http://common-rest-api:9999/ http://e.net:30200/;
}
location /security {
proxy_pass http://security-rest-api:9999/security;
proxy_redirect http://security-rest-api:9999/ http://e.net:30200/;
} }
It's working very well, but I have the one problem with 302-reply from app-pods:
If I try to login in my app, I get follow the 302 reply header:
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Date: Wed, 25 Apr 2018 10:37:50 GMT
Location: http://e.net:30200/security/rest/auth/login.html?callbackUrl=http://security-rest-api:9999/security/rest/namespace
Server: nginx/1.13.9
App-pods generated URL parameter "callbackUrl" from the Host request header Inside containers network and this URL parameter to get to the endpoint browser. Of course, next request get 404 code.
I can't to edit app-code (in app-pods don't use nginx), but I want to change 'security-rest-api:9999' to 'e.net:30200' parameter in the Location 302 reply header. How I can do it?
redirect isn't suitable since this generate new 302-reply and not solve my problem.
sub_filter change only reply body, but not reply head (where is Location parameter).
request_uri not working too, since this parameter work with request header only.
No, It's not working.
I tested this situation and finded work's config:
if ($args ~* (.*)(callbackUrl=http://security-rest-api:9999/)(.*)) {
set $args $1callbackUrl=http://e.net:30200/$3;
rewrite ^(.*)$ $1;
}
if ($args ~* (.*)(callbackURL=http%3A%2F%2Fsecurity-rest-api%3A9999%2F)(.*)) {
set $args $1callbackURL=http%3A%2F%2Fe.net%3A30200%2F$3;
rewrite ^(.*)$ $1;
}
location /security {
proxy_pass http://security-rest-api:9999/security;
proxy_set_header Host $http_host;
proxy_redirect http://security-rest-api:9999/ http://e.net:30200/;
}
Later, I will try to use this config on the pre-production stand and if this work (or work after corrects) - I will write it here.
Thanks for help information:
https://blog.imaginea.com/modifying-query-parameters-nginx-in-reverse-proxy-mode/
And thanks for you all, too!
Edit
I tested this config and have 2 edits:
If you to use un-standart port - you need write "proxy_set_header Host $http_host;" in location section;
URL in attributes can be like "http://security-rest-api:9999/" and like "http%3A%2F%2Fsecurity-rest-api%3A9999%2F". You need to use both conditions for each type of attribute.
I corrected code with this edits

Read form POST variables with nginx

I have a client program I cannot modify. It makes large POST (x-www-form-urlencoded) requests containing hundreds of variables across WAN links, but I only need 5 of them. I'm inserting nginx as a reverse proxy on the local client system. What's the easiest to get nginx to strip out the extra data?
Two ways I see so far:
1. Use Lua (If I did, should I do content_by_lua, rewrite the body, and then make a subrequest? Or is there a simpler way?)
2. Use form-input-nginx-module and proxy_set_body to parse and grab a few variables out.
I'm already using OpenResty, so Lua means no extra modules. But, it probably means writing more locations and so on to do subrequests.
In my opinion the easiest way will be using lua. The choice between content_by_lua, rewrite_by_lua, access_by_lua or any combination of them; will depend on how you use the response body of your subrequest. That decision will also determine if you would need additional locations or not.
Here are a couple of examples:
1. with content_by_lua targeting a local location.
(This approach requires the definition of the sub request location)
location /original/url {
lua_need_request_body on;
content_by_lua '
--Lots of params but I only need 5 for the subrequest
local limited_post_args, err = ngx.req.get_post_args(5)
if not limited_post_args then
ngx.say("failed to get post args: ", err)
return
end
local subreq_uri = "/test/local"
local subreq_response = ngx.location.capture(subreq_uri, {method=ngx.HTTP_POST,
body = ngx.encode_args(limited_post_args)})
ngx.print(subreq_response.body)
';
}
location ~/test/local {
lua_need_request_body on;
proxy_set_header Accept-Encoding "";
proxy_pass http://echo.200please.com;
}
2. with rewrite_by_lua to remote target
(No additional location is needed)
location /original/url/to/remote {
lua_need_request_body on;
rewrite_by_lua '
--Lost of params but I only need 5 for the subrequest
local limited_post_args, err = ngx.req.get_post_args(5)
if not limited_post_args then
ngx.say("failed to get post args: ", err)
return
end
--setting limited number of params
ngx.req.set_body_data(ngx.encode_args(limited_post_args))
--rewriting url
local subreq_path = "/test"
ngx.req.set_uri(subreq_path)
';
proxy_pass http://echo.200please.com;
}
Sample post request with 7 args limited to 5:
curl 'http://localhost/original/url/to/remote' --data 'param1=test&param2=2&param3=3&param4=4&param5=5&param6=6&param7=7' --compressed
response:
POST /test HTTP/1.0
Host: echo.200please.com
Connection: close
Content-Length: 47
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Accept: */*
Accept-Encoding: deflate, gzip
Content-Type: application/x-www-form-urlencoded
param3=3&param4=4&param1=test&param2=2&param5=5

Resources