Is it possible with nginx to rewrite "&" to "&" if it appears in a URL the client sends to nginx?
I'm currently kinda stuck, as some parts of my application (where I have plenty influence on) are calling s3 download links from my minio backend with "&" and others with "&" argument separation in the URL. I'm not sure why this issue exactly occurs, but my idea is to simply make nginx fix these URL calls internally, as they are practically the same. The problem is that minio (my S3 Server) interprets these wrong and denies access to a given resource as s3 signatures do not match anymore with & in the URL. They always have to be "&" instead of "&" !
To be a bit more specific, how the URL differs in some cases, please see the following example:
String I pass:
https://localhost/test/sprites.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=2PsuWGctH4UQmGvEQYjTDsZ2HqGM%2F20220601%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20220601T172937Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=6da7dc0137d25730d09bebbd54b0e1f0132d58cba318b3cfe11bcde5af608e05
Browser calls:
https://localhost/test/sprites.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=2PsuWGctH4UQmGvEQYjTDsZ2HqGM%2F20220601%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20220601T172937Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=6da7dc0137d25730d09bebbd54b0e1f0132d58cba318b3cfe11bcde5af608e05
The location at my NGINX config looks like this at the moment of writing:
location / {
auth_jwt_enabled off;
limit_req zone=s3 burst=100 nodelay;
proxy_buffer_size 256k;
proxy_buffers 4 512k;
proxy_busy_buffers_size 512k;
proxy_cache_convert_head off;
proxy_connect_timeout 180;
proxy_hide_header Set-Cookie;
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_http_version 1.1;
proxy_ignore_headers Set-Cookie;
proxy_pass http://127.0.0.1:7777;
proxy_read_timeout 86400;
proxy_send_timeout 86400;
proxy_set_header Authorization $http_authorization;
proxy_set_header Connection "";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Connection keep-alive;
add_header Content-disposition "attachment; filename=$1";
default_type application/octet-stream;
}
My guess here is that the browser converts the & to & for some reason, as the strings I'm passing does not contain & at all ...
Unfortunately nginx does not have an ability of globally replacing some strings. A workaround is possible, based on the rewrite ... last, when being triggered inside a location, starts over the NGX_HTTP_FIND_CONFIG_PHASE to find a new location for the rewritten URI. However there is a limitation of maximum 10 iteration rounds of this process, or nginx will throw an internal HTTP 500 error. Here is the idea:
location / {
if ($args ~ "^(?<prefix>.+)&(?<suffix>.+)$") {
rewrite ^ $uri?$prefix&$suffix? last;
}
...
}
Nevertheless I doubt the problem is on the nginx side, so most likely this won't help.
Related
I'm in the unfortunate situation that I need to extend my react application with an iframe containing an external application.
My application is hosted by a nginx reverse proxy that handles /api and signalr communication.
Unfortunately it also handles the outbout iframe src url.
in this example my site is on the url https://example.com
The iframe src url is in this case "https://external-site.com/someapp/session?token=1234"
When i see the requests in the browser the url has changed to https://example.com/esternal-site.com/someapp/session?token=1234, which needless to say is not working out of the box.
I've been toying with the nginx configuration but has been unable to just pass the request through without modification.
The iframe/destination works as expected when running locally.
I've attempted with a few different configuations inspired by stackoverflow and medium etc. but they've all returned various error codes.
the server runs on port 80, but https is handled by ingress on azure.
This is what i have currently:
upstream bff_service {
server ${BFF_HOST}:${BFF_PORT};
keepalive 32;
keepalive_requests 1000;
keepalive_timeout 75s;
}
server {
listen 80;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html;
add_header Set-Cookie "msal_client_id=${BFF_MSAL_CLIENT_ID};Path=/;Secure";
}
location /api {
proxy_read_timeout 300s;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host ${BFF_HOST};
proxy_set_header X-NginX-Proxy true;
proxy_pass ${BFF_PROTOCOL}://bff_service;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_redirect off;
}
location ^~ /external-site.com {
add_header Content-Security-Policy "frame-src 'self' https://external-site.com";
proxy_pass https://external-site.com/$request_uri;
}
}
I've also tried adding the lines below to the location:
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-NginX-Proxy true;
I'm looking for a configuration that allows me to embed an iframe with an external location. Perhaps even avoid nginx proxying it at all?
I am configuring a odoo server to block certain routes for people outside of some networks. I am using Nginx as a reverse proxy on this server.
My issue is with the route /web/session/lougout. When i add the two following blocks to my config, browsers start caching the 303 answer from the route and stop sending headers to the server. Since the headers are missing, it prevents the server from invalidating the session and leads to a bunch of issues.
location ~ ^/web/ {
add_header Content-Security-Policy upgrade-insecure-requests;
proxy_pass http://odoo-xxx-test;
proxy_buffering on;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
expires 864000;
allow *some ip* ;
allow *some ip* ;
allow *some ip* ;
deny all;
}
location ~ ^\/web\/(action\/|content\/|static\/|image\/|login|session\/|webclient\/) {
add_header Content-Security-Policy upgrade-insecure-requests;
proxy_pass http://odoo-xxx-test;
proxy_cache_valid 200 60m;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering on;
expires 864000;
}
From my understanding, the first location should deny access to all routes outside of the 3 allowed ips and the second should allow everyone outside of those 3 ips to access those 7 routes. I dont understand how those two block affect the caching of the /web/session/logout route, removing those two directives fixes the issues but this behavior is in the requirements for the projects.
Any help would be welcome!
Found the solution to my issue, turns out the expires directive was making the NGINX server cache the 303 response and serve it without contacting the server behind the proxy.
Removing the expires directive from both location fixed our issue.
So I have set up a reverse proxy to tunnel my application.
Unfortunately the application thinks it is served via http and not https and gives out URLs with port 80.
How can I handle this in the nginx reverse proxy? (by rewriting maybe)
When I go on the page:
https://my.server.com
index.php loads, everything is okay
after clicking something I have a URL like this:
https://my.server.com:80/page/stuff/?redirect_to
which throws an error within the browser because my reverse proxy doesn't serve SSL on port 80.
How can I migitate this?
My current nginx ssl vhost for the site:
... ssl stuff ...
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
location / {
proxy_pass http://localhost:22228;
proxy_buffering off;
proxy_redirect off;
proxy_read_timeout 43800;
proxy_pass_request_headers on;
proxy_set_header Connection "Keep-Alive";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass_header Content-Type;
proxy_pass_header Content-Disposition;
proxy_pass_header Content-Length;
proxy_set_header X-Forwarded-Proto https;
}
(yes I know my request headers look like a christmas tree 🎄)
Also bonus points if you show where the documentation addressing this issue is and what the mechanism is called.
For rewriting response body you can use http_sub_module:
location / {
proxy_pass http://localhost:22228;
sub_filter_once off;
sub_filter_types text/css application/javascript; # in addition to text/html
sub_filter "//my.server.com:80/" "//my.server.com/";
}
Many people says (1, 2) that you need to disable compression when using sub_filter directive:
proxy_set_header Accept-Encoding "";
For me, it works fine without this line in config, but it can be a feature of OpenResty which I use instead of nginx.
If your app generates HTTP 30x redirects with explicit indication of domain:port, you can rewrite Location header value with the proxy_redirect directive:
proxy_redirect //my.server.com:80/ //my.server.com/;
I'm having a very hard time understanding exactly when nginx overwrites my add_header directives.
I have the following:
include /etc/nginx/vhost.d/ihc.example.com;
location = /auth {
proxy_pass http://sso.example.com/auth/login;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location / {
auth_request /auth;
auth_request_set $saved_set_cookie $upstream_http_set_cookie;
add_header Set-Cookie $saved_set_cookie; #I don't see this header in the response
proxy_pass http://ihc.example.com;
include /etc/nginx/vhost.d/ihc.example.com_location;
}
The problem I'm having is that the auth_request_set $saved_set_cookie $upstream_http_set_cookie; cookie doesn't appear in the response.
Now if I remove the include /etc/nginx/vhost.d/ihc.example.com_location; line, it will appear, but this file contains a shared CORS fix that I would really, really like to keep:
if ($cors = 'trueGET') {
add_header 'Access-Control-Allow-Origin' $http_origin; #I see this header in the response
[...]
How can I achieve both without having to repeat the CORS headers in every location? And what is causing this to become overwritten? As I read the documentation, location should overwrite server, but in this case, what is causing this new "scope"?
I finally found the root cause of my issue:
Ifs are evil
Apparently, using if statements in location blocks leads to unspecified behaviours. Unfortunately for me, this means rethinking my entire setup, and potentially switching to OpenResty instead.
I want to use NGINX to as a proxy to get to Deluge which is inside my home network (NGINX is publically available).
This configuration works:
location 8112;
location / {
proxy_pass http://deluge_address:8112;
}
However I'd like to use an address in form of http://nginx_address/deluge to be proxied to internal http://deluge_address:8112.
I tried the following:
location /deluge/ {
proxy_pass http://deluge_address:8112/;
}
(I tried different combinations of trailing / - none work).
But I get 404 Not found instead.
I have some knowledge about networks, but not too much.
Does anybody have any idea what I'm doing wrongly?
I did find a solution for this, but found a bug also in Nginx in the same time
https://trac.nginx.org/nginx/ticket/1370#ticket
Edit-1
Seems like bug i logged was an invalid one, which even helped me understand few more things. So I edited the config a bit.
You need to use below config
location ~* /deluge/(.*) {
sub_filter_once off;
sub_filter_types text/css;
sub_filter '"base": "/"' '"base": "/deluge/"';
sub_filter '<head>' '<head>\n<base href="/deluge/">';
sub_filter 'src="/' 'src="./';
sub_filter 'href="/' 'href="./';
sub_filter 'url("/' 'url("./';
sub_filter 'url(\'/' 'url(\'./';
set $deluge_host 192.168.33.100;
set $deluge_port 32770;
proxy_pass http://$deluge_host:$deluge_port/$1;
proxy_cookie_domain $deluge_host $host;
proxy_cookie_path / /deluge/;
proxy_redirect http://$deluge_host:$deluge_port/ /deluge/;
}
The key was to insert a base url into the pages using below
sub_filter '<head>' '<head>\n<base href="/deluge/">';
And then make replacement in src and href attributes in html. And also url(' in css entries.
Luckily deluge has a JavaScript config which has the base url. So we can override the same by adding
sub_filter '"base": "/"' '"base": "/deluge/"';
I faced the same problem, luckily I found a better and official solution:
Reverse Proxy with Deluge WebUI
proxy_set_header X-Deluge-Base "/deluge/";
add_header X-Frame-Options SAMEORIGIN;
My final settings:
location /deluge {
proxy_pass http://127.0.0.1:8112/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 75;
proxy_send_timeout 3650;
proxy_read_timeout 3650;
proxy_buffers 64 512k;
client_body_buffer_size 512k;
client_max_body_size 0;
# https://dev.deluge-torrent.org/wiki/UserGuide/WebUI/ReverseProxy
proxy_set_header X-Deluge-Base "/deluge/";
add_header X-Frame-Options SAMEORIGIN;
}