NGINX Rewrite ignored / not working with proxy_pass - nginx

Bug in the upstream vendor app. Wrote a route in our node app to proxy the request and avoid the bug but can't get the NGINX rewrite to work correctly. I've tried many variations of rewrite and now at my wit's end. Spent more time on the rewrite than the actual code... =(
IN: /Txtranscription/transcription/TranscriptionHandler.ashx?q=c3R1ZHlfaWQ...
OUT: /Txtranscription/transcription/TranscriptionHandler.ashx?q=c3R1ZHlfaWQ...
EXPECTED: /transcription?encoded=c3R1ZHlfaWQ...
### override handling of /Txtranscription/transcription/TranscriptionHandler.ashx
location /Txtranscription/transcription/TranscriptionHandler.ashx {
add_header Access-Control-Allow-Origin $cors_header;
access_log logs/vapi.proxy.log lfupstream;
error_log logs/vapi.error.log error;
rewrite ^/Txtranscription/transcription/TranscriptionHandler\.ashx\?q=(.*)$ /transcription?encoded=$1 break;
proxy_pass http://vapi;
}

You shouldn't need to rewrite the request at all, you can append a path to a proxy_pass directive and Nginx will replace the matching part of the location block from the original request URI with the URI of your proxy_pass directive.
So this should work:
location /Txtranscription/transcription/TranscriptionHandler.ashx {
set $args encoded=$arg_q;
....
proxy_pass http://vapi/transcription$is_args$args;

Example:
location ~ ^/connector(/?)(.*)$ {
proxy_buffer_size 64k;
proxy_buffers 16 32k;
proxy_http_version 1.1;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
proxy_set_header Authorization "";
set $upstream_endpoint http://YOUR-END-POINT/$2$is_args$args;
proxy_pass $upstream_endpoint;
}
The magic is -> location ~ ^ /admin (/?)(.*)$
And then -> /$2$is_args$args;

Related

Hide a port from the content-security-policy so that script-src considers the url as self

I'm setting up the content-security-policy for my site , but the script url for my comments system contains a port number i.e http://dev.example.com:8080/client.js.
I have a few other URL's which use ports too and I'd prefer to hide which ports my site uses in case of possible security implications.
How can I hide these URL's containing port numbers so that the CSP considers the URL's as self?
For example instead of http://dev.example.com:8080/client.js I'd like it to be http://dev.example.com/comments/client.js.
I'm on an Nginx server and I've tried playing around with rewriting the URL, but can't get it working.
Here's what I've tried.
This allows me to change the script URL to http://dev.example.com/comments/client.js, but CSP still detects http://dev.example.com:8080/client.js.
location ~ /comments/ {
return 301 http://dev.example.com:8080/client.js;
}
Didn't work
location ~ /comments/ {
rewrite ^/comments/(.*) http://dev.example.com:8080/$1 last;
}
Just to be clear I know I can just throw the URL http://dev.example.com:8080/client.js straight into CSP e.g script-src 'self'
dev.example.com:8080/client.js;, but I want the URL to work with self e.g script-src 'self'.
Returns 502 bad gateway error
location /comments {
proxy_pass http://dev.example.com:8080/;
}
Also returns502 bad gateway.
location = /comments {
return 301 /comments/;
}
location ~ /comments/(?<ndpath>.*) {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_pass_request_headers on;
proxy_set_header Connection "keep-alive";
proxy_store off;
proxy_pass http://dev.example.com:8080/$ndpath$is_args$args;
gzip on;
gzip_proxied any;
gzip_types *;
}
What you are looking for is the concept of a reverse proxy. Described here.
In your case the simplest example is
location /comments {
proxy_pass http://localhost:8080/;
}

nginx reverse proxy "catch-all" location

EDIT: To be more clear, this is nginx version 1.13.8.
Take the following as an example nginx.conf file:
http {
upstream portal_backend {
server pc-loc43-01:15080;
}
upstream auth_backend {
server pc-loc43-01:16080;
}
server {
listen 9080 default_server;
server_name my-reverse-proxy;
location / {
auth_basic off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_pass http://portal_backend/;
}
location /auth {
auth_basic off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_pass http://auth_backend/auth;
}
}
}
I want to configure nginx to default to location / if it is unable to match the request to any of the locations, but I cannot find how to do this.
I don't see anything wrong with your code.
location / { is already the default location block for "unhandled" locations.
This would match all locations:
location / {
# ...
}
This would match the root only:
location = / {
# ...
}
This will match /auth and sub directories:
location /auth {
# ...
}
It must be related to how nginx does request matching -- somehow auth and authorize are too similar and it causes nginx problems (not a great explanation and maybe someone more experienced with nginx internals can chime in). The "solution" was to duplicate location / into location /authorize, so now the config file looks like:
...
location / {
auth_basic off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_pass http://portal_backend/;
}
location /authorize {
auth_basic off;
proxy_http_version 1.1;
proxy_set_header Host $http_host;
proxy_pass http://portal_backend/;
}
...
All the other routes work as I would have expected, e.g. /users, /customers, /whatever are all handled by location /

reroute nginx request url

I have a reverse nginx proxy where I want to route all request that come in with :
http://dns.com/content/xyz <—to—> http://dns.com/content/1.0/xyz
I have an upstream :
upstream backend_api.content.com {
server localhost:8080 max_fails=5 fail_timeout=30;
keepalive 100;
}
and location :
#Content Service
location ~* ^/content/?(.*) {
set $proxy_pass "http://backend_api.content.com";
rewrite ^/content/1.0(/.*)$ /content/1.0$1 break;
proxy_pass $proxy_pass
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host "api.stg.xxx.com";
proxy_set_header X-3scale-proxy-secret-token $secret_token;
proxy_set_header Original-Host $http_host;
proxy_set_header Authorization $outbound_auth_header;
proxy_set_header Original-Uri $scheme://$http_host$uri;
post_action /out_of_band_oauth_authrep_action;
}
but it seems like anything with http://dns/content/xyz fails and only when I give http://dns/content/1.0/xyz does it work.
You seem to be capturing part of the URI on the location ~* ^/content/?(.*) statement, but do nothing with it.
You also have a rewrite ^/content/1.0(/.*)$ /content/1.0$1 break; statement that does nothing, it simply writes the same URI back.
A quick and dirty solution might be to use two rewrite statements like this:
rewrite ^/content/1.0(/.*)$ /content/1.0$1 break;
rewrite ^/content(/.*)$ /content/1.0$1 break;
Which means that anything that does not match the first (non-) rewrite will be processed by the second, and get a /1.0 inserted.
Personally, I do not like it, and would rather use two location blocks:
location /content/1.0 {
set $proxy_pass "http://backend_api.content.com";
proxy_pass $proxy_pass;
proxy_http_version 1.1;
proxy_set_header ...
...
}
location /content {
rewrite ^/content(/.*)$ /content/1.0$1 last;
}
But check the evaluation order of your other location blocks. Note that prefix location blocks and regular expression location blocks have different evaluation rules. See this document for details.

Nginx Reverse Proxy Websocket Authentication - HTTP 403

I'm using Nginx as a reverse proxy of a Spring boot application. I also use Websockets with sockjs and stomp messages.
Here is the context configuration.
<websocket:message-broker application-destination-prefix="/app">
<websocket:stomp-endpoint path="/localization" >
<websocket:sockjs/>
</websocket:stomp-endpoint>
<websocket:simple-broker prefix="/topic" />
</websocket:message-broker>
Here is the client code:
var socket = new SockJS(entryPointUrl);
var stompClient = Stomp.over(socket);
var _this = this;
stompClient.connect({}, function () {
stompClient.subscribe('/app/some-url', function (message) {
// do some stuff
});
});
I also you Spring Security to protect some content.
#Configuration
#Order(4)
public static class FrontendSecurityConfig extends WebSecurityConfigurerAdapter {
#Override
protected void configure(HttpSecurity http) throws Exception {
http.authorizeRequests()
.antMatchers("/js/**", "/css/**", "/webjars/**").permitAll()
.anyRequest().authenticated()
.and()
.formLogin().loginPage("/login").permitAll()
.and()
.logout().permitAll();
}
}
Everything works great, expect when I run this application behind a Nginx reverse proxy. Here is the reverse configuration:
proxy_pass http://testsysten:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support (nginx 1.4)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
# Max body size
client_max_body_size 10M;
The connection always fails with a HTTP 403 code.
I'm using version 1.9.7.
Do you have any idea, why the client does not gets authenticated?
I know similar questions, like this one but the solutions do not work at all.
Update
I managed to run the application over HTTP. I need to pass the CSRF token in the Nginx configuration. New configuration is:
proxy_pass http://testsysten:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Pass the csrf token (see https://de.wikipedia.org/wiki/Cross-Site-Request-Forgery)
# Default in Spring Boot
proxy_pass_header X-XSRF-TOKEN;
# WebSocket support (nginx 1.4)
proxy_http_version 1.1;
Only missing is redirect over HTTPS. In the Spring logs is see following entry:
o.s.w.s.s.t.h.DefaultSockJsService - Processing transport request: GET http://testsystem:80/localization/226/3mbmu212/websocket
Seems like Nginx Proxy needs to rewrite the to the right port.
I solved the problem by myself. Basically, Nginx needs to pass some additional header values if you want to use Websocket and Spring Security. The following lines need to be added to location section in your Nginx config:
# Pass the csrf token (see https://de.wikipedia.org/wiki/Cross-Site-Request-Forgery)
# Default in Spring Boot and required. Without it nginx suppresses the value
proxy_pass_header X-XSRF-TOKEN;
# Set origin to the real instance, otherwise a of Spring security check will fail
# Same value as defined in proxy_pass
proxy_set_header Origin "http://testsysten:8080";
The accepted solution did not work for me although I was using a very classical HTTPS configuration:
server {
listen 443 ssl;
location /ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8888;
}
...
The problem is that Spring checks the origin and specifically that code was causing me trouble:
// in org.springframework.web.util.UriComponentsBuilder.adaptFromForwardedHeaders(HttpHeaders):
if ((this.scheme.equals("http") && "80".equals(this.port)) ||
(this.scheme.equals("https") && "443".equals(this.port))) {
this.port = null;
}
In that code the scheme is 'http' and the port is 8888, which is not discarded because it is not the standard port.
The browser however hits https://myserver/ and the 443 port is omitted because it is the default HTTPS one.
Therefore the ports do not match (empty != 8888) and origin check fails.
Either you can disable origin checks in Spring WebSockets:
registry.addHandler( resgisterHandler(), "/ws" ).setAllowedOrigins( "*" );
or (probably safer) you can add the scheme and port to the NGINX proxy configuration:
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
If you are interested, those headers are read in
org.springframework.web.util.UriComponentsBuilder.adaptFromForwardedHeaders(HttpHeaders)
For Spring Boot 2.2.2+
Starting with Spring Boot version 2.2.2 you should be adding following setting for these X-Forwarded-* headers to be taken into account:
server.forward-headers-strategy=native
(in application.properties for instance)
I had faced a similar problem. I was unable to use the basic Spring Security authentication with NGINX. Apart from setting the proxy_pass_header X-XSRF-TOKEN;, I also had to set underscores_in_headers on;, since NGINX by default does not allow headers with underscores and the CSRF token is named _csrf.
So my final configuration file looked like this:
server {
underscores_in_headers on;
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name localhost;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
location /example/ {
proxy_pass_header X-XSRF-TOKEN;
proxy_pass http://localhost:8080/;
}
}
I solved this problem without CSRF header in NGINX proxy.
My stack: spring-boot, spring-security (with redis session store), spring-boot-websocket with default STOMP implementation, NGINX to serve frontend and proxied to another services that frontend consume.
In first time I use the default configuration show in the NGINX Blog here and here (copy and paste for history):
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream websocket {
server 192.168.100.10:8010;
}
server {
listen 8020;
location / {
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
}
But dont work, still 403 Forbidden.
I fixed this issue with the configuration below (the real important part to fix websocket is # WebSocket Proxy):
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 30010;
server_name localhost;
client_max_body_size 10M;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
# Backend API Proxy
location /api {
proxy_pass http://192.168.0.100:30080;
proxy_set_header Host $http_host;
proxy_set_header Access-Control-Allow-Origin 192.168.0.100;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
rewrite ^/api/?(.*) /$1 break;
proxy_redirect off;
}
# CDN Proxy
location ~ ^/cdn/(.*) {
proxy_pass http://192.168.0.110:9000;
rewrite ^/cdn/(.*) /$1 break;
}
# This is the configuration that fix the problem with WebSocket
# WebSocket Proxy
location /ws {
proxy_pass http://192.168.0.120:30090;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header Access-Control-Allow-Origin 192.168.0.120;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
}
}
}
In my case (Spring Boot app), in addition to setting the Origin header as specified in the accepted answer, I had to set the Host header to match the ip:port of the Origin header, or to get rid of it altogether.
This is my working vhost config:
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl_certificate /etc/ssl/certs/<your-cert-file>.pem;
ssl_certificate_key /etc/ssl/private/<your-key-file>.key;
server_name <your-server-fqdn>;
access_log /var/log/nginx/<your-server-fqdn>.access.log;
error_log /var/log/nginx/<your-server-fqdn>.error.log error;
root /srv/www/<your-server-fqdn>;
index index.html index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /api {
proxy_pass http://127.0.0.1:8080/v1;
}
location /async-api {
proxy_pass http://127.0.0.1:8080/stomp;
proxy_http_version 1.1;
# either set Host header as follows or get rid of the directive altogether
#proxy_set_header Host "127.0.0.1:8080";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Set origin to the real instance, otherwise a of Spring security check will fail
# Same value as defined in proxy_pass
proxy_set_header Origin "http://127.0.0.1:8080";
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /admin-api {
proxy_pass http://127.0.0.1:8080/api;
}
}

Nginx - reverse proxy a Ghost blog with /subfolder redirect

I have a working nginx instance with the rules below. But I'm having difficulties pointing all the requests to domain.com/ghost
I tried modifying the location / {} block to location /ghost/ {} but with no success. I just get a 404 from the ghost app. Any suggestions?
server {
listen 80;
server_name domain.com;
root /home//user/ghost/;
index index.php;
# if ($http_host != "domain.com") {
# rewrite ^ http://domain.com$request_uri permanent;
# }
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:2368;
}
location ~* \.(?:ico|css|js|gif|jpe?g|png|ttf|woff)$ {
access_log off;
expires 30d;
add_header Pragma public;
add_header Cache-Control "public, mustrevalidate, proxy-revalidate";
proxy_pass http://127.0.0.1:2368;
}
location = /robots.txt { access_log off; log_not_found off; }
location = /favicon.ico { access_log off; log_not_found off; }
location ~ /\.ht {
deny all;
}
}
I'm using a regexp location directive for a similar proxy setup. This is the minified configuration file:
worker_processes 1;
pid /path/to/file.pid;
worker_priority 15;
events {
worker_connections 512;
accept_mutex on;
}
http {
server {
error_log /path/to/log/error.log error;
listen 127.0.0.1:9000;
server_name example.com;
location ~* (/ghost) {
expires epoch;
proxy_no_cache 1;
proxy_pass http://localhost:1234;
}
location / {
proxy_pass http://localhost:1234;
}
}
}
Have solved similar problem with other apps which have no support for subfolders. Both apps are built on one platform, so they both tries to work in /fx dir. I had to place one of them in to subfolder /gpms .
The idea is to redirect requests with referer from subfolder to destinations which links outside of subfolder - i just add subfolder to beginning of such uris. It is not ideal, but it works.
Here is my nginx config:
server {
listen 80;
server_name mydomain.com;
location / {
rewrite ^/$ /fx/;
proxy_pass http://127.0.0.1:56943/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300;
}
error_log /var/log/nginx/debug.log debug;
set $if_and_hack "";
if ( $http_referer ~ '^http://mydomain.com/gpms/.*$' ) {
set $if_and_hack "refgpms";
}
if ( $uri !~ '^/gpms/.*$' ) {
set $if_and_hack "${if_and_hack}_urinogpms";
}
if ( $if_and_hack = "refgpms_urinogpms" ) {
rewrite ^/(.*)$ http://$host/gpms/$1;
}
location /gpms/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_path /fx /;
proxy_pass http://127.0.0.1:12788/fx/;
proxy_redirect default;
}
}
External links will be broken, but it is not critical for me and i guess it may be corrected.
$if_and_hack is for overcome nginx limitation on nested conditions.
By the way i have got a cookies issue, because they was set with path, and hit browser bug with not sending cookies for a new path after redirect, so i just remove path from cookies.
Note full link form in rewrite - this form of rewrite immediately redirects browser to new page, you should not change it to just "/gpms/$1".
As alternative, i guess, it may be possible to use nginx module to inspect html content and modify links. I have not tried this. Or consider to use subdomains instead of subfolders.
Good news! As of version 0.4.0 Ghost now supports subdirectory installation. And there are already people who've figured this out and created tutorials.

Resources