nginx rewrite not working - nginx

I'm trying to set up a simple nginx server to act as a proxy between my front end ui and my back end api. The setup is fairly simple. The UI makes all api requests to /api/endpoint and the proxy server passes the request to the api. The proxy also needs to rewrite the request so that instead of going to http://api.location.net/api/endpoint, it goes to http://api.location.net/endpoint. The UI resides on http://api.location.net. This part isn't working (i get a 500 error) and I'm pretty sure it has to do with how I'm writing my rewrite rule. Here's my nginx config.
daemon off;
error_log off;
worker_processes 2;
worker_rlimit_nofile 100000;
events {
worker_connections 50000;
accept_mutex off;
}
http {
include /etc/nginx/mime.types;
access_log off;
sendfile on;
server {
listen 80 default_server;
server_name localhost _;
location / {
alias /srv/site/;
}
location /api/ {
rewrite ^/api ""; # I think this is the problem
proxy_pass http://api.location.net;
proxy_pass_request_headers on;
proxy_pass_header X-ResponseData;
proxy_redirect off;
}
}
}
Any help would be greatly appreciated, nginx is still fairly new for me and the documentation on nginx rewrite doesn't seem to have what I need.

If I understood you right, this should help
location /api/ {
proxy_pass http://api.location.net/;
proxy_pass_request_headers on;
proxy_pass_header X-ResponseData;
proxy_redirect off;
}
Note the URI part at proxy_pass directive
If the proxy_pass directive is specified with a URI, then when a
request is passed to the server, the part of a normalized request URI
matching the location is replaced by a URI specified in the directive:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass

Related

How do I prevent nginx from redirecting to HTTPS in this particular setup?

I have a somewhat messy setup (no choice) where a local computer is made available to the internet through port forwarding. It is only reachable through [public IP]:8000. I cannot get a Let's Encrypt certificate for an IP address, but the part of the app that will be accessed from the internet does not require encryption. So instead, I'm planning on making the app available from the internet at http://[public IP]:8000/, and from the local network at https://[local DNS name]/ (port 80). The certificate used in the latter is issued by our network's root CA. Clients within the network trust this CA.
Furthermore, some small changes are made to the layout of the page when accessed from the internet. These changes are made by setting an embedded query param.
In summary, I need:
+--------------------------+--------------------------+----------+--------------------------------------+
| Accessed using | Redirect to (ideally) | URL args | Current state |
+--------------------------+--------------------------+----------+--------------------------------------+
| http://a.b.c.d:8000 | no redirect | embedded | Arg not appended, redirects to HTTPS |
| http://localhost:8000 | no redirect | embedded | Arg not appended, redirects to HTTPS |
| http://[local DNS name] | https://[local DNS name] | no args | Working as expected |
| https://[local DNS name] | no redirect | no args | Working as expected |
+--------------------------+--------------------------+----------+--------------------------------------+
For the two top rows, I don't want the redirection to HTTPS, and I need ?embedded to be appended to the URL.
Here's my config:
upstream channels-backend {
server api:5000;
}
# Connections from the internet (no HTTPS)
server {
listen 8000;
listen [::]:8000;
server_name [PUBLIC IP ADDRESS] localhost;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
# Do not allow access to /admin/ from the internet.
return 404;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
proxy_pass http://channels-backend/;
}
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend/;
# I want to add "?embedded" to the URL if accessed through http://[public IP]:8000.
# I do not want to redirect to HTTPS.
try_files $uri $uri/ /$uri.html?embedded =404;
}
}
# Upgrade requests from local network to HTTPS
server {
listen 80;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
server_name [local DNS name] [local IP] localhost;
# This works; it redirects to HTTPS.
return 301 https://$http_host$request_uri;
}
# Server for connections from the local network (uses HTTPS)
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name [local DNS name] [local IP] localhost;
ssl_password_file /etc/nginx/certificates/global.pass;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_protocols TLSv1.2 TLSv1.1;
ssl_certificate /etc/nginx/certificates/certificate.crt;
ssl_certificate_key /etc/nginx/certificates/privatekey.key;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
proxy_pass http://channels-backend/admin/;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
# Proxy to backend
proxy_read_timeout 30;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $server_name;
proxy_redirect off;
proxy_pass http://channels-backend/;
}
# ignore cache frontend
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend/;
# Do not add "?embedded" argument.
try_files $uri $uri/ /$uri.html =404;
}
}
The server serves both the frontend and an API developed using React and Django RF, in case it matters. It's deployed using Docker.
Any pointers would be greatly appreciated.
Edit: I commented out everything except the first server (port 8000), and requests are still being redirected to https://localhost:8000 from http://localhost:8000. I don't understand why. I'm using an incognito tab to rule out cache as the problem.
Edit 2: I noticed that Firefox sets an Upgrade-Insecure-Requests header with the initial request to http://localhost:8000. How can I ignore this header and not upgrade insecure requests? This request was made by Firefox, and not the frontend application.
Edit 3: Please take a look at the below configuration, which I'm now using to try to figure out the issue. How can this possibly result in redirection from HTTP to HTTPS? There's now only one server block, and there's nothing here that could be interpreted as a wish to redirect to https://localhost:8000 from http://localhost:8000. Where does the redirect come from? Notice that I replaced some parts with redirects to Google, Yahoo and Facebook. I'm not redirected to any of these. I'm immediately upgraded to HTTPS, which should not be supported at all with this configuration. It's worth mentioning that the redirect ends in SSL_ERROR_RX_RECORD_TOO_LONG. The certificate is accepted when accessing https://localhost/ (port 80) using the original configuration.
upstream channels-backend {
server api:5000;
}
# Server for connections from the internet (does not use HTTPS)
server {
listen 8000;
listen [::]:8000 default_server;
server_name localhost [public IP];
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
ssl off;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
# Do not allow access to /admin/ from the internet.
return 404;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
proxy_pass http://channels-backend/;
}
location / {
if ($args != "embedded") {
return 301 https://google.com;
# return 301 http://$http_host$request_uri?embedded;
}
return 301 https://yahoo.com;
# root /var/www/frontend/;
# try_files $uri $uri/ /$uri.html =404;
}
}
Boy, do I feel stupid.
In my docker-compose.yml file, I had accidentally mapped port 8000 to 80:
nginx-server:
image: nginx-server
build:
context: ./
dockerfile: .docker/dockerfiles/NginxDockerfile
restart: on-failure
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
- "0.0.0.0:8000:80" # Oops
So any request on port 8000 was received by nginx as a request on port 80. Thus, even a simple config like...
server {
listen 8000;
return 301 https://google.com;
}
... would result in an attempt to upgrade to HTTPS (causes include unexpected caching of redirects, possibly default behavior, etc.) on port 80. I was thoroughly confused, but fixing my compose instructions fixed the problem:
nginx-server:
image: nginx-server
build:
context: ./
dockerfile: .docker/dockerfiles/NginxDockerfile
restart: on-failure
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
- "0.0.0.0:8000:8000" # Fixed

Redirect NiFi Azure AD OAuth Request usinig NGINX

I have been plugging away at this for a good number of days now and I have been unable to find a solution. We have deployed NiFi onto an HDInsight cluster Edge Node, which comes with NGINX (free) pre-installed. We have installed NiFi onto this node and had the basic, unsecured NiFi UI being served by NGINX. So far, so good.
The requirement is that NiFi be secured using Azure AD and it is this that is giving me issues. The problem is that no matter what I try, I am unable to configure NGINX such that it redirects the OAuth request from NiFi to https://login.microsoftonline.com. The closest I have got is to redirect to the required domain, and include the query string, but missing the location part of the URL (AD tenant ID), which means authentication fails.
So my question is how can I configure NGINX to correctly redirect the OAuth request? I am either able to get:
https://login.microsoftonline.com/?client_id=<ID>&response_type=code&scope=openid+email&state=4bibmlfesmgbmi9o1p8blibd4q&redirect_uri=https%3A%2F%2F<NODE HOSTNAME>%3A443%2Fnifi-api%2Faccess%2Foidc%2Fcallback
or
https://<NODE HOSTNAME>/<TENANT ID>/oauth2/v2.0/authorize?client_id=<ID>&response_type=code&scope=openid+email&state=4bibmlfesmgbmi9o1p8blibd4q&redirect_uri=https%3A%2F%2Fems-poc-tri-hdi-nfi.apps.azurehdinsight.net%3A443%2Fnifi-api%2Faccess%2Foidc%2Fcallback
I am unable to get:
https://login.microsoftonline.com/<TENANT ID>/oauth2/v2.0/authorize?client_id=<ID>&response_type=code&scope=openid+email&state=4bibmlfesmgbmi9o1p8blibd4q&redirect_uri=https%3A%2F%2Fems-poc-tri-hdi-nfi.apps.azurehdinsight.net%3A443%2Fnifi-api%2Faccess%2Foidc%2Fcallback
I have tried all sorts, but I have finally had to admit defeat. The current NGINX config is given below:
user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
events {
worker_connections 768;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_names_hash_bucket_size 1024;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log notice;
rewrite_log on;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
#upstream nifi {
# server 10.1.0.10:9443;
# keepalive 512;
#}
server
{
listen 443 default_server;
#server_name localhost;
server_name ems-poc-tri-hdi-nfi.apps.azurehdinsight.net;
proxy_set_header X-ProxyHost ems-poc-tri-hdi-nfi.apps.azurehdinsight.net;
proxy_set_header X-ProxyScheme https;
proxy_set_header X-ProxyPort 443;
location = / {
return 301 https://10.1.0.10:9443/nifi;
}
location /nifi {
proxy_pass https://10.1.0.10:9443/nifi;
}
location /nifi-api {
proxy_pass https://10.1.0.10:9443/nifi-api;
}
location ~ "\/<AAD TENANT ID>" {
access_log /var/log/nginx/special.access.log;
proxy_set_header Host login.microsoftonline.com;
add_header X-uri "$uri";
add_header X-host "$host";
add_header X-request_uri "$request_uri";
add_header X-args "$args";
if ($request_uri ~ "/[^?]+\?[^?]+callback$") {
#return 307 https://login.microsoftonline.com$uri?$args&redirect=true;
#rewrite ^(?<location>/[^?]+) $location redirect;
#rewrite ^(?<location>/[^?]+) https://login.microsoftonline.com$location redirect;
rewrite ^ $scheme://login.microsoftonline.com$uri?$args&redirect=true? last;
}
if ($request_uri ~ "/[^?]+\?[^?]+true$") {
# proxy_pass https://login.microsoftonline.com;
rewrite ^ https://login.microsoftonline.com last;
#return 307 https://login.microsoftonline.com$uri?$args;
}
}
}
}

Nginx supervisord configuration

I have a supervisord server running on localhost:9001.
I am trying to serve it at localhost/supervisord.
The nginx config is like this:
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /tmp/nginx.pid;
#daemon off;
events {
worker_connections 1024;
}
http {
# MIME / Charset
default_type application/octet-stream;
charset utf-8;
# Logging
access_log /var/log/nginx/access.log;
# Other params
server_tokens off;
tcp_nopush on;
tcp_nodelay off;
sendfile on;
upstream supervisord {
server localhost:9001;
}
server {
listen 80;
client_max_body_size 4G;
keepalive_timeout 5;
location ^~ /stylesheets {
alias /Users/ocervell/.virtualenvs/ndc-v3.3/lib/python2.7/site-packages/supervisor/ui/stylesheets;
access_log off;
}
location ^~ /images {
alias /Users/ocervell/.virtualenvs/ndc-v3.3/lib/python2.7/site-packages/supervisor/ui/images;
access_log off;
}
location /supervisord {
# Set client IP / Proxy IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
# Set host header
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://supervisord/;
}
}
}
Before adding ^~ /images and ^~ /stylesheets locations the page was returning a 502 Bad Gateway.
With the above config I am able to access localhost/supervisord but the CSS is missing on the page.
I see the css / images are loaded correctly in the browser:
But I see an error message in the browser console and it seems to be the culprit:
The mimetype in the browser for localhost/stylesheets/supervisor.css it shows as octet-stream instead of text/css.
The mimetype in the browser for localhost:9001/stylesheets/supervisor.css it shows as the correct text/css.
How can I fix this error ?
I thought about dynamically rewriting the mimetype for static files, but I am not an expert in nginx and have no idea how to do that from nginx config.
It's really interesting such an obvious function like putting web interfaces behind transparent reverse proxy is not that straightforward to configure as it should be.
Anyway this is what I do to get reverse proxy working with applications where root can't be modified, like supervisor:
location /supervisor {
proxy_pass http://127.0.0.1:9001/;
}
location / {
if ($http_referer ~ "^.*/supervisor"){
return 301 /supervisor/$request_uri;
}
}
Application-side requests will hit the main end-point but then NginX will re-direct them to the /supervisor EP
This works in most cases but not always.The following supervisor's web functions will fail:
getting action confirmation - you can start / stop services but the result page will fail to load; just go to the /supervisor EP to check the result
live tail does not work; however there is a log display with manual refresh which works under the link with the program name.
Anyway this partial support is good engough for me and you may find it also useful.
I was able to get it working simply with this:
upstream supervisor {
server 127.0.0.1:9001;
}
server {
# ...
location /supervisor/ {
proxy_pass http://supervisor/;
}
}
Even worked in the browser with and without ending slash in url (ie both http://example.com/supervisor and http://example.com/supervisor/ worked).
This was a must for me!

Nginx loosing header sometime

I am using nginx as reverse proxy server for android application(get/post requests only). Some of the data contained in the headers. In some cases nginx loses "id" or "fail_id" header.
config:
user user;
worker_processes 4;
error_log /var/log/nginx/error.log;
events {
worker_connections 100000;
use epoll;
}
http {
upstream myproject {
server 192.168.88.246:2053;
}
server {
listen 2054;
ssl on;
ssl_certificate /home/user/android/cert/cert.pem;
ssl_certificate_key /home/user/android/cert/key.pem;
proxy_read_timeout 600;
proxy_send_timeout 600;
location / {
proxy_pass http://myproject;
proxy_pass_request_headers on;
}
}
}
Could i set original headers of request?
Updated:
A more detailed study found that nginx miss "fail_id" header. All other headers are working.
Problem solved!
Nginx default config misses headers with underscore.
This directive solved the problem:
underscores_in_headers on;
Thank You For the underscore directive. I have used underscores_in_headers on; directive and the header value with underscore passed to my node app.
Now I am able to access header value (api_key) using postman request and angular request from the web.
But now when The request is raise from the android app and I have set the api_key in the android request header, I am unable to access the api_key.
My config is:
server {
listen 80;
server_name uat.api.myserver.com;
underscores_in_headers on;
location / {
proxy_pass http://localhost:9102;
}
}

Nginx as reverse-proxy: serving files without extension

I'm currently using Nginx coupled with Apache, serving following static files with success.
Extract from my /sites-enabled/default:
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|xml)$ {
root /home/website/public_html/app/public;
}
But I also have some cache files located here:
/home/website/public_html/app/storage/cache
/home/website/public_html/app/storage/views
/home/website/public_html/app/storage/sessions
/cache and /sessions also have sub-directories. All sub-directories and files have random filename, and no extension.
I want Nginx to also serve these files.
I tried this (example for /views folder), but without success. I even have nothing in logs, but Nginx restart correctly, and website load with no errors.
location /views {
alias /home/website/public/app/app/storage/views;
access_log /var/log/nginx/web.views.access.log;
error_log /var/log/nginx/web.views.error.log;
}
Tried this also, but same effects as above:
location ~* ^.$ {
root /home/website/public/app/app/storage/views;
access_log /var/log/nginx/web.views.access.log;
error_log /var/log/nginx/web.views.error.log;
}
Also tried adding add_header Content-Type application/octet-stream; in these 2 tries, but no change.
Finally, here is the http part of my nginx.conf
http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/access.log;
sendfile on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
(/var/log/nginx/access.log and /var/log/nginx/error.log don't show up anything related to my issue, too).
Thanks for any clue and help !
EDIT
Complete current /sites-enabled/default file (and yes, there's a double /app/, it's normal ;) )
# You may add here your
# server {
# ...
# }
# statements for each of your virtual hosts
server {
listen 80; ## listen for ipv4
listen [::]:80 default ipv6only=on; ## listen for ipv6
server_name www.website.com website.com;
#access_log /var/log/nginx/localhost.access.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080/;
access_log off;
#root /var/www;
#index index.html index.htm;
}
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|xml)$ {
root /home/website/public_html/app/public;
expires 30d;
}
location ~* ^.$ {
add_header Content-Type application/octet-stream;
root /home/website/public/app/app/storage/views;
access_log /var/log/nginx/bt.views.access.log;
error_log /var/log/nginx/bt.views.error.log;
}
}
The problem is, that you need something to identify the files without any extension. A sub-directory or something that’s always present within the request. Your regular expression only matched for requests that start end end with a dot (e.g. http://example.com/.). The following server configuration assumes that all URLs start with storage, as this would be the only possibility to identify those files.
Please note that I’m using the try_files directive to rewrite the internal path where nginx should look for the file. The root directive is not meant for what you want to achieve.
And last but not least, you should always nest location blocks with regular expressions. There is no limit in the nesting level. nginx will create some kind of tree data structure to search for the best matching location. So think of a tree while writing the blocks.
server {
listen 80 default;
listen [::]:80 default ipv6only=on;
server_name www.website.com website.com;
root /home/website/public_html/app;
location / {
# Matches any request for a URL ending on any of the extension
# listed in the regular expression.
location ~* \.(jpe?g|gif|css|png|js|ico|txt|xml)$ {
expires 30d
access_log off;
try_files /public$uri =404;
}
# Matches any request starting with storage.
location ~* ^/storage {
access_log /var/log/nginx/bt.views.access.log;
error_log /var/log/nginx/bt.views.error.log;
try_files /app$uri;
}
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080/;
access_log off;
}
}

Resources