Nginx supervisord configuration - nginx

I have a supervisord server running on localhost:9001.
I am trying to serve it at localhost/supervisord.
The nginx config is like this:
worker_processes 1;
error_log /var/log/nginx/error.log;
pid /tmp/nginx.pid;
#daemon off;
events {
worker_connections 1024;
}
http {
# MIME / Charset
default_type application/octet-stream;
charset utf-8;
# Logging
access_log /var/log/nginx/access.log;
# Other params
server_tokens off;
tcp_nopush on;
tcp_nodelay off;
sendfile on;
upstream supervisord {
server localhost:9001;
}
server {
listen 80;
client_max_body_size 4G;
keepalive_timeout 5;
location ^~ /stylesheets {
alias /Users/ocervell/.virtualenvs/ndc-v3.3/lib/python2.7/site-packages/supervisor/ui/stylesheets;
access_log off;
}
location ^~ /images {
alias /Users/ocervell/.virtualenvs/ndc-v3.3/lib/python2.7/site-packages/supervisor/ui/images;
access_log off;
}
location /supervisord {
# Set client IP / Proxy IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
# Set host header
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://supervisord/;
}
}
}
Before adding ^~ /images and ^~ /stylesheets locations the page was returning a 502 Bad Gateway.
With the above config I am able to access localhost/supervisord but the CSS is missing on the page.
I see the css / images are loaded correctly in the browser:
But I see an error message in the browser console and it seems to be the culprit:
The mimetype in the browser for localhost/stylesheets/supervisor.css it shows as octet-stream instead of text/css.
The mimetype in the browser for localhost:9001/stylesheets/supervisor.css it shows as the correct text/css.
How can I fix this error ?
I thought about dynamically rewriting the mimetype for static files, but I am not an expert in nginx and have no idea how to do that from nginx config.

It's really interesting such an obvious function like putting web interfaces behind transparent reverse proxy is not that straightforward to configure as it should be.
Anyway this is what I do to get reverse proxy working with applications where root can't be modified, like supervisor:
location /supervisor {
proxy_pass http://127.0.0.1:9001/;
}
location / {
if ($http_referer ~ "^.*/supervisor"){
return 301 /supervisor/$request_uri;
}
}
Application-side requests will hit the main end-point but then NginX will re-direct them to the /supervisor EP
This works in most cases but not always.The following supervisor's web functions will fail:
getting action confirmation - you can start / stop services but the result page will fail to load; just go to the /supervisor EP to check the result
live tail does not work; however there is a log display with manual refresh which works under the link with the program name.
Anyway this partial support is good engough for me and you may find it also useful.

I was able to get it working simply with this:
upstream supervisor {
server 127.0.0.1:9001;
}
server {
# ...
location /supervisor/ {
proxy_pass http://supervisor/;
}
}
Even worked in the browser with and without ending slash in url (ie both http://example.com/supervisor and http://example.com/supervisor/ worked).
This was a must for me!

Related

How do I prevent nginx from redirecting to HTTPS in this particular setup?

I have a somewhat messy setup (no choice) where a local computer is made available to the internet through port forwarding. It is only reachable through [public IP]:8000. I cannot get a Let's Encrypt certificate for an IP address, but the part of the app that will be accessed from the internet does not require encryption. So instead, I'm planning on making the app available from the internet at http://[public IP]:8000/, and from the local network at https://[local DNS name]/ (port 80). The certificate used in the latter is issued by our network's root CA. Clients within the network trust this CA.
Furthermore, some small changes are made to the layout of the page when accessed from the internet. These changes are made by setting an embedded query param.
In summary, I need:
+--------------------------+--------------------------+----------+--------------------------------------+
| Accessed using | Redirect to (ideally) | URL args | Current state |
+--------------------------+--------------------------+----------+--------------------------------------+
| http://a.b.c.d:8000 | no redirect | embedded | Arg not appended, redirects to HTTPS |
| http://localhost:8000 | no redirect | embedded | Arg not appended, redirects to HTTPS |
| http://[local DNS name] | https://[local DNS name] | no args | Working as expected |
| https://[local DNS name] | no redirect | no args | Working as expected |
+--------------------------+--------------------------+----------+--------------------------------------+
For the two top rows, I don't want the redirection to HTTPS, and I need ?embedded to be appended to the URL.
Here's my config:
upstream channels-backend {
server api:5000;
}
# Connections from the internet (no HTTPS)
server {
listen 8000;
listen [::]:8000;
server_name [PUBLIC IP ADDRESS] localhost;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
# Do not allow access to /admin/ from the internet.
return 404;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
proxy_pass http://channels-backend/;
}
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend/;
# I want to add "?embedded" to the URL if accessed through http://[public IP]:8000.
# I do not want to redirect to HTTPS.
try_files $uri $uri/ /$uri.html?embedded =404;
}
}
# Upgrade requests from local network to HTTPS
server {
listen 80;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
server_name [local DNS name] [local IP] localhost;
# This works; it redirects to HTTPS.
return 301 https://$http_host$request_uri;
}
# Server for connections from the local network (uses HTTPS)
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name [local DNS name] [local IP] localhost;
ssl_password_file /etc/nginx/certificates/global.pass;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_protocols TLSv1.2 TLSv1.1;
ssl_certificate /etc/nginx/certificates/certificate.crt;
ssl_certificate_key /etc/nginx/certificates/privatekey.key;
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
proxy_pass http://channels-backend/admin/;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
# Proxy to backend
proxy_read_timeout 30;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $server_name;
proxy_redirect off;
proxy_pass http://channels-backend/;
}
# ignore cache frontend
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend/;
# Do not add "?embedded" argument.
try_files $uri $uri/ /$uri.html =404;
}
}
The server serves both the frontend and an API developed using React and Django RF, in case it matters. It's deployed using Docker.
Any pointers would be greatly appreciated.
Edit: I commented out everything except the first server (port 8000), and requests are still being redirected to https://localhost:8000 from http://localhost:8000. I don't understand why. I'm using an incognito tab to rule out cache as the problem.
Edit 2: I noticed that Firefox sets an Upgrade-Insecure-Requests header with the initial request to http://localhost:8000. How can I ignore this header and not upgrade insecure requests? This request was made by Firefox, and not the frontend application.
Edit 3: Please take a look at the below configuration, which I'm now using to try to figure out the issue. How can this possibly result in redirection from HTTP to HTTPS? There's now only one server block, and there's nothing here that could be interpreted as a wish to redirect to https://localhost:8000 from http://localhost:8000. Where does the redirect come from? Notice that I replaced some parts with redirects to Google, Yahoo and Facebook. I'm not redirected to any of these. I'm immediately upgraded to HTTPS, which should not be supported at all with this configuration. It's worth mentioning that the redirect ends in SSL_ERROR_RX_RECORD_TOO_LONG. The certificate is accepted when accessing https://localhost/ (port 80) using the original configuration.
upstream channels-backend {
server api:5000;
}
# Server for connections from the internet (does not use HTTPS)
server {
listen 8000;
listen [::]:8000 default_server;
server_name localhost [public IP];
keepalive_timeout 70;
access_log /var/log/nginx/access.log;
underscores_in_headers on;
ssl off;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /admin/ {
# Do not allow access to /admin/ from the internet.
return 404;
}
location /static/rest_framework/ {
alias /home/docker/backend/static/rest_framework/;
}
location /static/admin/ {
alias /home/docker/backend/static/admin/;
}
location /files/media/ {
alias /home/docker/backend/media/;
}
location /api/ {
proxy_pass http://channels-backend/;
}
location / {
if ($args != "embedded") {
return 301 https://google.com;
# return 301 http://$http_host$request_uri?embedded;
}
return 301 https://yahoo.com;
# root /var/www/frontend/;
# try_files $uri $uri/ /$uri.html =404;
}
}
Boy, do I feel stupid.
In my docker-compose.yml file, I had accidentally mapped port 8000 to 80:
nginx-server:
image: nginx-server
build:
context: ./
dockerfile: .docker/dockerfiles/NginxDockerfile
restart: on-failure
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
- "0.0.0.0:8000:80" # Oops
So any request on port 8000 was received by nginx as a request on port 80. Thus, even a simple config like...
server {
listen 8000;
return 301 https://google.com;
}
... would result in an attempt to upgrade to HTTPS (causes include unexpected caching of redirects, possibly default behavior, etc.) on port 80. I was thoroughly confused, but fixing my compose instructions fixed the problem:
nginx-server:
image: nginx-server
build:
context: ./
dockerfile: .docker/dockerfiles/NginxDockerfile
restart: on-failure
ports:
- "0.0.0.0:80:80"
- "0.0.0.0:443:443"
- "0.0.0.0:8000:8000" # Fixed

nginx rewrite not working

I'm trying to set up a simple nginx server to act as a proxy between my front end ui and my back end api. The setup is fairly simple. The UI makes all api requests to /api/endpoint and the proxy server passes the request to the api. The proxy also needs to rewrite the request so that instead of going to http://api.location.net/api/endpoint, it goes to http://api.location.net/endpoint. The UI resides on http://api.location.net. This part isn't working (i get a 500 error) and I'm pretty sure it has to do with how I'm writing my rewrite rule. Here's my nginx config.
daemon off;
error_log off;
worker_processes 2;
worker_rlimit_nofile 100000;
events {
worker_connections 50000;
accept_mutex off;
}
http {
include /etc/nginx/mime.types;
access_log off;
sendfile on;
server {
listen 80 default_server;
server_name localhost _;
location / {
alias /srv/site/;
}
location /api/ {
rewrite ^/api ""; # I think this is the problem
proxy_pass http://api.location.net;
proxy_pass_request_headers on;
proxy_pass_header X-ResponseData;
proxy_redirect off;
}
}
}
Any help would be greatly appreciated, nginx is still fairly new for me and the documentation on nginx rewrite doesn't seem to have what I need.
If I understood you right, this should help
location /api/ {
proxy_pass http://api.location.net/;
proxy_pass_request_headers on;
proxy_pass_header X-ResponseData;
proxy_redirect off;
}
Note the URI part at proxy_pass directive
If the proxy_pass directive is specified with a URI, then when a
request is passed to the server, the part of a normalized request URI
matching the location is replaced by a URI specified in the directive:
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass

configuring nginx as proxy to work with devpi mirror on HP-cloud

I'm trying to create a devpi mirror on HP-cloud that will be accessed via nginx, i.e - nginx listens to port 80 and used as a proxy to devpi that is using port 4040 on the same machine.
I have configured an HP-cloud security group that is opened for all ports (inbound and outbound) in hp-cloud (just for the beginning, I'll change it later of-course), and started an ubuntu 14 instance.
I have allocated a public IP to the instance that I have created.
I have installed devpi-server using pip, and nginx using apt-get.
I have followed the instructions on devpi's tutuorial page here:
ran devpi-server --port 4040 --gen-config, and copied the contents that was created in nginx-devpi.conf into nginx.conf.
Then, I have started the server using devpi-server --port 4040 --start.
Started nginx using sudo nginx.
My problem is as follows:
When I'm SSHing to the hp-instance on which the nginx and devpi are running, and executing pip install -i http://<public-ip>:80/root/pypi/ simplejson it succeeded.
But, when I'm running the same command from my laptop I get
Downloading/unpacking simplejson
Cannot fetch index base URL http://<public-ip>:80/root/pypi/
http://<public-ip>:80/root/pypi/simplejson/ uses an insecure transport scheme (http). Consider using https if <public-ip>:80 has it available
Could not find any downloads that satisfy the requirement simplejson
Cleaning up...
No distributions at all found for simplejson
Storing debug log for failure in /home/hagai/.pip/pip.log
I thought it might be security/network issue, but I think that this is not the case, because curl http://<public-ip>:80 returns the same thing when I'm executing it from my laptop and from the HP instance:
{
"type": "list:userconfig",
"result": {
"root": {
"username": "root",
"indexes": {
"pypi": {
"type": "mirror",
"bases": [],
"volatile": false
}
}
}
}
}
I have also tried to start another instance in HP-cloud and execute pip install -i http://<public-ip>:80/root/pypi/ simplejson, but I got the same error as in my laptop.
I can't understand what is the difference between these two cases, and I'd be happy if someone would have a solution for this case, or any idea what might be the problem.
My nginx.conf file:
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
server {
server_name localhost;
listen 80;
gzip on;
gzip_min_length 2000;
gzip_proxied any;
#gzip_types text/html application/json;
proxy_read_timeout 60s;
client_max_body_size 64M;
# set to where your devpi-server state is on the filesystem
root /home/ubuntu/.devpi/server;
# try serving static files directly
location ~ /\+f/ {
error_page 418 = #proxy_to_app;
if ($request_method != GET) {
return 418;
}
try_files /+files$uri #proxy_to_app;
}
# try serving docs directly
location ~ /\+doc/ {
try_files $uri #proxy_to_app;
}
location / {
error_page 418 = #proxy_to_app;
return 418;
}
location #proxy_to_app {
proxy_pass http://localhost:4040;
#dynamic: proxy_set_header X-outside-url $scheme://$host:$server_port;
proxy_set_header X-outside-url http://localhost:80;
proxy_set_header X-Real-IP $remote_addr;
}
}
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
#passenger_root /usr;
#passenger_ruby /usr/bin/ruby;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
#include /etc/nginx/sites-enabled/*;
}
edit:
I have tried to use devpi-client from my laptop, and when I've executed devpi use http://<public-ip>:80 from my laptop I get the following:
using server: http://localhost/ (not logged in)
no current index: type 'devpi use -l' to discover indices
~/.pydistutils.cfg : no config file exists
~/.pip/pip.conf : no config file exists
~/.buildout/default.cfg: no config file exists
always-set-cfg: no
You can try modify from this:
location #proxy_to_app {
proxy_pass http://localhost:4040;
#dynamic: proxy_set_header X-outside-url $scheme://$host:$server_port;
proxy_set_header X-outside-url http://localhost:80;
proxy_set_header X-Real-IP $remote_addr;
}
To this
location #proxy_to_app {
proxy_pass http://localhost:4040;
proxy_set_header X-outside-url $scheme://$host;
proxy_set_header X-Real-IP $remote_addr;
}
This has been work for me :-).

Nginx: cache only specific urls and named location

I have rails application. There are parts of nginx config of it:
upstream app_server {
server unix:/var/www/app/shared/unicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name app hostname;
keepalive_timeout 5;
root /var/www/app/current/public;
try_files $uri/index.html $uri.html $uri #app;
location #app {
proxy_pass http://app_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_cache off;
}
location ~ /assets/*\.(png|gif|jpg|jpeg|css|js|swf|ico|gz)(\?[0-9]+)?$ {
access_log off;
}
}
I want to cache several pages of my apps (for example all *.json urls). How I can do that?
According to nginx docs I can't:
use nested location in named location
use proxy_cache in if block
You can get away with adding Json to the extensions in the static file block if they are not virtual URLs.
If they are, you need to either setup differently by forwarding everything to your app by default and making exceptions, so you avoid having to use named locations. Or you can set variables based on if statements inside the named location:
location #app {
set $proxy_cache_cfg "off";
if($request_uri ~ \.json$) {
set $proxy_cache_cfg "json_zone";
}
}
Untested, not sure if "off" should be quoted and whether it even would work here. If this won't work, you can always use the reverse approach and set proxy_no_cache based on a variable, since that is in effect for anything non-empty and non-zero.

Nginx as reverse-proxy: serving files without extension

I'm currently using Nginx coupled with Apache, serving following static files with success.
Extract from my /sites-enabled/default:
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|xml)$ {
root /home/website/public_html/app/public;
}
But I also have some cache files located here:
/home/website/public_html/app/storage/cache
/home/website/public_html/app/storage/views
/home/website/public_html/app/storage/sessions
/cache and /sessions also have sub-directories. All sub-directories and files have random filename, and no extension.
I want Nginx to also serve these files.
I tried this (example for /views folder), but without success. I even have nothing in logs, but Nginx restart correctly, and website load with no errors.
location /views {
alias /home/website/public/app/app/storage/views;
access_log /var/log/nginx/web.views.access.log;
error_log /var/log/nginx/web.views.error.log;
}
Tried this also, but same effects as above:
location ~* ^.$ {
root /home/website/public/app/app/storage/views;
access_log /var/log/nginx/web.views.access.log;
error_log /var/log/nginx/web.views.error.log;
}
Also tried adding add_header Content-Type application/octet-stream; in these 2 tries, but no change.
Finally, here is the http part of my nginx.conf
http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/access.log;
sendfile on;
keepalive_timeout 65;
tcp_nodelay on;
gzip on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
(/var/log/nginx/access.log and /var/log/nginx/error.log don't show up anything related to my issue, too).
Thanks for any clue and help !
EDIT
Complete current /sites-enabled/default file (and yes, there's a double /app/, it's normal ;) )
# You may add here your
# server {
# ...
# }
# statements for each of your virtual hosts
server {
listen 80; ## listen for ipv4
listen [::]:80 default ipv6only=on; ## listen for ipv6
server_name www.website.com website.com;
#access_log /var/log/nginx/localhost.access.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080/;
access_log off;
#root /var/www;
#index index.html index.htm;
}
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|xml)$ {
root /home/website/public_html/app/public;
expires 30d;
}
location ~* ^.$ {
add_header Content-Type application/octet-stream;
root /home/website/public/app/app/storage/views;
access_log /var/log/nginx/bt.views.access.log;
error_log /var/log/nginx/bt.views.error.log;
}
}
The problem is, that you need something to identify the files without any extension. A sub-directory or something that’s always present within the request. Your regular expression only matched for requests that start end end with a dot (e.g. http://example.com/.). The following server configuration assumes that all URLs start with storage, as this would be the only possibility to identify those files.
Please note that I’m using the try_files directive to rewrite the internal path where nginx should look for the file. The root directive is not meant for what you want to achieve.
And last but not least, you should always nest location blocks with regular expressions. There is no limit in the nesting level. nginx will create some kind of tree data structure to search for the best matching location. So think of a tree while writing the blocks.
server {
listen 80 default;
listen [::]:80 default ipv6only=on;
server_name www.website.com website.com;
root /home/website/public_html/app;
location / {
# Matches any request for a URL ending on any of the extension
# listed in the regular expression.
location ~* \.(jpe?g|gif|css|png|js|ico|txt|xml)$ {
expires 30d
access_log off;
try_files /public$uri =404;
}
# Matches any request starting with storage.
location ~* ^/storage {
access_log /var/log/nginx/bt.views.access.log;
error_log /var/log/nginx/bt.views.error.log;
try_files /app$uri;
}
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080/;
access_log off;
}
}

Resources