Airflow + Nginx set up gives Airflow 404 = lots of circles - nginx

I'm trying to set up Airflow behind nginx, using the instructions given here.
airflow.cfg file
base_url = https://myorg.com/airflow
web_server_port = 8081
.
.
.
enable_proxy_fix = True
nginx configuration
server {
listen 443 ssl http2 default_server;
server_name myorg.com;
.
.
.
location /airflow {
proxy_pass http://localhost:8081;
proxy_set_header Host $host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Forwarded-Proto "https";
}
}
Airflow webserver and scheduler are up and running as systemd. When I try to access https://myorg.com/airflow/, it gives Airflow 404 = lots of circles.
What could be wrong? Really appreciate your help in getting this running.

I just had the same problem and fixed it by adding a tailing / to the location: location /airflow/ { instead of location /airflow {. The tailing backslash tells nginx to remove the preceeding /airflow in uri paths to the corresponding python app.
My overall config looks as follows:
server_name my_server.my_org.net;
location /airflow/ {
proxy_pass http://localhost:9997;
proxy_set_header Host $host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
In airflow.cfg I additionally specified:
base_url = http://my_server.my_org.net/airflow
enable_proxy_fix = False # Seems to be deprecated?
web_server_port = 9997

I run into the same problem using https.
But using the configuration in the solution led me to another problem.
Anything other than /airflow/ location falls back to / location.
Returning 404 errors to assets.
Using the configuration bellow solved the issue:
location ^~ /airflow/ {
proxy_pass_header Authorization;
proxy_pass http://localhost:8080/airflow/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_http_version 1.1;
proxy_redirect off;
proxy_set_header Connection "";
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
}

Related

How do I configure nginx and ShinobiCCTV?

Link to Shinobi:
https://shinobi.video/
I have a Shinobi which is at 127.0.0.1.
And also the domain example.com on / is the backend, I want example.com/shinobi to host Shinobi.
I tried to do this via nginx, here is my configuration:
server {
server_name example.com;
listen 443 ssl;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
}
location /shinobi/ {
proxy_pass http://127.0.0.1:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
proxy_redirect off;
}
location /socket.io/ {
proxy_pass http://127.0.0.1:8080/socket.io;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade;
proxy_set_header Host $host;
}
}
This doesn't work for me, I found the answer on reddit:
https://www.reddit.com/r/ShinobiCCTV/comments/fgmce0/problem_with_shinobi_behind_nginx_reverse_proxy/
I changed baseURL in /home/Shinobi/conf.json to https://example.com/shinobi/ and restarted Shinobi pm2 restart all. I get this response:
[PM2] Applying action restartProcessId on app [all](ids: [ 0 ])
[PM2] [camera](0) ✓
When I go to https://example.com/shinobi/{TOKEN}/embed/{GROUP}/{CAMERA}/fullscreen%7Cjquery I get:
Cannot GET /shinobi/{TOKEN}/embed/{GROUP}/{CAMERA}/fullscreen%7Cjquery
That didn't work for me. Can you please tell me how I can fix this and what could be the problem?

Websockets in NGINX not working with server with internet proxy: Error: No pong received in 3 seconds

I was trying to host flask application in NGINX which uses websockets.
It is working fine with the servers which do not use any proxy servers.
When it is hosted in a server that passes requests to proxy servers, client does not receive any message sent via websocket.
Initially none of the external API calls were working which started working when I added environ variable http_proxy and https_proxy for the service.
But the socket is still not working.
Got error: "no pong received in 3 seconds" in the server when trying to connect to websocket
This is what I get in browser
The following is the nginx configuration.
log_format upstreamlog '$server_name to: $upstream_addr [$request] '
'upstream_response_time $upstream_response_time '
'msec $msec request_time $request_time';
upstream socket_nodes {
ip_hash;
server 127.0.0.1:4000;
server 127.0.0.1:4001;
server 127.0.0.1:4002;
}
server {
listen 80;
listen [::]:80;
access_log /var/log/nginx/access.log upstreamlog;
add_header Strict-Transport-Security max-age=15768000;
location /static/* {
alias /file_path;
}
location / {
include uwsgi_params;
proxy_pass http://socket_nodes;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
add_header Front-End-Https on;
proxy_buffer_size 16k;
proxy_busy_buffers_size 16k;
}
location /socket.io {
proxy_pass http://socket_nodes/socket.io;
proxy_http_version 1.1;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
Try changing the socket configuration as below,
location /socket.io {
proxy_pass http://socket_nodes/socket.io;
proxy_http_version 1.1;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $proxy_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}

Unable to use reverse proxy nginx

I have two node js apps that I want to put behind nginx.
I access application 1 which has "/" as a base and I access its pages which are on /be/ but when I try to go to application 2 automatically I am referred to "/".
summary
App1:
based:"/"
url: localhost:3003
application pages can be found in /be/
App2:
base:"/be/"
url: localhost:3000/be/login
the application pages can also be found on /be/
here is my nginx config
server {
listen 80;
server_name 192.168.1.64;
access_log /var/log/nginx/portalerr.logs;
error_log /var/log/nginx/portalaccess.logs;
location / {
proxy_pass http://192.168.1.64:3003/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
}
location /login/ {
proxy_pass http://192.168.1.64:3000/be/login;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
rewrite /be/login(.*) /$1 break;
proxy_cache_bypass $http_upgrade;
}
}

Rewrite a subdomain to a backend proxy whith nginx

if have a subdomain on my nginx webserver configuration: sub.mydomain.com
and i have a backend server which listen on port 5000: http://127.0.0.1:5000
is it possible to pass all subdomain calls to the backend?
Like: https://sub.mydomain.com/list to http://127.0.0.1:5000/sub/list
This should work with all methods: POST, PUT, GET, DELETE
UPDATE:
when i call my server: https://mysubdomain.mydomain.com
with the following configuration:
upstream http_backend {
server 127.0.0.1:5000;
}
server_name ~^(?<subdomain>[^.]+)\.mydomain\.com;
This does not work (error: 404):
location / {
proxy_pass http://http_backend/$subdomain/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
This works fine:
location / {
proxy_pass http://http_backend/mysubdomain/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
When i log the $subdomain variable in the access_log, it seems to be correct.
nginx version: nginx/1.9.15
To pass all subdomains you need to set it in server name by putting dot before domain.
server_name .mydomain.com;
Yes, you can use variables in proxy_pass. And you can extract part of domain using regexp server name.
server {
server_name ~^(?<sub>[^.]+)\.example\.com;
# now subdomain of example.com placed to $sub
# please, note, this rule do not work for http://example.com
location / {
proxy_pass http://127.0.0.1:5000/$sub/;
# Path part of proxy_par URI will replace path
# part of location directive (so / -> /$sub/, /xxxx/ -> /$sub/xxxx/)
}
}
Thats all :)
It seems nginx does not add the $uri to the proxy_pass if i use the $subdomain variable.
The following solution works:
location / {
proxy_pass http://http_backend/$subdomain/$uri;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}

Nginx reverse proxy works only for landing page, any other links won't work

I have a reverse proxy for /forums setup like so:
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location /forums {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://127.0.0.1:4567/;
proxy_redirect off;
# Sockect.IO Support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
It works, kind of, when I try to go to localhost/forums I see the page, but all the static content 404's and if I try to click on a link, e.g. /login it takes me to localhost/login instead of localhost/forums/login, any idea how I can fix this?
You should add a slash character at the end of location /forums so as to get:
location /forums/ {
...
}
According to the doc:
If a location is defined by a prefix string that ends with the slash
character, and requests are processed by one of proxy_pass then the
special processing is performed. In response to a request with URI
equal to this string, but without the trailing slash, a permanent
redirect with the code 301 will be returned to the requested URI with
the slash appended.

Resources