Nginx Reverse Proxy Websocket Authentication - HTTP 403 - nginx

I'm using Nginx as a reverse proxy of a Spring boot application. I also use Websockets with sockjs and stomp messages.
Here is the context configuration.
<websocket:message-broker application-destination-prefix="/app">
<websocket:stomp-endpoint path="/localization" >
<websocket:sockjs/>
</websocket:stomp-endpoint>
<websocket:simple-broker prefix="/topic" />
</websocket:message-broker>
Here is the client code:
var socket = new SockJS(entryPointUrl);
var stompClient = Stomp.over(socket);
var _this = this;
stompClient.connect({}, function () {
stompClient.subscribe('/app/some-url', function (message) {
// do some stuff
});
});
I also you Spring Security to protect some content.
#Configuration
#Order(4)
public static class FrontendSecurityConfig extends WebSecurityConfigurerAdapter {
#Override
protected void configure(HttpSecurity http) throws Exception {
http.authorizeRequests()
.antMatchers("/js/**", "/css/**", "/webjars/**").permitAll()
.anyRequest().authenticated()
.and()
.formLogin().loginPage("/login").permitAll()
.and()
.logout().permitAll();
}
}
Everything works great, expect when I run this application behind a Nginx reverse proxy. Here is the reverse configuration:
proxy_pass http://testsysten:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support (nginx 1.4)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
# Max body size
client_max_body_size 10M;
The connection always fails with a HTTP 403 code.
I'm using version 1.9.7.
Do you have any idea, why the client does not gets authenticated?
I know similar questions, like this one but the solutions do not work at all.
Update
I managed to run the application over HTTP. I need to pass the CSRF token in the Nginx configuration. New configuration is:
proxy_pass http://testsysten:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Pass the csrf token (see https://de.wikipedia.org/wiki/Cross-Site-Request-Forgery)
# Default in Spring Boot
proxy_pass_header X-XSRF-TOKEN;
# WebSocket support (nginx 1.4)
proxy_http_version 1.1;
Only missing is redirect over HTTPS. In the Spring logs is see following entry:
o.s.w.s.s.t.h.DefaultSockJsService - Processing transport request: GET http://testsystem:80/localization/226/3mbmu212/websocket
Seems like Nginx Proxy needs to rewrite the to the right port.

I solved the problem by myself. Basically, Nginx needs to pass some additional header values if you want to use Websocket and Spring Security. The following lines need to be added to location section in your Nginx config:
# Pass the csrf token (see https://de.wikipedia.org/wiki/Cross-Site-Request-Forgery)
# Default in Spring Boot and required. Without it nginx suppresses the value
proxy_pass_header X-XSRF-TOKEN;
# Set origin to the real instance, otherwise a of Spring security check will fail
# Same value as defined in proxy_pass
proxy_set_header Origin "http://testsysten:8080";

The accepted solution did not work for me although I was using a very classical HTTPS configuration:
server {
listen 443 ssl;
location /ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8888;
}
...
The problem is that Spring checks the origin and specifically that code was causing me trouble:
// in org.springframework.web.util.UriComponentsBuilder.adaptFromForwardedHeaders(HttpHeaders):
if ((this.scheme.equals("http") && "80".equals(this.port)) ||
(this.scheme.equals("https") && "443".equals(this.port))) {
this.port = null;
}
In that code the scheme is 'http' and the port is 8888, which is not discarded because it is not the standard port.
The browser however hits https://myserver/ and the 443 port is omitted because it is the default HTTPS one.
Therefore the ports do not match (empty != 8888) and origin check fails.
Either you can disable origin checks in Spring WebSockets:
registry.addHandler( resgisterHandler(), "/ws" ).setAllowedOrigins( "*" );
or (probably safer) you can add the scheme and port to the NGINX proxy configuration:
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
If you are interested, those headers are read in
org.springframework.web.util.UriComponentsBuilder.adaptFromForwardedHeaders(HttpHeaders)
For Spring Boot 2.2.2+
Starting with Spring Boot version 2.2.2 you should be adding following setting for these X-Forwarded-* headers to be taken into account:
server.forward-headers-strategy=native
(in application.properties for instance)

I had faced a similar problem. I was unable to use the basic Spring Security authentication with NGINX. Apart from setting the proxy_pass_header X-XSRF-TOKEN;, I also had to set underscores_in_headers on;, since NGINX by default does not allow headers with underscores and the CSRF token is named _csrf.
So my final configuration file looked like this:
server {
underscores_in_headers on;
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name localhost;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
location /example/ {
proxy_pass_header X-XSRF-TOKEN;
proxy_pass http://localhost:8080/;
}
}

I solved this problem without CSRF header in NGINX proxy.
My stack: spring-boot, spring-security (with redis session store), spring-boot-websocket with default STOMP implementation, NGINX to serve frontend and proxied to another services that frontend consume.
In first time I use the default configuration show in the NGINX Blog here and here (copy and paste for history):
http {
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream websocket {
server 192.168.100.10:8010;
}
server {
listen 8020;
location / {
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
}
But dont work, still 403 Forbidden.
I fixed this issue with the configuration below (the real important part to fix websocket is # WebSocket Proxy):
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 30010;
server_name localhost;
client_max_body_size 10M;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
# Backend API Proxy
location /api {
proxy_pass http://192.168.0.100:30080;
proxy_set_header Host $http_host;
proxy_set_header Access-Control-Allow-Origin 192.168.0.100;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
rewrite ^/api/?(.*) /$1 break;
proxy_redirect off;
}
# CDN Proxy
location ~ ^/cdn/(.*) {
proxy_pass http://192.168.0.110:9000;
rewrite ^/cdn/(.*) /$1 break;
}
# This is the configuration that fix the problem with WebSocket
# WebSocket Proxy
location /ws {
proxy_pass http://192.168.0.120:30090;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header Access-Control-Allow-Origin 192.168.0.120;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
}
}
}

In my case (Spring Boot app), in addition to setting the Origin header as specified in the accepted answer, I had to set the Host header to match the ip:port of the Origin header, or to get rid of it altogether.
This is my working vhost config:
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl_certificate /etc/ssl/certs/<your-cert-file>.pem;
ssl_certificate_key /etc/ssl/private/<your-key-file>.key;
server_name <your-server-fqdn>;
access_log /var/log/nginx/<your-server-fqdn>.access.log;
error_log /var/log/nginx/<your-server-fqdn>.error.log error;
root /srv/www/<your-server-fqdn>;
index index.html index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /api {
proxy_pass http://127.0.0.1:8080/v1;
}
location /async-api {
proxy_pass http://127.0.0.1:8080/stomp;
proxy_http_version 1.1;
# either set Host header as follows or get rid of the directive altogether
#proxy_set_header Host "127.0.0.1:8080";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
# Set origin to the real instance, otherwise a of Spring security check will fail
# Same value as defined in proxy_pass
proxy_set_header Origin "http://127.0.0.1:8080";
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /admin-api {
proxy_pass http://127.0.0.1:8080/api;
}
}

Related

Daphne websocket erro r 200 when used with nginx

im trying to implement a system that comprise of nginx , daphne and gunicorn. So far , i have gotten gunicorn to work , however im facing an issue connecting the websockets for daphne. This issue does not arise when im in developing mode , only when i utilize nginx. This is the error code from my console :
reconnecting-websocket.js:199 WebSocket connection to 'ws://192.168.8.31/' failed: Error during WebSocket handshake: Unexpected response code: 200
This is my nginx config file :
upstream crm_server { server unix:/home/user/project/venv/run/gunicorn.sock fail_timeout=0;
}
upstream channels-backend {
server localhost:8001;
}
server {
listen 80;
# add here the ip address of your server
# or a domain pointing to that ip (like example.com or www.example.com)
server_name 192.168.8.31;
keepalive_timeout 5;
client_max_body_size 4G;
access_log /home/user/project/venv/logs/nginx-access.log;
error_log /home/user/project/venv/logs/nginx-error.log;
location /staticfiles/ {
root /home/user/roject/crm/staticfiles/;
}
# checks for static file, if not found proxy to app
location / {
try_files $uri #proxy_to_app; }
location /ws/ {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
include proxy_params;
proxy_pass http://channels-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_buffering off;
proxy_redirect off;
proxy_pass http://crm_server;
}
This is my front end thats generating the socket
<script type="text/javascript">
var loc = window.location
var wsStart = 'ws://'
if (loc.protocol == 'https'){
wsStart = 'wss://' } var endpoint = wsStart + loc.host + loc.pathname var socket = new ReconnectingWebSocket(endpoint)
This is my routing:
from channels.routing import ProtocolTypeRouter , URLRouter
from django.urls import path
from rnd.consumers import NotificationConsumer
from django.conf.urls import url
from channels.auth import AuthMiddlewareStack
from channels.security.websocket import AllowedHostsOriginValidator, OriginValidator
application = ProtocolTypeRouter({
"websocket": AllowedHostsOriginValidator(
AuthMiddlewareStack(
URLRouter(
[
path('',NotificationConsumer),
] ) )
) })
I would greatly appreciate any form of feed back!
When you proxy pass to open a Web-socket connection you need to proxy pass all of the web-socket http headers. https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism#WebSocket-specific_headers you will also need to pass the Origin header since you are using AllowedHostsOriginValidator.
--
also to debug these I suggest using a api client directly rather than the browser. Paw or Insomnia can be used to hit api to check that your Nginx config is routing to channels and not to Gunicorn.

Flask Restplus Swagger Not Loading Behind Nginx

I have a Flask API and a Swagger UI generated with Flask Restplus. The API runs in a Docker container behind an Nginx container which serves it over HTTP.
Here is a health check endpoint which confirms the API is running:https://mobydq.net/mobydq/api/v1/health
{"message":"MobyDQ API running in production mode"}
However, the Swagger which is supposed to load at the following URL does not load at all: https://mobydq.net/mobydq/api/doc
Here is the Nginx configuration:
http {
upstream api {
server api:5434;
}
upstream app {
server app:3000;
}
# Server for https
server {
listen 443 ssl http2;
server_name mobydq.net;
ssl_certificate /etc/letsencrypt/live/mobydq.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mobydq.net/privkey.pem;
# Location for MobyDQ Flask API
location /mobydq {
limit_req zone=default burst=20;
proxy_pass http://api;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Location for MobyDQ Web App
location / {
limit_req zone=default burst=20;
proxy_pass http://app;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# Default server to redirect http requests to https
server {
listen 80 default_server;
server_name mobydq.net;
listen [::]:80 default_server;
location ~ /.well-known {
root /var/www/letsencrypt;
}
location / {
return 301 https://$host$request_uri;
}
}
}
Any idea why the Swagger is not loading? I looked into the http requests sent when loading the page but it did not help much. I can only see the favicon loading:
I also looked at the console and saw an error but I'm not able to tell what it means:
The problem was that Nginx did not properly redirect the http requests when trying to get the resources from Swagger (the JSON configuration file in particular).
The issue has been fixed by changing the Nginx configuration as follow:
[...]
# Location for MobyDQ Flask API
location ~ ^/(mobydq|swaggerui) {
limit_req zone=default burst=20;
proxy_pass http://api;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
[...]

RoR 5.0.0 ActionCable wss WebSocket handshake: Unexpected response code: 301

Hello I'm trying to serve a simple chat using ror 5.0.0 beta (with puma)
working on production mode (in localhost there are no problems).
This is my Nginx configuration:
upstream websocket {
server 127.0.0.1:28080;
}
server {
listen 443;
server_name mydomain;
ssl_certificate ***/server.crt;
ssl_certificate_key ***/server.key;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers
HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/jenkins.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://localhost:3000;
proxy_read_timeout 90;
proxy_redirect http://localhost:3000 https://mydomain;
location /cable/{
proxy_pass http://websocket/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
break;
}
}
This is config/redis/cable.yml
production:
url: redis://localhost:6379/1
development:
url: redis://localhost:6379/2
test:
url: redis://localhost:6379/3
and config/environments/production.rb
# Action Cable endpoint configuration
config.action_cable.url = 'wss://mydomain/cable'
# config.action_cable.allowed_request_origins = [ 'http://example.com', /http:\/\/example.*/ ]
# Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
config.force_ssl = false
And this is the error i'm receiving:
application-[...].js:27 WebSocket connection to 'wss://mydomain/cable' failed: Error during WebSocket handshake: Unexpected response code: 301
Any tips? :) Thanks
I solved adding phusion passenger.
nginx config is now :
server{
listen 80;
passenger_enabled on;
passenger_app_env production;
passenger_ruby /../ruby-2.3.0/ruby;
root /path to application/public;
client_max_body_size 4G;
keepalive_timeout 10;
[...]
location /cable{
passenger_app_group_name websocket;
passenger_force_max_concurrent_requests_per_process 0;
}
}
You have to remove default folder config/redis/cable.yml and move that file to /config only.
For SSL just enable default ssl options and it will works .-)
Thanks everyone for the help
Your websocket URI is /cable/ and not /cable, so the latter will hit the location / block. Try:
location /cable {
rewrite ^/cable$ / break;
rewrite ^/cable(.*)$ $1 break;
proxy_pass http://websocket;
...
}
Also, not sure you need a break; in there. I presume the missing } between the two location blocks is just a typo in the question.
EDIT1: Added rewrite to restore correct upstream mapping.
EDIT2: Alternative solution is to explicitly rewrite /cable to /cable/ like this:
location = /cable { rewrite ^ /cable/ last; }
location /cable/ {
proxy_pass http://websocket/;
...
}
I spend almost 5 hours yesterday trying to solve this particular problem. I ended up using a separate domain for the websocket connection called ws.example.com as everything else resulted in a 301 redirect.
Here's is my nginx.conf file. I've removed the SSL parts, but you could just insert your own. Note that you need nginx 1.4+ as everything prior to this version doesn't support websocket proxying.
upstream socket {
server unix:/mysocket fail_timeout=0;
}
upstream websocket {
server 127.0.0.1:28080;
}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 443 ssl;
server_name ws.example.com;
access_log off;
# SSL configs here
location / {
proxy_pass http://websocket/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
server {
listen 443 ssl;
server_name example.com;
# SSL configs here
location #app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://socket;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
I read somewhere that allowed_request_origins didn't work as expected so I went the safe way (until the bug is fixed) and turned the checker of completely using ActionCable.server.config.disable_request_forgery_protection = true.
Here's my cable.ru file for starting action cable.
require ::File.expand_path('../../config/environment', __FILE__)
Rails.application.eager_load!
require 'action_cable/process/logging'
Rails.logger.level = 0
ActionCable.server.config.disable_request_forgery_protection = true
run ActionCable.server
I'm also using the latest rails version from Github.
gem "rails", github: "rails/rails"

nginx redirect all http to https with exceptions

I would like to redirect all http traffic to https with a handful of exceptions. Anything with /exception/ in the url I would like to keep on http.
Have tried the following suggested by Redirect all http to https in nginx, except one file
but it's not working. The /exception/ urls will be passed from nginx to apache for some php processing in a laravel framework but that shouldn't matter.
Any suggestions for improvement much appreciated!
server {
listen 127.0.0.1:80;
location / {
proxy_pass http://127.0.0.1:7080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Accel-Internal /internal-nginx-static-location;
access_log off;
}
location /exception/ {
# empty block do nothing
# I've also tried adding "break;" here
}
return 301 https://localhost$request_uri;
}
Nginx finds the longest matching location and processes it first, but your return at the end of the server block was being processed regardless. This will redirect everything but /exception/ which is passed upstream.
server {
listen 127.0.0.1:80;
access_log off;
location / {
return 301 https://localhost$request_uri;
}
location /exception/ {
proxy_pass http://127.0.0.1:7080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Accel-Internal /internal-nginx-static-location;
}
}

Nginx wildcard proxy, pass subdomain to the server (upstream proxy)

I would like to be able to pass subdomain.domain.com to .domain.com apache server, with subdomain info too.
I would like to make a nginx cache for domain, acting like wildcard, but passing subdomain to the destination (there is apache witch wildcard too). Up to now, I pass the info via proxy_set_header Host $host; but I would like to have request with subdomain at the apache server.
upstream domain.com {
server 172.1.1.1:80 weight=50 fail_timeout=30s;
}
server {
server_name *.domain.com;
location / {
proxy_pass http://domain.com;
#proxy_pass $request;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
}
location ~* ^.+. (jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js|swf)$ {
proxy_pass http://topmanagergame.com;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cache my-cache;
proxy_cache_valid 200 302 30m;
proxy_cache_valid 404 1m;
}
access_log /var/log/nginx/domain.com.log main;
error_log off;
}
Do you think I can use proxy_pass with upstream ?
Nginx (*wildcard_domain.com) --(cache)--> Apache (*wildcard_domain.com)
Nginx (anything.domain.com) --(cache)--> Apache (anything.domain.com)
upstream somestring {
server domain2.com:80 weight=50 fail_timeout=30s;
}
server {
listen 80;
server_name *.domain.com;
server_name ~^(?<subdomain>.+)\.domain\.com$;
location / {
proxy_pass http://somestring;
proxy_set_header Host $subdomain.domain2.com;
}
}
So I was trying to find the answer to this problem and kept finding this post. But I think dmytrivv answer is out of date. In our scenario, we have both wildcard domains (e.g. *.mydomain.com) and custom domains (e.g. fullycustomdomain.com). But you can solve both by using proxy_set_header Host $host; and having default at the end of your listen.
upstream qaweb {
# Servers in the web farm
server ip-notreal-name.ec2.internal:80;
}
server {
listen 443 ssl default;
ssl_certificate certs/mydomain.com.crt;
ssl_certificate_key certs/mydomain.com.key;
# Support for wildcard domains
server_name admin.mydomain.com *.mydomain.com "";
location / {
# Turn off access logging so we don't fill the hardrive
access_log off;
proxy_pass http://qaweb;
proxy_set_header Host $host;
# So that the correct IP shows up in the log once libapache2-mod-rpaf is installed
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Note, we are also using it as a TLS termination proxy.
You can also find more examples on how to use proxy_pass here https://www.liaohuqiu.net/posts/nginx-proxy-pass/

Resources