how not to ssl terminate at NginX load balacner - nginx

I want to know how to have NginX (load balancer) accept traffic on 443 and forward it to port 443 on the load balanced web server nodes.
I am using NginX as a load balancer where the SSL termination occurs at NginX level. And then NginX sends unencrypted traffic to my web serves at port 80.
This is my current ngnx configuration:
upstream appserver {
server 10.0.1.132;
server 10.0.1.243;
}
server {
listen 443 ssl;
server_name localhost;
ssl_certificate /etc/nginx/cert.pem;
ssl_certificate_key /etc/nginx/cert.key;
ssl_session_cache shared:SSL:1m;
ssl_session_timeout 5m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location / {
proxy_pass http://appserver;
}
}
I have gone through: Nginx load balance with upstream SSL
My real issue is, If I want NginX to listen traffic on 443, then I need to configure the ssl termination on nignx. Else nginx service won't start and will complain about missing ssl cert/keys.
In short, I want Nginx to simply accept traffic on 443 and forward it to 443 on load balanced Web server nodes. Then let my webservers do the SSL work.
The best practice is to do SSL offloading at load balancer level but I want to do otherwise.
Thanks.

This requires Layer 4 snooping/routing on NGINX's part, which is apparently not entirely supported. NGINX seems to support SNI, but for some reason I could not prevent it from terminating the TLS connection.
I ended up using HAProxy.

For this you need to use stream instead of http.

It is possible but this requires an extra module namely ngx_stream_ssl_preread_module it is available as of Nginx 1.11.5
It allows access to the SNI from which you can extract server name found in the client's ClientHello message via the $ssl_preread_server_name variable based on that you can route a TCP ("stream") connection to another enpoint. The downside is that you can only rely on the hostname for routing.
The documentation for the module provides an example of how to do that.

Related

Bind SSL certificate to a port number -- Nginx

Sorry for the limited understanding of Nginx and SSL. I have a React and Django app deployed on a server running on Nginx.
The React app is accessible using "example.org"(name is faked for demo purpose) and for the Django app, I have configured it to be accessible with port 3000 ie "example.org:3000".
The domain has SSL certificates installed and certificates are seen in "example.org" but while accessing "example.org:3000", the certificates are not available to this port.
I have been trying to allow ssl certificates to the port as well but couldnt succeed. I changed nginx conf file with listen 3000 ssl without success.
Please help, is there a way or should we need to modify the ssl certificates?
Nginx config at the moment is:
server {
listen 80 default_server;
server_name example.org;
return 301 https://example.org;
}
server {
listen 443 ssl;
server_name example.org;
ssl_certificate /etc/nginx/ssl/ssl_bundle.crt;
ssl_certificate_key /etc/nginx/ssl/example.key;
location / {
root /home/ubuntu/example/build;
index index.html index.htm;
}
}
The Port has nothing to do with the certs OR TLS Termination in general. IN case my assumptions are correct and your Django app is exposing its port 3000 by itself you need a proxy configuration that terminates the TLS for you.
server {
listen 8080 ssl;
server_name example.org;
ssl_certificate /etc/nginx/ssl/ssl_bundle.crt;
ssl_certificate_key /etc/nginx/ssl/example.key;
location / {
proxy_pass http://127.0.0.1:3000/;
proxy_set_header Host $host;
.....
}
}
This will terminate the TLS Session for you on Port 8080 and forwards the traffic to your Django app. There are other, more advanced options, proxying traffic to your appserver but this one will do it.
Note: In case you want to proxy the traffic through NGINX make sure Port 3000 is not exposed to the public anymore.

Nginx - Upstream Configuration Issue

I noticed something on an nginx config. There are 2 upstream blocks configured that are exactly the same:
upstream test1.example.com {
server flaskapp.example.com:5000
}
server {
listen 443 ssl;
proxy_pass test1.example.com;
ssl_certificate /opt/certs/example1.com.crt;
ssl_certificate_key /opt/example1.com.key;
ssl_protocols TLSv1.2;
ssl ciphers "ECDHE-ECDSA-AES128-GCM-SHA256"
}
upstream test2.example.com {
server flaskapp.example.com:5000
}
server {
listen 443 ssl;
proxy_pass test2.test.com;
ssl_certificate /opt/certs/test.com.crt;
ssl_certificate_key /opt/test.com.key;
ssl_protocols TLSv1.2;
ssl ciphers "ECDHE-ECDSA-AES128-GCM-SHA256"
}
I have 2 server blocks listening on port 443. So I have the same server listening for 2 separate connections on the same block... if that makes sense.
My thought was that this would fail because the same server listening for incoming https connections to test1 and test2.example.com wouldn't know 'where' to route the requests too. But that's not what's happening.
If I go to https://test1.example.com I am routed to the correct app. And https works as expected.
If I go to https://test2.example.com I am routed to the correct app. But https does not work as expected. This is confusing because both certs are wildcard certs. I am unsure why 1 succeeded and one failed.
If I comment out the first upstream block:
# upstream test1.example.com { server flaskapp.example.com:5000 }
# server {proxy_pass test1.example.com; }
Something stranger happens. Connecting to https://test2.test.com gives me a 'failed to connect to server' error message in my web browser. And the logs show this as the error:
No "ssl_certificate" is defined in server listening on SSL port while SSL handshaking
This is for test1.example.com, and I know the wildcard cert works. I'm using it elsewhere. So I'm unsure why I'm getting a 'failed to connect to server' error when I go to test1.example.com in this manner.
A few things to note:
Both test1.example.com and test2.test.com point to the same nginx server.
If both upstream/server blocks are working then test1.example.com shows the site is ssl secure. That is expected. But test2.test.com shows the website is insecure. This leads me to believe that only the first server/upstream block is working as expected. And the 2nd server/upstream block is being ignored.
actually does make sense, in that a server shouldn't be listening for incoming connections to the same port, and route to different servers. The proxy doesn't know what to do with 1 of the connections (bad explanation on my part).
But that doesn't explain why the 2nd server/upstream block would outright fail. Even when test2.example.com is the only server/upstream block configured.
Any advice is appreciate, thank you for your time and consideration. This is something I've been struggling to understand and make heads/tails of.
bossrhino
I think you need to use server_name directive. Because your web server listens on same ip and the same port for two subdomains.
I guess this config file should work properly:
upstream test1.example.com {
server flaskapp.example.com:5000
}
server {
listen 443 ssl;
server_name test1.example.com;
proxy_pass test1.example.com;
ssl_certificate /opt/certs/example1.com.crt;
ssl_certificate_key /opt/example1.com.key;
ssl_protocols TLSv1.2;
ssl ciphers "ECDHE-ECDSA-AES128-GCM-SHA256"
}
upstream test2.example.com {
server flaskapp.example.com:5000
}
server {
listen 443 ssl;
server_name test2.example.com;
proxy_pass test2.test.com;
ssl_certificate /opt/certs/test.com.crt;
ssl_certificate_key /opt/test.com.key;
ssl_protocols TLSv1.2;
ssl ciphers "ECDHE-ECDSA-AES128-GCM-SHA256"
}

SSL certificate returned for multiple server blocks

I have an nginx configuration with multiple virtual hosts and subdomains. Each subdomain needs to have a different SSL certificate bound. Here is the configuration for my first subdomain:
server {
listen 443;
server_name a.website.com;
ssl on;
ssl_certificate /etc/nginx/ssl/a/a.crt;
ssl_certificate_key /etc/nginx/ssl/a/a.rsa;
.....
The configuration for my second:
server {
listen 443;
listen 3443;
server_name b.website.com;
ssl on;
ssl_certificate /etc/nginx/ssl/b/b.crt;
ssl_certificate_key /etc/nginx/ssl/b/b.key;
....
The problem is if I go to b.website.com, the SSL certificate for both a.website.com and b.website.com are returned when I expect only b.website.com to be bound. I validated this using ssllabs.
Any advice?
I didn't notice that in ssllabs the second certificate was only returned if SNI wasn't enabled which makes sense because both certs are on the same IP. Apparently the integration we're working with doesn't support SNI (crazy I know) so I guess I have to spin up another server.

https works for local IP address but not for local IP with application port

I have Mattermost installed in my server, currently I can login to it by browsing through http://192.168.x.x:8066, I've installed a self-signed cerrtificate for this IP, but when I tried to browse it with https://192.168.x.x:8065, it failed to redirect to the Mattermost page.
Below is the configuration of my nginx.conf:
server {
listen 443 http2 ssl;
listen [::]:443 http2 ssl;
listen 443;
server_name 192.168.3.201:8066;
ssl on;
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
}
However, when I just browse the URL without port 8066 , it displays the default nginx page with no errors.
What's wrong with my nginx.conf file? I'm still new to nginx FYI.
Any suggestions will be very much appreciated.
I suggest you follow the example nginx configuration from the documentation here. Start with that config file, updating server_name to be the domain name you want mattermost to be reachable from, and server to be the IP address and port on which mattermost is listening.
Once you've got that working, you can continue through the instructions to #9 which covers setting up SSL.

Can I redirect non-SSL traffic that comes in on an SSL port with Nginx

I have a fairly standard setup with nginx fronting a django app. I want the django app to be SSL only, and so I have two listen blocks in my nginx conf, with the traffic coming in on port 80 (HTTP) being redirected to port 443 (SSL). This is working as expected.
I am running this setup inside a VM that has port forwarding turned on, such that I can browser the site from the host machine by going to port 8080 (HTTP) or 8081 (SSL). Again, this work fine, as expected.
The problem comes when I am redirected internally from the Django app during a registration workflow. Because Django never sees the SSL status (SSL is terminated at nginx, and traffic to the app is forwarded on port 5000 over HTTP), but does see the port, the redirect is getting mangled**.
The net result of all of this is that I have traffic being directed into nginx on the SSL port, that is not SSL, e.g. http://127.0.0.1:443/. Is there any way to configure nginx to handle this?
** NB I am setting the X-Forwarded-Proto header in Nginx, and Django is picking up the correct .is_secure() value, this is a specific problem with an external library not checking is_secure and just redirecting on the incoming URL scheme.
[UPDATE 1]
Attached are the relevant config settings. This is from the Vagrantfile itself, showing the port forwarding:
config.vm.forward_port 80, 8080 # website, via nginx (redirects to SSL:8081)
config.vm.forward_port 443, 8081 # website, via nginx (accepts SSL)
config.vm.forward_port 5000, 8180 # website, via gunicorn (direct)
Using the above port forwarding configuration, if I browse to the site on the host machine on the HTTP port (8080), then the request is accepted, and nginx (see below) redirects this request to HTTPS (running on port 8081). Once I am on HTTPS the site itself works fine:
(host) http://127.0.0.1:8080 -> forwarded to -> (guest vm) http://127.0.0.1:80
(host) https://127.0.0.1:8081 -> forwarded to -> (guest vm) https://127.0.0.1:443
The problem occurs when I get a redirect internally from Django which mixed scheme & protocol, and ends up with a request to http:\\127.0.0.1:8081\..., which fails, as nginx is expecting traffic on 8081 to be SSL.
What I really want is a rule that says 'listen on 443 for both SSL and non-SSL and redirect non-SSL'.
This is the relevant nginx configuration:
# Django app is served by Gunicorn, running under port 5000 (via Foreman)
upstream gunicorn {
server 127.0.0.1:5000 fail_timeout=0;
}
server {
listen 80;
# 8081 is the port I am forwarding to on the host machine
rewrite ^ https://127.0.0.1:8081$request_uri? permanent;
}
server {
listen 443;
ssl on;
ssl_protocols SSLv3 TLSv1;
ssl_ciphers HIGH:!ADH:!MD5;
ssl_prefer_server_ciphers on;
ssl_certificate /etc/nginx/ssl/self-signed.crt;
ssl_certificate_key /etc/nginx/ssl/self-signed.key;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location /static/ {
alias /app/static/;
}
location /media/ {
alias /app/media/;
}
location / {
# everything else is to be served by the django app (upstream 'gunicorn')
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# this is set to ensure that django is_secure returns True
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_pass http://gunicorn;
}
}
You should check out "Error Processing" section of this document:
http://nginx.org/en/docs/http/ngx_http_ssl_module.html
Non-standard error code 497 may be used to process plain HTTP request which has been sent to HTTPS port.
Something like this should work (untested):
error_page 497 https://$host$request_uri;
Named locations may also be used in error_page, see http://nginx.org/r/error_page for details.

Resources