I am working on AWS and using an EC2 instance for CI operations and more particularly for Jenkins. This server lies behind an nginx server which is used as a proxy for requests to ci.domain.com
Everything is working fine until a failure occurs on the CI server.
Once that happens, another instance boots up, upserts its IP to the DNS service pointing the new IP to ci.domain.com
The problem is that I need to restart the nginx service in order to discover the CI server again. Before that I get 5xx Bad Request or Timeout responses.
Below is part of my nginx configuration concerning the particular upstream server:
upstream jenkins {
server ci.domain.com max_fails=3 fail_timeout=100s;
}
server {
listen 80;
server_name ci.domain.com;
location / {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Connection "";
proxy_pass http://jenkins;
proxy_read_timeout 300;
proxy_send_timeout 300;
proxy_connect_timeout 300;
# Required for new HTTP-based CLI
proxy_http_version 1.1;
proxy_request_buffering off;
}
}
Related
I'm running an nginx reverse proxy to be able to run multiple servers behind my firewall. I noticed on my mail server the error log is filled with "failed login from < local ip of nginx >" and I was wondering how can I set it so I get the remote IP of the person/bot that is trying to login so I might use that information for auto blocking those addresses (for example)?
This is my current config:
server {
listen 8443 ssl http2;
server_name mail.domain.com;
location / {
proxy_set_header Host $host;
proxy_pass https://<internal ip>/;
client_max_body_size 0;
proxy_connect_timeout 3600;
proxy_send_timeout 3600;
proxy_read_timeout 3600;
send_timeout 3600;
}
}
I think you're looking for one of these
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
you can add these to http, server or location block and read the header in your app to filter the request
Just found out my mail server (Kerio) does nothing with the information forwarded by the reverse proxy, so the only thing I can do is hope for an update that does.
We are using NGINX on our cPanel server to reverse proxy ZKTeco ZKBioSecurity servers. Due to compatibility with some of their devices not supporting HTTPS, all our servers use HTTP, but, of course, all sessions to our NGINX server is secured with HTTPS and a Sectigo certificate provided by cPanel’s AutoSSL.
Here’s the problem: it seems that the ZKBioSecurity servers are detecting that the client is using HTTPS to connect to them through NGINX, and because of this, give the following prompt each time you want to log in, advising you to download and install the ISSOnline driver and certificate. The certificate, however, is issued to the ZKBioSecurity server for 127.0.0.1, so of course this is rather pointless as we are connecting to the NGINX server using a FQDN. This does not happen if we use HTTP:
So my question: is there something in the request (the HTTP header perhaps?) that NGINX forwards to the upstream server that contains the protocol (HTTPS) the client used to connect to the server? Because this somehow seems to be the case.
Here’s our NGINX config for ZKBioSecurity servers:
location /.well-known {
root /home/novacloud/public_html/subdomain/.well-known;
allow all;
try_files $uri = 404;
}
location / {
if ($scheme = http) {
return 301 https://$host$request_uri;
}
proxy_pass http://192.168.0.1:8080;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
}
The server_name directive is, of course, managed by cPanel. The above is an example of the include files we use in the main cPanel NGINX configuration file. I thought it was the proxy_set_header X-Forwarded-Proto $scheme, but even if I remove this, I still get the Driver Detection Exception prompt.
Here’s a Pastebin of a cURL of the ZKBioSecurity server from our cPanel/NGINX server
I am trying to route requests such that those requiring websockets will route to a long-lived nginx process, and all others will go to the general reverse-proxy which handles all other traffic. These nginx processes exist in our AWS cloud behind an ELB that has been configured to use Proxy Protocol. Note that all of this works correctly with our current setup which uses only one nginx process that is configured to use proxy_protocol.
The change to this setup is as follows:
The first nginx server handling all ingress uses proxy_protocol and forwards requests to either the websocket or non-websocket nginx servers locally:
server {
listen 8080 proxy_protocol;
real_ip_header proxy_protocol;
charset utf-8;
client_max_body_size 20M;
#send to websocket process
location /client {
proxy_pass http://localhost:8084;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $proxy_protocol_addr;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Proxy-Scheme $scheme;
proxy_set_header X-Proxy-Port $proxy_port;
proxy_set_header X-ELB-Proxy-Scheme "https";
proxy_set_header X-ELB-Proxy-Port "443";
# Always support web socket connection upgrades
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
#send to non-websocket process
location / {
proxy_pass http://localhost:8082;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $proxy_protocol_addr;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Proxy-Scheme $scheme;
proxy_set_header X-Proxy-Port $proxy_port;
proxy_set_header X-ELB-Proxy-Scheme "https";
proxy_set_header X-ELB-Proxy-Port "443";
# Always support web socket connection upgrades
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
When any non-websocket request is sent to localhost:8082, I get an empty reply. If I remove proxy_protocol from the first server, I get a response as expected. Obviously, I need proxy_protocol to support the ingress from our ELB, so removing it is not an option. However, I would like to know what pieces I am missing to route traffic correctly -- and I would also like to know why proxying a request locally from a proxy_protocol enabled server to another nginx process (regardless of this second process using proxy_protocol or not) fails.
For reference, the basic configuration of this secondary nginx process is below:
upstream console {
server localhost:3000 max_fails=3 fail_timeout=60 weight=1;
}
server {
listen 8082;
client_max_body_size 20M;
location /console {
proxy_pass http://console
}
.
.
.
}
Turns out the non-websocket proxy block should not set the various proxy and upgrade headers:
location / {
proxy_pass http://localhost:8082;
proxy_set_header Host $host;
}
I have a scenario that i'm trying to configure in nginx where I have a number of processes each listening from ports 8000,8001... Upon establishing a http connection to one of these ports I then get the client (within javascript) to establish a WebSocket connection. All the listening processes have the same /SS websocket endpoint. However if a http connection initially makes a connection to 8000 it needs to also establish the websocket connection to 8000 too. I have the following nginx configuration:
upstream backends {
server 127.0.0.1:8000;
server 127.0.0.1:8001;
server 127.0.0.1:8002;
}
include /etc/nginx/conf.d/*.conf;
server {
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://backends;
}
location /SS {
proxy_set_header Host $host;
proxy_pass http://backends;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
However this doesn't route the the websocket to the same place as the initial connection.
I thought of a way to do this where I setup different endpoints for each process and pass this through the initial http request. I would get the client to then use this endpoint for the WebSocket connection. This would however need me to configure all the different endpoints in nginx. I was wondering if there's a better way to solve this just within the nginx configuration?
I'm working on a Google Cloud Compute Engine instance. Ubuntu 12.04.
I have a Tornado app installed on the server working on port 8888 and I have nginx configuration as shown below:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream chat_servers {
server 127.0.0.1:8888;
}
server {
listen 80;
server_name chat.myapp.com;
access_log /home/ubuntu/logs/nginx_access.log;
error_log /home/ubuntu/logs/nginx_error.log;
location /talk/ {
proxy_set_header X-Real-IP $remote_addr; # http://wiki.nginx.org/HttpProxyModule
proxy_set_header Host $host; # pass the host header - http://wiki.nginx.org/HttpProxyModule#proxy_pass
proxy_http_version 1.1; # recommended with keepalive connections - http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_http_version
# WebSocket proxying - from http://nginx.org/en/docs/http/websocket.html
proxy_connect_timeout 7d;
proxy_send_timeout 7d;
proxy_read_timeout 7d;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_pass http://chat_servers;
}
}
When I try to connect to ws://chat.myapp.com/talk/etc/ via Javascript, Tornado app's open() method on WebSocketHandler gets called and I print the log on the server successfully, but on the client side, the code never enters the onopen() and after some time I get 1006 error code,WebSocket opening handshake timed out`.
This app was working fine on Amazon (AWS) EC2 server with the same configuration but after I moved to Google Cloud, somehow the handshake cannot be done.
Is there any configuration specific to Google Cloud? Or any nginx update on the file?
I am confused and I spent two days on this but couldn't solve the problem.
Default nginx's version on Ubuntu was nginx/1.1.19. I updated it to nginx/1.8.0. The problem is solved.