"Not Found" problem with QuestDB behind NGINX - nginx

I am running a QuestDB 6.6.1-server. Now I want to increase the protection of this server and put the web gui behind an NGINX reverse proxy as described in QuestDB blog post where setting up basic authentication is shown.
When I try to open the QuestDB web gui, the login popup is displayed, I can enter name and password without issues. However, after having successfully passed the login popup, I only see a bare text "Not Found" in the browser (Note: but NOT the NGINX 403 Not Found screen, which I now in other cases). Neither nginx.log, nor questdb.log show entries.
It is POSSIBLE to reach the QuestDB web gui via <server.domain>:9000, no issues there.
The "location" settings are defined in a file reverse_proxy.conf:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name server.domain;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
ssl_certificate <path>/nginx.crt;
ssl_certificate_key <path>/nginx.key;
root /var/www/server.domain/html;
index index.html index.htm;
server_name server.domain;
location /location1 {
proxy_pass https://localhost:port1;
proxy_set_header Host $host;
}
location /location2 {
proxy_pass http://localhost:port2/location2;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10M;
}
location = /questdb/ {
auth_basic "Restricted QuestDB";
auth_basic_user_file <path>/.htpasswd;
proxy_pass http://localhost:9000;
proxy_set_header Host $host;
proxy_read_timeout 300;
proxy_connect_timeout 120;
proxy_send_timeout 300;
proxy_set_header Host $host;
}
}
reverse_proxy.conf is imported into nginx.conf. nginx.conf looks like this:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# For suppression of server version number
server_tokens off;
server_names_hash_bucket_size 64;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
##
# Virtual Host Configs
##
map_hash_max_size 262144;
map_hash_bucket_size 262144;
include /etc/nginx/conf.d/*.conf;
}

I see you are using questdb on a directory. You are proxying to http://localhost:9000/questdb, and QuestDB is saying "Not found"
To avoid that, you would need to add a slash at the end of proxy_pass http://localhost:9000; as in proxy_pass http://localhost:9000/;
Problem then is that relative URLs (/assets, /exec...) will not work and you will need to rewrite them on nginx.
It would be probably easier to just use a subdomain rather than a directory.
update: this is the nginx config I use. As explained, relative links are broken
location /questdb/ {
proxy_pass http://localhost:9000/;
index index.html index.htm;
auth_basic "Restricted Content";
auth_basic_user_file /opt/homebrew/etc/nginx.htpasswd;
}

Related

I can't get my asp.net mvc to use ssl when being hosted on a digitalocean droplet

I can't get my dotnet mvc app to be hosted correctly over ssl (https). It only works over http. The following is my relevant nginx files (with "example.org" used instead of my domain)
/etc/nginx/sites-enabled/default
# Default server configuration
#
server {
server_name example.org *.example.org;
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# SSL configuration
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
listen 80 default_server;
# listen [::]:80 default_server deferred;
return 444;
}
server {
if ($host = example.org) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name example.org *.example.org;
return 404; # managed by Certbot
}
/etc/nginx/nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
/etc/nginx/proxy.conf
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffers 32 4k;
This makes my site work over "http://example.org" but not over "https://example.org". I don't know why it won't work over https? I tried altering my /etc/nginx/nginx.conf file to make it like the recommended documentation for asp.net hosting via Microsoft. Here's my new /etc/nginx/nginx.conf file.
/etc/nginx/nginx.conf
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
include /etc/nginx/proxy.conf;
limit_req_zone $binary_remote_addr zone=one:10m rate=5r/s;
server_tokens off;
sendfile on;
# Adjust keepalive_timeout to the lowest possible value that makes sense
# for your use case.
keepalive_timeout 29;
client_body_timeout 10; client_header_timeout 10; send_timeout 10;
upstream my-app{
server 127.0.0.1:5000;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name example.org *.example.org;
ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
ssl_session_timeout 1d;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;
ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_tickets off;
ssl_stapling off;
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
#Redirects all traffic
location / {
proxy_pass http://my-app;
limit_req zone=one burst=10 nodelay;
}
}
}
When I change my /etc/nginx/nginx.conf file to the above, both "http://example.org" and "https://example.com" fail. So how do I get this app to work over https?
The problem was actually my ufw firewall. When I was setting up the droplet I did the commands:
sudo ufw enable
sudo ufw allow OpenSSH
sudo ufw default deny incoming
sudo ufw allow 'Nginx HTTP'
The problem above is that was supposed to do sudo ufw allow 'Nginx Full' then sudo reboot. After this, my original nginx configuration worked!
Try the config below (don't forget to disable https redirection in your app - remove app.UseHttpsRedirection(); from your Program.cs (Startup.cs) and remove applicationUrl https reference from launchSettings.json "https://localhost:5001"):
worker_processes 1;
events { worker_connections 1024; }
http {
sendfile on;
access_log /var/log/nginx/access.log combined;
error_log /var/log/nginx/error.log;
upstream web-api {
server 127.0.0.1:5000;
}
server {
listen 80;
server_name yourdomain.com;
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 443 ssl;
server_name yourdomain.com;
ssl_certificate /etc/ssl/certs/yourdomain.com.crt;
ssl_certificate_key /etc/ssl/private/yourdomain.com.key;
location / {
proxy_pass http://web-api;
proxy_redirect off;
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
}
}
}

How to find the page nginx is serving

I have inherited an nginx instance. I am barely more than a newbie when it comes to nginx.
When I navigate to the IP address of the box "http://192.168.1.10" I get a custom page back created by the previous dev somehow.
The nginx.conf looks like this:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
The conf.d directory is empty.
The sites-enabled directory contains a link default which points to default in sites-enabled. Here is that file:
server {
location / {
proxy_pass http://localhost:3002;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
listen 80;
listen [::]:80;
}
I have looked in /var/www/html and found the default index.nginx-debian.html that is not the one being served.
So, my question is, where is the file I'm getting back being served from? I'm running on a ubuntu server distro.

Nginx is not able to handle a large number of requests

I have an scenario on where a nginx is in front of an Artifactory server.
Recently, while trying to pull a big number of docker images in a for loop, all at the same time (first test was with 200 images, second test with 120 images), access to Artifactory gets blocked, as nginx is busy processing all the requests and users will not be able to reach it.
My nginx server is running with 4 cpu cores and 8192 of ram.
I have tried to improve the handling of files in the server, by adding the bellow:
sendfile on;
sendfile_max_chunk 512k;
tcp_nopush on;
This made it a bit better (but of course, pull's of images with 1gb+ take much more time, due to the chunk size) - still, access to the UI would cause a lot of timeouts.
Is there something else that i can do to improve the nginx performance, whenever a bigger load is pushed thru it?
I think that my last option is to increase the size of the machine (more cpu's) aswell as the number of processes on nginx (8 to 16).
The full nginx.conf file follows bellow:
user www-data;
worker_processes 8;
pid /var/run/nginx.pid;
events {
worker_connections 19000;
}
http {
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
include /etc/nginx/mime.types;
default_type application/octet-stream;
gzip on;
gzip_disable "msie6";
sendfile on;
sendfile_max_chunk 512k;
tcp_nopush on;
set_real_ip_from 138.190.190.168;
real_ip_header X-Forwarded-For;
log_format custome '$remote_addr - $realip_remote_addr - $remote_user [$time_local] $request_time'
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"';
server {
listen 80 default;
listen [::]:80 default;
server_name _;
return 301 https://$server_name$request_uri;
}
###########################################################
## this configuration was generated by JFrog Artifactory ##
###########################################################
## add ssl entries when https has been set in config
ssl_certificate /etc/ssl/certs/{{ hostname }}.cer;
ssl_certificate_key /etc/ssl/private/{{ hostname }}.key;
ssl_session_cache shared:SSL:1m;
ssl_prefer_server_ciphers on;
## server configuration
server {
listen 443 ssl;
server_name ~(?<repo>.+)\.{{ hostname }} {{ hostname }} _;
if ($http_x_forwarded_proto = '') {
set $http_x_forwarded_proto $scheme;
}
## Application specific logs
access_log /var/log/nginx/{{ hostname }}-access.log custome;
error_log /var/log/nginx/{{ hostname }}-error.log warn;
rewrite ^/$ /webapp/ redirect;
rewrite ^//?(/webapp)?$ /webapp/ redirect;
rewrite ^/(v1|v2)/(.*) /api/docker/$repo/$1/$2;
chunked_transfer_encoding on;
client_max_body_size 0;
location / {
proxy_read_timeout 900;
proxy_max_temp_file_size 10240m;
proxy_pass_header Server;
proxy_cookie_path ~*^/.* /;
proxy_pass http://{{ appserver }}:8081/artifactory/;
proxy_set_header X-Artifactory-Override-Base-Url $http_x_forwarded_proto://$host:$server_port;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
Thanks for the tips.
Cheers,
Ricardo

Tornado behind Nginx for websockets - timeout

We are running our server in AWS EC2 M4.xlarge instance and we are seeing unexplained behavior with the websocket connections.
Problem: Our clients (running python websocket-client to connect) that are connected to the tornado websocket server behind nginx, are ALL being dropped at the same time and for the same reason of ping/pong timedout after some time (between 3-6hrs) of being connected. Not sure if we are setting some configuration incorrectly.
Here is the configuration for our sysctl.conf:
net.core.somaxconn = 65536
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_fin_timeout = 15
/etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 65000;
multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 15;
keepalive_requests 100000;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
This is our configuration for the application server.
upstream ourserver {
server 127.0.0.1:9999;
}
server {
listen 80;
listen [::]:80 ipv6only=on;
return 301 https://$host$request_uri;
}
server {
listen 443 default_server;
listen [::]:443 default_server ipv6only=on;
server_name **DNS**;
ssl on;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_certificate /etc/ssl/certs/ssl-bundle.crt;
ssl_certificate_key /etc/ssl/private/my.key;
client_max_body_size 10m;
client_body_buffer_size 128k;
client_header_buffer_size 1k;
keepalive_timeout 15s;
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains;";
location /secure/ {
proxy_pass http://ourserver;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://ourserver;
proxy_read_timeout 120s;
proxy_connect_timeout 120s;
}
}
Any help is greatly appreciated.
My on_message had a blocking call which takes between 20ms to 200ms due to DB call. This delay propagates to the remaining events coming in, which eventually led to sockets being dropped by clients.

Nginx https reverse proxy another 502

Hi I am trying to setup a nginx to work as a reverse proxy to an application that I am running on a tomcat server. when I try to access my application through http it works fine, but when I try to access it over https I am getting a 502 error
here follows my nginx config file
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log notice;
gzip on;
gzip_disable "msie6";
rewrite_log on;
server{
ssl on;
listen 80;
listen 443 ssl;
server_name myapp.local;
ssl_certificate max.local.crt;
ssl_certificate_key server.key;
#ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
#ssl_ciphers RC3:HIGH:!aNULL:!MD5;
#ssl_prefer_server_ciphers on;
ssl_session_timeout 5m;
keepalive_timeout 60;
error_log /var/log/nginx/hybris.log;
rewrite_log on;
set $my_port 9001;
set $my_protocol "http";
if ($scheme = https){
set $myport 9002;
set $my_protocol "https";
}
location / {
if ( $http_user_agent ~ "Chrome"){
#just a proof of concept
return 301 http://$host/AE/en;
}
if ( $http_user_agent ~ "Firefox"){
#just a proof of concept
return 301 http://google.com/;
}
}
location /AE/en {
proxy_pass $scheme://10.0.2.2:$my_port;
proxy_set_header Host $host;
}
location ~(?:/..)?/_ui/(.*) {
proxy_pass http://10.0.2.2:9001/_ui/$1;
proxy_set_header Host $host;
}
}
}
When using https you are changing the port and also scheme for connecting to the tomcat server - this does not really make sense. You would only use https for a backend server if it is in another datacenter, not within a local network. It should work fine if you remove the $my_port and $my_protocol definitions and change your /AE/en location block to
location /AE/en {
proxy_pass http://10.0.2.2:9001;
proxy_set_header Host $host;
}
I think you need to create two server sections. One for listening on port 80 and the other for listening on port 453 which is for https.

Resources