Duplicated log on single request with a minus sign at the end - nginx

I have a setup where one nginx does reverse proxy to a second one.
I observed in my logs, that one nginx sees two request but the second one sees just one of them. I also observed that the browser is just sending one request, so the duplication if exists, does not come from the client.
Here is a sample of the logs:
nginx_1 | 192.168.64.8 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.0" 200 9402 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.1" 200 111773 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" "-"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.1" 200 111773 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.1" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" "-"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.1" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
nginx_1 | 192.168.64.8 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.0" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
As you can see, proxy_1 is the nginx working as reverse proxy, and nginx_1 is the one receiving and processing the requests.
In the end of every duplicated line you can see a "-" that I don't know what it means. I suspect is doing some sort of rewriting I'm not aware of but don't understand how it happens.
Could you give me some clue about what is going on here?
Here I also provide the server config:
access_log /dev/stdout;
error_log /dev/stdout;
server {
listen 7000;
listen 80;
client_max_body_size 24M;
server_name *.rareconnect.org;
include /etc/nginx/params/gzip_on;
location / {
include /etc/nginx/params/proxy_pass_local_rareconnect_org_8000;
}
}
This is on top of the standard configuration of nginx:1.9.5 as appears in the official docker registry.
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Thanks in advance.

Looks like there are few records access_log. Can you check nginx.conf as well?

Related

How can I access to payara 5 administration on port 4848 on a fresh installation where the FQDN is used by Payara and a mail server?

How can I access to payara administration on port 4848 on a fresh installation where the FQDN is used by Payara and a mail server?
I am trying to set a full new platform with Payara, I can't manage to access to the administration console on port 4848.
Environment:
• OS: Debian GNU/Linux 11 (bullseye)
• Java: openjdk version "11.0.16" 2022-07-19
• Payara: Payara Server 5.2022.2 #badassfish (build 306)
• Ngnix: nginx/1.18.0
On a fresh Debian installation I first set ufw to be able to open necessary port. Then I started to install IredMail (1.6.0 MARIADB edition.). Once mail serveur was working I installed openjdk 11, then Payara. In Payara I created a domain with adminport set to 4848 and instance port set to 8888. change-admin-password and enable-secure-admin has been run for this domain.
Taking in count that my FQDN is my.domain.com, I managed to have the Payara welcome page on https: // my.domain.com, IredMail administration on https: // my.domain.com/ireadmin and IredWebMail on https: // my.domain.com/mail.
While trying to access the administation console https: // my.domain.com:4848 goes on error:
This site can’t be reached - ERR_CONNECTION_TIMED_OUT
After long search for a solution on the net, I created a dedicated url /gfadmin, see configuration below, where the page seems to be reached but I got a white page displayed. The console log shows:
gfadmin:18 GET .... /theme/com/sun/webui/jsf/suntheme/css/safari.css net::ERR_ABORTED 404
gfadmin:28 GET .... /theme/META-INF/prototype/prototype.js net::ERR_ABORTED 404
gfadmin:27 GET .... /theme/META-INF/json/json.js net::ERR_ABORTED 404
gfadmin:29 GET .... /theme/META-INF/com_sun_faces_ajax.js net::ERR_ABORTED 404
gfadmin:26 GET .... /theme/META-INF/dojo/dojo.js net::ERR_ABORTED 404
gfadmin:17 GET .... /theme/com/sun/webui/jsf/suntheme/css/css_master.css net::ERR_ABORTED 404
gfadmin:31 Uncaught ReferenceError: dojo is not defined
at gfadmin:31:1
(anonymous) # gfadmin:31
gfadmin:34 GET .... /resource/css/css_ns6up.css net::ERR_ABORTED 404
gfadmin:46 GET .... /resource/community-theme/images/login-product_name_open.png 404
gfadmin:89 GET .... /resource/js/cj.js net::ERR_ABORTED 404
as well as the nginx log shows:
0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:26 +0000] "GET / HTTP/2.0" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:29 +0000] "GET /gfadmin HTTP/2.0" 200 1705 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/com/sun/webui/jsf/suntheme/css/safari.css HTTP/2.0" 404 548 ".... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/META-INF/prototype/prototype.js HTTP/2.0" 404 548 ".... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/META-INF/json/json.js HTTP/2.0" 404 548 ".... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/META-INF/com_sun_faces_ajax.js HTTP/2.0" 404 548 "... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/META-INF/dojo/dojo.js HTTP/2.0" 404 548 ".... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /theme/com/sun/webui/jsf/suntheme/css/css_master.css HTTP/2.0" 404 548 ".... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:05:59 +0000] "GET /resource/css/css_ns6up.css HTTP/2.0" 404 548 "... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:06:30 +0000] "GET /resource/community-theme/images/login-product_name_open.png HTTP/2.0" 404 548 "... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
94.43.88.148 - - [12/Aug/2022:10:06:30 +0000] "GET /resource/js/cj.js HTTP/2.0" 404 548 "... my.server.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"
Thinking that the issue could come from the added url /gfadmin, I set, in location {...} root to /opt/payara5/ with not result.
From /etc/ngnix/sites-vailable I remove 00-default-ssl.conf and created a new file my.server.com.conf with he following content:
upstream glassfish {
server 127.0.0.1:8888;
}
upstream gfadmin {
server 127.0.0.1:4848;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name my.server.com;
gzip on;
gzip_types text/css text/javascript text/plain application/xml;
gzip_min_length 1000;
location ^~ /.well-known/acme-challenge/ {
allow all;
root /var/www/my.server.com/;
default_type "text/plain";
try_files $uri =404;
}
location / {
proxy_pass http: // localhost:8888;
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;
}
location ~* .(png|ico|gif|jpg|jpeg|css|js)$ {
#proxy_pass https: // localhost:8888/$request_uri;
proxy_pass https: // localhost:8888;
}
location /gfadmin {
root /opt/payara5/;
charset utf-8;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_max_temp_file_size 0;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_pass https://127.0.0.1:4848;
proxy_connect_timeout 300;
send_timeout 300;
}
location /mail {
root /var/www/html;
index index.php index.html;
}
location /iredadmin {
root /var/www/html;
index index.php index.html;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
#root /usr/share/nginx/html;
root /usr/share/nginx/base;
}
#listen 80;
#listen 4848;
ssl_certificate /etc/letsencrypt/live/my.server.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/my.server.com/privkey.pem; # managed by Certbot
include /etc/nginx/templates/misc.tmpl;
include /etc/nginx/templates/ssl.tmpl;
include /etc/nginx/templates/iredadmin.tmpl;
include /etc/nginx/templates/roundcube.tmpl;
include /etc/nginx/templates/sogo.tmpl;
include /etc/nginx/templates/netdata.tmpl;
include /etc/nginx/templates/php-catchall.tmpl;
include /etc/nginx/templates/stub_status.tmpl;
}
I would appreciate any help that will allow me to fix this issue.
Thank you
Unfortunately the Payara web administration console uses absolute paths which leads to the problem that after loading the HTML file the browser tries to load the CSS and JS files from root directory (instead of from "/gfadmin").
There is no direct workaround for that as stated out in this answer.
However there is the possibility to use a subdomain especially for the Payara web administration console which redirects any request to the root directory to the Payara server at port 4848. Just add the following lines before your other server configuration:
# subdomain redirecting to Payara admin console
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl_certificate _path_to_certificate_;
ssl_certificate_key _path_to_certificate_key_;
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
server_name _subdomain_;
# Redirect Payara admin console
location / {
proxy_pass https://127.0.0.1:4848/;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_set_header Connection "";
include /etc/nginx/proxy_params;
}
}
# main server configuraion
server {
...

Nginx reverse proxy unable to load Harbor page

I am trying to place harbor behind another nginx reverse proxy
nginx reverse proxy (server1) --> harbor (in another server2 using docker-compose)
unable to reach harbor server
configuration as below
server {
listen 443 ssl;
include snippets/self-signed.conf;
include snippets/ssl-params.conf;
location /harbor {
proxy_pass https://172.21.205.245/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_ssl_verify off;
}
}
snippets/self-signed.conf
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
snippets/ssl-params.conf
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/nginx/dhparam.pem;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;
ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0
ssl_session_timeout 10m;
#ssl_session_cache shared:SSL:10m;
ssl_session_tickets off; # Requires nginx >= 1.5.9
ssl_stapling on; # Requires nginx >= 1.3.7
ssl_stapling_verify on; # Requires nginx => 1.3.7
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable strict transport security for now. You can uncomment the following
# line if you understand the implications.
# add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
nginx logs as below harbor page stuck in loading
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /harbor/ HTTP/1.1" 200 477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /light-theme.css?buildTimestamp=1635159453616 HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /dark-theme.css?buildTimestamp=1635159453616 HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /runtime.723165ddb219c3510b4c.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /scripts.fc1928a0f22676249790.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /styles.e71e5822ddf4adf262c4.css HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /polyfills.a5e9bc0ea6dbbbdc0878.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /main.cae0d8ae343d0dc7e111.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
It is actually working, one can see in the logs and on the webpage that data is fetched. If you hit F12 on your keyboard to see what the application is trying to fetch, you will notice some 404 errors.
The problem in your case is that your content is served from /harbor/*.js|css but the angular application is trying to fetch from /*.js|css
if you open the webconsole (F12) you'll see it yorself.

Nginx: Add $host to log

How can I (or why can't I) add the $host variable to my nginx configuration file to see what domain (or subdomain) a client is requesting?
Current Format:
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$host" "$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
Current Output:
10.7.2.104 - - [17/Feb/2017:14:31:15 -0600] "GET /crazystairs HTTP/1.1" 404 2327 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393"
Desired Ouput:
10.7.2.104 - - [17/Feb/2017:14:31:15 -0600] "GET example.com/crazystairs HTTP/1.1" 404 2327 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393"
Notice the host in front of the request
I found the problem. You must declare main after setting the directory where the log will be saved for the main format to be used.
access_log logs/crazystairs/portal/access.log main;

http proxy behind nginx redirecting to wrong url

I have nginx listening on a server to serve a website www.example.com
I'm trying to redirect www.example.com/iPython to an http-server which serves a temporary iPython notebook and is sitting on the server at port 8000 https://github.com/jupyterhub/configurable-http-proxy
server {
listen 80;
server_name http://xxx.xxx.xxx.xxx;
charset utf-8;
location /iPython {
proxy_pass http://xxx.xxx.xxx.xxx:8000/;
}
}
Here is the log:
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:09 +0000] "GET /iPythontest HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
web_1 | Not Found: /spawn/iPythontest
nginx_1 | xxx.xxx.xxx.xxx - - [19/Sep/2016:16:14:10 +0000] "GET /spawn/iPythontest HTTP/1.1" 404 9298 "http://xxx.xxx.xxx.xxx/iPythontest" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36" "-"
The server then gets redirected to http://www.example.com/spawn/iPython rather than http://www.example.com/iPython/spawn which returns a 404 error on my site. I would like to make sure all traffic gets redirected in this way, but I'm not sure which directive can accomplish that

Nginx stops responding after few seconds

Debian 7.8, nginx 1.8.0
Reboot my server, call a page, got the page, access logs are correct.
5.49.32.xxx - - [06/Aug/2015:14:22:30 +0200] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36"
5.49.32.xxx - - [06/Aug/2015:14:22:31 +0200] "GET /favicon.ico HTTP/1.1" 200 26 "http://f1.mydomain.fr/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.130 Safari/537.36"
And after about 1 min of server up, nginx stops responding.
Nothing in access_log, nothing in error_log.
service nginx restart doesn't solve the problem. If I want to get a response, the only way is to reboot my server.
I checked my syslog and kern.log, I saw nothing interesting.
No idea where the problem could be...
Here is my conf:
server {
listen 80;
server_name
f1.mydomain.fr
;
root /var/www/mydomain/current/web;
access_log /var/log/nginx/mydomain-access.log;
error_log /var/log/nginx/mydomain-error.log error;
location / {
# For Symfony2
try_files $uri /app.php$is_args$args;
}
location ~ ^/(app|app_dev|config|app_test|clear|ocp|apcu)\.php(/|$) {
include /etc/nginx/fastcgi_params;
include /etc/nginx/conf/fastcgi;
}
location ~ /\.ht {
deny all;
}
}

Resources