How can I (or why can't I) add the $host variable to my nginx configuration file to see what domain (or subdomain) a client is requesting?
Current Format:
log_format main '$remote_addr - $remote_user [$time_local] $status '
'"$host" "$request" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
Current Output:
10.7.2.104 - - [17/Feb/2017:14:31:15 -0600] "GET /crazystairs HTTP/1.1" 404 2327 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393"
Desired Ouput:
10.7.2.104 - - [17/Feb/2017:14:31:15 -0600] "GET example.com/crazystairs HTTP/1.1" 404 2327 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393"
Notice the host in front of the request
I found the problem. You must declare main after setting the directory where the log will be saved for the main format to be used.
access_log logs/crazystairs/portal/access.log main;
Related
I am trying to place harbor behind another nginx reverse proxy
nginx reverse proxy (server1) --> harbor (in another server2 using docker-compose)
unable to reach harbor server
configuration as below
server {
listen 443 ssl;
include snippets/self-signed.conf;
include snippets/ssl-params.conf;
location /harbor {
proxy_pass https://172.21.205.245/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_ssl_verify off;
}
}
snippets/self-signed.conf
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
snippets/ssl-params.conf
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/nginx/dhparam.pem;
ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384;
ssl_ecdh_curve secp384r1; # Requires nginx >= 1.1.0
ssl_session_timeout 10m;
#ssl_session_cache shared:SSL:10m;
ssl_session_tickets off; # Requires nginx >= 1.5.9
ssl_stapling on; # Requires nginx >= 1.3.7
ssl_stapling_verify on; # Requires nginx => 1.3.7
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
# Disable strict transport security for now. You can uncomment the following
# line if you understand the implications.
# add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
nginx logs as below harbor page stuck in loading
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /harbor/ HTTP/1.1" 200 477 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /light-theme.css?buildTimestamp=1635159453616 HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /dark-theme.css?buildTimestamp=1635159453616 HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /runtime.723165ddb219c3510b4c.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /scripts.fc1928a0f22676249790.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /styles.e71e5822ddf4adf262c4.css HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /polyfills.a5e9bc0ea6dbbbdc0878.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
172.21.192.1 - - [01/Dec/2021:22:00:52 +0530] "GET /main.cae0d8ae343d0dc7e111.js HTTP/1.1" 404 197 "https://172.21.196.91/harbor/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36 Edg/96.0.1054.34"
It is actually working, one can see in the logs and on the webpage that data is fetched. If you hit F12 on your keyboard to see what the application is trying to fetch, you will notice some 404 errors.
The problem in your case is that your content is served from /harbor/*.js|css but the angular application is trying to fetch from /*.js|css
if you open the webconsole (F12) you'll see it yorself.
I've used Nginx docs and set access log format.
https://docs.nginx.com/nginx/admin-guide/monitoring/logging/
access_log /var/log/nginx/access.log;
log_format compression '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" "$gzip_ratio"';
When I access the site using Firefox I get the following log:
192.168.xx.xxx - - [13/Apr/2021:14:52:16 +0200] "GET /home HTTP/1.1" 200 21408 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0"
But when I access the site using Chrome or Opera I get following logs:
Opera:
192.168.xx.xxx - - [13/Apr/2021:14:53:19 +0200] "GET /me HTTP/1.1" 200 24062 "https://my-site.com/me" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.120 Safari/537.36 OPR/64.0.3417.92"
Chrome:
192.168.xx.xxx - - [13/Apr/2021:15:06:56 +0200] "GET /me HTTP/1.1" 200 115426 "https://my-site.com/me" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36"
Did I properly set my Nginx logging configuration?
I am trying to include a request header of name My-Key into access log
I tried to add it as "$http_My-Key":
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_My-Key" "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" '
'"$request_time" "$upstream_connect_time" "$upstream_header_time" "$upstream_response_time"';
however, it access log for that header, I get this value "--Key" logged:
X.X.X.X - - [22/Jul/2020:10:26:18 +0000] "POST /v1/my/app HTTP/1.1" 404 732 "--Key" "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36" "X.X.X.X" "0.009" "0.004" "0.008" "0.008"
instead of the expected value my test which I sent with postman, any idea why?
I am trying to change the log format /etc/nginx/nginx.conf to:
http {
log_format custom '3,$time_iso8601,$cookie_binuDid,,IPS,,0,$remote_addr,??,Unknown,N,N,$content_type,content,fetch,N'
'$status,Y,$upstream_response_time,$status,$upstream_response_time,$upstream_response_length,$upstream_response_length'
'"$upstream_http_cache-control",$request_uri,,,,,000000,0,Unknown'
access_log /var/log/nginx/access.log custom;
Yet my access log prints:
220.233.181.158 - - [08/Dec/2019:14:26:08 +0000] "GET /static/js/5.32912c95.chunk.js HTTP/1.1" 304 0 "http://example-3.com/auth" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
I did service nginx restart after the change, with no effect.
I guess there is another configuration file that needs to be changed, but I couldn't figure. Can someone please help? Am I doing something wrong?
I have a setup where one nginx does reverse proxy to a second one.
I observed in my logs, that one nginx sees two request but the second one sees just one of them. I also observed that the browser is just sending one request, so the duplication if exists, does not come from the client.
Here is a sample of the logs:
nginx_1 | 192.168.64.8 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.0" 200 9402 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.1" 200 111773 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" "-"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic1.jpg HTTP/1.1" 200 111773 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.1" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" "-"
proxy_1 | 192.168.64.1 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.1" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
nginx_1 | 192.168.64.8 - - [15/Jul/2016:11:43:32 +0000] "GET /images/mypic2.jpg HTTP/1.0" 200 25619 "http://localhost/es" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
As you can see, proxy_1 is the nginx working as reverse proxy, and nginx_1 is the one receiving and processing the requests.
In the end of every duplicated line you can see a "-" that I don't know what it means. I suspect is doing some sort of rewriting I'm not aware of but don't understand how it happens.
Could you give me some clue about what is going on here?
Here I also provide the server config:
access_log /dev/stdout;
error_log /dev/stdout;
server {
listen 7000;
listen 80;
client_max_body_size 24M;
server_name *.rareconnect.org;
include /etc/nginx/params/gzip_on;
location / {
include /etc/nginx/params/proxy_pass_local_rareconnect_org_8000;
}
}
This is on top of the standard configuration of nginx:1.9.5 as appears in the official docker registry.
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Thanks in advance.
Looks like there are few records access_log. Can you check nginx.conf as well?