Excuse me if I am being dumb or this is a duplicate question (I didn't find a solution that worked)
I am trying to use a static site with nginx, and my file structure is like this:
root
img
some image.jpg
some image.jpg
js
some code.js
some code.js
css
some code.css
some code.css
and here is my config:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name xilog.xyz www.xilog.xyz;
root /var/www/xilog.xyz/public_html;
index index.html index.htm index.php;
autoindex on;
}
If you visit the site you can see that it half works, with css working but when I try to access js and images that should be there, they don't appear (eg: xilog.xyz/img/logo.png) hence the loader which is stuck on and missing background.
Error log when I load site:
2020/04/27 09:29:22 [error] 24648#24648: *6 open() "/var/www/xilog.xyz/public_html/cgi-bin/mainfunction.cgi" failed (2: No such file or directory), client: 68.132.136.198, server: xilog.xyz, request: "GET /_img/portfolio/corkboard.png HTTP/1.1", host: "xilog.xyz"
Last several access log lines:
141.101.99.65 - - [27/Apr/2020:11:52:21 +0000] "GET /img/logo.png HTTP/1.1" 200 585687 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.155.230 - - [27/Apr/2020:11:52:22 +0000] "GET /favicon.ico HTTP/1.1" 200 38078 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:52:28 +0000] "GET / HTTP/1.1" 200 12979 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.158.221 - - [27/Apr/2020:11:52:28 +0000] "GET /js/portfolio.js HTTP/1.1" 200 803 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.158.115 - - [27/Apr/2020:11:52:28 +0000] "GET /js/easteregg.js HTTP/1.1" 200 1484 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
141.101.107.186 - - [27/Apr/2020:11:52:28 +0000] "GET /css/style.css HTTP/1.1" 200 16356 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.155.26 - - [27/Apr/2020:11:52:28 +0000] "GET /js/main.js HTTP/1.1" 200 4866 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
141.101.99.87 - - [27/Apr/2020:11:52:28 +0000] "GET /css/animations.css HTTP/1.1" 200 732 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.94 - - [27/Apr/2020:11:52:28 +0000] "GET /img/portfolio/corkboard.png HTTP/1.1" 200 4313 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.155.134 - - [27/Apr/2020:11:52:28 +0000] "GET /img/logo-white.png HTTP/1.1" 200 13837 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
141.101.107.162 - - [27/Apr/2020:11:52:28 +0000] "GET /css/bootstrap.min.css HTTP/1.1" 200 140942 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.154.31 - - [27/Apr/2020:11:52:29 +0000] "GET /img/portfolio/shift.jpg HTTP/1.1" 200 525361 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
141.101.98.220 - - [27/Apr/2020:11:52:29 +0000] "GET /img/portfolio/extioh.jpg HTTP/1.1" 200 362925 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.155.74 - - [27/Apr/2020:11:52:30 +0000] "GET /img/bg-med.png HTTP/1.1" 200 237622 "https://xilog.xyz/css/style.css" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.142 - - [27/Apr/2020:11:53:07 +0000] "GET / HTTP/1.1" 200 12979 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:53:07 +0000] "GET /css/bootstrap.min.css.map HTTP/1.1" 200 562427 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:53:13 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:53:15 +0000] "GET /css/bootstrap.min.css.map HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:53:18 +0000] "GET /css/bootstrap.min.css.map HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.159.42 - - [27/Apr/2020:11:53:20 +0000] "GET /photography HTTP/1.1" 404 153 "https://xilog.xyz/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0" "122.167.110.18"
162.158.90.174 - - [27/Apr/2020:11:54:10 +0000] "GET /robots.txt HTTP/1.1" 200 24 "-" "Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)" "77.75.78.169"
141.101.96.180 - - [27/Apr/2020:11:54:12 +0000] "GET /_img/portfolio/corkboard.png HTTP/1.1" 404 153 "-" "Mozilla/5.0 (compatible; SeznamBot/3.2; +http://napoveda.seznam.cz/en/seznambot-intro/)" "2a02:598:aaaa:2::8209"
162.158.119.7 - - [27/Apr/2020:11:54:39 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36" "2001:8f8:1329:dd06:e7c1:a5de:1c56:a3ee"
Dang it, started working after waiting. I think it was a cache issue.
Related
I am setting up a Nginx reverse proxy using these docker images
nginx
jwilder/docker-gen
jrcs/letsencrypt-nginx-proxy-companion
The host where these containers are located is given public IP address so that the web app can be accessed from the internet. We registered the subdomain and set the public IP address to that subdomain.
When I test the configuration, the web app worked and https also worked but from LAN only (we have local DNS server that has the subdomain paired to a local server IP address.
But when I tried accessing it from internet it gives 301 redirected too many times. And yes I saw in the Nginx log the server logged around 20s 301 redirections and then stopped.
I am still clueless about what is wrong with the configuration. I used this template https://raw.githubusercontent.com/jwilder/nginx-proxy/master/nginx.tmpl for the auto-generated configuration files with a little modification in the location part (proxy_connect_timeout, proxy_send_timeout, proxy_read_timeout, proxy_send_lowat) to make our web app not 502 gateway timeout from long processing.
Can anyone point where my configuration failed?
# fhij.abcd.co.id
upstream fhij.abcd.co.id {
## Can be connected with "nginx-proxy" network
# fhid_web_1
server 172.20.0.8:8069;
}
server {
server_name fhij.abcd.co.id;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
return 301 https://$host$request_uri;
}
server {
server_name fhij.abcd.co.id;
listen 443 ssl http2 ;
access_log /var/log/nginx/access.log vhost;
ssl_session_timeout 5m;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;
ssl_certificate /etc/nginx/certs/fhij.abcd.co.id.crt;
ssl_certificate_key /etc/nginx/certs/fhij.abcd.co.id.key;
ssl_dhparam /etc/nginx/certs/fhij.abcd.co.id.dhparam.pem;
ssl_stapling on;
ssl_stapling_verify on;
ssl_trusted_certificate /etc/nginx/certs/fhij.abcd.co.id.chain.pem;
add_header Strict-Transport-Security "max-age=31536000" always;
include /etc/nginx/vhost.d/default;
location / {
proxy_pass http://fhij.abcd.co.id;
proxy_connect_timeout 9990;
proxy_send_timeout 9990;
proxy_read_timeout 9990;
proxy_send_lowat 12000;
}
}
These are the access.log I got from accessing from internet
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:42 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:43 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:43 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:43 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:43 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:43 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:44 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:44 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:44 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:45 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:45 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:46 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:46 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:46 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:46 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:46 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:47 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:47 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:47 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
fhij.abcd.co.id 162.158.178.186 - - [22/Nov/2019:03:57:47 +0000] "GET / HTTP/1.1" 301 169 "-" "Mozilla/5.0 (Linux; Android 9; YAL-L21) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.96 Mobile Safari/537.36"
And these are from LAN
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET / HTTP/2.0" 303 215 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET /web HTTP/2.0" 303 227 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET /web/login HTTP/2.0" 200 4383 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET /web/content/269-ceb8fb1/web.assets_common.0.css HTTP/2.0" 499 0 "https://fhij.abcd.co.id/web/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET /web/content/276-771d39e/web.assets_frontend.0.css HTTP/2.0" 499 0 "https://fhij.abcd.co.id/web/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "GET /web/webclient/qweb?mods= HTTP/2.0" 304 0 "https://fhij.abcd.co.id/web/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
fhij.abcd.co.id 192.168.130.127 - - [22/Nov/2019:05:21:29 +0000] "POST /web/webclient/bootstrap_translations HTTP/2.0" 200 87 "https://fhij.abcd.co.id/web/login" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0"
Why it is so different? Nginx response to internet request was 301 and response to LAN was 303
Turned out there is a mistake in our Cloudflare DNS and WAF configuration. Modifying these configuration fixed this issue.
I set in my airflow.cfg:
authenticate = True
auth_backend = airflow.contrib.auth.backends.password_auth
And I think, that i`d able to see login page then I open http://localhost:8089/admin, but I can only see "Too many redirects" page, logs looks like a:
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/ HTTP/1.1" 302 281 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
airflow_webserver_1 | 172.21.0.1 - - [15/Jul/2019:06:35:53 +0000] "GET /admin/airflow/login?next=%2Fadmin%2F HTTP/1.1" 302 221 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
Have no ideas why it doesn`t work...
I wrote a script that configures a captive portal and everything is working fine. The problem I'm having is because nginx is redirecting all the requests to http://hotspot.localnet/index.php is also redirecting all requests for images. So none of the images in index.php work.
Is there a way to exclude png,pdf files from being redirected to index.php and be displayed? Tried many regex examples found on the internet but having no luck.
The source code of the nginx config file is here: https://github.com/tretos53/Captive-Portal/blob/master/default_nginx
This is access.log with the above configuration:
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /index.php HTTP/1.1" 200 582 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/1.jpg HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/2.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/3.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/4.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/5.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/6.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
Add this to your config:
location ~ \.(?:pdf|png)$ {
try_files $uri =404;
}
I am trying to find the bandwidth used by the most prevalent ip addresses making requests within nginx access logs. This is what I have started out with:
$ cat /path/to/access.log |awk '{print $1}' |sort |uniq -c |sort -n |tail
($1 is the ip address, while the bytes of request is $10) - which will output:
# of requests | IP Address
1220 xxx.xxx.xxx.xxx
1347 xxx.xxx.xxx.xxx
1420 xxx.xxx.xxx.xxx
2104 xxx.xxx.xxx.xxx
etc...
What I am trying to accomplish is to identify how much bandwidth each one of these addresses is requesting. For example:
# of requests | IP Address | total bytes requested (unique to ip)
1220 xxx.xxx.xxx.xxx 45626026
1347 xxx.xxx.xxx.xxx 49565157
1420 xxx.xxx.xxx.xxx 56689122
2104 xxx.xxx.xxx.xxx 76665299
etc...
My restrictions are not too limited. So, with that said, if the possible solution would be to use more than one command to resolve upon the final query (i.e. find total bandwidth by ip), so be it. Thanks for any help provided!
With single GNU awk solution:
Sample access.log for demonstration purpose:
127.0.0.1 - - [15/Aug/2017:09:38:35 +0300] "GET / HTTP/1.1" 200 111 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
127.0.0.1 - - [15/Aug/2017:09:38:46 +0300] "GET / HTTP/1.1" 200 171 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
127.0.0.1 - - [15/Aug/2017:09:59:38 +0300] "GET /favicon.ico HTTP/1.1" 404 152 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
127.0.0.1 - - [15/Aug/2017:09:59:39 +0300] "GET /favicon.ico HTTP/1.1" 404 1502 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
127.0.0.1 - - [15/Aug/2017:11:04:45 +0300] "GET / HTTP/1.1" 200 23976 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
127.0.0.2 - - [15/Aug/2017:09:38:35 +0300] "GET / HTTP/1.1" 200 14111 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.2 - - [15/Aug/2017:09:38:46 +0300] "GET / HTTP/1.1" 200 1414 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.2 - - [15/Aug/2017:09:59:38 +0300] "GET /favicon.ico HTTP/1.1" 404 1522 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.2 - - [15/Aug/2017:09:59:39 +0300] "GET /favicon.ico HTTP/1.1" 404 1332 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.3 - - [15/Aug/2017:11:04:45 +0300] "GET / HTTP/1.1" 200 23976 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) G$
127.0.0.1 - - [15/Aug/2017:09:38:35 +0300] "GET / HTTP/1.1" 200 141 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.1 - - [15/Aug/2017:09:38:46 +0300] "GET / HTTP/1.1" 200 1041 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.3 - - [15/Aug/2017:09:59:38 +0300] "GET /favicon.ico HTTP/1.1" 404 1529 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.1 - - [15/Aug/2017:09:59:39 +0300] "GET /favicon.ico HTTP/1.1" 404 1026 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.1 - - [15/Aug/2017:11:04:45 +0300] "GET / HTTP/1.1" 200 23976 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) G$
127.0.0.3 - - [15/Aug/2017:09:38:35 +0300] "GET / HTTP/1.1" 200 1414 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.1 - - [15/Aug/2017:09:38:46 +0300] "GET / HTTP/1.1" 200 13341 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gec$
127.0.0.3 - - [15/Aug/2017:09:59:38 +0300] "GET /favicon.ico HTTP/1.1" 404 172 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.3 - - [15/Aug/2017:09:59:39 +0300] "GET /favicon.ico HTTP/1.1" 404 1502 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; r$
127.0.0.3 - - [15/Aug/2017:11:04:45 +0300] "GET / HTTP/1.1" 200 23976 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) G$
The job:
awk 'BEGIN{ PROCINFO["sorted_in"]="#val_num_desc" }
{ a[$1]++; b[$1]+=$10 }
END{
for(i in a) { if(++c>10) break; print i,b[i] }
}' /path/to/access.log
PROCINFO["sorted_in"]="#val_num_desc" - comparison of array values, to sort by IP address frequency in descending order
if(++c>10) - ensures iterating over only first 10 items, which is emulation of tail command (gets the last 10 lines) The loop starts from the most frequent IP address
The output:
127.0.0.1 65437
127.0.0.3 52569
127.0.0.2 18379
today my Wordpress site was compromised so now serve a JS script that redirect to SPAM site.
I looked inside the Apache logs to reconstruct what happened, but i can't know how to interpretate this:
xx.xx.xx.xx - - [09/Jan/2017:10:24:42 +0100] "GET /wp-login.php HTTP/1.1" 200 6111 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:42 +0100] "GET /wp-login.php HTTP/1.1" 200 6111 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:43 +0100] "GET /wp-login.php HTTP/1.1" 200 6111 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:43 +0100] "GET /wp-login.php HTTP/1.1" 200 6111 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:43 +0100] "POST /wp-login.php HTTP/1.1" 302 4 "/wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:43 +0100] "POST /wp-login.php HTTP/1.1" 302 4 "/wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:44 +0100] "GET /wp-admin/ HTTP/1.1" 302 4 "http://my.host.name/wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:44 +0100] "GET /wp-admin/ HTTP/1.1" 200 219966 "/wp-login.php" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:47 +0100] "GET /wp-admin/theme-editor.php HTTP/1.1" 200 183974 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:48 +0100] "GET /wp-admin/theme-editor.php?file=404.php&theme=twentyfourteen HTTP/1.1" 500 3427 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
xx.xx.xx.xx - - [09/Jan/2017:10:24:49 +0100] "GET /wp-admin/theme-install.php?upload HTTP/1.1" 200 161448 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:50.0) Gecko/20100101 Firefox/50.0"
As you can see somebody has tried to login, but he can't. But after two simple GET request to "/wp-admin/" seems to be logged in and he able to modify and install new theme.
At this point I have tried to find differences between the wp-admin directory that is present on the server with the original one downloaded from the official site wordpress.org, but i didn't find any differences.
I compared all file present into wp-admin with "diff" utility and i can't find any difference. For example:
diff /var/www/html/original.wordpress/wp-admin/themes.php /var/www/html.hacked/wp-admin/themes.php didn't output any code
Could you help me to find evidence?
Looks like your exploit happened here:
/wp-admin/theme-install.php?upload
Make sure your themes and plugins are updated. That last request returned a 200 response from your webserver to the attacker, and was likely used to upload a backdoor. I'd audit the contents of /wp-content/uploads/ to see if there are any out of place PHP files (backdoor shells) in there. If so, it's possible they used that to pivot within your site, so there may be other directories containing PHP backdoors elsewhere.