My website is getting attacked with such request as
66.249.75.242 - - [12/Jan/2023:00:29:11 +0800] "GET /?bailiffry/1529595 HTTP/1.1" 200 57100 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.5304.115 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.236 - - [12/Jan/2023:00:29:14 +0800] "GET /?Diphysite-7105-hwfLs/328609048 HTTP/1.1" 200 57097 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.236 - - [12/Jan/2023:00:29:16 +0800] "GET /?hypothermal/sealant313919.html HTTP/1.1" 200 57100 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.5304.115 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.75.236 - - [12/Jan/2023:00:29:17 +0800] "GET /?dianilid/elated357845.html HTTP/1.1" 200 57100 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.5304.115 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
I have blocked other patterns, I just wish to block this for now as I have been solving this for hours and wish a quick fix for now.
How do I block request starting with question mark?
Yes you can. See the question below
Drop unwanted connections
if (condition) ) {
return 444;
}
You Can also put a WAF (Web Application Firewall) in your front, if your request is coming for a security issue you're facing.
you can see NAXSI. It's Open-source and compatible with any nginx version.
Related
I am trying to clear the cache of amp page but I am getting this error:
Public key not found due to ingestion error: 499 error from origin
That’s all we know.
What I have checked:
.well-known/amphtml/apikey.pub is publicly available
the file is not roboted(allowed for google bots)
the file has content type plain/text
bypassed cloudflare cache
Suspiction:
There are also 301 requests from google bots. I guess it is because google bot is requesting the file in HTTP initially and redirected to https.
64.233.173.70 - - [13/Jun/2022:16:08:22 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 301 193 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Google-AMPHTML)"
64.233.173.204 - - [13/Jun/2022:16:08:22 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 200 451 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Google-AMPHTML)"
66.249.70.56 - - [13/Jun/2022:16:10:55 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 301 193 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.21 - - [13/Jun/2022:16:11:14 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 200 451 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.55 - - [13/Jun/2022:17:07:18 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 301 193 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.114 - - [13/Jun/2022:17:07:19 +0800] "GET /.well-known/amphtml/apikey.pub HTTP/1.1" 200 451 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
Another suspicion:
The .well-known/amphtml/apikey.pub cannot be curled and it returns 403. But from above logs from Google bot, it seems it does not have any problem fetching the file.
I don't understand what I miss here. Please help!
Whenever I'm trying to open web application my Nginx access log shows "GET /false HTTP/1.1" 404 206 "https://www.example.com/FeedifySW.js" "Mozilla/5.0 (Linux; Android 6.0; Micromax Q4260 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.85 Mobile Safari/537.36" - -
I have searched through all way possible and still couldn't find the solution. Can anyone help?
This is for an nginx server where i have hosted my web application. due to high request counts, sometimes the application crashes
"GET /false HTTP/1.1" 404 206 "https://www.example.com/FeedifySW.js" "Mozilla/5.0 (Linux; Android 6.0; Micromax Q4260 Build/MRA58K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.85 Mobile Safari/537.36" - -
When I access a page in the browser I get a proper 200 from the server:
xx.xxx.xxx.xxx - - [02/May/2019:19:53:50 +0200] "GET /retourneren HTTP/1.1" 200 2889 "https://mysite.nl/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15" "-"
However when I add the url in prerender I get a 400:
3.90.111.223 - - [02/May/2019:19:50:39 +0200] "GET /retourneren HTTP/1.1" 404 10050 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/61.0.3163.59 Safari/537.36 Prerender (+https://github.com/prerender/prerender)" "-"
therefor the page is not getting cached. Does anyone have an idea?
That seems like you might be setting the prerender-status-code meta tag on the page and setting it to "404", which would make Prerender.io return a 404 response code directly.
Can you confirm whether or not that meta tag is being set in the HTML of the page?
I wrote a script that configures a captive portal and everything is working fine. The problem I'm having is because nginx is redirecting all the requests to http://hotspot.localnet/index.php is also redirecting all requests for images. So none of the images in index.php work.
Is there a way to exclude png,pdf files from being redirected to index.php and be displayed? Tried many regex examples found on the internet but having no luck.
The source code of the nginx config file is here: https://github.com/tretos53/Captive-Portal/blob/master/default_nginx
This is access.log with the above configuration:
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /index.php HTTP/1.1" 200 582 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/1.jpg HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/2.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/3.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/4.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/5.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
192.168.137.1 - - [03/Dec/2018:19:53:16 +0000] "GET /images/6.png HTTP/1.1" 302 161 "http://hotspot.localnet/index.php" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"
Add this to your config:
location ~ \.(?:pdf|png)$ {
try_files $uri =404;
}
An abandoned outdated wordpress website that i thought the internet didn't know about got hacked. The attack got resolved quickly and had no real damage as for as I know now. But it does give an opportunity to study the attack used.
One thing that caught my attention is an upload of a malicous sitemap.xml causing google to do many (250k/day) requests to strange urls matching a specific pattern. 6 digits - 8 digits - 6 char hex
Examples:
66.249.76.33 - - [03/Oct/2018:14:12:13 +0200] "GET /035742-41258563-3329f7 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:13 +0200] "GET /042913-72193084-e8a20a HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:14 +0200] "GET /012527-34165946-30e419 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:14 +0200] "GET /064248-52623737-8691d5 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.2 - - [03/Oct/2018:14:12:15 +0200] "GET /052839-44405924-68722a HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.64.2 - - [03/Oct/2018:14:12:15 +0200] "GET /065830-65437791-de5b61 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:16 +0200] "GET /013227-70693694-023293 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:16 +0200] "GET /125539-43521853-8481a2 HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:17 +0200] "GET /033515-14477539-24816a HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.76.33 - - [03/Oct/2018:14:12:17 +0200] "GET /104450-28458335-28053c HTTP/1.1" 302 244 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
I've verified it's a real google bot by using host and ping. I'm curious if anyone has seen this attack before and how it works!
Never seen the attack before but it's execution is pretty simple: upload a massive sitemap of randomly generated "page urls" and let google do the rest.
Google will go "ooh look at that they must have added loads of new content, I want that" and will hit the site a LOT to try and get it, hence crippling the site. We see this when taking large sites to a new url structure all the time.
The only way I've found to compensate is to use NGINX rate limiting to stop any single IP making too many requests per second.