How do I block a user agent using nginx.
so far I have something like this:
if ($http_user_agent = "Mozilla/5.0 (Linux; Android 4.2.2; SGH-M919 Build/JDQ39) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.169 Mobile Safari/537.22") {
return 403;}
this is from a similar thread on this stack overflow.
I run nginx as a reverse proxy for cherrypy server. I intend to filter a certain user agent using nginx alone but the above code doesn't work on my server.
is that the correct way to do this?
It wasn't included in any block in the nginx config. Should I add it to the "http" block or the "server" block
in order to block the specific user agent I included this code in the "server" block:
if ($http_user_agent = "Mozilla/5.0 (Linux; Android 4.2.2; SGH-M919 Build/JDQ39) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.169 Mobile Safari/537.22"){
return 403;
}
and it worked as expected.
If's are evil - use the map directive.
Directive if has problems when used in location context, in some cases
it doesn’t do what you expect but something completely different
instead. In some cases it even segfaults. It’s generally a good idea
to avoid it if possible.
Nginx Ultimate Bad Bot Blocker makes blocking bots easy with support for Debian / Centos / Alpine Linux / FreeBSD.
Related
I have a site that have some plugins, and one of those plugins (facebook for woocommerce) is loading until it returns a timeout error (504). I can change some constants in wp-config.php, but none of them works when I need to debug a timeout.
I tried to remove every configuration and file that I found from this plugin and then reinstall it, but the error is still there.
I tried to deactivate every other plugin, but woocommerce and the error is still there.
I looked for some debug plugins, but I only found plugins that change wp-config.php constants and do some logs at files. It is useless, I can do this.
I tried to put some "die" with messages in plugin's code, but nothing changed.
Server log just shows this:
x.x.x.x - - [09/Nov/2020:17:52:56 -0300] "xxxxx.com" "GET /wp-admin/admin.php?page=wc-facebook HTTP/1.1" 504 160 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0" "-"
I don't know what I can to do to debug this timeout, I've tried everything I know with the wordpress.
I solved by asking on plugin's forum: https://wordpress.org/support/topic/plugin-page-giving-timeout-504/#post-13687667
I just needed to activate WP_DEBUG and WP_DEBUG_LOG flags. I discovered the line that was breaking the site and then I could properly debug and find the problem.
I have an issue with changing the user agent.
I am trying to use the following line in my runner.js file in the browsers array :
chrome:headless:userAgent=Mozilla/5.0\ \(Linux\;\ Android\ 5.0\;\ SM-G900P\ Build/LRX21T\)\ AppleWebKit/537.36\ \(KHTML,\ like\ Gecko\)\ Chrome/57.0.2987.133\ Mobile\ Safari/537.36
However, the best I can get is Mozilla/5.0 (Linux in the actual user agent.
The guide doesn't say anything explicit about user agents and how to escape them.
Could someone help me with using a custom user agent for the headless chrome? I can't seem to get over the escaping problem. Thanks.
I actually found the answer, you need to escape with \\ every ; character.
E.g:
chrome:headless:userAgent=Mozilla/5.0 (X11\\; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.50 Safari/537.36
will work.
In case of using in cli command you need to double escape. (I didn't have success in that)
Nor do nginx or php-fpm reports a error 500 output, In fact the 500 response is on the access log and not on the nginx error log. The script is running ok in my dev environment.
nginx version: nginx/1.6.2
PHP 5.5.19 (fpm-fcgi)
Tried this
catch_workers_output = 1
Restarted everything, still not working
nginx access log shows:
x.x.x.x - - [12/Dec/2014:19:25:08 -0200] "GET /merchant/customer/mobile/data?sEcho=1&iColumns=3&sColumns=%2C%2C&iDisplayStart=0&iDisplayLength=10&mDataProp_0=0&sSearch_0=&bRegex_0=false&bSearchable_0=true&mDataProp_1=1&sSearch_1=&bRegex_1=false&bSearchable_1=true&mDataProp_2=2&sSearch_2=&bRegex_2=false&bSearchable_2=true&sSearch=&bRegex=false&_=1418418256370 HTTP/1.1" 500 589 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36"
Note the 500 error, which should be on error.log and with the php trace that's is happening on other errors.
Any clues?
When PHP display_errors are disabled, PHP errors can return Nginx 500 error.
Take a look into your php-fpm logs, i'm sure you'll find the error there. With CentOS 7 :
tail -f /var/log/php-fpm/www-error.log
You can finally show PHP errors. In /etc/php.ini, change :
display_errors = Off
to :
display_errors = On
Hope it helps.
In case it helps someone else (Google brought me here), I had a similar problem (although in Apache, not nginx).
An older app I was installing was giving a 500 error with no output anywhere, in spite of every conceivable error logging setting turned to the most verbose level.
It turns out the problem was the controversial error-control operator, "#". As the red warning box in the docs states, no matter how verbose your logging is, if an #-prefixed command causes PHP to stop because of a typo, or because it is not available (say, because you have forgotten to install a critical module like php-mysql), PHP will exit with absolutely no indication as to why.
So, if you find yourself with a 500-error and no logs, check your codebase for "#" symbols.
Verify php-fpm service is running
sudo service php-fpm status
This will display the hostnames that can execute php.
Next edit php.ini to display errors. this will display errors on page..to make things easy to diagnose
sudo vim /etc/php.ini
then hit key [/] and type
display_errors
hit key [i] user arrow keys and backspace to set display_errors = on
then restart php-fpm service
On system type redhat /centos etc..
sudo service php-fpm restart
I'm trying to create a fail2ban filter that is going to ban the host when it sends over 100 POST requests over 30 seconds interval.
jail.local:
[nginx-postflood]
enabled = false
filter = nginx-postflood
action = myaction
logpath = /var/log/nginx/access.log
findtime = 30
bantime = 100
maxretry = 100
nginx-postflood.conf
[Definition]
failregex = ^<HOST>.*"POST.*
ignoreregex =
Using GREP i was able to test the regular expressions and indeed it matches Host and POST requests.
Problem is that it bans any Host that performs at least one POST request. This means likely that it's not taking findttime or maxretry options into consideration. In my opinion it's timestamp issue.
Sample line of nginx log:
5.5.5.5 - user [05/Aug/2014:00:00:09 +0200] "POST /auth HTTP/1.1" 200 6714 "http://referer.com" "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0"
Any help?
I guess it maybe to late for the answer but anyway...
The excerpt you have posed has the filter disabled.
enabled = false
As there is not mentioning of Fail2Ban version and syslog/fail2ban logs are missing for this jail.
I tested your Filter on fail2ban 0.9.3-1 and it works fine although I had to enable it and had to drop the line with action = myaction as you have not provided what you are expecting fail2ban to do.
Therefore this filter should work fine, based that it's enabled and the action is correct as well.
What is happening in the provided example is that Your Filter is disabled and fail2ban is using another Filter which checks the same log file and matches your regex but has more restrictive rules i.e ban after 1 request.
I am a bit confused about syslog message format. I have to write a program that parses syslog messages. When I read what I get in my syslog-ng instance I get messages like this:
Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
I can clearly determine the real message (which is, in this case an Apache access log message) The rest is metadata about the syslog message itself.
However when I read the RFC 5424 the message examples look like:
without structured data
<34>1 2003-10-11T22:14:15.003Z mymachine.example.com su - ID47 - BOM'su root' failed for lonvick on /dev/pts/8
or with structured data
<165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID#32473 iut="3" eventSource="Application" eventID="1011"] BOMAn application event log entry...
So now I am a bit confused. What is the correct syslog message format ? It is a matter of spec version where RFC 5424 obsoleted RFC 3164 ?
The problem in this case is that apache is logging via the standard syslog(3) or via logger. This only supports the old (RFC3164) syslog format, i.e. there is no structured data here.
In order to have the fields from the apache log show up as RFC5424 structured data, apache would need to format the log that way.
The first example is not proper RFC3164 syslog, because the priority value is stripped from the header. Proper RFC3164 format would look like this:
<34>Jan 12 06:30:00 1.2.3.4 apache_server: 1.2.3.4 - - [12/Jan/2011:06:29:59 +0100] "GET /foo/bar.html HTTP/1.1" 301 96 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729)" PID 18904 Time Taken 0
Traditionally rfc3164 syslog messages are saved to files with the priority value removed.
The other two are in RFC5424 format.
If you have access to the installed syslog-daemon on the system you could configure it to write the logs (received both locally or via network) in a different format. rsyslogd for instance allows to configure your own format (just write a template) and also if I remember correctly has a built-in template to store in json format. And there are libraries in almost any language to parse json.
EDIT: You could also make rsyslogd part of your program. rsyslog is very good in reading incoming syslogs in either of the two RFC formats. You can then use rsyslog to output the message in JSON. This way rsyslog does all the decompositioning of the message for you.