this is my first question here so I apologize if don't comply with the best practices.
Recently I've been a victim of a POST HTTP slow DDoS attack using different IPs on similar and different ranges on my server.
I've managed to mitigate it using fail2ban by creating a personalized filter for my specific needs:
[Definition]
failregex = ^<HOST> .*POST .*xmlrpc\.php.*
^<HOST> .*HEAD .*m.y.i.p.:80.*
ignoreregex =
These two were the most recursive attempts at my server: 'POST xmlrpc.php' request and a 'HEAD http://m.y.i.p/{phpmyadmin|phpwhatevervariation|etc...}'.
I successfully managed to block them using my local jail as this
[nginx-xmlrpc]
enabled = true
filter = nginx-xmlrpc
action = route
logpath = /var/log/nginx/access.log
maxretry = 3
findtime = 10800
bantime = 86400
The problem is that I was keeping myself locked out every so often during development. So I decided to whitelist myself. Unfortunately, my ISP provides dynamic ips, so I had to associate a hostname to zonomi and use DDNS to update my subdomain with my new assigned IP addresses every so often. I then added my hostname to my ignoreip entry on the local jail as this:
# MISCELLANEOUS OPTIONS
#
# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
# ban a host which matches an address in this list. Several addresses can be
# defined using space separator.
ignoreip = 127.0.0.1/8 my.hostname.sub.domain
Today I was working on the webserver and I got blocked, so I checked my hostname and it hasn't updated my IP. I manually did it and after the DNS spread over and the hostname's ip changed, I tried to access my website/server with no success.
It appears to me that either: (1) once the ban was set I would have to restart fail2ban to flush the block on my IP (which I dislike due to the fact that all the other IP's blocked are forgotten, the real threats) or (2) somehow fail2ban wasn't able to update my hostname's associated IP.
My question is: If it's (1), is there a way to lift the block automatically without restarting fail2ban or, in case it's (2), is there a way to update my hostname's ip automatically?
Does fail2ban uses IPTABLES? Should I cron a chain flush with my hostname's IP on iptables every minute?
Kindly,
A.
Fail2Ban uses iptables. As per fail2ban's documentation, it allows whitelisting based on hostname or ip addresses:
http://www.fail2ban.org/wiki/index.php/Whitelist
You should use a Dynamic DNS service, set a small TTL for your hostname (like 600 which amounts for 10 minutes). You can go even with 300 (it's not complaint with the standard but it will the job). Then check and see. If your DDNS hostname was created with a default TTL which in most of the cases for A records is between 3600 and 14400 (1 hour - 4 hours) then that might be the reason.
Related
One of our clients has a wrong configured device which sends a wrong username. He can not find the device, so fail2ban keeps blocking him. When we add the ip to the whitelist it works, but he gets banned again, when his router sets a new ip address.
So, my idea was to set up a fail2ban filter to ignore his specific username in postfix and doevecot.
How would such a filter look like?
This is not a filter what you need, but rather an ignoreregex in jail configuration (one per jail dovecot or postfix).
Note that how the regex may look is depending on how the log-line (which cause the ban) would look.
If your log-line (which cause ban) looks for example like this:
... auth-worker(123): Info: sql(certain-user#example.de,192.0.2.1,<sessionid>): unknown user
you can add to the jail following config (in your jail.local):
[dovecot]
...
_ignore_users = (?:certain-user|some-user|another-user)#example.de
ignoreregex = sql\(%(_ignore_users)s,[^\)]*\): unknown user$
...
And fail2ban would stop banning such users if ban occurs only by this kind of messages, otherwise you have to write ignoreregex for several message formats.
Just by the way:
One of our clients has a wrong configured device which sends a wrong username. He can not find the device...
His system e. g. some of his hosts may be part of some botnet (always trying a bruteforce for some single username), so it is probably not advisable to white-list him.
When we add the ip to the whitelist it works, but he gets banned again, when his router sets a new ip address
If he has some kind of DynDns (or can configure it in his router), you can add its FQDN to ignoreip instead of IP.
I have a couple of web sites and an hmailserver on my VPS. I recently noticed that comcast, and I guess others, are rejecting email where there is no PTR or rDNS. I can set up the PTR in DNS but not sure about the rDNS. I came across an old blog that shows how to do it in older versions of IIS but I'm not sure if this is the right way or how to do it in IIS-10
Say I have 3 domains
myDomain1.com and MyDomain2.com both use Ip x.x.x.y
and myMailServer.com uses Ip x.x.x.z
DNS for these are all set up on cloudflare. MX records for the MyDomain1 and MyDomain2 both point to mail.MyMailserver.com. I use hmailserver and all this works fine.
Comcast says I need a PTR record which I can set up on Cloudflare, and rDNS which is what I'm asking about.
This blog
http://www.expta.com/2010/01/how-to-enable-reverse-dns-lookup-in-iis.html
shows how to do it in earlier versions of IIS (I am on IIS-10) using IP Address and Domain Restrictions
So my questions are :
1 - if this is not the right way then please redirect me! else
2 - Do I use Allow Entry or Deny Entry?
3 - Do I use the mail domain x.x.x.z for all or the ip assigned in DNS - or both?
4 - do I do this for all domains?
Thanks
You need to go to your VPS provider to check their control panel for the PTR/rDNS setup. Usually all the VPS providers has this functionality in their control panels. If not, then ask them to add a proper PTR record for your IP address pointed to your hostname.
The hostname itself should be with proper A record pointed to the IP address of your VPS server.
Separate PTR record setup should be created for all your IPs.
I'm trying to setup some new hosts in munin for monitoring. For some reason it ain't happening!
Here's what I've tried so far.
On the munin server, which is already monitoring several other hosts, I've added the host I want in /etc/munin/munin.conf
[db1]
address 10.10.10.25 # <- obscured the real IP address
use_node_name yes
And on the db1 host I have this set in /etc/munin/munin-node.conf
host_name db1.example.com
allow ^127\.0\.0\.1$
allow ^10\.10\.10\.26$
allow ^::1$
port 4949
And I made sure to restart the services on both machines.
From the monitoring host I can telnet to the new server I want to monitor on the munin port:
[root#monitor3:~] #telnet db1.example.com 4949
Trying 10.10.10.26...
Connected to db1.example.com.
Escape character is '^]'.
# munin node at db1.example.com
Wait a few minutes.. and nothing! The new server won't appear in the munin dashboard on the munin monitoring host.
In the /var/log/munin/munin-update.log log on the db1 host (the one I'm trying to monitor) I find this:
2015/11/30 03:20:02 [INFO] starting work in 14199 for db1/10.10.10.26:4949.
2015/11/30 03:20:02 [FATAL] Socket read from db1 failed. Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.
2015/11/30 03:20:02 [ERROR] Munin::Master::UpdateWorker<db1;db1> died with '[FATAL] Socket read from db1 failed. Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.
What could be going on here? And how can I solve this ?
Since you have already verified that your network connection is ok, as a first step of investigation, I would surely simplify the munin-node.conf. Currently you have:
host_name db1.example.com
allow ^127\.0\.0\.1$
allow ^10\.10\.10\.26$
allow ^::1$
port 4949
From these I would remove:
host_name (it is probably redundant.)
The IPv6 loopback address. (I don't think you need it, but you can add it back later if you do need it)
The IPv4 loopback address. (same as above)
If it still not working, you could completely outrule any issue with the allow config by replacing the direct IPs with:
cidr_allow 10.10.10.0/24
This would allow connection from a full range of IPs in case your db1 host appears to be connecting from a different IP.
Given a target URI, how can I programmatically determine whether an HTTP GET of that URI would be making a request to the local machine?
Context: There are two reasons I need to do this. One is that I have a mod_perl2 application that responds to HTTP requests. In doing so, it sometimes needs to make an HTTP request to retrieve some data from a target URI. To avoid an infinite recursion of HTTP requests, I need to avoid making the HTTP request if the target URI would actually resolve to the current machine. This is to prevent users from accidentally shooting themselves in the foot. It is not intended as a security check.
The second reason is that, if my application receives an HTTP request, I need to look up some metadata using the request URI as key. The problem is that any of several URI synonyms could have been used as keys in creating the metadata, so I need a way to resolve the synonyms, but only for URIs on the local host machine.
The problem is not as simple as looking at the URI to see if the domain is "localhost", or its IP address is 127.0.0.1 (or 127.0.1.1 or 127.*), because: (a) the target URI might use a fully qualified domain name (e.g., foo.example.com) that resolves to an IP address on the current machine; and (b) a machine can have several IP addresses.
The OS must have the information needed to figure this out, since it has to know the IP addresses and ports on which it listens. This post discusses the problem of trying to determine the local machine's IP address (or addresses, since it may have several). Maybe I could do that to determine the local machine's IP addresses, and then perhaps I could compare those IP addresses against the IP address in the target URI (or the IP address returned by gethostbyname of the URI's domain). Do I really need to do that? Are there problems with that approach? Is there a better way?
This post indicates that C# has a function HttpContext.Current.Request.IsLocal to do what I need, but I have been unable to find anything similar in perl.
I previously asked this question on perlmonks.org (because I'm using perl) but found no satisfactory answer. If there is a solution available in some other programming language that is commonly available on Linux, such as C, bash or python, that would be adequate also. I do not need a solution that is guaranteed to work in every possible case, but it would be nice if it would work for most cases.
Since I found no better solution I ended up implementing this almost exactly as suggested by #EightBitTony and someone else on perlmonks. After getting the host out of the URI, which can be done using the perl URI module, here is the perl code I used to determine whether the host is local:
#! /usr/bin/perl -w
use strict;
use Socket;
use IO::Interface::Simple;
print "127.0.1.1 is local\n" if &IsLocalHost("127.0.1.1");
print "google.com is local\n" if &IsLocalHost("google.com");
exit 0;
################ IsLocalHost #################
# Is the given host name, which may be either a domain name or
# an IP address, hosted on this local host machine?
# Results are cached in a hash for fast repeated lookup.
sub IsLocalHost
{
my $host = shift || return 0;
our %isLocal; # Cache
return $isLocal{$host} if exists($isLocal{$host});
my $packedIp = gethostbyname($host);
if (!$packedIp) {
$isLocal{$host} = 0;
return 0;
}
my $ip = inet_ntoa($packedIp) || "";
our %localIps; # Another cache
%localIps = map { ($_, 1) } &GetIps() if !%localIps;
my $isLocal = $localIps{$ip} || $ip =~ m/^127\./ || 0;
# TODO: Check for IPv6 loopback also. See:
# http://ipv6exchange.net/questions/16/what-is-the-loopback-127001-equivalent-ipv6-address
$isLocal{$host} = $isLocal;
return $isLocal;
}
################ GetIps #################
# Lookup IP addresses on this host.
sub GetIps
{
my #interfaces = IO::Interface::Simple->interfaces;
my #ips = grep {$_} map { $_->address } #interfaces;
return #ips;
}
There is a naive solution to this, described as,
extract the fully qualified domain name, hostname or IP address from the URI in question.
resolve that to an IP address
compare that against a list of IP addresses on the current host
if there is a match, then this URI points to this host
This works, as long as,
the URI doesn't resolve to another host which then redirects to this one
the URI doesn't resolve to a load balancer which then balances back to this host
the host doesn't use a Proxy which could handle the request (caching proxy) or some other device in the chain.
However, I think your question is too broad and you would be better placed breaking down into two questions,
How do I extract the IP address, hostname or FQDN from a URI (and ask that on a programming site)
How do I enumerate all the IP addresses on a single host (and if that host is a Linux server, ask that question here).
This isn't really an answer, but it's too long for a comment, and I suspect your question is going to be closed.
start cmd: # ip route get 192.168.1.2
local 192.168.1.2 dev lo src 192.168.1.2
cache <local>
I want to host a new subdomain on an Ec2 Instance(ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com) like blog.somesite.com
I have the DNS settings on a 3rd party host(like Godaddy) that look like:
site ip addr as shown above, is the value of the ec2 server e.g. xxx.xxx.xx.xx and not
ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com
If I try to do an mxtoolbox lookup on DNS for blog.myapp.com, it seems to have properly propogated the A-Record, do I need a CNAME record instead of A-Record?
If I try to access blog.myapp.com via browser, it is just a never ending connection. If I access myapp.com , it has always been working fine.
On my ec2 box, I'm running nginx, does something need to be configured on nginx too?
Sorry about the newbieness - still learning.
Thank you!
To start with, you should assign an elastic IP to your instance. IP addresses will change if the instance is ever stopped. With an elastic IP, you can re-associate the ip address to the instance if you need to stop it.
If you are setting up a DNS record for the apex, it needs to be an A record (Apex records is your domain with no subdomain).
For the domain blog.yourdomain.com you can set up either an A or CNAME record.
You will likely need to configure your host within nginx to respond to requests with your domain name.
You will also need to make sure port 80 is open on your security group, and system firewall if your OS has one configured.