I have a web app and I would like to prevent DOS attacks by blocking an IP address if it make many request in a short period of time.
For example, if the same IP address makes 100 request in a second, I can assume that it's some kind of attack and I would like to block this IP.
However, making this check in the application layer seems too expensive - what is the optimal way to make this check?
Should I make this kind of check at my:
firewall
router
apache config
someplace else entirely ...
If you want to block IP addresses when they make a certain number of requests, this is best done at the Network layer. This would suggest that you do this either in your host machine's network stack or using a router (which operates at the network layer).
Some things you might want to consider though are:
- Are you really wanting to block access to the entire host based on an IP address, or do you want to block access to a specific application running on a specific port.
- Sometimes, by using NATs, one IP address may be making requests on behalf of many real hosts.
With any security application you need to have many layers of defence, so it would be a good idea to invest in a good firewall as well.
Some apps for generating APIs in django implement some methods for limiting the amount of request per second.
For example django-piston use throttling method to do that.
django-piston throttling
Thats an easy way to solve the problem.
Related
Here's the issue...
I work in a remote area of Alaska where cell service can be... questionable. We do have decent WiFi, however, is not openly available to staff because it has a low enough data limit that we don't want to deal with people streaming Netflix and running out the company data.
The big issue is that we want to use an app like Slack or Discord to communicate more effectively across the business. Because the cell service is spotty and the WiFi is currently off-limits, I was wondering if there was a way for me to create a WiFi network that was whitelisted to only allow Slack, for example. Then we get the benefits of using the WiFi without risking running out of data.
Thoughts? I was thinking about setting up a network proxy, but I wanted to get the internet's take on it before I dive down the rabbit hole.
The best way I can think of to handle something like this is to use a router that you can configure the dns server settings and block all dns entries that aren’t on your allowlist. This doesn’t strictly block traffic to everywhere but it will do a pretty good job.
You could also block all dns traffic that’s not going to the local dns server which would help not allow people around it. To have a hard block on things you would need to block specific options addresses which with services such as slack or discord could change randomly which would be hard to keep up on.
Another option that would work well is use your own self hosted version of mattermost, rocket.chat, or riot/matrix that you would have control over and knowledge of the IP address so that you can allowlist only those up addresses. The other advantage with this is if the business is just localized communication and you don’t need to chat across long distances then you could set this to work completely on a network with no internet access so you wouldn’t have to do any blocking because the wifi is completely separated from the internet.
Things are heavily based on your situation but I hope this gives you a good place to start
Supposed I have two services that need to share and / or exchange data. Both instances are separate from each other, and they shall not know anything about where the other part is located.
Now in order for them to be able to share and / or exchange data, they need to connect to each other.
How do they find each other without the need to configure the IP addresses explicitly? In other words: How could they detect each other automatically?
Basically, I have two ideas:
Pull: You need to have a central service where they register. Then you can ask that service for the address of a service, and that service then returns those data. While this works, it has the drawback that it only shifts the problem to the next level: What if I have multiple instances of that service, and I don't want them to know each other in advance?
Push: Each service broadcasts its own address, so that other services get it to know. Each service repeats this from time to time. Drawback: This does hardly work in the internet.
Any idea of how I could solve this in an intelligent way?
PS: If you want to say so, I'm looking for a way to handle dynamic IPs without the need for a central DNS server.
The usual way is to have some fault-tolerant server where services register and can then look for other services - Curator framework implements that over zookeeper.
If you want autodiscovery then you should probably implement some sort of gossip protocol so that the servers would know which other servers are out there in a reliable way. You should keep in mind that getting gossip protocols right is tricky (e.g. some of past Amazon cloud failures where due to problems in their implementation)
"broadcast packets are not forwarded everywhere on a network, but only to devices within a broadcast domain."
If your devices are on different broadcast domains then broadcasting is not going to work.
You are probably going to have to implement your own central service, unless you can use one of the free dynamic dns servers, for example: Free
I am building a service which requires me to dynamically launch and close servers at many locations around the world, (for example using AWS). When a user visits my domain they need to be assigned to a local server with the lowest latency.
By assignment, I mean that for example the client makes an ajax call to example.com/getData, it should go directly to one particular server that is has been assigned to. Different servers will be doing different computation, so it is not sufficient to have some kind of general load balancing.
What general mechanisms/technology would allow me to 1) Assess the latency between a particular client and any server under my control? 2) Assign a particular client to a particular server? I cannot use just the IP addresses for example, since javascript has domain name based restrictions.
Thanks
Note: I do not have enough reputation to link all the technologies in the response, therefore sometimes you will see the links copied in plain text.
1) Assign users to a local server with the lowest latency is not always possible.
Sometimes the geographically closest server to a user is unexpectedly the one with the highest latency.
To find the lowest latency between your (running) servers and the users is not an easy task.
There might be many different hops (routers) between the client and the server, and any of them at any time can have problems, routes update, packet congestions and so on.
The quickest way to assess the latency is a ping, but it can be that the firewalls block this.
So the best way to achieve this is to use the anycast
All the major CDN providers implement this method. Some use the TCP anycast, which seems to be not recommended, and others UDP anycast. It is an open debate.
Anyway in order to implement anycast you need to be able to peer with the ISP routers, and normally this is not possible. Additionally there are good peers and bad peers.
Finally All this requires a deep knowledge of the routing protocols and the TCP/IP stack.
A quick and dirty solution could be to use BIND with the GEO-IP patch.
So you can define specific dns query responses per country.
What I mean is that, for instance, if you have a server in UK and one in US you can configure BIND to respond to users coming from europe to hit the UK server and users coming from US to hit the US server.
2) To assign a particular client to a particular server you can use the technique I described on the point 1 or you can use a proxy and sticky sessions.
HA-Proxy is a good product to achieve this. (the URL: xy.1wt.eu )
3) if you use the point 1, you will not have problems with cross domain ajax calls. In fact it is completely transparant for the client. For instance for the same domain example.com a user coming from US will resolve it to 1.1.1.1 whereas a user coming from Germany will resolve example.com to 2.2.2.2 (ip addresses are fake and used just as an example).
On a side note, a solution to do cross domain ajax call is JSON-P which has though some drawbacks, like the lack of support for POST.
If I were you I would go with the BIND and GEO-IP, because it would solve all three problems in once. (a part for the latency because is not always true that the geographically closest server is the one with the lowest latency.)
I have a network of computers connected in form of a graph.
I want to ping from one computer(A) to another computer(B). A and B are connected to each other through many different ways, but I want to PING via only a particular edges only. I have the information of the edges to be followed during pinging available at both A and B.
How should I do this?
You could source route the ping but the return would choose its own path.
Furthermore, source-routed packets are often filtered due to security concerns. (Not always, they are useful and sometimes even required at edge routers.)
If the machines are under your local administrative control, then you could ensure that source-routed packets are permitted. As long as you are able to start a daemon on machine B, you could also easily enough design your own ping protocol that generates source-routed echo returns.
Well, this is actually done by routing protocols that are configured on the media in between the computers (routers I expect). I think there isn't a way where you can say "use that specific route". The routers have different protocols (OSPF, EIGRP, RIPv2) and they do the load balancing. The only way you would be sure of one specific route is to use static routing, but this isn't dynamically done where your computer decides the route.
This is normal because :
if you would be able to chose a route, DoS would be quite easy to do to kill one route.
We're implementing a SIP-based solution and have configured the setup to work with RTPProxy. Right now, we're routing everything through RTPProxy as we were having some issues with media transport relying on ICE. If we're not mistaken, a central relay server is necessary for relaying streaming data between two clients if they're behind symmetric NATs. In practice, is this a large percentage of all consumer users? How much bandwidth woudl we save if we implemented proper routing to skip the relay server when not necessary. Are there better solutions we're missing?
In falling order of usefulness:
There is a direct connection between the two endpoints in both directions. You just connect and you are essentially done.
There is a direct connection between the two endpoints in one direction. In that case you just connect via the right direction by trying both.
Both parties are behind NATs of some kind.
Luckily, UPnP works in one end, you can then upgrade the connection to the above scheme
UPnP doesn't work, but STUN does. Use it to punch a hole in the NAT. There are a couple of different protocols but the general trick is to negotiate via a middle man that coordinates the NAT-piercing.
You fall back to let another node on the network act as a relaying proxy.
If you implement the full list above, then you have to give up very few connections and don't have to spend much time on bandwidth utilization at proxies. The BitTorrent protocol, of which I am somewhat familiar, usually stops at UPnP, but provides a built-in test to test for connectivity through the NAT.
One really wonders why IPv6 did not get implemented earlier - this is a waste of programmers time.
Real world NAT types survey (not a huge dataset, though):
http://nattest.net.in.tum.de/results.php
According to Google, about 8% of the traffic has to be relayed: http://code.google.com/apis/talk/libjingle/important_concepts.html
A large percentage (if not the majority) of home users uses NAT, as that is what those xDSL/cable routers use to provide network access to the local network.
You can theoretically use UPnP to open ports and set-up forwarding rules on the router to go through the NAT transparently. Unfortunately (or fortunately, depending on who you are) many users disable UPnP as a matter of course on their router and may not appreciate having to add forwarding rules manually.
What you might be able to do (and what Skype does AFAIK) is to have some of the users that have clear network paths and enough bandwidth act as relay nodes. Apart from the routing and QoS issues, you would at least have to find some way to ensure the privacy of any relayed data from anyone, including the owner of the relay node. In addition, there might be legal issues to settle with this approach, apart from the technical ones.