I want to deny users access to my site if they have made X requests in Y milliseconds. According to Microsoft I can use dynamic ip security in the web config.
This is how my config, that I use for testing the ip throttle, looks:
<security>
<dynamicIpSecurity>
<denyByRequestRate enabled="true" maxRequests="2" requestIntervalInMilliseconds="10000"></denyByRequestRate>
</dynamicIpSecurity>
</security>
Now to my problem: Since I'm using Cloudflare I won't see the real IP of my visitors. According to Cloudflare, they provide the real IP in a couple of headers. Although, it does not seem that Azure looks at these headers when checking if the IP should be denied.
My question is: Is there some way I can still use the web config way of denying requests by IP or do I need to use a code solution instead?
I think that I found the answer to my own question. I'll post my findings here for people that might have the same problem.
According to this there is an attribute called "enableProxyMode" which
Enables IIS not only to block requests from a client IP that is seen
by IIS, but also to block requests from IP addresses that are received
in the x-forwarded-for HTTP header.
Since Cloudflare puts the real IP of the visitor in the X-Forwarded-For HTTP header it works perfectly.
Related
Let's say I have this DNS entry: mysite.sample. I am developing, and have a copy of my website running locally in http://localhost:8080. I want this website to be reachable using the (fake) DNS: http://mysite.sample, without being forced to remember in what port this site is running. I can setup /etc/hosts and nginx to do proxing for that, but ... Is there an easier way?
Can I somehow setup a simple DNS entry using /etc/hosts and/or dnsmasq where also a non-standard port (something different than :80/:443) is specified? Without the need to provide extra configuration for nginx?
Or phrased in a simpler way: Is it possible to provide port mappings for dns entries in /etc/hosts or dnsmasq?
DNS has nothing to do with the TCP port. DNS is there to resolv names (e.g. mysite.sample) into IP addresses - kind of like a phone book.
So it's a clear "NO". However, there's another solution and I try to explain it.
When you enter http://mysite.sample:8080 in your browser URL bar, your client (e.g. browser) will first try to resolve mysite.sample (via OS calls) to an IP address. This is where DNS kicks in, as DNS is your name resolver. If that happened, the job of DNS is finished and the browser continues.
This is where the "magic" in HTTP happens. The browser is connecting to the resolved IP address and the desired port (by default 80 for http and 443 for https), is waiting for the connection to be accepted and is then sending the following headers:
GET <resource> HTTP/1.1
Host: mysite.sample:8080
Now the server reads those headers and acts accordingly. Most modern web servers have something called "virtual hosts" (i.e. Apache) or "sites" (i.e. nginx). You can configure multiple vhosts/sites - one for each domain. The web server will then provide the site matching the requested host (which is retreived by the browser from the URL bar and passed to the server via Host HTTP header). This is pure HTTP and has nothing to do with TCP.
If you can't change the port of your origin service (in your case 8080), you might want to setup a new web server in front of your service. This is also called reverse proxy. I recommend reading the NGINX Reverse Proxy docs, but you can also use Apache or any other modern web server.
For nginx, just setup a new site and redirect it to your service:
location mysite.example {
proxy_pass http://127.0.0.1:8080;
}
There is a mechanism in DNS for discovering the ports that a service uses, it is called the Service Record (SRV) which has the form
_service._proto.name. TTL class SRV priority weight port target.
However, to make use of this record you would need to have an application that referenced that record prior to making the call. As Dominique has said, this is not the way HTTP works.
I have written a previous answer that explains some of the background to this, and why HTTP isn't in the standard. (the article discusses WS, but the underlying discussion suggested adding this to the HTTP protocol directly)
Edited to add -
There was actually a draft IETF document exploring an official way to do this, but it never made it past draft stage.
This document specifies a new URI scheme called http+srv which uses a DNS SRV lookup to locate a HTTP server.
There is an specific SO answer here which points to an interesting post here
I have a web server cluster behind a proxy/load balancer. That proxy contains my SSL certs and hands the web servers the decrypted traffic, and along the way adds an "x-forwarded-for" header into the HTTP header the web application receives. This application has seen millions of IP addresses over the past decade, but something weird happened today.
For the first time, I saw an x-forwarded-for that contained a second address reach the application [addressed altered]:
x-forwarded-for: 62.211.19.218, 177.168.159.85
This indicates that the traffic came through a proxy, and I understand this is normal for x-f-f. I would have thought this was impossible (or at least unlikely) with https as the protocol.
Can someone explain how this is legit?
As per RFC 7239, this HTTP header is specified as
X-Forwarded-For: client, proxy1, proxy2, ...
Where client is the IP of the original client and then each proxy adds the IP it received the request from, at the end of the list. In the above example, you would see IP of proxy3 in your webserver and proxy2 is the IP which connected to the proxy3.
As anyone can put anything inside this header, you should accept it only from known sources like your own reverse proxy or whitelist of known legit proxies. For example Apache has mod_rpaf, which transparently changes client IP address to the one provided in this header, but only if the request is received from the IP of known proxy server.
On corporate networks you can easily do transparent proxying for HTTPS traffic without any notice from normal users. Just create your own certification authority, use for example Windows Group Policy to install & trust this CA on all corporate workstations. Then redirect all HTTPS connections to your proxy which will generate certificate for all visited domains on the fly. This is something which is happening and you can even buy enterprise hardware proxies using this method.
So to summarize the reasons why you could see multiple IPs in the X-Forwarded-For header:
Transparent HTTPS proxy as mentioned above
The header was added by the requestor itself (browser, wget, script) for whatever reason, for example to hide its own IP
Some CDN like Cloudflare could add that header if used
Multiple reverse proxies defined either intentionally or by mistake
Conclusion: You should only trust this header if it originates from your own proxy (in case of multiple IPs, trust only the last one).
MAYBE it's using the Proxy protocol for HTTPS. Granted you may not be using httproxy, but this seems to be a decent description:
http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt
I'm not sure about the SSL cert, but there's no guarantee someone is doing something pathalogical (maybe unintentionally) like running all their HTTPS traffic through a proxy and then accepting all the invalid certificates. But I suspect the proxy protocol might make this work; it does expose the HTTP headers to the proxy in some sense.
I'm trying to use nginx behind a Compute Engine http load balancer. I would like to allow Health Check requests to come through a port unauthorized and all other requests to be authorized with basic auth.
The Health Check requests come from IP block: 130.211.0.0/22. If I see requests coming from this IP block with no X-forwarded-for header, then it is a health check from the load balancer.
I'm confused on how to set this up with nginx.
Have you tried using Nginx header modules? Googling around I found these:
HttpHeadersMoreModule
Headers
There's also a similar question here.
Alternative. In the past I worked with a software (RT), which had thought of this possibility in the software itself, providing a subdirectory for unauthorized access (/noauth/). Maybe your software might have the same, and you could configure GCE health check to point to something like /noauth/mycheck.html.
Please remember that headers can be easily forged, so an attacker who knows your vulnerability could access your server without auth.
Scenario: I want a staging environment at a customer's site. The customer owns www.example.com. I want to map the site to staging.example.com reachable from the outside, but I haven't got time to wait for the bureaucracy surrounding either the purchase of the new subdomain or opening of secondary HTTP ports.
Assumption: If I spoof the HTTP Header param Host to be staging.example.com on the client side, but actually make the request to the IP of www.example.com, IIS will redirect the request to the configured site for staging.example.com. Am I right?
So is there any client tool that can help me with that? I'm fairly famailiar with Fiddler, but it seem to override my rewrites of the host parameter. Also I would need to configure it to do it for every request, not just one, to make it trivial to test.
Are there simpler solutions to this problem?
I'm not entirely sure what you're asking.
Inside Fiddler, by clicking Tools > HOSTS and you can send all traffic targeting one site, e.g. dev.example.com to the IP of your choice. The target site (namely dev.example.com) doesn't need to exist at all in this case. Your client (e.g. the browser) has no idea that Fiddler is retargeting the traffic, it just thinks that it is talking to dev.example.com.
If you have the Fiddler book, check out the Retargeting Traffic section for many other ways to retarget traffic.
I simply want both example.com and www.example.com to go to the same website. Currently we have just added a binding to each. This creates a problem for WCF which then breaks because it says it can only accept one http address.
Is there a better way to configure a site to accept addresses with or without 'www'?
this has to be changed in the dns using a host or a record.
You shouldn't be hosting a site on the root domain. Better approach here is to redirect traffic to the root (.example.com) to the correct site (www.example.com) using an HTTP 302.
The WCF service should only receive traffic on its single binding. Use DNS, HTTP redirection, or some Traffic management application/appliance to get traffic from multiple points to the correct one.