I've tried using an x-forwarded-for header with IP addresses from Mexico, London, etc... I've used the google sensor tool to change my IP but Marklogic somehow sees through those techniques when I log the results of this function xdmp:get-request-client-address.
Because xdmp:get-request-client-address() returns
the internet address of the client from which the HTTP server request
is issued
If you want the value of the x-forwarded-for header, then use xdmp:get-request-header(). You can use the request client address as a default value in case that header is not found:
xdmp:get-request-header("x-forwarded-for", xdmp:get-request-client-address())
Related
I am trying to get the IPV4 address of the user trying to access a web server using the request object. Which header stores the IPV4 address of the requester?
I tried these headers but all have blank values.
HTTP_X_FORWARDED_FOR
Remote_Addr
HTTP_CLIENT
Try this:
HTTP_CLIENT_IP
HTTP_X_FORWARDED
HTTP_FORWARDED
I tried an application, and I used a way to ban those who send more than 5 empty requests to the server, but the problem then, is that everybody got blocked, and this is because everybody was seen as a ONE UNIQUE IP.
In the code, I used the way to get the X-Real-IP but it doesent work on OpenShift, so how to do that then?
Here is how I get the IP:
x_real_ip = self.request.headers.get("X-Real-IP")
remote_ip = self.request.remote_ip if not x_real_ip else x_real_ip
Update: I get '127.3.165.129', None) when doing print(self.request.remote_ip, x_real_ip)
You want to look for the "x-forwarded-for" header to get the visitors ip address. What you are seeing is the ip address of the reverse proxy that users go through before ending up at your application/gear.
You can refer to this article in the Developer Center for more information about how requests are routed on OpenShift: https://developers.openshift.com/en/managing-port-binding-routing.html
I've found some interesting reading on the X-Forwarded-* headers, including the Reverse Proxy Request Headers section in the Apache documentation, as well as the Wikipedia article on X-Forwarded-For.
I understand that:
X-Forwarded-For gives the address of the client which connected to the proxy
X-Forwarded-Port gives the port the client connected to on the proxy (e.g. 80 or 443)
X-Forwarded-Proto gives the protocol the client used to connect to the proxy (http or https)
X-Forwarded-Host gives the content of the Host header the client sent to the proxy.
These all make sense.
However, I still can't figure out a real life use case of X-Forwarded-Host. I understand the need to repeat the connection on a different port or using a different scheme, but why would a proxy server ever change the Host header when repeating the request to the target server?
If you use a front-end service like Apigee as the front-end to your APIs, you will need something like X-FORWARDED-HOST to understand what hostname was used to connect to the API, because Apigee gets configured with whatever your backend DNS is, nginx and your app stack only see the Host header as your backend DNS name, not the hostname that was called in the first place.
This is the scenario I worked on today:
Users access certain application server using "https://neaturl.company.com" URL which is pointing to Reverse Proxy. Proxy then terminates SSL and redirects users' requests to the actual application server which has URL of "http://192.168.1.1:5555". The problem is - when application server needed to redirect user to other page on the same server using absolute path, it was using latter URL and users don't have access to this. Using X-Forwarded-Host (+ X-Forwarded-Proto and X-Forwarded-Port) allowed our proxy to tell application server which URL user used originally and thus server started to generate correct absolute path in its responses.
In this case there was no option to stop application server to generate absolute URLs nor configure it for "public url" manually.
I can tell you a real life issue, I had an issue using an IBM portal.
In my case the problem was that the IBM portal has a rest service which retrieves an url for a resource, something like:
{"url":"http://internal.host.name/path"}
What happened?
Simple, when you enter from intranet everything works fine because internalHostName exists but... when the user enter from internet then the proxy is not able to resolve the host name and the portal crashes.
The fix for the IBM portal was to read the X-FORWARDED-HOST header and then change the response to something like:
{"url":"http://internet.host.name/path"}
See that I put internet and not internal in the second response.
For the need for 'x-forwarded-host', I can think of a virtual hosting scenario where there are several internal hosts (internal network) and a reverse proxy sitting in between those hosts and the internet. If the requested host is part of the internal network, the requested host resolves to the reverse proxy IP and the web browser sends the request to the reverse proxy. This reverse proxy finds the appropriate internal host and forwards the request sent by the client to this host. In doing so, the reverse proxy changes the host field to match the internal host and sets the x-forward-host to the actual host requested by the client. More details on reverse proxy can be found in this wikipedia page http://en.wikipedia.org/wiki/Reverse_proxy.
Check this post for details on x-forwarded-for header and a simple demo python script that shows how a web-server can detect the use of a proxy server: x-forwarded-for explained
One example could be a proxy that blocks certain hosts and redirects them to an external block page. In fact, I’m almost certain my school filter does this…
(And the reason they might not just pass on the original Host as Host is because some servers [Nginx?] reject any traffic to the wrong Host.)
X-Forwarded-Host just saved my life. CDNs (or reverse proxy if you'd like to go down to "trees") determine which origin to use by Host header a user comes to them with. Thus, a CDN can't use the same Host header to contact the origin - otherwise, the CDN would go to itself in a loop rather than going to the origin. Thus, the CDN uses either IP address or some dummy FQDN as the Host header fetching content from the origin. Now, the origin may wish to know what was the Host header (aka website name) the content is asked for. In my case, one origin served 2 websites.
Another scenario, you license your app to a host URL then you want to load balance across n > 1 servers.
On any Heroku stack, I want to get the client's IP. my first attempt might be:
request.headers['REMOTE_ADDR']
This does not work, of course, because all requests are passed through proxies. So the alternative was to use:
request.headers['X-Forwarded-For']
But this is not quite safe, is it?
If it contains only one value, I take this. If it contains more than one value (comma-separated), I could take the first one.
But what if someone manipulates this value? I cannot trust request.headers['X-Forwarded-For'] as I could with request.headers['REMOTE_ADDR']. And there is no list of trusted proxies that I could use, either.
But there must be some way to reliably get the client's IP address, always. Do you know one?
In their docs, Heroku describes that X-Forwarded-For is "the originating IP address of the client connecting to the Heroku router".
This sounds as if Heroku could be overwriting the X-Forwarded-For with the originating remote IP. This would prevent spoofing, right? Can someone verify this?
From Jacob, Heroku's Director of Security at the time:
The router doesn't overwrite X-Forwarded-For, but it does guarantee that the real origin will always be the last item in the list.
This means that, if you access a Heroku app in the normal way, you will just see your IP address in the X-Forwarded-For header:
$ curl http://httpbin.org/ip
{
"origin": "123.124.125.126",
}
If you try to spoof the IP, your alleged origin is reflected, but - critically - so is your real IP. Obviously, this is all we need, so there's a clear and secure solution for getting the client's IP address on Heroku:
$ curl -H"X-Forwarded-For: 8.8.8.8" http://httpbin.org/ip
{
"origin": "8.8.8.8, 123.124.125.126"
}
This is just the opposite of what is described on Wikipedia, by the way.
PHP implementation:
function getIpAddress() {
if (isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
$ipAddresses = explode(',', $_SERVER['HTTP_X_FORWARDED_FOR']);
return trim(end($ipAddresses));
}
else {
return $_SERVER['REMOTE_ADDR'];
}
}
I work in Heroku's support department and have spent some time discussing this with our routing engineers. I wanted to post some additional information to clarify some things about what's going on here.
The example provided in the answer above just had the client IP displayed last coincidentally and that's not really guaranteed. The reason it wasn't first is because the originating request claimed that it was forwarding for the IP specified in the X-Forwarded-For header. When the Heroku router received the request, it just appended the IP that was directly connecting to the X-Forwarded-For list after the one that had been injected into the request. Our router always adds the IP that connected to the AWS ELB in front of our platform as the last IP in the list. This IP could be the original one (and in the case where there's only one IP, it almost certainly is), but the instant there are multiple IPs chained, all bets are off. Convention is always to add the latest IP in the chain to the end of the list (which is what we do), but at any point along the chain that chain can be altered and different IPs could be inserted. As such, the only IP that's reliable (from the perspective of our platform) is the last IP in the list.
To illustrate, let's say someone initiates a request and arbitrarily adds 3 additional IPs to the X-Forwarded-For header:
curl -H "X-Forwarded-For: 12.12.12.12,15.15.15.15,4.4.4.4" http://www.google.com
Imagine this machine's IP was 9.9.9.9 and that it had to pass through a proxy (e.g., a university's campus-wide proxy). Let's say that proxy had an IP of 2.2.2.2. Assuming it wasn't configured to strip X-Forwarded-For headers (which it likely wouldn't be), it would just tack the 9.9.9.9 IP to the end of the list and pass the request on to Google. At this point, the header would look like this:
X-Forwarded-For: 12.12.12.12,15.15.15.15,4.4.4.4,9.9.9.9
That request will then pass through Google's endpoint, which will append the university proxy's IP of 2.2.2.2, so the header will finally look like this in Google's logs:
X-Forwarded-For: 12.12.12.12,15.15.15.15,4.4.4.4,9.9.9.9,2.2.2.2
So, which is the client IP? It's impossible to say from Google's standpoint. In reality, the client IP is 9.9.9.9. The last IP listed is 2.2.2.2 though and the first is 12.12.12.12. All Google would know is that the 2.2.2.2 IP is definitely correct because that was the IP that actually connected to their service – but they wouldn't know if that was the initial client for the request or not from the data available. In the same way, when there's just one IP in this header – that is the IP that directly connected to our service, so we know it's reliable.
From a practical standpoint, this IP will likely be reliable most of the time (because most people won't be bothering to spoof their IP). Unfortunately, it's impossible to prevent this sort of spoofing and by the time a request gets to the Heroku router, it's impossible for us to tell if IPs in an X-Forwarded-For chain have been tampered with or not.
All reliability issues aside, these IP chains should always be read from left-to-right. The client IP should always be the left-most IP.
You can never really trust any information coming from the client. It's more of a question of who do you trust and how do you verify it. Even Heroku can possibly be influenced to provide a bad HTTP_X_FORWARDED_FOR value if they have a bug in their code, or they get hacked somehow. Another option would be some other Heroku machine connecting to your server internally and bypassing their proxy altogether while faking REMOTE_ADDR and/or HTTP_X_FORWARDED_FOR.
The best answer here would depend on what you're trying to do. If you're trying to verify your clients, a client-side certificate might be a more appropriate solution. If all you need the IP for is geo-location, trusting the input might be good enough. Worst case, someone will fake the location and get the wrong content... If you have a different use case, there are many other solutions in between those two extremes.
If I make a request with multiple X-Forwarded-For headers: curl -s -v -H "X-Forwarded-For: 1.1.1.1, 1.1.1.2, 1.1.1.3" -H "X-Forwarded-For: 2.2.2.2" -H "X-Forwarded-For: 3.3.3.3" https://foo.herokuapp.com/
> X-Forwarded-For: 1.1.1.1, 1.1.1.2, 1.1.1.3
> X-Forwarded-For: 2.2.2.2
> X-Forwarded-For: 3.3.3.3
The X-Forwarded-For header passed along to the app will be:
1.1.1.1, 1.1.1.2, 1.1.1.3, <real client IP>, 2.2.2.2, 3.3.3.3
so picking the last from that list does not hold up :/
I need to log the user ip ady's for every request to our JBoss Netty server. I thought:
MessageEvent e;
e.getChannel().getRemoteAddress();
was the correct answer, but this always returns 127.0.0.1 and I need the actual client ip. Coming from Rails I checked how they find out the ip, from the docu:
Determines originating IP address.
REMOTE_ADDR is the standard but will
fail if the user is behind a proxy.
HTTP_CLIENT_IP and/or
HTTP_X_FORWARDED_FOR are set by
proxies so check for these if
REMOTE_ADDR is a proxy.
HTTP_X_FORWARDED_FOR may be a comma-
delimited list in the case of multiple
chained proxies; the last address
which is not trusted is the
originating IP.
So should I check for all the headers in Netty or is there an easier way?
Ok I have the answer. Using ChannelHandlerContext instead of MessageEvent does the trick.
SocketAddress remoteAddress = ctx.getChannel().getRemoteAddress();