I want to prevent users by spoofing referrer header to access my API services.
In other words, I want to allow API being called only from particular domain (i.e. www.abcd.ef).
Documentation states:
The Referer request header contains the address of the previous web
page from which a link to the currently requested page was followed.
The Referer header allows servers to identify where people are
visiting them from and may use that data for analytics, logging, or
optimized caching, for example.
Is there any way I can make sure that AJAX (javascript) call comes from certain domain, even though it is possible to edit http referer header field to act like you come from www.abcd.ef?
Related
We have a third party service that implemented single-sign-on on some route to it’s web-app.
In order to use this single-sign-on, we provided with a POST API, and we need to pass on that route some credentials- including an organization secret that we got from him (yes, we need to pass it in query params), and the user’s email address.
In order to not to expose credentials over the browser, we tried to mimic that request by creating our own backend endpoint, and return the same result as their enpoint (kind of a proxy) with redirection (status 307 with Location header) cause there API returns plain HTML.
It seems like when client send a post request with redirection he can’t add the authorization header (JWT) required by our backend to operate (backend return Authorization required). the request is been done to our server and the result is redirection to another server.
How can we bypass it? Or maybe we can secure it with different approach?
I know that the header is usually been removed for a good reason, who knows where I can be redirected to?
but can't I tell him somehow that I trust the redirection?
We tried to use phantom-form only to use from submit post. We tried to make our endpoint to be GET and making the redirection other way (react-router) but I think that natively js does not allow to attach Authorization header.
I thought it would be a common pattern, but I didn't find someone that talks about it exactly, since we are trying to query our backend and not some external redirection API.
When a user clicks a link to an external site from my site, is my sites URL available to the external site? How about if my site redirects the user to the external site?
Also what about the site prior to mine? If the previous URL is available, is the one one step back also available?
There is Referer in Request header: Link
The Referer request header contains the address of the previous web page from which a link to the currently requested page was followed. The Referer header allows servers to identify where people are visiting them from and may use that data for analytics, logging, or optimized caching, for example.
Let's say you are on a site say google, and you clicked on the link of your site:
This is what you will receive in Request Headers
**Referer**: https://www.google.com/
And if you click on https://www.apple.com from google, and apple sites redirect you to let say https://www.apple.com/lae/ then on Request header's Referer you will receive:
**Referer**: https://www.apple.com
or, you can also receive google.com and others, it depends
NOTE: The HTTP referer header can be spoofed So it's not fixed what you received!
I own a url shortening service and I want to detect whether the request that I received was from a web crawler or not. In response to the request, I send a HTTP header 302 that redirects the requester to the original link. I was thinking that I could provide an invisible link with the response, so that a bot would send me a request for that page too but a normal user wont. This is based on the hypothesis that even if bots read the header and redirects, they still scans the page and send requests to the links found in it. Is the hypothesis correct? If it is not, I could also redirect them via Javascript but that would not be the standard way of redirecting(I suppose).
Yes, crawlers definitely follow redirects. Their purpose is to find as many pages (or content) as possible. Following redirects is a basic requirement for that goal. However, I do not know if commercial crawlers read the body of a redirect response. I think they don't since information displayed on a redirect page is never shown to a user since they are always redirected away from that page.
There are other crawlers like Crawljax that are build for testing web applications. They will read all the data but those crawlers aren't (or shouldn't be) used to crawl the public web.
I was using Fiddler see on-the-field how web sites use cookies in their login systems. Although I have some HTTP knowledge, I'm just just learning about cookies and how they are used within sites.
Initially I assumed that when submitting the form I'd see no cookies sent, and that the response would contain some cookie info that would then be saved by the browser.
In fact, just the opposite seems to be the case. It is the request that's sending in info, and the server returns nothing.
When fiddling about the issue, I noticed that even with a browser cleaned of cookies, the client seems to always be sending a RequestVerificationToken to the server, even when just looking around withot being signed in.
Why is this so?
Thanks
Cookies are set by the server with the Set-Cookie HTTP response header, and they can also be set through JavaScript.
A cookie has a path. If the path of a cookie matches the path of the document that is being requested, then the browser will include all such cookies in the Cookie HTTP request header.
You must make sure to be careful when setting or modifying cookies in order to avoid XSS attacks against your users. As such, it might be useful to include a hidden and unique secret within your login forms, and use such secret prior to setting any cookies. Alternatively, you can simply check that HTTP Referer header matches your site. Otherwise, a malicious site can copy your form fields, and create a login form to your site on their site, and do form.submit(), effectively logging out your user, or performing a brute-force attack on your site through unsuspecting users that happen to be visiting the malicious web-site.
The RequestVerificationToken that you mention has nothing to do with HTTP Cookies, it sounds like an implementation detail that some sites written in some specific site-scripting language use to protect their cookie-setting-pages against XSS attacks.
When you hit a page on a website, usually the response(the page that you landed on) contains instructions from the server in the http response to set some cookies.
Websites may use these to track information about your behavior or save your preferences for future or short term.
Website may do so on your first visit to any page or on you visit to a particular page.
The browser would then send all cookies that have been set with subsequent request to that domain.
Think about it, HTTP is stateless. You landed on Home Page and clicked set by background to blue. Then you went to a gallery page. The next request goes to your server but the server does not have any idea about your background color preference.
Now if the request contained a cookie telling the server about your preference, the website would serve you your right preference.
Now this is one way. Another way is a session. Think of cookies as information stored on client side. But what if server needs to store some temporary info about you on server side. Info that is maybe too sensitive to be exposed in cookies, which are local and easily intercepted.
Now you would ask, but HTTP is stateless. Correct. But Server could keep info about you in a map, whose is the session id. this session id is set on the client side as a cookie or resent with every request in parameters. Now server is only getting the key but can lookup information about you, like whether you are logged in successfully, what is your role in the system etc.
Wow, that a lot of text, but I hope it helped. If not feel free to ask more.
I am successfully able to authenticate Facebook and Google accounts using my Oauth2 servlets. I am using state with a timer and a session cookie to try to verify that it is indeed a legitimate Oauth callback.
Is there any benefit if I also examine the HTTP Referer header to ensure that I was redirected from the provider's OAuth page?
If no benefit, could there be a problem if I also examine the HTTP Referer field?
No.
I can simulate any headers I want as a malicious attacker. I can make it look like I'm coming from http://cia.fbi.gov.vpn/uber1337h4x. This is obvious and well known.
Any pages coming from HTTPS do not send a refer header as per RFC2616 sec15:
Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.
Breaks usability as per RFC2616 sec15:
Because the source of a link might be private information or might reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent.
In short, you are not given greater security. Your security is not in inspecting a vastly insecure transport protocol, it's in the OAuth layer. You also break usability.
Don't do it.
The answer is:
No, you shouldn't use it, and there is NO valuable benefit of doing it.
Authorization Servers are very aware of this also. And here was stated.
From the mailing list of OAuth-WG:
Callback URL pages SHOULD redirect to a trusted page immediately after receiving the authorization code in the URL. This prevents the authorization code from remaining in the browser history, or from inadvertently leaking in a referer header.
If you are worry about CSRF, you SHOULD NOT use the HTTP Referer as a technique to verify the origin of an authorization, that's why the parameter state is (which sound you're using).
If you worry about an specific security concern of the oauth2 protocol, there is a full section inside the draft.
If you worry about other Security Considerations, this is the source.
I suggest you give all your effort implementing all the validations around the param: state.
Edit:
After reading the nuances of the question, you are really answered your own question. The use of cookies (probably HTML5 local storage) for both cases, is the best solution we know so far.
The first nuance is about CSRF and one of the possible countermeasures available is Checking the HTTP Referer header, and this was already addressed in the protocol.
The second nuance, I'm not completly sure, but is probably a case of Extension Grant, this is because it sounds that you may work as an "auth proxy requester", same as SAML oauth2 extension.
Don't verify the HTTP referer; the "state" parameter (which it sounds you're using) is the approach OAuth 2.0 defines to defend against cross-site request forgery (CSRF) attacks.
You may want to have a look at the new O'Reilly book Getting Started with OAuth 2.0 by Ryan Boyd. It describes this and related security considerations.
Plain security was not a concern of the question because the state parameter is being used.
The main concerns I had in mind were:
Whether it is the same browser that my app sent to Facebook that's coming back to present a candidate token?
Whether the agent (browser-like agent) or agents are repeatedly doing OAuth requests and presenting me with bad OAuth tokens that cause my app to repeatedly contact Facebook with bad tokens leading to potentially adverse treatment by Facebook.
The only possible solution to the first problem is to also set a cookie in addition to using state. referer would help if most providers weren't using https.
The second problem has a nuance. The mis-behaving agents need not be directly controlled by a malicious entity. They may be normal users browsers redirected via some indirect means (a popular hijacked website, social engineering).
Because of the nuance there is a chance that the referer header may not be forged. However, https precludes any meaningful benefit.
Cookies definitely help in the second case also because if you are setting cookies in a POST no third-party website can cause them to be set and you cannot be flooded with bad OAuth responses due to hacked websites redirecting users en masse to OAuth you.
This is not a clear answer (or question) but hopefully this shows the nuances behind the question.