We use the URLReferrer and a code passed in on the query string to produce online videos so that only our paid clients can link to our video playback page. This system has worked well for some time. I know the URL referrer can be spoofed, but who would tell their clients to do such a thing to access a video ? It's worked well for us.
However, today I was asked about someone for whom it did not work. The URLReferrer is null, and their site is HTTPS. I have done some reading online and I get the impression there's no way to access the URL referrer when the source page is https. Is this correct ? If I made a https version of our site, would that resolve it ? Or is there any other way for me to get around this ?
Thanks
Your online research is correct. The main reason for not setting an HTTP Referrer header or equivalent is that this could be a security issue. The referrer contains "where you come from", this is private information, and should not be exposed to others, what use is it otherwise to have a secure site if everyone can track where you have been?
So: you cannot get the referrer if the referrer is encrypted (with SSL or otherwise).
Update: here's what the HTTP specification says about coming from a secure site:
Clients SHOULD NOT include a Referer header field in a (non-secure)
HTTP request if the referring page was transferred with a secure
protocol.
As you might have guessed, there's no way around this restriction. Your only option is to use a different verification model. One such method is giving your users a key and require them to send that as a parameter with the request. Several other methods can be thought of.
Related
Just had a quick question: right now when you are doing cross domain tracking with Analytics, the URL has parameters added to them to track it. I was just wondering, is there a way to use something like POST or any other method to pass the cookie info from one domain to the next so the URL does not seem so messy with all the URL parameters being added.
E.g. going from google.com to yahoo.com/lots_of_paramters, could we go to just yahoo.com/gclid=123
Cheers,
Yuri
The parameters are read on the other domain to recreate the same Google Analytics cookies.
Data sent on POST is not available on the posted page through Javascript only on the back end. So you would need the backend to actually store this data and send back to the interface, maybe through AJAX to recreate the cookies there.
Another idea is probably to try to parse the cookies server side and send them as new cookies, so the Javascript code on the destination domain wouldn't need to recreate the cookies, they would be normal HTTP Header set cookies.
Still it's a lot of work and fuzz just to pass the cookies on. Urls are ugly but are still the easiest and portable way to handle the problem.
I am successfully able to authenticate Facebook and Google accounts using my Oauth2 servlets. I am using state with a timer and a session cookie to try to verify that it is indeed a legitimate Oauth callback.
Is there any benefit if I also examine the HTTP Referer header to ensure that I was redirected from the provider's OAuth page?
If no benefit, could there be a problem if I also examine the HTTP Referer field?
No.
I can simulate any headers I want as a malicious attacker. I can make it look like I'm coming from http://cia.fbi.gov.vpn/uber1337h4x. This is obvious and well known.
Any pages coming from HTTPS do not send a refer header as per RFC2616 sec15:
Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.
Breaks usability as per RFC2616 sec15:
Because the source of a link might be private information or might reveal an otherwise private information source, it is strongly recommended that the user be able to select whether or not the Referer field is sent.
In short, you are not given greater security. Your security is not in inspecting a vastly insecure transport protocol, it's in the OAuth layer. You also break usability.
Don't do it.
The answer is:
No, you shouldn't use it, and there is NO valuable benefit of doing it.
Authorization Servers are very aware of this also. And here was stated.
From the mailing list of OAuth-WG:
Callback URL pages SHOULD redirect to a trusted page immediately after receiving the authorization code in the URL. This prevents the authorization code from remaining in the browser history, or from inadvertently leaking in a referer header.
If you are worry about CSRF, you SHOULD NOT use the HTTP Referer as a technique to verify the origin of an authorization, that's why the parameter state is (which sound you're using).
If you worry about an specific security concern of the oauth2 protocol, there is a full section inside the draft.
If you worry about other Security Considerations, this is the source.
I suggest you give all your effort implementing all the validations around the param: state.
Edit:
After reading the nuances of the question, you are really answered your own question. The use of cookies (probably HTML5 local storage) for both cases, is the best solution we know so far.
The first nuance is about CSRF and one of the possible countermeasures available is Checking the HTTP Referer header, and this was already addressed in the protocol.
The second nuance, I'm not completly sure, but is probably a case of Extension Grant, this is because it sounds that you may work as an "auth proxy requester", same as SAML oauth2 extension.
Don't verify the HTTP referer; the "state" parameter (which it sounds you're using) is the approach OAuth 2.0 defines to defend against cross-site request forgery (CSRF) attacks.
You may want to have a look at the new O'Reilly book Getting Started with OAuth 2.0 by Ryan Boyd. It describes this and related security considerations.
Plain security was not a concern of the question because the state parameter is being used.
The main concerns I had in mind were:
Whether it is the same browser that my app sent to Facebook that's coming back to present a candidate token?
Whether the agent (browser-like agent) or agents are repeatedly doing OAuth requests and presenting me with bad OAuth tokens that cause my app to repeatedly contact Facebook with bad tokens leading to potentially adverse treatment by Facebook.
The only possible solution to the first problem is to also set a cookie in addition to using state. referer would help if most providers weren't using https.
The second problem has a nuance. The mis-behaving agents need not be directly controlled by a malicious entity. They may be normal users browsers redirected via some indirect means (a popular hijacked website, social engineering).
Because of the nuance there is a chance that the referer header may not be forged. However, https precludes any meaningful benefit.
Cookies definitely help in the second case also because if you are setting cookies in a POST no third-party website can cause them to be set and you cannot be flooded with bad OAuth responses due to hacked websites redirecting users en masse to OAuth you.
This is not a clear answer (or question) but hopefully this shows the nuances behind the question.
I was recently looking around at some of the features my current web host offers, and am now wondering about a few things. Even if you can only answer part of this, I appreciate any help you can provide.
I have a domain, mydomian.com, and the host offers shared SSL so I can use HTTPS by using this address https://mydomain.myhost.com. The SSL certificate is good for *.myhost.com.
I don't know a lot about SSL, but I'm assuming this means that the data between site users and ANY domain on myhost.com is encrypted. So was curious if this meant that if someone else on the same host as me somehow intercepted the data from my site would they be able to view it, since they would also have a https://theirdomain.myhost.com address, which uses the same SSL certificate? I may have no idea at all, and this was pretty much a guess.
If HTTPS is used on a login page, but after logging in the other pages are viewed over HTTP, is this a security issue?
Is there any way to show a web form via HTTP for bots like Google, but have real users redirected to the HTTPS version? Would be ideal if this could be done via .htaccess. I currently have some rewrite rules that redirect certain pages to HTTPS, but the rest as HTTP. So if a visitor visits the contact form they get the HTTPS version automatically, but it automatically switches back to HTTP for pages that don't contain forms. So, via htaccess, is there a way to direct real users to the HTTPS version, but have bots directed to the HTTP version? I would like these pages to still be indexed by the search engines, but would like users to see it via HTTPS.
Thanks in advance for any help you can provide.
I'm going to guess you'll be okay for number one. If your host does it correctly, individual subdomains never get to see the SSL keys. Here's how it would work:
Some guy with a browser sends an encrypted request to your subdomain server.
Your host's master server receives the request and decrypts it.
The master server sends the decrypted request to your subdomain server.
And any HTTPS responses you send back go through that process in reverse. It should be easy to check if they've set things up that way: If you can set up shared SSL without personally handling any key files, you're good. If you actually get your hands on some key files... not good.
For two: If you encrypt the login, you protect the passwords, which is good. But if you switch back to HTTP afterwards, you open yourself up to other attacks. See: Firesheep. There may be others.
And for three. Yes - definitely doable. Check out mod_rewrite. Can't give you an example, as I've never used this particular case, but I can point you to this page - particularly the section entitled "Browser Dependent Content."
Hope that helps!
Every traffic is encrypted, when you use https:// as protocol. (Except for some uncommon circumstances I won't talk about here). An SSL certificate's purpose is to prove the identity of the server, by combining it's public key with an identity. This certificate is only usable with the private key that belongs to the public one. In your case it seems that this certificate as well as the key-pair is provided by your hosting provider. I guess that neither you nor the other customers on the host have access to this private key. That means that only your provider is able to decrypt the traffic. Since that's always the case (he's running the server, so has access to every data), that should be no problem.
In most cases it is a security issue. On every further unencrypted http-request the client has to provide some information of the session to the server. These can be intercepted and used by an attacker. (simply speaking)
The bots should support https, why not redirect them? Anyhow: The important part is not to provide the page containing the form via https. To protect your user's data you should take care that the response is transferred via https.
I have scenario where My webpage is requested by another website. That website will have Hyperlink to my webpage.
I Need to check whether request is coming from valid website or not. I have done this by checking URL Referer of that website, and working fine.
Another way to validate this request is to validate client certificate(x.509).
I want to know which is the best/secure way to validate referer website? Is there any other way to validate referer site excepting url referer and certificate validation?
Thanks
Fenil
The client certificate would identify the person clicking on the link, but not the referring page, so it should be ruled out.
As for the referrer, it does work, but with a couple of caveats:
1 - it's not secure (for big values of "secure"). The http_referrer is an optional field that the browser inserts in the request for your site. So it's controlled at the client's side and can be easily forged. So, if the level of security you want is "make sure that somebody has not posted my link on another page, where unaware users may click on it" then you're quite fine with checking the referrer. If you're relying on this for anything more (like making sure the incoming person is authorized to do something on your site) then you probably want a form of user authentication
2- some software that may be installed on your users' computers (like "Norton Internet Security") masks the http_referrer out of privacy concerns, so some of your users may not have a http_referrer.
I have a directory with my media files and I need no to display them on other sites.
Server doesn't support .htaccess, because it uses nginx.
How can I enable hotlink protection for my files??
Thank you.
Easiest way would be to check for the Referer header in HTTP request. Basically if that header does not have URL from your site, then this could be hot linking.
This has following problems:
Referrer header can be forged -> hot linking works
All user agents do not necessarily send the Referrer header -> legitimate user might not get the content.
You could also set a cookie when user is browsing your site, and check for existence of that cookie when user is accessing the streaming content.
The details may be dated, but Igor gives an example of referrer mapping for image hotlink protection that might be useful here: http://nginx.org/pipermail/nginx/2007-June/001082.html
If you decide to go the referrer route.
If you are using memcached you could also store store client IP addresses for a time and only serve up your streaming media if an unexpired client IP is found in the cache. The client IP gets cached during normal browsing ensuring that the person viewing your streaming content has also recently been visiting your site.
On my hostgator site, they used nginx as a proxy to Apache(nginx+apache). maybe that will help you. Also if you have access to the logs, if you see a lot of traffic that way from a ip I would investigate, and if it points to a site, then block the other web server. Php's file_get_contents doesn't get stopped by htaccess or anything else I know besides blocking the ip.