Ok, so I get the rule that browsers should not confuse history store and web cache: Clicking back should not send a request to the server. I also get that browsers manufacturers have the poetic licence to break this rule.
What I don't get is the following (please stay with me here)...
OK: Browsing our web site in HTTP, the history buttons did not send requests to server. Great! Behaviour as expected.
NOK: Browsing history on the same site in HTTPS mode, Chrome faired well but IE9/10 and FF did not. They would send the request for the HTML page to the server and then correctly use the store for the static files. Why the difference?
So after a little head scratching and testing, I found that the presence of the Pragma:no-cache header in the responses we were sending was responsible for this behaviour. After removing the header, that should never have been there in the first place, IE and FF faired well when using history buttons in HTTPS - no more sent requests.
Now, how can the presence of a header which should be ignored by modern browsers and only used in requests, be causing this strange issue in browser history navigation?
Related
I have a web server which returns 200 OK with a bunch of set-cookies, and an HTML page which loads a bunch of scripts from the same server.
However, the subsequent loads that was spawned from that HTML page submits a different cookie on their HTTP request headers.
What could be causing that? Surely there's some policy I'm missing out on, but I don't see why it works on some pages and not others?
I'm using chrome as the browser, but this behavior also happened from iOS, so I'm guessing it's not browser specific.
So after a lot more reading and troubleshooting, it turns out that when you don't set a cookie path, it'll default to whatever path the original request set-cookie was sent to. And because my resource paths had a different path, the cookie was not sent.
Adding Path=/ fixed it for my issue. Of course, if you don't want your cookie to be accessible to all pages this is bad, but my web-server requires requests to come with cookies because they are sensitive data (for security reasons).
Basically, http2 push using http2_push_preload doesn't work if you set header Vary: Accept on your response because you are doing content negotiation using the Accept request header. I'm using content negotiation to send (http2 push) webp pics instead of jpg to clients that support it.
HTTP/2 Push works for .js, .css files and all in the same call and shows "Push/Other" in Chrome DevTools, but fails for this one unique case (jpg content negotiated to webp), and shows just "Other" (not pushed) in Chrome DevTools.
Content negotiation for brotli, gzip compressions all work fine and get pushed properly using the Vary: Accept-Encoding and same for languages using the Vary: Accept-Language.
Only Vary: Accept fails.
Please help I'm at the point of giving up.
P.S: I was going through nginx source https://github.com/nginx/nginx/blob/master/src/http/v2/ngx_http_v2.c. Do a Crtl+F and you will find cases for only "Accept-Encoding" and "Accept-Language", nothing for "Accept". So I think "Accept" case is not yet supported by nginx??
P.P.S: I'm not overpushing, only using http2 push for the hero image.
Edit: Here's bug ticket on nginx site for those who want to track it:
https://trac.nginx.org/nginx/ticket/1851
https://trac.nginx.org/nginx/ticket/1817
Edit 2: Nginx team has responded by saying they are not going to support it due to security reasons (you can find the response in the duplicate bug post), which I believe is due to pushing from different origins like CDNs? Anyway, I need this feature, so the only option left is to:
Create a custom patch or package.
Use some other server software that supports it.
Manually implement in website code a feature to rewrite .jpg paths to .jpg.webp if requests are coming from clients that support webp.
(I don't give up :P)
I'm not entirely surprised by this and Apache does the same. If you want this to change suggest to raise a bug with nginx but wouldn't be surprised if they didn't prioritise it.
It also seems the browsers don't handle this situation very well either.
HTTP/2 push is fraught with opportunities to over push and this is one example. You should not push if client does not support WebP and you often won't know that with the information that you have at this point. Chrome seems to send webp in the accept header when you ask for the HTML for example, but Firefox does not.
Preload is a much better, safer, option that will respect vary headers and also cache status.
I have a website with a page containing a lot of images. Those images will never change on the server (if they change, the URI on the page referencing them will have a different version parameter, in an URI such as http://cdn.example.com/image1.jpg?v=123).
The images are served with Cache-Control: max-stale=31536000, public header as advised in an answer on StackOverflow.
ETag header is disabled, as recommended in another answer.
When I press F5 in Chrome, everything works as expected: the page reloads, and there are no requests for images. However, when I press Ctrl+F5, the browser does the requests to the server.
I want it to behave as for the favicon: do a request to the server only when explicitly asked (that is a Ctrl+F5 on the specific image), and avoid a roundtrip to the server when I do Ctrl+F5 on a page containing the images.
Is it possible?
If yes, what headers should I put in the response?
Notes:
The developer console remains closed during the tests.
The content is directly served from a server; no reverse proxies.
When the image is requested after I press Ctrl+F5 on a page, the request contains Cache-Control: no-cache, making it impossible to respond with HTTP 304.
I'm exploring the idea of HTTP Origin checks as CSRF protection for Drupal at https://www.drupal.org/node/1803712
Now I was testing how the Origin header arrives with a POST request, but Firefox does not send the Origin header on the user login form submission. Chromium and Chrome work fine, they send the Origin header.
Firefox version is 36.0.1. I also tested with a clean Firefox installation because I thought maybe some of my browser plugins suppress the Origin header, but no luck - no Origin header there either.
Is there a documentation page that describes when Firefox sends the Origin header and when not?
Is isn't implemented yet. There's a discussion here: https://bugzilla.mozilla.org/show_bug.cgi?id=446344
The default on Firefox is not to send HTTP_ORIGIN.
The reason is a bug that causes hangs on some mobile Firefox versions if the network.http.sendOriginHeader configuration variable (accessible via about:config) is enabled. (For details see https://developer.mozilla.org/en-US/Firefox/Experimental_features#Security and the link provided by Marco's comment https://bugzilla.mozilla.org/show_bug.cgi?id=446344.)
There is a proposal to enable FF sending HTTP_ORIGIN by default, but the TODO list is long (see https://bugzilla.mozilla.org/show_bug.cgi?id=1424076).
So it will probably take years until FF will generally send HTTP_ORIGIN even without Javascript code enabling CSRF.
Some FOSS OSes preconfigure their FF ports to send HTTP_ORIGIN by default.
BTW, MS Edge also does not send HTTP_ORIGIN without explicitly enabling CSRF using Javascript.
For this reason I have implemented a security setting of my site which enables the users to disallow POST transactions from browsers that do not provide HTTP_ORIGIN.
I send back an image with the following HTTP response header:
Cache-Control: private,max-age=86400
My understanding is that the browser should not even ask for this file for 24 hours (86,400 = 60s * 60m * 24h).
What I'm seeing on subsequent requests is that it still asks for the file, but gets back a "304 Not Modified." This is good, but I want to remove even that request/response.
What header is required to prevent the browser from even bothering to ask for the file, and just have it blindly use the file it has in local cache, until that file expires?
It all really depends on how you're testing this. On Firefox 3.6 and IE8, clicking on a link and then on a link that move you back to the first page will use the cache correctly with max-age. Hitting the Return key again in the URL field will show the same behavior.
However, hitting F5 will ask again for the file but allows 304 responses.
Hitting Ctrl+F5 will always ask again for the file, with Cache-Control and Pragma set to no-cache, forcing a 200 response.
This simply can't be done reliably in HTML < 5.
You could use client side storage in HTML5 or use a browser extension such as Gears to provide this functionality.