I've been trying to refresh my understanding of HTTP/1.1 caching - something I find I have to do every once in a while, as there seems to be too many possible combinations for my brain to remember reliably.
I took it as a given that the Expires or max-age cache control directives were used by browsers as hints, not ironclad laws: if a cache entry is stale (older than its max age), the browser MAY validate it.
A colleague and I had a bit of a row about this and he forced me to read the RFC, which I felt was a bit harsh but proved him entirely right: if I understand it correctly, clients are not allowed to use a cache entry that they know to be stale.
In other words: if a document specifies a max-age cache header, and no other directives such as must-validate affect caching behaviour, and that document's cache entry becomes stale, the browser will always re-validate it.
This is entirely clear and logical, and to put that argument to rest, I set out to confirm it empirically by having a server serve a test JavaScript file with the following headers:
< HTTP/1.1 200 OK
< Content-Type: application/x-javascript; charset=UTF-8
< Cache-Control: public, max-age=30
< Date: Thu, 30 Jan 2014 22:11:28 GMT
< Accept-Ranges: bytes
< Server: testServer/1.0
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
This JavaScript file is included by an HTML page hosted on the same server through a <script> element in the document's <head>. The HTML page is served with the following cache control header:
Cache-Control: no-cache, must-revalidate, no-store, max-age=0
After loading the HTML page, I clicked through to a different page, waited 5 minutes for good measure and clicked on a link back to the original page, monitoring all HTTP requests - and the test file was never requested. This was reproduced consistently with both Firefox and Safari at their latest versions.
This got a bit long-winded, but the gist of my question is: my tests seem to show that mainstream browsers do not respect the RFC and will not revalidate stale cache entries. Did I misinterpret the RFC? Just as likely, did I botch my tests, and can someone prove them wrong? Or do browsers really not respect the max-age cache directive?
After much digging and help from #CodeCaster, the canonical answer to this question is that browsers do appear to respect the RFC: stale cache entries are always re-validated, except in the very specific case that they were accessed, directly or indirectly, through the browser's history. In this case, section 13.3 of the RFC applies:
History mechanisms and caches are different. In particular history mechanisms SHOULD NOT try to show a semantically transparent view of the current state of a resource. Rather, a history mechanism is meant to show exactly what the user saw at the time when the resource was retrieved.
I cannot reproduce your issue. I have tried to, using ASP.NET MVC, with the following code:
public ActionResult Index()
{
Response.AddHeader("Cache-Control",
"no-cache, must-revalidate, no-store, max-age=0");
return View();
}
public ActionResult JavaScript()
{
Response.AddHeader("Cache-Control", "public, max-age=30");
return View();
}
public ActionResult Page2()
{
return View();
}
The first action returns the Index page (relevant headers only):
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate, max-age=0
Pragma: no-cache
Expires: -1
Vary: Accept-Encoding
Content-Length: 182
<html>
<head>
<script src="/Home/JavaScript" type="text/javascript"></script>
</head>
<body>
Page 2
</body>
</html>
The JavaScript is returned like this:
HTTP/1.1 200 OK
Cache-Control: public, max-age=30
Vary: Accept-Encoding
Content-Length: 24
document.write('Foo');
And Page2 merely contains a link back to Home.
Now when I, using Internet Explorer 11, Chrome 32 or Firefox 26, I see the following behavior:
Upon the first request to /, the Index document is requested as well as the JavaScript file, and they are returned like shown above as verified with Fiddler.
When I click the "Page 2" link, only Page 2 is requested, because it doesn't contain anything but a link back to Page 1.
Now when I click the "Home" link from Page 2 within 30 seconds, the JS file is not requested again in any browser.
When I wait a while (> 30 seconds) on Page 2 and then click the "Home" link, the JS file is requested in all three browsers.
However, when I click the Back button from Page 2 in any browser after any period of time (either greater or less than 30 seconds), the Index file is always, but the JS file is never requested again in any browser.
It requires a refresh (F5) or navigating away and back by clicking "Page 2" followed by "Home" again to make the browser perform a new request for the JS file after it became stale.
Related
When I visit chesseng.herokuapp.com I get a response header that looks like
Cache-Control:private
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/css
Date:Tue, 16 Oct 2012 06:37:53 GMT
Last-Modified:Tue, 16 Oct 2012 03:13:38 GMT
Status:200 OK
transfer-encoding:chunked
Vary:Accept-Encoding
X-Rack-Cache:miss
and then I refresh the page and get
Cache-Control:private
Connection:keep-alive
Date:Tue, 16 Oct 2012 06:20:49 GMT
Status:304 Not Modified
X-Rack-Cache:miss
so it seems like caching is working. If that works for caching then what is the point of Expires and Cache-Control:max-age. To add to confusion, when I test the page at https://developers.google.com/speed/pagespeed/insights/ it tells me to "Leverage browser caching".
Cache-Control: private
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache, such as a proxy server.
From RFC2616 section 14.9.1
To answer your question about why caching is working, even though the web-server didn't include the headers:
Expires: [a date]
Cache-Control: max-age=[seconds]
The server kindly asked any intermediate proxies to not cache the contents (i.e. the item should only be cached in a private cache, i.e. only on your own local machine):
Cache-Control: private
But the server forgot to include any sort of caching hints:
they forgot to include Expires (so the browser knows to use the cached copy until that date)
they forgot to include Max-Age (so the browser knows how long the cached item is good for)
they forgot to include E-Tag (so the browser can do a conditional request)
But they did include a Last-Modified date in the response:
Last-Modified: Tue, 16 Oct 2012 03:13:38 GMT
Because the browser knows the date the file was modified, it can perform a conditional request. It will ask the server for the file, but instruct the server to only send the file if it has been modified since 2012/10/16 3:13:38:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
The server receives the request, realizes that the client has the most recent version already. Rather than sending the client 200 OK, followed by the contents of the page, it instead tells you that your cached version is good:
304 Not Modified
Your browser did have to suffer the round-trip delay of sending a request to the server, and waiting for the response, but it did save having to re-download the static content.
Why Max-Age? Why Expires?
Because Last-Modified sucks.
Not everything on the server has a date associated with it. If I'm building a page on the fly, there is no date associated with it - it's now. But I'm perfectly willing to let the user cache the homepage for 15 seconds:
200 OK
Cache-Control: max-age=15
If the user hammers F5, they'll keep getting the cached version for 15 seconds. If it's a corporate proxy, then all 67,198 users hitting the same page in the same 15-second window will all get the same contents - all served from close cache. Performance win for everyone.
The virtue of adding Cache-Control: max-age is that the browser doesn't even have to perform a "conditional" request.
if you specified only Last-Modified, the browser has to perform a If-Modified-Since request, and watch for a 304 Not Modified response
if you specified max-age, the browser won't even have to suffer the network round-trip; the content will come right out of the caches.
The difference between "Cache-Control: max-age" and "Expires"
Expires is a legacy (c. 1998) equivalent of the modern Cache-Control: max-age header:
Expires: you specify a date (yuck)
max-age: you specify seconds (goodness)
And if both are specified, then the browser uses max-age:
200 OK
Cache-Control: max-age=60
Expires: 20180403T192837
Any web-site written after 1998 should not use Expires anymore, and instead use max-age.
What is ETag?
ETag is similar to Last-Modified, except that it doesn't have to be a date - it just has to be a something.
If I'm pulling a list of products out of a database, the server can send the last rowversion as an ETag, rather than a date:
200 OK
ETag: "247986"
My ETag can be the SHA1 hash of a static resource (e.g. image, js, css, font), or of the cached rendered page (i.e. this is what the Mozilla MDN wiki does; they hash the final markup):
200 OK
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
And exactly like in the case of a conditional request based on Last-Modified:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
304 Not Modified
I can perform a conditional request based on the ETag:
GET / HTTP/1.1
If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4"
304 Not Modified
An ETag is superior to Last-Modified because it works for things besides files, or things that have a notion of date. It just is
RFC 2616, section 14.9.1:
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache...A private (non-shared) cache MAY cache the response.
Browsers could use this information. Of course, the current "user" may mean many things: OS user, a browser user (e.g. Chrome's profiles), etc. It's not specified.
For me, a more concrete example of Cache-Control: private is that proxy servers (which typically have many users) won't cache it. It is meant for the end user, and no one else.
FYI, the RFC makes clear that this does not provide security. It is about showing the correct content, not securing content.
This usage of the word private only controls where the response may be cached, and cannot ensure the privacy of the message content.
The Expires entity-header field gives the date/time after which the response is considered stale.The Cache-control:maxage field gives the age value (in seconds) bigger than which response is consider stale.
Althought above header field give a mechanism to client to decide whether to send request to the server. In some condition, the client send a request to sever and the age value of response is bigger then the maxage value ,dose it means server needs to send the resource to client? Maybe the resource never changed.
In order to resolve this problem, HTTP1.1 gives last-modifided head. The server gives the last modified date of the response to client. When the client need this resource, it will send If-Modified-Since head field to server. If this date is before the modified date of the resouce, the server will sends the resource to client and gives 200 code.Otherwise,it will returns 304 code to client and this means client can use the resource it cached.
I want browsers to always add (except first time) "If-Modified-Since" request header to avoid unnecessary traffic.
Response headers are:
Accept-Ranges:bytes
Cache-Control:max-age=0, must-revalidate
Connection:Keep-Alive
Content-Length:2683
Content-Type:text/html; charset=UTF-8
Date:Thu, 05 Apr 2012 13:06:19 GMT
Keep-Alive:timeout=15, max=497
Last-Modified:Thu, 05 Apr 2012 13:05:11 GMT
Server:Apache/2.2.21 (Red Hat)
FF 11 and IE 9 both send "If-Modified-Since" and get 304 in response but Chrome 18 doesn't and get 200.
Why? How to force Chrome to sent "If-Modified-Since" header?
I do not know if it important or not but all requests going through HTTPS.
I've been chasing this issue for some time, thought I'd share what I found.
"The rule is actually quite simple: any error with the certificate means the page will not be cached."
https://code.google.com/p/chromium/issues/detail?id=110649
If you are using a self-signed certificate, even if you tell Chrome to add an exception for it so that the page loads, no resources from that page will be cached, and subsequent requests will not have an If-Modified-Since header.
I just now found this question, and after puzzling over Chrome's If_Modified_Since behavior, I've found the answer.
Chrome's decision to cache files is based on the Expires header that it receives. The Expires header has two main requirements:
It must be in Greenwich Mean Time (GMT), and
It must be formatted according to RFC 1123 (which is basically RFC 822 with a four-digit year).
The format is as follows:
Expires: Sat, 07 Sep 2013 05:21:03 GMT
For example, in PHP, the following outputs a properly formatted header.
$duration = time() + 3600 // Expires in one hour.
header("Expires: " . gmdate("D, d M Y H:i:s", $duration) . " GMT");
("GMT" is appended to the string instead of the "e" timezone flag because, when used with gmdate(), the flag will output "UTC," which RFC 1123 considers invalid. Also note that the PHP constants DateTime::RFC1123 and DATE_RFC1123 will not provide the proper formatting, since they output the difference to GMT in hours [i.e. +02:00] rather than "GMT".)
See the W3C's date/time format specifications for more info.
In short, Chrome will only recognize the header if it follows this exact format. This, combined with the Cache-Control header...
header("Cache-Control: private, must-revalidate, max-age=" . $duration);
...allowed me to implement proper cache control. Once Chrome recognized those headers, it began caching the pages I sent it (even with query strings!), and it also began sending the If_Modified_Since header. I compared it to a stored "last-modified" date, sent back HTTP/1.1 304 Not Modified, and everything worked perfectly.
Hope this helps anyone else who stumbles along!
I've noticed almost the same behaviour and my findings are:
First of all the 200 status indicator in chrome is not the whole truth, you need to look at the "Size Content" column as well. If this says "(from cache)" the resource was take directly from cache without even asking if it was modified.
This caching behaviour of resources that lack any indication of expires or max-age seems to apply when requesting static files that have a last-modified header. I've noted that chrome (ver. 22):
Asks for the file the first time (obviously since it is not in cache).
Asks if it is modified the second time (since it is in cache but has no indication of freshness).
Uses it directly the third time and then on (even if it is a new browser session).
I'm a bit puzzled by this behaviour but it is fairly reasonable, if it is static, was modified a long time ago, and hasn't changed since last check you could assume that it is going to be valid for a while longer (don't know how they calculate it though).
I had the same problem, in Chrome all requests were always status code 200, in other browsers 304.
It turned out I had the disable cache (while DevTools is open) checked in on Devtools - Settings - General page..:)
Don't disable cache in Chrome Dev Tools (on "Network" tab).
Cache-Control should be Cache-Control: public. Use true as second parameter to header PHP function: header("Cache-Control: public", true);
I have also found that Chrome (recent v95+) also returns a cached 200 response if I have DevTools open. It never even sends the request on to the server! If I close DevTools, the behaviour is as it should, and the server receives the expected request.
I'm trying to figure out the best HTTP headers to send for four use cases. I'm hoping to come up with headers that do not depend on user agent / protocol version sniffing but I'll accept that if nothing else fits. All URLs are fetched through fully custom handler so I can select all headers as I like, this is all about intermediate proxies and user agents. If possible, this should be compatible with both HTTP/1.0 and HTTP/1.1 clients. If multiple solutions exists, the best one will be the shortest one when sent over the wire.
Static public content
All "Static public content" is stuff that HTTP is really all about: if the URL is the same, the content is the same. I can do this easily: for example, I put user profile icon into http://domain.com/profiles/xyz/icon/1234abcd where "1234abcd" is the SHA-1 of the file contents of the icon. If I change to icon in the future, I'll create a new URL and and modify all existing referrers that should use the new icon. What are the best headers to declare that this may be cached forever and may be shared? I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time + one year>
Is this enough to allow caching by user agents and proxies? Do I need Last-Modified or Pragma?
Static non-public content
All "Static non-public content" is stuff that is static but may not be available to everybody. In fact, this content will be available only to selected logged in users (session is kept with session cookie holding session UUID). If the URL is the same, the content is the same. However, the response is not public. An use case could be an image shared to selected friends in a social network service. I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=<huge number>, s-maxage=0
Is this enough to allow caching by user agents and and disable proxies? Do I need Pragma?
Volatile public content
All "Volatile public content" is stuff that is volatile and available to everybody. Something like frontpage of http://slashdot.org/ when not logged in. The intent is to allow rapidly updating content in a non-changing URL. Note that I do NOT want to break the user agent history mechanism (that is, clicking something from a volatile page and then hitting the back button should not result in fetching the volatile page from the server -- however, clicking a link that goes to front page should fetch the resource from the server). I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: public, max-age=0, s-maxage=0
Is this enough to prevent caching but to allow history mechanism (back button)? I know that if I send Cache-Control: no-store, must-revalidate I can force reloading but this is not what I want because that will break the back button, too. Do I need Last-Modified or Pragma?
Even though this is public, it probably does not make sense to allow intermediate proxies to cache this because it's volatile.
Volatile non-public content
All "Volatile non-public content" is stuff that is volatile and not available to everybody (private). Something like frontpage of http://slashdot.org/ when you are logged in. The intent is to allow rapidly updating content in a non-changing URL. Note that I do NOT want to break the user agent history mechanism (that is, clicking something from a volatile page and then hitting the back button should not result in fetching the volatile page from the server -- however, clicking a link that goes to front page should fetch the resource from the server). I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=0, s-maxage=0
Is this enough to prevent caching but to allow history mechanism (back button)? Do I need Pragma?
Things that still need testing with my suggested headers:
Verify that private content will not be leaked through HTTP/1.0 proxies.
Verify that caching works correctly in proxies.
Verify that caching works correctly in user agents.
Verify that user agent history mechanism works in user agents (all cases).
Verify that following a link to a volatile page fetches fresh content from the server.
Verify all the results when using HTTPS instead of HTTP.
I'll answer my own question:
Static public content
Date: <current time>
Expires: <current time + one year>
Rationale: This is compatible with the HTTP/1.0 proxies and RFC 2616 Section 14: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21
The Last-Modified header is not needed for correct caching (because conforming user agents follow the Expires header) but may be included for the end user consumption. Including the Last-Modified header may also decrease the server data transfer in case user hits the Reload/Refresh button. If Last-Modified header is added, it should reflect real data instead of something invented up. If you want to decrease server data transfer (in case user hits Reload/Refresh button) and cannot include real Last-Modified header, you may add ETag header to allow conditional GET (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.26). If you already include Last-Modified also adding ETag is just waste. Note that Last-Modified is clearly superior because it's supported by HTTP/1.0 clients and proxies, too. A suitable value for ETag in case of dynamic pages is SHA-1 of the contents of the page/resource. Note that using Last-Modified or ETag will not help with the server load, only with the server outgoing internet pipe / data transfer rate.
Static non-public content
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=31536000, s-maxage=0
Vary: Cookie
Rationale: The Date and Expires headers are for HTTP/1.0 compatibility and because there's no sensible way to specify that the response is private, these headers communicate that the response may not be cached. The Cache-Control header tells that this response may be cached by private cache but shared cache may not cache the response. The s-maxage=0 is added because private may not be supported by all proxies that support Cache-Control (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3 - I have no idea which proxies are broken). The max-age is set to value of 60*60*24*365 (1 year) because the HTTP/1.1 specification does not define any upper limit for this parameter, I guess that this is implementation dependant. The Expires headers SHOULD be limited to one year in the future, so using the same logic here should be okay. The Vary: Cookie header is required because the session that is used to check if the visitor is allowed to see the content is transferred in a cookie; because the returned response depends on the cookie value the cache may not use cached response if cookie header is changed.
I might personally break the last part. By not including the Vary: Cookie header I can improve caching a lot. For example: I have a profile image at http://example.com/icon/12 which is returned only for selected authenticated users. I have a visitor X with session id 5f2 and I allow the image to that user. Visitor X logs out and then later logs in again. Now X has session id 2e8 stored in his session cookie. If I have Vary: cookie, the user agent of X cannot use the cached image and is forced to reload this to its cache. Because the content varies by Cookie, a conditional GET with last modification time cannot be used. I haven't tested if using ETag could help in this case because in that case, the server response would be the same (match the SHA-1 ETag computed from the contents of the response). Be warned that Internet Explorer (at least up to version 9) always forces conditional GET for resources that include Vary: Cookie even if suitable response were already in cache (source: http://blogs.msdn.com/b/ie/archive/2010/07/14/caching-improvements-in-internet-explorer-9.aspx). This is because internal cache implementation of MSIE does not remember which Cookie it sent the first time so it cannot know if the current Cookie is the same one.
However, here's an example of a problem that is caused by dropping the Vary: Cookie header to show why this is indeed required for technically correct behavior: see the example above and imagine that after X has logged out, visitor Y logs in with the same user agent (the user agent may have been restarted between X and Y, it does not matter). If Y views a page that includes a link to http://example.com/icon/12 then Y will see the icon embedded inside the page even though Y wouldn't be able to see the icon if X had not been using the same user agent previously. In my case I don't consider this a big enough problem because Y would be able to access the icon manually by inspecting the user agent cache regardless of possibly added Vary: Cookie. However, this issue may prevent Y from noticing that he wouldn't technically have access to this content (this may be important e.g. if Y is co-authoring the content). If the content is considered sensitive, the server must send no-store regardless of the problems caused by this Cache-Control directive.
Here too, adding Last-Modified header will help with users hitting Reload/Refresh button (see discussion above).
Volatile public content
Date: <current time>
Expires: <current time>
Cache-Control: public, max-age=0, s-maxage=0
Last-Modified: <real-last-modification-time>
Rationale: Tell HTTP/1.0 clients and proxies that this response should be considered stale immediately. The Last-Modified time is included to allow skipping content data transmission when the resource is accessed again and client supports conditional GET. If the Last-Modified cannot be used, ETag may be used as a replacement (see discussion above). It's critical to use Last-Modified to allow conditional GET with HTTP/1.0 compatible clients.
If the content may be delayed even slightly, then Expires, max-age and s-maxage [sic] should be adjusted suitably. For example, adding 5 seconds to those might help a lot for highly popular site, as suggested by symcbean's answer. Note that unlike conditional GET, increasing the expiry time will decrease server load instead of just decreasing server outgoing data traffic (because the server will see less requests in total).
Volatile non-public content
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=0, s-maxage=0
Last-Modified: <real-last-modification-time>
Vary: Cookie
Rationale: Tell HTTP/1.0 clients and proxies that this response should be considered stale immediately. The Last-Modified time is included to allow skipping content data transmission when the resource is accessed again and client supports conditional GET. If the Last-Modified cannot be used, ETag may be used as a replacement (see discussion above). It's critical to use Last-Modified to allow conditional GET with HTTP/1.0 compatible clients. Also note that Cache-Control must not include no-cache, must-revalidate or no-store because using any of these directives will break the back button in at least one user agent. However, if the content the server is transferring contains sensitive material that should not be stored in permanent storage, the no-store flag MUST be used regardless of breaking the back button. Warning: note that the use of no-store cannot prevent sensitive material ending up on the hard disk without encryption if the operating system has swapping enabled and the swap is not encrypted! Also note that using no-store makes very little sense unless the connection is encrypted (HTTPS/SSL).
Mostly OK, however you do need to bear in mind that HTTP/1.0 proxies may cache content served up as
Cache-Control: private
So you should set an explicit Date-modified header as well as the expires header.
For your 'Static non-public content' you should add a 'Varies: Cookie' header.
For your 'Volatile public content': How fast is it changing? Setting an TTL of +5 seconds may offload a lot of effort from your servers.
For 'Volatile non-public content' you should probably add no-cache,must-revalidate to the Cache-control header.
Pragma headers issued from the server should have no effect on clients nor proxies.
Do test out what happens when your cache expires (IME you can end up with a system even slower than one accessed with no populated cache due to all the conditional requests / 304 responses)
From a browser perspective,
What occur if a component (image, script, stylesheet...) is served without a Last-Modified HTTP header field...
Is it however cached by the browser even if it won't be able to perform a validity check(If-Modified-Since) in future, due to his lack of date/time information?
Eg:
GET /foo.png HTTP/1.1
Host: example.org
--
200 OK
Content-Type: image/png
...
Is foo.png however cached?
--
Would you know any online service to serve my raw HTTP response that I can write myself in order to test what I'm asking ?
Thank you.
Generally speaking, responses can be cached unless they explicitly say that they can't (e.g., with cache-control: no-store).
However, most caches will not store responses that don't have something that they can base freshness on, e.g., Cache-Control, Expires, or Last-Modified.
For the complete rules, see:
https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-p6-cache-13#section-2.1
See:
http://www.mnot.net/blog/2009/02/24/unintended_caching
for an example of how this can surprise some people.
Yes, the image may get cached even without a Last-Modified response header.
The browser will then cache the image until its TTL expires. You can set the image's Time To Live using appropriate response headers, e.g. this would set the TTL to one hour:
Cache-Control: max-age=3600
Date: Tue, 29 Mar 2011 20:18:17 GMT
Expires: Tue, 29 Mar 2011 21:18:17 GMT
Even without any Last-Modified in the response, the browser may still use the Date header for subsequent If-Modified-Since requests.
I disabled the Last-Modified header on a large site and FF 13 doesn't take the contents from cache, although a max-age is given etc. Contents without a Last-Modified header ALWAYS get a status 200 ok when requested, not a 304. So the browser looks for it in the cache.
This is just making me angry. I can't figure out why all the resources in my page are being requested EVERY single time.
E.g. my site.css returns the following headers (using fiddler):
HTTP/1.1 200 OK
Server: ASP.NET Development Server/9.0.0.0
Date: Mon, 29 Nov 2010 17:36:21 GMT
X-AspNet-Version: 2.0.50727
Content-Length: 9093
Cache-Control: public, max-age=2592000
Expires: Wed, 29 Dec 2010 17:36:21 GMT
Last-Modified: Mon, 08 Nov 2010 17:20:16 GMT
Content-Type: text/css
Connection: Close
But every time I hit refresh I see all the resources (css,js,images) getting re-requested. I have control over the headers returned for any and all of these resources, but I haven't figured it out yet.
I have even debugged my ASP.NET app and the HttpModule is definitely being asked for the resources again.
Can someone give me an idea of what to do?
EDIT:
Ok, I removed must-revalidate, proxy-revalidate from the headers and that is getting me closer to where I want to be, now when I press back it still requests my css/js files when I press back.
Is there anything more I can do to avoid this?
The following links might be of help to you.
Differences in reload behavior between FF and IE
http://blog.httpwatch.com/2008/10/15/two-important-differences-between-firefox-and-ie-caching/
In a nutshell, your caching behavior is going to be determined by the headers and the browser you are using.
What browser are you using for testing? The back button is also handled differently.
Back Button (Browser Behavior)
And, finally, a breakdown of f5/ctrl f5, click, shift click, etc behavior between browsers:
What requests do browsers' "F5" and "Ctrl + F5" refreshes generate?
If you are handling the requests in your own module - which seems to be the case - and the request contains an If-Modified-Since header, you can use that to determine whether to respond with a 200 and sending the whole resource again, or just send a 304 and skip sending the js/css/etc contents.
Other than that, expect browsers to re-query resources on hitting F5 / Refresh. It is just that you may skip sending the whole js/css/etc and return a 304 if everything is OK.
Other than that, #Chris's answer covers pretty much everything else.
What are the request headers you see when you hit back?