HTTP Headers: Controlling Cache and History Mechanism - http

I'm trying to figure out the best HTTP headers to send for four use cases. I'm hoping to come up with headers that do not depend on user agent / protocol version sniffing but I'll accept that if nothing else fits. All URLs are fetched through fully custom handler so I can select all headers as I like, this is all about intermediate proxies and user agents. If possible, this should be compatible with both HTTP/1.0 and HTTP/1.1 clients. If multiple solutions exists, the best one will be the shortest one when sent over the wire.
Static public content
All "Static public content" is stuff that HTTP is really all about: if the URL is the same, the content is the same. I can do this easily: for example, I put user profile icon into http://domain.com/profiles/xyz/icon/1234abcd where "1234abcd" is the SHA-1 of the file contents of the icon. If I change to icon in the future, I'll create a new URL and and modify all existing referrers that should use the new icon. What are the best headers to declare that this may be cached forever and may be shared? I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time + one year>
Is this enough to allow caching by user agents and proxies? Do I need Last-Modified or Pragma?
Static non-public content
All "Static non-public content" is stuff that is static but may not be available to everybody. In fact, this content will be available only to selected logged in users (session is kept with session cookie holding session UUID). If the URL is the same, the content is the same. However, the response is not public. An use case could be an image shared to selected friends in a social network service. I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=<huge number>, s-maxage=0
Is this enough to allow caching by user agents and and disable proxies? Do I need Pragma?
Volatile public content
All "Volatile public content" is stuff that is volatile and available to everybody. Something like frontpage of http://slashdot.org/ when not logged in. The intent is to allow rapidly updating content in a non-changing URL. Note that I do NOT want to break the user agent history mechanism (that is, clicking something from a volatile page and then hitting the back button should not result in fetching the volatile page from the server -- however, clicking a link that goes to front page should fetch the resource from the server). I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: public, max-age=0, s-maxage=0
Is this enough to prevent caching but to allow history mechanism (back button)? I know that if I send Cache-Control: no-store, must-revalidate I can force reloading but this is not what I want because that will break the back button, too. Do I need Last-Modified or Pragma?
Even though this is public, it probably does not make sense to allow intermediate proxies to cache this because it's volatile.
Volatile non-public content
All "Volatile non-public content" is stuff that is volatile and not available to everybody (private). Something like frontpage of http://slashdot.org/ when you are logged in. The intent is to allow rapidly updating content in a non-changing URL. Note that I do NOT want to break the user agent history mechanism (that is, clicking something from a volatile page and then hitting the back button should not result in fetching the volatile page from the server -- however, clicking a link that goes to front page should fetch the resource from the server). I'm currently thinking something along the lines:
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=0, s-maxage=0
Is this enough to prevent caching but to allow history mechanism (back button)? Do I need Pragma?
Things that still need testing with my suggested headers:
Verify that private content will not be leaked through HTTP/1.0 proxies.
Verify that caching works correctly in proxies.
Verify that caching works correctly in user agents.
Verify that user agent history mechanism works in user agents (all cases).
Verify that following a link to a volatile page fetches fresh content from the server.
Verify all the results when using HTTPS instead of HTTP.

I'll answer my own question:
Static public content
Date: <current time>
Expires: <current time + one year>
Rationale: This is compatible with the HTTP/1.0 proxies and RFC 2616 Section 14: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21
The Last-Modified header is not needed for correct caching (because conforming user agents follow the Expires header) but may be included for the end user consumption. Including the Last-Modified header may also decrease the server data transfer in case user hits the Reload/Refresh button. If Last-Modified header is added, it should reflect real data instead of something invented up. If you want to decrease server data transfer (in case user hits Reload/Refresh button) and cannot include real Last-Modified header, you may add ETag header to allow conditional GET (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.26). If you already include Last-Modified also adding ETag is just waste. Note that Last-Modified is clearly superior because it's supported by HTTP/1.0 clients and proxies, too. A suitable value for ETag in case of dynamic pages is SHA-1 of the contents of the page/resource. Note that using Last-Modified or ETag will not help with the server load, only with the server outgoing internet pipe / data transfer rate.
Static non-public content
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=31536000, s-maxage=0
Vary: Cookie
Rationale: The Date and Expires headers are for HTTP/1.0 compatibility and because there's no sensible way to specify that the response is private, these headers communicate that the response may not be cached. The Cache-Control header tells that this response may be cached by private cache but shared cache may not cache the response. The s-maxage=0 is added because private may not be supported by all proxies that support Cache-Control (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3 - I have no idea which proxies are broken). The max-age is set to value of 60*60*24*365 (1 year) because the HTTP/1.1 specification does not define any upper limit for this parameter, I guess that this is implementation dependant. The Expires headers SHOULD be limited to one year in the future, so using the same logic here should be okay. The Vary: Cookie header is required because the session that is used to check if the visitor is allowed to see the content is transferred in a cookie; because the returned response depends on the cookie value the cache may not use cached response if cookie header is changed.
I might personally break the last part. By not including the Vary: Cookie header I can improve caching a lot. For example: I have a profile image at http://example.com/icon/12 which is returned only for selected authenticated users. I have a visitor X with session id 5f2 and I allow the image to that user. Visitor X logs out and then later logs in again. Now X has session id 2e8 stored in his session cookie. If I have Vary: cookie, the user agent of X cannot use the cached image and is forced to reload this to its cache. Because the content varies by Cookie, a conditional GET with last modification time cannot be used. I haven't tested if using ETag could help in this case because in that case, the server response would be the same (match the SHA-1 ETag computed from the contents of the response). Be warned that Internet Explorer (at least up to version 9) always forces conditional GET for resources that include Vary: Cookie even if suitable response were already in cache (source: http://blogs.msdn.com/b/ie/archive/2010/07/14/caching-improvements-in-internet-explorer-9.aspx). This is because internal cache implementation of MSIE does not remember which Cookie it sent the first time so it cannot know if the current Cookie is the same one.
However, here's an example of a problem that is caused by dropping the Vary: Cookie header to show why this is indeed required for technically correct behavior: see the example above and imagine that after X has logged out, visitor Y logs in with the same user agent (the user agent may have been restarted between X and Y, it does not matter). If Y views a page that includes a link to http://example.com/icon/12 then Y will see the icon embedded inside the page even though Y wouldn't be able to see the icon if X had not been using the same user agent previously. In my case I don't consider this a big enough problem because Y would be able to access the icon manually by inspecting the user agent cache regardless of possibly added Vary: Cookie. However, this issue may prevent Y from noticing that he wouldn't technically have access to this content (this may be important e.g. if Y is co-authoring the content). If the content is considered sensitive, the server must send no-store regardless of the problems caused by this Cache-Control directive.
Here too, adding Last-Modified header will help with users hitting Reload/Refresh button (see discussion above).
Volatile public content
Date: <current time>
Expires: <current time>
Cache-Control: public, max-age=0, s-maxage=0
Last-Modified: <real-last-modification-time>
Rationale: Tell HTTP/1.0 clients and proxies that this response should be considered stale immediately. The Last-Modified time is included to allow skipping content data transmission when the resource is accessed again and client supports conditional GET. If the Last-Modified cannot be used, ETag may be used as a replacement (see discussion above). It's critical to use Last-Modified to allow conditional GET with HTTP/1.0 compatible clients.
If the content may be delayed even slightly, then Expires, max-age and s-maxage [sic] should be adjusted suitably. For example, adding 5 seconds to those might help a lot for highly popular site, as suggested by symcbean's answer. Note that unlike conditional GET, increasing the expiry time will decrease server load instead of just decreasing server outgoing data traffic (because the server will see less requests in total).
Volatile non-public content
Date: <current time>
Expires: <current time>
Cache-Control: private, max-age=0, s-maxage=0
Last-Modified: <real-last-modification-time>
Vary: Cookie
Rationale: Tell HTTP/1.0 clients and proxies that this response should be considered stale immediately. The Last-Modified time is included to allow skipping content data transmission when the resource is accessed again and client supports conditional GET. If the Last-Modified cannot be used, ETag may be used as a replacement (see discussion above). It's critical to use Last-Modified to allow conditional GET with HTTP/1.0 compatible clients. Also note that Cache-Control must not include no-cache, must-revalidate or no-store because using any of these directives will break the back button in at least one user agent. However, if the content the server is transferring contains sensitive material that should not be stored in permanent storage, the no-store flag MUST be used regardless of breaking the back button. Warning: note that the use of no-store cannot prevent sensitive material ending up on the hard disk without encryption if the operating system has swapping enabled and the swap is not encrypted! Also note that using no-store makes very little sense unless the connection is encrypted (HTTPS/SSL).

Mostly OK, however you do need to bear in mind that HTTP/1.0 proxies may cache content served up as
Cache-Control: private
So you should set an explicit Date-modified header as well as the expires header.
For your 'Static non-public content' you should add a 'Varies: Cookie' header.
For your 'Volatile public content': How fast is it changing? Setting an TTL of +5 seconds may offload a lot of effort from your servers.
For 'Volatile non-public content' you should probably add no-cache,must-revalidate to the Cache-control header.
Pragma headers issued from the server should have no effect on clients nor proxies.
Do test out what happens when your cache expires (IME you can end up with a system even slower than one accessed with no populated cache due to all the conditional requests / 304 responses)

Related

Understand Request HTTP Cache-Control [duplicate]

The header Cache-Control: max-age=0 implies that the content is considered stale (and must be re-fetched) immediately, which is in effect the same thing as Cache-Control: no-cache.
I had this same question, and found some info in my searches (your question came up as one of the results). Here's what I determined...
There are two sides to the Cache-Control header. One side is where it can be sent by the web server (aka. "origin server"). The other side is where it can be sent by the browser (aka. "user agent").
When sent by the origin server
I believe max-age=0 simply tells caches (and user agents) the response is stale from the get-go and so they SHOULD revalidate the response (eg. with the If-Not-Modified header) before using a cached copy, whereas, no-cache tells them they MUST revalidate before using a cached copy. From 14.9.1 What is Cacheable:
no-cache
...a cache MUST NOT use the response
to satisfy a subsequent request
without successful revalidation with
the origin server. This allows an
origin server to prevent caching even
by caches that have been configured to
return stale responses to client
requests.
In other words, caches may sometimes choose to use a stale response (although I believe they have to then add a Warning header), but no-cache says they're not allowed to use a stale response no matter what. Maybe you'd want the SHOULD-revalidate behavior when baseball stats are generated in a page, but you'd want the MUST-revalidate behavior when you've generated the response to an e-commerce purchase.
Although you're correct in your comment when you say no-cache is not supposed to prevent storage, it might actually be another difference when using no-cache. I came across a page, Cache Control Directives Demystified, that says (I can't vouch for its correctness):
In practice, IE and Firefox have
started treating the no-cache
directive as if it instructs the
browser not to even cache the page.
We started observing this behavior
about a year ago. We suspect that
this change was prompted by the
widespread (and incorrect) use of this
directive to prevent caching.
...
Notice that of late, "cache-control:
no-cache" has also started behaving
like the "no-store" directive.
As an aside, it appears to me that Cache-Control: max-age=0, must-revalidate should basically mean the same thing as Cache-Control: no-cache. So maybe that's a way to get the MUST-revalidate behavior of no-cache, while avoiding the apparent migration of no-cache to doing the same thing as no-store (ie. no caching whatsoever)?
When sent by the user agent
I believe shahkalpesh's answer applies to the user agent side. You can also look at 13.2.6 Disambiguating Multiple Responses.
If a user agent sends a request with Cache-Control: max-age=0 (aka. "end-to-end revalidation"), then each cache along the way will revalidate its cache entry (eg. with the If-Not-Modified header) all the way to the origin server. If the reply is then 304 (Not Modified), the cached entity can be used.
On the other hand, sending a request with Cache-Control: no-cache (aka. "end-to-end reload") doesn't revalidate and the server MUST NOT use a cached copy when responding.
max-age=0
This is equivalent to clicking Refresh, which means, give me the latest copy unless I already have the latest copy.
no-cache
This is holding Shift while clicking Refresh, which means, just redo everything no matter what.
Old question now, but if anyone else comes across this through a search as I did, it appears that IE9 will be making use of this to configure the behaviour of resources when using the back and forward buttons. When max-age=0 is used, the browser will use the last version when viewing a resource on a back/forward press. If no-cache is used, the resource will be refetched.
Further details about IE9 caching can be seen on this msdn caching blog post.
In my recent tests with IE8 and Firefox 3.5, it seems that both are RFC-compliant. However, they differ in their "friendliness" to the origin server. IE8 treats no-cache responses with the same semantics as max-age=0,must-revalidate. Firefox 3.5, however, seems to treat no-cache as equivalent to no-store, which sucks for performance and bandwidth usage.
Squid Cache, by default, seems to never store anything with a no-cache header, just like Firefox.
My advice would be to set public,max-age=0 for non-sensitive resources you want to have checked for freshness on every request, but still allow the performance and bandwidth benefits of caching. For per-user items with the same consideration, use private,max-age=0.
I would avoid the use of no-cache entirely, as it seems it has been bastardized by some browsers and popular caches to the functional equivalent of no-store.
Additionally, do not emulate Akamai and Limelight. While they essentially run massive caching arrays as their primary business, and should be experts, they actually have a vested interest in causing more data to be downloaded from their networks. Google might not be a good choice for emulation, either. They seem to use max-age=0 or no-cache randomly depending on the resource.
max-age
When an intermediate cache is forced, by means of a max-age=0 directive, to revalidate
its own cache entry, and the client has supplied its own validator in the request, the
supplied validator might differ from the validator currently stored with the cache entry.
In this case, the cache MAY use either validator in making its own request without
affecting semantic transparency.
However, the choice of validator might affect performance. The best approach is for the
intermediate cache to use its own validator when making its request. If the server replies
with 304 (Not Modified), then the cache can return its now validated copy to the client
with a 200 (OK) response. If the server replies with a new entity and cache validator,
however, the intermediate cache can compare the returned validator with the one provided in
the client's request, using the strong comparison function. If the client's validator is
equal to the origin server's, then the intermediate cache simply returns 304 (Not
Modified). Otherwise, it returns the new entity with a 200 (OK) response.
If a request includes the no-cache directive, it SHOULD NOT include min-fresh,
max-stale, or max-age.
courtesy: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.4
Don't accept this as answer - I will have to read it to understand the true usage of it :)
I'm hardly a caching expert, but Mark Nottingham is. Here are his caching docs. He also has excellent links in the References section.
Based on my reading of those docs, it looks like max-age=0 could allow the cache to send a cached response to requests that came in at the "same time" where "same time" means close enough together they look simultaneous to the cache, but no-cache would not.
By the way, it's worth noting that some mobile devices, particularly Apple products like iPhone/iPad completely ignore headers like no-cache, no-store, Expires: 0, or whatever else you may try to force them to not re-use expired form pages.
This has caused us no end of headaches as we try to get the issue of a user's iPad say, being left asleep on a page they have reached through a form process, say step 2 of 3, and then the device totally ignores the store/cache directives, and as far as I can tell, simply takes what is a virtual snapshot of the page from its last state, that is, ignoring what it was told explicitly, and, not only that, taking a page that should not be stored, and storing it without actually checking it again, which leads to all kinds of strange Session issues, among other things.
I'm just adding this in case someone comes along and can't figure out why they are getting session errors with particularly iphones and ipads, which seem by far to be the worst offenders in this area.
I've done fairly extensive debugger testing with this issue, and this is my conclusion, the devices ignore these directives completely.
Even in regular use, I've found that some mobiles also totally fail to check for new versions via say, Expires: 0 then checking last modified dates to determine if it should get a new one.
It simply doesn't happen, so what I was forced to do was add query strings to the css/js files I needed to force updates on, which tricks the stupid mobile devices into thinking it's a file it does not have, like: my.css?v=1, then v=2 for a css/js update. This largely works.
User browsers also, by the way, if left to their defaults, as of 2016, as I continuously discover (we do a LOT of changes and updates to our site) also fail to check for last modified dates on such files, but the query string method fixes that issue. This is something I've noticed with clients and office people who tend to use basic normal user defaults on their browsers, and have no awareness of caching issues with css/js etc, almost invariably fail to get the new css/js on change, which means the defaults for their browsers, mostly MSIE / Firefox, are not doing what they are told to do, they ignore changes and ignore last modified dates and do not validate, even with Expires: 0 set explicitly.
This was a good thread with a lot of good technical information, but it's also important to note how bad the support for this stuff is in particularly mobile devices. Every few months I have to add more layers of protection against their failure to follow the header commands they receive, or to properly interpet those commands.
This is answered directly in the MDN docs about cache control:
Most HTTP/1.0 caches don't support no-cache directives, so
historically max-age=0 was used as a workaround. But only
max-age=0 could cause a stale response to be reused when caches
disconnected from the origin server. must-revalidate addresses that.
That's why the example below is equivalent to no-cache.
Cache-Control: max-age=0, must-revalidate
But for now, you can simply use no-cache instead.
And also in the MDN docs about cache validation:
It is often stated that the combination of max-age=0 and
must-revalidate has the same meaning as no-cache.
Cache-Control: max-age=0, must-revalidate
max-age=0 means that the
response is immediately stale, and must-revalidate means that it must
not be reused without revalidation once it is stale — so in
combination, the semantics seem to be the same as no-cache.
However, that usage of max-age=0 is a remnant of the fact that many
implementations prior to HTTP/1.1 were unable to handle the no-cache
directive — and so to deal with that limitation, max-age=0 was used as
a workaround.
But now that HTTP/1.1-conformant servers are widely deployed, there's
no reason to ever use that max-age=0-and-must-revalidate combination —
you should instead just use no-cache.
One thing that (surprisingly) hasn't been mentioned is that a request can explicitly indicate that it will accept stale data, using the max-stale directive. In that case, if the server responded with max-age=0, the cache would merely consider the response stale, and would be free to use it to satisfy the client's request [which asked for potentially-stale data]. By contrast, if the server sends no-cache that really does trump any request by the client (with max-stale) for stale data, as the cache MUST revalidate.

What is the difference between no-cache and no-store in Cache-control?

I don't find get the practical difference between Cache-Control:no-store and Cache-Control:no-cache.
As far as I know, no-store means that no cache device is allowed to cache that response. In the other hand, no-cache means that no cache device is allowed to serve a cached response without validate it first with the source. But what is that validation about? Conditional get?
What if a response has no-cache, but it has no Last-Modified or ETag?
Regards.
See the below flow chart for better understanding
Ref: (https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching?hl=en#cache-control)
But what is that check about?
Exactly checking Last-Modified or ETag. Client would ask server if it has new version of data using those headers and if the answer is no it will serve cached data.
Update
From RFC
no-cache
If the no-cache directive does not specify a field-name, then a cache MUST NOT use
the response to satisfy a subsequent request without successful revalidation with the
origin server. This allows an origin server to prevent caching even by caches that
have been configured to return stale responses to client requests.
As you identified, no-cache doesn't mean there is never caching, but rather that the user agent has to always ask the server if it's OK to use what it cached. By contrast, no-store says to not even keep a copy, which means there's nothing to ask about. If you know the answer to "Can I reuse this?" is always no, you get a performance boost by skipping cache validation and saving room in the cache for other data.
Aside from performance, there is a behavior difference with browser history. HTTP 1.1 section 13.13 says that "expiration time does not apply to history mechanisms." The no-cache header describes expiration, and so doesn't apply to history mechanisms such as the back button. Thus, the user can navigate backward to a previous page with no-cache without the server being contacted.
The no-store header, on the other hand, prevents the data from being stored outside of a session, in which case it simply isn't available for a history mechanism to use. With no-store, if the user ends his session by navigating to another domain and then goes back, the only way for browser to know what to display is to get the initial page again from the server.
Here's how a Chromium issue on this topic makes the distinction:
no-cache doesn't mean "don't cache this" (that would be no-store). no-cache means don't use this for normal loads unless the resource is revalidated for freshness. History navigations are not normal loads.
No-store : Client will not make any caching operation.
No-cache : Client will cache the response, but client will check server before using that cached data: "data has changed on the server or not?" :with help of 'If-Modified-Since' or 'If-None-Match' header.

Ideal HTTP cache control headers for different types of resources

I want to find a minimal set of headers, that work with "all" caches and browsers (also when using HTTPS!)
On my web site, I'll have three kinds of resources:
(1) Forever cacheable (public / equal for all users)
Example: 0A470E87CC58EE133616F402B5DDFE1C.cache.html (auto generated by GWT)
These files are automatically assigned a new name, when they change content (based on the MD5).
They should get cached as much as possible, even when using HTTPS (so I assume, I should set Cache-Control: public, especially for Firefox?)
They shouldn't require the client to make a round-trip to the server to validate, if the content has changed.
(2) Changing occasionally (public / equal for all users)
Examples: index.html, mymodule.nocache.js
These files change their content without changing the URL, when a new version of the site is deployed.
They can be cached, but probably need a round-trip to be revalidated every time.
(3) Individual for each request (private / user specific)
Example: JSON responses
These resources should never be cached unencrypted to disk under no circumstances. (Except maybe I'll have a few specific requests that could be cached.)
I have a general idea on which headers I would probably use for each type, but there's always something I could be missing.
I would probably use these settings:
Cache-Control: max-age=31556926 – Representations may be cached by any cache. The cached representation is to be considered fresh for 1 year:
To mark a response as "never expires," an origin server sends an
Expires date approximately one year from the time the response is
sent. HTTP/1.1 servers SHOULD NOT send Expires dates more than one
year in the future.
Cache-Control: no-cache – Representations are allowed to be cached by any cache. But caches must submit the request to the origin server for validation before releasing a cached copy.
Cache-Control: no-store – Caches must not cache the representation under any condition.
See Mark Nottingham’s Caching Tutorial for further information.
Cases one and two are actually the same scenario.
You should set Cache-Control: public and then generate a URL with includes the build number / version of the site so that you have immutable resources that could potentially last forever.
You also want to set the Expires header a year or more in the future so that the client will not need to issue a freshness check.
For case 3, you could all of the following for maximum flexibility:
"Cache-Control", "no-cache, must-revalidate"
"Expires", 0
"Pragma", "no-cache"

What's the difference between Cache-Control: max-age=0 and no-cache?

The header Cache-Control: max-age=0 implies that the content is considered stale (and must be re-fetched) immediately, which is in effect the same thing as Cache-Control: no-cache.
I had this same question, and found some info in my searches (your question came up as one of the results). Here's what I determined...
There are two sides to the Cache-Control header. One side is where it can be sent by the web server (aka. "origin server"). The other side is where it can be sent by the browser (aka. "user agent").
When sent by the origin server
I believe max-age=0 simply tells caches (and user agents) the response is stale from the get-go and so they SHOULD revalidate the response (eg. with the If-Not-Modified header) before using a cached copy, whereas, no-cache tells them they MUST revalidate before using a cached copy. From 14.9.1 What is Cacheable:
no-cache
...a cache MUST NOT use the response
to satisfy a subsequent request
without successful revalidation with
the origin server. This allows an
origin server to prevent caching even
by caches that have been configured to
return stale responses to client
requests.
In other words, caches may sometimes choose to use a stale response (although I believe they have to then add a Warning header), but no-cache says they're not allowed to use a stale response no matter what. Maybe you'd want the SHOULD-revalidate behavior when baseball stats are generated in a page, but you'd want the MUST-revalidate behavior when you've generated the response to an e-commerce purchase.
Although you're correct in your comment when you say no-cache is not supposed to prevent storage, it might actually be another difference when using no-cache. I came across a page, Cache Control Directives Demystified, that says (I can't vouch for its correctness):
In practice, IE and Firefox have
started treating the no-cache
directive as if it instructs the
browser not to even cache the page.
We started observing this behavior
about a year ago. We suspect that
this change was prompted by the
widespread (and incorrect) use of this
directive to prevent caching.
...
Notice that of late, "cache-control:
no-cache" has also started behaving
like the "no-store" directive.
As an aside, it appears to me that Cache-Control: max-age=0, must-revalidate should basically mean the same thing as Cache-Control: no-cache. So maybe that's a way to get the MUST-revalidate behavior of no-cache, while avoiding the apparent migration of no-cache to doing the same thing as no-store (ie. no caching whatsoever)?
When sent by the user agent
I believe shahkalpesh's answer applies to the user agent side. You can also look at 13.2.6 Disambiguating Multiple Responses.
If a user agent sends a request with Cache-Control: max-age=0 (aka. "end-to-end revalidation"), then each cache along the way will revalidate its cache entry (eg. with the If-Not-Modified header) all the way to the origin server. If the reply is then 304 (Not Modified), the cached entity can be used.
On the other hand, sending a request with Cache-Control: no-cache (aka. "end-to-end reload") doesn't revalidate and the server MUST NOT use a cached copy when responding.
max-age=0
This is equivalent to clicking Refresh, which means, give me the latest copy unless I already have the latest copy.
no-cache
This is holding Shift while clicking Refresh, which means, just redo everything no matter what.
Old question now, but if anyone else comes across this through a search as I did, it appears that IE9 will be making use of this to configure the behaviour of resources when using the back and forward buttons. When max-age=0 is used, the browser will use the last version when viewing a resource on a back/forward press. If no-cache is used, the resource will be refetched.
Further details about IE9 caching can be seen on this msdn caching blog post.
In my recent tests with IE8 and Firefox 3.5, it seems that both are RFC-compliant. However, they differ in their "friendliness" to the origin server. IE8 treats no-cache responses with the same semantics as max-age=0,must-revalidate. Firefox 3.5, however, seems to treat no-cache as equivalent to no-store, which sucks for performance and bandwidth usage.
Squid Cache, by default, seems to never store anything with a no-cache header, just like Firefox.
My advice would be to set public,max-age=0 for non-sensitive resources you want to have checked for freshness on every request, but still allow the performance and bandwidth benefits of caching. For per-user items with the same consideration, use private,max-age=0.
I would avoid the use of no-cache entirely, as it seems it has been bastardized by some browsers and popular caches to the functional equivalent of no-store.
Additionally, do not emulate Akamai and Limelight. While they essentially run massive caching arrays as their primary business, and should be experts, they actually have a vested interest in causing more data to be downloaded from their networks. Google might not be a good choice for emulation, either. They seem to use max-age=0 or no-cache randomly depending on the resource.
max-age
When an intermediate cache is forced, by means of a max-age=0 directive, to revalidate
its own cache entry, and the client has supplied its own validator in the request, the
supplied validator might differ from the validator currently stored with the cache entry.
In this case, the cache MAY use either validator in making its own request without
affecting semantic transparency.
However, the choice of validator might affect performance. The best approach is for the
intermediate cache to use its own validator when making its request. If the server replies
with 304 (Not Modified), then the cache can return its now validated copy to the client
with a 200 (OK) response. If the server replies with a new entity and cache validator,
however, the intermediate cache can compare the returned validator with the one provided in
the client's request, using the strong comparison function. If the client's validator is
equal to the origin server's, then the intermediate cache simply returns 304 (Not
Modified). Otherwise, it returns the new entity with a 200 (OK) response.
If a request includes the no-cache directive, it SHOULD NOT include min-fresh,
max-stale, or max-age.
courtesy: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.4
Don't accept this as answer - I will have to read it to understand the true usage of it :)
I'm hardly a caching expert, but Mark Nottingham is. Here are his caching docs. He also has excellent links in the References section.
Based on my reading of those docs, it looks like max-age=0 could allow the cache to send a cached response to requests that came in at the "same time" where "same time" means close enough together they look simultaneous to the cache, but no-cache would not.
By the way, it's worth noting that some mobile devices, particularly Apple products like iPhone/iPad completely ignore headers like no-cache, no-store, Expires: 0, or whatever else you may try to force them to not re-use expired form pages.
This has caused us no end of headaches as we try to get the issue of a user's iPad say, being left asleep on a page they have reached through a form process, say step 2 of 3, and then the device totally ignores the store/cache directives, and as far as I can tell, simply takes what is a virtual snapshot of the page from its last state, that is, ignoring what it was told explicitly, and, not only that, taking a page that should not be stored, and storing it without actually checking it again, which leads to all kinds of strange Session issues, among other things.
I'm just adding this in case someone comes along and can't figure out why they are getting session errors with particularly iphones and ipads, which seem by far to be the worst offenders in this area.
I've done fairly extensive debugger testing with this issue, and this is my conclusion, the devices ignore these directives completely.
Even in regular use, I've found that some mobiles also totally fail to check for new versions via say, Expires: 0 then checking last modified dates to determine if it should get a new one.
It simply doesn't happen, so what I was forced to do was add query strings to the css/js files I needed to force updates on, which tricks the stupid mobile devices into thinking it's a file it does not have, like: my.css?v=1, then v=2 for a css/js update. This largely works.
User browsers also, by the way, if left to their defaults, as of 2016, as I continuously discover (we do a LOT of changes and updates to our site) also fail to check for last modified dates on such files, but the query string method fixes that issue. This is something I've noticed with clients and office people who tend to use basic normal user defaults on their browsers, and have no awareness of caching issues with css/js etc, almost invariably fail to get the new css/js on change, which means the defaults for their browsers, mostly MSIE / Firefox, are not doing what they are told to do, they ignore changes and ignore last modified dates and do not validate, even with Expires: 0 set explicitly.
This was a good thread with a lot of good technical information, but it's also important to note how bad the support for this stuff is in particularly mobile devices. Every few months I have to add more layers of protection against their failure to follow the header commands they receive, or to properly interpet those commands.
This is answered directly in the MDN docs about cache control:
Most HTTP/1.0 caches don't support no-cache directives, so
historically max-age=0 was used as a workaround. But only
max-age=0 could cause a stale response to be reused when caches
disconnected from the origin server. must-revalidate addresses that.
That's why the example below is equivalent to no-cache.
Cache-Control: max-age=0, must-revalidate
But for now, you can simply use no-cache instead.
And also in the MDN docs about cache validation:
It is often stated that the combination of max-age=0 and
must-revalidate has the same meaning as no-cache.
Cache-Control: max-age=0, must-revalidate
max-age=0 means that the
response is immediately stale, and must-revalidate means that it must
not be reused without revalidation once it is stale — so in
combination, the semantics seem to be the same as no-cache.
However, that usage of max-age=0 is a remnant of the fact that many
implementations prior to HTTP/1.1 were unable to handle the no-cache
directive — and so to deal with that limitation, max-age=0 was used as
a workaround.
But now that HTTP/1.1-conformant servers are widely deployed, there's
no reason to ever use that max-age=0-and-must-revalidate combination —
you should instead just use no-cache.
One thing that (surprisingly) hasn't been mentioned is that a request can explicitly indicate that it will accept stale data, using the max-stale directive. In that case, if the server responded with max-age=0, the cache would merely consider the response stale, and would be free to use it to satisfy the client's request [which asked for potentially-stale data]. By contrast, if the server sends no-cache that really does trump any request by the client (with max-stale) for stale data, as the cache MUST revalidate.

Why both no-cache and no-store should be used in HTTP response?

I'm told to prevent user-info leaking, only "no-cache" in response is not enough. "no-store" is also necessary.
Cache-Control: no-cache, no-store
After reading this spec http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html, I'm still not quite sure why.
My current understanding is that it is just for intermediate cache server. Even if "no-cache" is in response, intermediate cache server can still save the content to non-volatile storage. The intermediate cache server will decide whether using the saved content for following request. However, if "no-store" is in the response, the intermediate cache sever is not supposed to store the content. So, it is safer.
Is there any other reason we need both "no-cache" and "no-store"?
I must clarify that no-cache does not mean do not cache. In fact, it means "revalidate with server" before using any cached response you may have, on every request.
must-revalidate, on the other hand, only needs to revalidate when the resource is considered stale.
If the server says that the resource is still valid then the cache can respond with its representation, thus alleviating the need for the server to resend the entire resource.
no-store is effectively the full do not cache directive and is intended to prevent storage of the representation in any form of cache whatsoever.
I say whatsoever, but note this in the RFC 2616 HTTP spec:
History buffers MAY store such responses as part of their normal operation
But this is omitted from the newer RFC 7234 HTTP spec in potentially an attempt to make no-store stronger, see:
https://www.rfc-editor.org/rfc/rfc7234#section-5.2.1.5
Under certain circumstances, IE6 will still cache files even when Cache-Control: no-cache is in the response headers.
The W3C states of no-cache:
If the no-cache directive does not
specify a field-name, then a cache
MUST NOT use the response to satisfy a
subsequent request without successful
revalidation with the origin server.
In my application, if you visited a page with the no-cache header, then logged out and then hit back in your browser, IE6 would still grab the page from the cache (without a new/validating request to the server). Adding in the no-store header stopped it doing so. But if you take the W3C at their word, there's actually no way to control this behavior:
History buffers MAY store such responses as part of their normal operation.
General differences between browser history and the normal HTTP caching are described in a specific sub-section of the spec.
From the HTTP 1.1 specification:
no-store:
The purpose of the no-store directive is to prevent the inadvertent release or retention of sensitive information (for example, on backup tapes). The no-store directive applies to the entire message, and MAY be sent either in a response or in a request. If sent in a request, a cache MUST NOT store any part of either this request or any response to it. If sent in a response, a cache MUST NOT store any part of either this response or the request that elicited it. This directive applies to both non- shared and shared caches. "MUST NOT store" in this context means that the cache MUST NOT intentionally store the information in non-volatile storage, and MUST make a best-effort attempt to remove the information from volatile storage as promptly as possible after forwarding it.
Even when this directive is associated with a response, users might explicitly store such a response outside of the caching system (e.g., with a "Save As" dialog). History buffers MAY store such responses as part of their normal operation.
The purpose of this directive is to meet the stated requirements of certain users and service authors who are concerned about accidental releases of information via unanticipated accesses to cache data structures. While the use of this directive might improve privacy in some cases, we caution that it is NOT in any way a reliable or sufficient mechanism for ensuring privacy. In particular, malicious or compromised caches might not recognize or obey this directive, and communications networks might be vulnerable to eavesdropping.
no-store should not be necessary in normal situations, and can harm both speed and usability. It is intended for use where the HTTP response contains information so sensitive it should never be written to a disk cache at all, regardless of the negative effects that creates for the user.
How it works:
Normally, even if a user agent such as a browser determines that a response shouldn't be cached, it may still store it to the disk cache for reasons internal to the user agent. This version may be utilised for features like "view source", "back", "page info", and so on, where the user hasn't necessarily requested the page again, but the browser doesn't consider it a new page view and it would make sense to serve the same version the user is currently viewing.
Using no-store will prevent that response being stored, but this may impact the browser's ability to give "view source", "back", "page info" and so on without making a new, separate request for the server, which is undesirable. In other words, the user may try viewing the source and if the browser didn't keep it in memory, they'll either be told this isn't possible, or it will cause a new request to the server. Therefore, no-store should only be used when the impeded user experience of these features not working properly or quickly is outweighed by the importance of ensuring content is not stored in the cache.
My current understanding is that it is just for intermediate cache server. Even if "no-cache" is in response, intermediate cache server can still save the content to non-volatile storage.
This is incorrect. Intermediate cache servers compatible with HTTP 1.1 will obey the no-cache and must-revalidate instructions, ensuring that content is not cached. Using these instructions will ensure that the response is not cached by any intermediate cache, and that all subsequent requests are sent back to the origin server.
If the intermediate cache server does not support HTTP 1.1, then you will need to use Pragma: no-cache and hope for the best. Note that if it doesn't support HTTP 1.1 then no-store is irrelevant anyway.
If you want to prevent all caching (e.g. force a reload when using the back button) you need:
no-cache for IE
no-store for Firefox
There's my information about this here:
http://blog.httpwatch.com/2008/10/15/two-important-differences-between-firefox-and-ie-caching/
For chrome, no-cache is used to reload the page on a re-visit, but it still caches it if you go back in history (back button). To reload the page for history-back as well, use no-store. IE needs must-revalidate to work in all occasions.
So just to be sure to avoid all bugs and misinterpretations I always use
Cache-Control: no-store, no-cache, must-revalidate
if I want to make sure it reloads.
If a caching system correctly implements no-store, then you wouldn't need no-cache. But not all do. Additionally, some browsers implement no-cache like it was no-store. Thus, while not strictly required, it's probably safest to include both.
Note that Internet Explorer from version 5 up to 8 will throw an error when trying to download a file served via https and the server sending Cache-Control: no-cache or Pragma: no-cache headers.
See http://support.microsoft.com/kb/812935/en-us
The use of Cache-Control: no-store and Pragma: private seems to be the closest thing which still works.
Originally we used no-cache many years ago and did run into some problems with stale content with certain browsers... Don't remember the specifics unfortunately.
We had since settled on JUST the use of no-store. Have never looked back or had a single issue with stale content by any browser or intermediaries since.
This space is certainly dominated by reality of implementations vs what happens to have been written in various RFCs. Many proxies in particular tend to think they do a better job of "improving performance" by replacing the policy they are supposed to be following with their own.
Just to make things even worse, in some situations, no-cache can't be used, but no-store can:
http://faindu.wordpress.com/2008/04/18/ie7-ssl-xml-flex-error-2032-stream-error/
To answer the question, there are two players here, the client (request) and the server (response).
Client:
The client can only request with ONE cache method. There are different methods and if not specified, will use default.
default: Inspect browser cache:
If cached and "fresh": Return from cache.
If cached, stale, but still "valid": Return from cache, and schedule a fetch to update cache (for next use).
If cached and stale: Fetch with conditions, cache, and return.
If not cached: Fetch, cache, and return.
no-store: Fetch and return.
reload: Fetch, cache, and return. (default-4)
no-cache: Inspect browser cache:
If cached: Fetch with conditions, cache, and return. (default-3)
If not cached: Fetch, cache, and return. (default-4)
force-cache: Inspect browser cache:
If cached: Return it regardless if stale.
If not cache: Fetch, cache, and return. (default-4)
only-if-cached: Inspect browser cache:
If cached: Return it regardless if stale.
If not cached: Throw network error.
Notes:
Still "valid" means the current age is within the stale-while-revalidate lifetime. It needs "revalidation", but is still acceptable to return.
"Fetch" here, for simplicity, is short for "non-conditional network
fetch".
"Fetch with conditions" means fetch using headers like
If-Modified-Since, or ETag so the server can respond with 304: (Not Modified).
https://fetch.spec.whatwg.org/#concept-request-cache-mode
Server::
Now that we understand what the client can do, the server responses make more sense.
Looking at the Cache-Control header, if the server returns:
no-store: Tells client to not use cache at all
no-cache: Tells client it should do conditional requests and ignore freshness
max-age: Tells client how long a cache is "fresh"
stale-while-revalidate: Tells client how long cache is "valid"
immutable: Cache forever
Now we can put it all together. That means the only possibilities are:
Non-conditional network fetch
Conditional network fetch
Return stale cache
Return stale but valid cache
Return fresh cache
Return any cache
Any combination of client, or server can dictate what method, or set of methods, to use. If the server returns no-store, it's not going to hit the cache, no matter what the client request type. If the client request was no-store, it doesn't matter what the server returns, it won't cache. If the client doesn't specify a request type, the server will dictate it with Cache-Control.
It makes no sense for a server to return both no-cache and no-store since no-store overrides everything. Yes, you've probably seen both together, and it's useless outside of broken browser implementations. Still, no-store has been part of spec since 1999: https://datatracker.ietf.org/doc/html/rfc2616#section-14.9.2
In real life usage, if your server supports 304: Not Modified, and you want to use client cache as a way to improve speed, but still want to force a network fetch, use no-cache. If don't support 304, and want to force a network fetch, use no-store. If you're okay with cache sometimes, use freshness and revalidation headers.
In reality, if you're mixing up no-cache and no-store on the client, very little would change. Then, just a couple of headers get sent and there will different internal responses handled by the browser. An issue can occur if you use no-cache and then forget to use it later. no-cache tells it to store the response in the cache, and a later request without it might trigger internal cache.
There are times when you may want to mix methods even on the same resource based on context. For example, you may want to use reload on a service worker and background sync, but use default for the web page itself. This is where you can manipulate the user agent (browser) cache to your liking. Just remember that the server generally has the final say as to how the cache should work.
To clarify some possible future confusion. The client can use the Cache-Control header on the request, to tell the server to not use its own cache system when responding. This is unrelated to the browser/server dynamic, and more about the server/database dynamic.
Also no-store technically means must not store to any non-volatile storage (disk) and release it from volatile storage (memory) ASAP. In practice, it means don't use a cache at all. The command actually goes both ways. A client request with no-store shouldn't write to disk or database and is meant to transient.
TL;DR: no-store overrides no-cache. Setting both is useless, unless we are talking out-of-spec or HTTP/1.0 browsers that don't support no-store (Maybe IE11?). Use no-cache for 304 support.
A pretty old topic but I'll share some recent ideas:
no-store: Must not attempt to store anything, and must also take action to delete any copy it might have.
no-cache: Never use a local copy without first validating with the origin server. It prevents all possibility of a cache hit, even with fresh resources.
So, answering the question, using only one of them is enough.
Also, some (not very) recent works prove that browsers are more Cache-Control compatible nowadays.
OWASP discusses this:
What's the difference between the cache-control directives: no-cache, and no-store?
The no-cache directive in a response indicates that the response must not be used to serve a subsequent request i.e. the cache must not display a response that has this directive set in the header but must let the server serve the request. The no-cache directive can include some field names; in which case the response can be shown from the cache except for the field names specified which should be served from the server. The no-store directive applies to the entire message and indicates that the cache must not store any part of the response or any request that asked for it.
Am I totally safe with these directives?
No. But generally, use both Cache-Control: no-cache, no-store and Pragma: no-cache, in addition to Expires: 0 (or a sufficiently backdated GMT date such as the UNIX epoch). Non-html content types like pdf, word documents, excel spreadsheets, etc often get cached even when the above cache control directives are set (although this varies by version and additional use of must-revalidate, pre-check=0, post-check=0, max-age=0, and s-maxage=0 in practice can sometimes result at least in file deletion upon browser closure in some cases due to browser quirks and HTTP implementations). Also, 'Autocomplete' feature allows a browser to cache whatever the user types in an input field of a form. To check this, the form tag or the individual input tags should include 'Autocomplete="Off" ' attribute. However, it should be noted that this attribute is non-standard (although it is supported by the major browsers) so it will break XHTML validation.
Source here.

Resources