Are both "Cache request directives" and "Cache response directives" needed? - http

If I already have "Cache request directives," what is the point of "Cache response directives." Do they add anything? Will my application run the same without them?
I looking for proof whether "Cache response directives" are redundant. If they are redundant, I will not bother with them.
GC_

I assume you are asking as an application developer and if so, you should not bother with any Cache-Control header your application receives in a request.
Why?
Because that Cache-Control header is intended for caches before the request reaches your application.
It is not for your application.
This is explained in RFC7234 Section 5.2 (emphasis mine):
The "Cache-Control" header field is used to specify directives for caches along the request/response chain.
The purpose of the header is to tell caches what to do with the request.
Your application receives the header because it is attached to a request.
But just because you receive it, it doesn't mean it is for you.
Bottom line: ignore any Cache-Control header in a request.
Cache-Control in a response comes from your application and it is also intended for caches.
You use it to tell caches what to do with the response.
Basically, you use the header to specify whether the response is cacheable and if it is, for how long.
It is not merely a copy of the Cache-Control header received in a request.
Do they add anything?
Yes, they do.
Cache-Control in a response tells caches whether the response is cacheable and if it is,
it allows caches to serve an equivalent request immediately with a cached response.
This reduces your application's load and improves response times from a client's point of view.
RFC7234 Section 4.2 states:
When a response is "fresh" in the cache, it can be used to satisfy subsequent requests without contacting the origin server, thereby improving efficiency.
Your next question:
Will my application run the same without them?
It depends.
If your application doesn't add appropriate Cache-Control header for responses that must not be cached, future requests may receive stale responses.
So, I recommend that at the very least, add Cache-Control: no-cache to responses that must not be cached.
Additional explanation for your question in the comment section
The header should generally come from your backend, not your frontend.
This allows caches to accurately accelerates requests to your backend and keeps your frontend request code simple.
There is one exception: if the backend isn't yours and its response freshness policy doesn't match your requirement.
An example scenario may be in order:
Let's say, that in addition to sending requests to your own backend, your frontend also sends requests to someone else's backend.
This particular backend specifies that its responses are cacheable for at most 5 minutes, by either sending Cache-Control: max-age=300 or appropriate Expires header.
Let's also say, that you want the responses to be no more than 10 seconds stale, because 5 minutes is too stale for you.
Since the backend isn't yours, you can't change the 5-minutes directive, but you can send your requests with Cache-Control: max-age=10 thereby forcing the caches to fetch a fresh response if a cached response is older than 10 seconds, despite the 5-minutes directive from the backend.
That is the appropriate situation to send Cache-Control header from your frontend: the backend isn't yours and its response freshness policy doesn't match your requirement.

Are both "Cache request directives" and "Cache response directives" needed?
Yes. Cache-Control in request header and Cache-Control in response header are both needed. Even if you already have Cache-Control in request header, Cache-Control in response is not redundant. They are 2 different things. According to RFC7234:
cache directives are unidirectional in that the presence of a directive in a request does not imply that the same directive is to be given in the response.
Generally speaking, Cache-Control in response header controls the cache behaviour from resource provider's point of view. -- should the resource stored in cache? How long would it be valid? When requested, does it need to be revalidated? etc. As response headers can be configured for all HTTP requests, "Cache response directives" provides a way to define cache policy for all resources.
Cache-Control in request header, however, controls the cache behaviour from resource consumer's point of view. It's more like defining exceptional case where the cache policy of specific resource should be adjusted. If you check RFC7234, most of the "Request Cache-Control Directives" indicates that the client is willing to... or indicates that the client is unwilling to...
Also, as request headers can only be configured in some cases (e.g. Ajax), "Cache request directives" doesn't exist for many HTTP requests. For example, after HTML file is parsed, many HTTP requests will be created to fetch static resources (image files, css files etc.), there is no way to configure Cache-Control header for these requests manually in program.
If I already have "Cache request directives", what is the point of "Cache response directives"?
If you only have "Cache request directives" and never get Cache-Control response header, some problems will happen:
Without Cache-Control response header, the cache behaviour of all resources are decided by browser (e.g. calculate valid-time through LM-Factor algorithm). In the worst case, there would be no cache at all.
For static resources (e.g. image files, css files), as you can't configure Cache-Control in request, you lost cache control ability.

Related

Caching strategy using ETag and Expires/Cache-control with no assets version/ID

After reading a lot about caching validators (more intensively after reading this answer on SO), I had a doubt that didn't find the answer anywhere.
My use-case is to serve a static asset (a javascript file, ie: https://example.com/myasset.js) to be used in other websites, so messing with their Gpagespeed/gmetrix score matters the most.
I also need their users to receive updated versions of my static asset every time I deploy new changes.
For this, I have the following response headers:
Cache-Control: max-age=10800
etag: W/"4efa5de1947fe4ce90cf10992fa"
In short, we can see the following flow in terms of how browser behaves using etag
For the first request, the browser has no value for the If-None-Match Request Header, so the Server will send back the status code 200 (Ok), the content itself, and a Response header with ETag value.
For the subsequent requests, the browser will add the previously received ETag value in a form of the If-None-Match Request Header. This way, the server can compare this value with the current value from ETag and, if both match, the server can return 304 (Not Modified) telling the browser to use the latest version of the file, or just 200 followed by the new content and the related ETag value instead.
However, I couldn't find any information in regards to using the Cache-Control: max-age header and how will this affect the above behavior, like:
Will the browser request for new updates before max-age has met? Meaning that I can define a higher max-age value (pagespeed/gmetrix will be happy about it) and force this refresh using only etag fingerprint.
If not, then what are the advantages of using etag and adding extra bits to the network?
No, the browser will not send any requests until max-age has passed.
The advantage of using ETag is that, if the file hasn't changed, you don't need to resend the entire file to the client. The response will be a small 304.
Note that you can achieve the best of both worlds by using the stale-while-revalidate directive, which allows stale responses to be served while the cache silently revalidates the resource in the background.

Does the ETag header make the Cache-Control header obsolete? How to make sure Cache-Control is not harmful then?

Definition of ETag header (https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag):
The ETag HTTP response header is an identifier for a specific version
of a resource. It allows caches to be more efficient, and saves
bandwidth, as a web server does not need to send a full response if
the content has not changed. On the other side, if the content has
changed, etags are useful to help prevent simultaneous updates of a
resource from overwriting each other ("mid-air collisions").
Definition of Cache-Control header (https://developer.mozilla.org/de/docs/Web/HTTP/Headers/Cache-Control):
The Cache-Control general-header field is used to specify directives
for caching mechanisms in both requests and responses.
So the ETag header tells the browser for a resource to send a single HTTP request to the server and ask if the file hash has changed. If yes, download a new one. Great. So if the ETag header is set why should I need Cache-Control any more (beside of the Expires header which may help to avoid this single request)?
So if I have to set the Cache-Control header anyway it can only be harmful right? I think the most appropriate value would be:
Cache-Control: must-revalidate
But I am not sure if this triggers unecessary additional actions.
After some research, I found a great tutorial on Medium by Alex Barashkov: "Best practices for cache control settings for your website".
Alex writes:
I recommend you apply Cache-Control: no-cache to html files. Applying
“no-cache” does not mean that there is no cache at all, it simply
tells the browser to validate resources on the server before use it
from the cache. That’s why we need to use it with Etag, so browsers
will send a simple request and load the extra 80 bytes to verify the
state of the file.
Presence of ETag header does not tell the browser to do anything. Browser decides what to do based on the Cache-Control header it receives in the request and cached response. If it decides that resource is stale or needs to be re-validated, then it can use the ETag value to create a conditional request to the server and either get a new resource (status code 200), or a notification that things have not changed (status code 304)
Both headers are necessary for your cache to work optimally.

Why isn't ETag alone enough to invalidate the browser cache?

I've read a lot of related articles on the matter and also the very good article about HTTP caching here:
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching?hl=en#invalidating-and-updating-cached-responses
but it is still not clear to me:
Why isn't sending an ETag header enough to invalidate the browser cache for a particular resource? Why does everyone recommend actually changing the URL/filename of the resource to force the browser to re-download the file? If the browser has already cached the file with a particular ETag and the ETag is modified on the server, wouldn't that suffice?
I find the following pages helpful:
https://jakearchibald.com/2016/caching-best-practices/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag
This line from MDN's ETag page shares the key point (emphasis added):
If a user visits a given URL again (that has an ETag set), and it is stale, that is too old to be considered usable, the client will send the value of its ETag along in an If-None-Match header field...
The ETag will be used by the client to revalidate resources once they become "stale". But what constitutes "stale"?
This is where the Cache-Control header comes in handy. The Cache-Control header can be sent with a response to let the client know how long the client may cache an item until it should be considered stale. For example, Cache-Control: no-cache would indicate that the resource should be considered stale immediately. See the MDN Cache-Control page for more information on available Cache-Control values.
When the browser attempts to process a request for a cached resource that is considered stale, it will first send a revalidation request to the server with the resource's last ETag value included via the If-None-Match request header, as described on MDN's ETag page. It can also use the Last-Modified response header sent as the If-Modified-Since request header as a secondary option.
If the server determines that the client's ETag value (in the If-None-Match request header) is current, then it will respond with a 304 (Not Modified) HTTP status code and an empty body, indicating that the client can use the cached entry. Otherwise, the server will respond with a 200 HTTP status code and the new response body.
Other resources:
Difference between no-cache and must-revalidate
What's default value of cache-control?
To answer your questions directly:
Why isn't sending an ETag header enough to invalidate the browser cache for a particular resource? -- Because the ETag header is not validated until the cached entry is considered stale, such as via an expiration date set in the Cache-Control response header.
Why does everyone recommend actually changing the URL/filename of the resource to force the browser to re-download the file? -- Changing the URL/filename or adding a query string will force the client to avoid using a cache. This is simple and is a virtually guaranteed way of cache-busting. This does not mean it's necessary, but it tends to be safe in the realm of inconsistent browser behaviors.
If the browser has already cached the file with a particular ETag and the ETag is modified on the server, wouldn't that suffice? -- Technically it should suffice as long as the appropriate Cache-Control headers (including the Pragma and Expires headers) are included. See How to control web page caching, across all browsers? for more details.

Is Cache-Control:must-revalidate obliging to validate all requests, or just the stale ones?

I have a mess with this header, I have read that Cache-Control:must-revalidate oblige to validate all requests with the source before serving a cached item, but just the stale ones? or all no matter if stale or fresh? I have read both things in different places.
What is the difference with Cache-Control:no-cache ? Because these headers look equivalent to me.
UPDATE 1: I have read this from a book:
The Cache-Control: must-revalidate response header tells the cache
to bypass the freshness calculation mechanisms and revalidate on every
access:
#Peter O. has pointed out what the RFC says. So that old book is wrong.
UPDATE 2: In this tutorial : http://www.mnot.net/cache_docs/
no-cache — forces caches to submit the request to the origin server
for validation before releasing a cached copy, every time. This is
useful to assure that authentication is respected (in combination with
public), or to maintain rigid freshness, without sacrificing all of
the benefits of caching.
must-revalidate — tells caches that they must
obey any freshness information you give them about a representation.
HTTP allows caches to serve stale representations under special
conditions; by specifying this header, you’re telling the cache that
you want it to strictly follow your rules.
Section 14.9.4 of HTTP/1.1:
When the must-revalidate directive is present in a response
received by
a cache, that cache MUST NOT use the entry after it becomes
stale
to respond to a subsequent request without first revalidating it
with the
origin server
Section 14.8 of HTTP/1.1:
If the response includes the "must-revalidate" cache-control
directive, the cache MAY use that response in replying to a
subsequent request. But if the response is stale, all caches
MUST first revalidate it with the origin server...
So it appears that only stale responses must be revalidated if
must-revalidate is received.
For no-cache, see section 14.9.1:
If the no-cache directive does not specify a field-name [which is
the case
here], then a cache MUST NOT use the response to satisfy a
subsequent
request without successful revalidation with the origin server...
Thus, no-cache applies both to fresh and stale responses.
EDIT:
This phrase may be relevant here (section 13.3):
When a cache has a stale entry that it would like to use as a response
to a client's request, it first has to check with the origin server
(or possibly an intermediate cache with a fresh response) to see if
its cached entry is still usable.
So, must-revalidate is probably relevant when the cache has intermediate
caches, since otherwise the cache can check the intermediate cache for a
fresh response rather than check the origin server directly.

ETag vs Header Expires

I've looked around but haven't been able to figure out if I should use both an ETag and an Expires Header or one or the other.
What I'm trying to do is make sure that my flash files (and other images and what not only get updated when there is a change to those files.
I don't want to do anything special like changing the filename or putting some weird chars on the end of the url to make it not get cached.
Also, is there anything I need to do programatically on my end in my PHP scripts to support this or is it all Apache?
They are slightly different - the ETag does not have any information that the client can use to determine whether or not to make a request for that file again in the future. If ETag is all it has, it will always have to make a request. However, when the server reads the ETag from the client request, the server can then determine whether to send the file (HTTP 200) or tell the client to just use their local copy (HTTP 304). An ETag is basically just a checksum for a file that semantically changes when the content of the file changes.
The Expires header is used by the client (and proxies/caches) to determine whether or not it even needs to make a request to the server at all. The closer you are to the Expires date, the more likely it is the client (or proxy) will make an HTTP request for that file from the server.
So really what you want to do is use BOTH headers - set the Expires header to a reasonable value based on how often the content changes. Then configure ETags to be sent so that when clients DO send a request to the server, it can more easily determine whether or not to send the file back.
One last note about ETag - if you are using a load-balanced server setup with multiple machines running Apache you will probably want to turn off ETag generation. This is because inodes are used as part of the ETag hash algorithm which will be different between the servers. You can configure Apache to not use inodes as part of the calculation but then you'd want to make sure the timestamps on the files are exactly the same, to ensure the same ETag gets generated for all servers.
Etag and Last-modified headers are validators.
They help the browser and/or the cache (reverse proxy) to understand if a file/page, has changed, even if it preserves the same name.
Expires and Cache-control are giving refresh information.
This means that they inform, the browser and the reverse in-between proxies, up to what time or for how long, they may keep the page/file at their cache.
So the question usually is which one validator to use, etag or last-modified, and which refresh infomation header to use, expires or cache-control.
Expires and Cache-Control are "strong caching headers"
Last-Modified and ETag are "weak caching headers"
First the browser check Expires/Cache-Control to determine whether or not to make a request to the server
If have to make a request, it will send Last-Modified/ETag in the HTTP request. If the Etag value of the document matches that, the server will send a 304 code instead of 200, and no content. The browser will load the contents from its cache.
Another summary:
You need to use both. ETags are a "server side" information. Expires are a "Client side" caching.
Use ETags except if you have a load-balanced server. They are safe and will let clients know they should get new versions of your server files every time you change something on your side.
Expires must be used with caution, as if you set a expiration date far in the future but want to change one of the files immediatelly (a JS file for instance), some users may not get the modified version until a long time!
One additional thing I would like to mention that some of the answers may have missed is the downside to having both ETags and Expires/Cache-control in your headers.
Depending on your needs it may just add extra bytes in your headers which may increase packets which means more TCP overhead. Again, you should see if the overhead of having both things in your headers is necessary or will it just add extra weight in your requests which reduces performance.
You can read more about it on this excellent blog post by Kyle Simpson: http://calendar.perfplanet.com/2010/bloated-request-response-headers/
In my view, With Expire Header, server can tell the client when my data would be stale, while with Etag, server would check the etag value for client' each request.
ETag is used to determine whether a resource should use the copy one. and Expires Header like Cache-Control is told the client that before the cache decades, client should fetch the local resource.
In modern sites, There are often offer a file named hash, like app.98a3cf23.js, so that it's a good practice to use Expires Header. Besides this, it also reduce the cost of network.
Hope it helps ;)
Etag is a hash for indicating the version of the resource. When the server returns data, it hashes the data and set this hash value under ETAG. When you send a "PUT" request to the server to update a record, maybe simultaneously another user made the same "PUT" request and its request has been processed. The server will check your "PUT" data and will see that it is the same update so it wont make another update, it will send you the updated data (by another user) and you will update your cache.
when the time for caching expires, the browser automatically makes a new request to get the fresh data. That is why "Expires" header is used
If a response includes both an Expires header and a max-age directive,
the max-age directive overrides the Expires header, even if the
Expires header is more restrictive. This rule allows an origin server
to provide, for a given response, a longer expiration time to an
HTTP/1.1 (or later) cache than to an HTTP/1.0 cache. This might be
useful if certain HTTP/1.0 caches improperly calculate ages or
expiration times, perhaps due to desynchronized clocks.

Resources