This is just making me angry. I can't figure out why all the resources in my page are being requested EVERY single time.
E.g. my site.css returns the following headers (using fiddler):
HTTP/1.1 200 OK
Server: ASP.NET Development Server/9.0.0.0
Date: Mon, 29 Nov 2010 17:36:21 GMT
X-AspNet-Version: 2.0.50727
Content-Length: 9093
Cache-Control: public, max-age=2592000
Expires: Wed, 29 Dec 2010 17:36:21 GMT
Last-Modified: Mon, 08 Nov 2010 17:20:16 GMT
Content-Type: text/css
Connection: Close
But every time I hit refresh I see all the resources (css,js,images) getting re-requested. I have control over the headers returned for any and all of these resources, but I haven't figured it out yet.
I have even debugged my ASP.NET app and the HttpModule is definitely being asked for the resources again.
Can someone give me an idea of what to do?
EDIT:
Ok, I removed must-revalidate, proxy-revalidate from the headers and that is getting me closer to where I want to be, now when I press back it still requests my css/js files when I press back.
Is there anything more I can do to avoid this?
The following links might be of help to you.
Differences in reload behavior between FF and IE
http://blog.httpwatch.com/2008/10/15/two-important-differences-between-firefox-and-ie-caching/
In a nutshell, your caching behavior is going to be determined by the headers and the browser you are using.
What browser are you using for testing? The back button is also handled differently.
Back Button (Browser Behavior)
And, finally, a breakdown of f5/ctrl f5, click, shift click, etc behavior between browsers:
What requests do browsers' "F5" and "Ctrl + F5" refreshes generate?
If you are handling the requests in your own module - which seems to be the case - and the request contains an If-Modified-Since header, you can use that to determine whether to respond with a 200 and sending the whole resource again, or just send a 304 and skip sending the js/css/etc contents.
Other than that, expect browsers to re-query resources on hitting F5 / Refresh. It is just that you may skip sending the whole js/css/etc and return a 304 if everything is OK.
Other than that, #Chris's answer covers pretty much everything else.
What are the request headers you see when you hit back?
Related
I'm trying to add HTTP Conditional Get functionality to the first page of a mobile site. However, what I'm seeing is making me think that Safari simply doesn't support HTTP Conditional Get for the first document it loads.
I tested this by making a new Rails 3.2.13 application, with one action that looks like this:
def index
last_modified = Date.today
etag_obj = "something something"
fresh_when last_modified: last_modified, etag: etag_obj, public: false
end
The request headers contain both an Etag and Last-Modified, as expected:
[cache_test]$ curl -I http://localhost:5000/test
HTTP/1.1 200 OK
Etag: "79b547467286b3e20fad13f73fc1bf78"
Last-Modified: Wed, 24 Apr 2013 00:00:00 GMT
Content-Type: text/html; charset=utf-8
Cache-Control: max-age=0, private, must-revalidate
X-Ua-Compatible: IE=Edge
X-Request-Id: 8ace0b71533b2bc036bd84b12289cda6
X-Runtime: 0.012269
Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-04-20)
Date: Wed, 24 Apr 2013 21:06:49 GMT
Content-Length: 0
Connection: Keep-Alive
Set-Cookie: _cache_test_session=BAh7B0kiD3Nlc3Npb25faWQGOgZFRkkiJTM0NWRjNDgyZGQwNWJmMGZkNmIzNmY4NjVmNWEzMDkwBjsAVEkiEF9jc3JmX3Rva2VuBjsARkkiMTlYK0VrMERGdEsvOVlzbVJ5N1FPQlZXVDRwak9zQ3QzS3ZXZ1pvbnJocjg9BjsARg%3D%3D--0902ae8a505527bd49d969753ddf1d2bd4c716eb; path=/; HttpOnly
On Firefox 20.0 and Chrome 26.0.1410.65 this acts like I expect: On the second load, the browser sends If-Modified-Since and If-None-Match in the request, and receives a 304 Not Modified.
However, it's not doing this on Safari. On Safari it sends neither If-Modified-Since nor If-None-Match, and as you'd expect, it then fully reloads the page from the server. I am seeing this on Safari 6.0.3 on my MacBook Pro, Safari in the iOS Simulator using an iPhone with iOS 6.1, and on an iPad Mini with iOS 6.1.3.
And what's really strange about this is that Safari clearly supports Conditional Get for subsequent requests (CSS, JS, images, etc) -- I can see them clearly after the main HTML page comes through. But I can't for the life of me figure out how to get Safari to make a Conditional Get request for the first document.
So, I guess my question is two parts:
Am I missing something obvious? Because it actually seems a little crazy that Safari would act this way by design. But maybe I'm expecting too much from Apple here.
If I'm not missing something obvious, does anybody know of any workarounds?
When I visit chesseng.herokuapp.com I get a response header that looks like
Cache-Control:private
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/css
Date:Tue, 16 Oct 2012 06:37:53 GMT
Last-Modified:Tue, 16 Oct 2012 03:13:38 GMT
Status:200 OK
transfer-encoding:chunked
Vary:Accept-Encoding
X-Rack-Cache:miss
and then I refresh the page and get
Cache-Control:private
Connection:keep-alive
Date:Tue, 16 Oct 2012 06:20:49 GMT
Status:304 Not Modified
X-Rack-Cache:miss
so it seems like caching is working. If that works for caching then what is the point of Expires and Cache-Control:max-age. To add to confusion, when I test the page at https://developers.google.com/speed/pagespeed/insights/ it tells me to "Leverage browser caching".
Cache-Control: private
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache, such as a proxy server.
From RFC2616 section 14.9.1
To answer your question about why caching is working, even though the web-server didn't include the headers:
Expires: [a date]
Cache-Control: max-age=[seconds]
The server kindly asked any intermediate proxies to not cache the contents (i.e. the item should only be cached in a private cache, i.e. only on your own local machine):
Cache-Control: private
But the server forgot to include any sort of caching hints:
they forgot to include Expires (so the browser knows to use the cached copy until that date)
they forgot to include Max-Age (so the browser knows how long the cached item is good for)
they forgot to include E-Tag (so the browser can do a conditional request)
But they did include a Last-Modified date in the response:
Last-Modified: Tue, 16 Oct 2012 03:13:38 GMT
Because the browser knows the date the file was modified, it can perform a conditional request. It will ask the server for the file, but instruct the server to only send the file if it has been modified since 2012/10/16 3:13:38:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
The server receives the request, realizes that the client has the most recent version already. Rather than sending the client 200 OK, followed by the contents of the page, it instead tells you that your cached version is good:
304 Not Modified
Your browser did have to suffer the round-trip delay of sending a request to the server, and waiting for the response, but it did save having to re-download the static content.
Why Max-Age? Why Expires?
Because Last-Modified sucks.
Not everything on the server has a date associated with it. If I'm building a page on the fly, there is no date associated with it - it's now. But I'm perfectly willing to let the user cache the homepage for 15 seconds:
200 OK
Cache-Control: max-age=15
If the user hammers F5, they'll keep getting the cached version for 15 seconds. If it's a corporate proxy, then all 67,198 users hitting the same page in the same 15-second window will all get the same contents - all served from close cache. Performance win for everyone.
The virtue of adding Cache-Control: max-age is that the browser doesn't even have to perform a "conditional" request.
if you specified only Last-Modified, the browser has to perform a If-Modified-Since request, and watch for a 304 Not Modified response
if you specified max-age, the browser won't even have to suffer the network round-trip; the content will come right out of the caches.
The difference between "Cache-Control: max-age" and "Expires"
Expires is a legacy (c. 1998) equivalent of the modern Cache-Control: max-age header:
Expires: you specify a date (yuck)
max-age: you specify seconds (goodness)
And if both are specified, then the browser uses max-age:
200 OK
Cache-Control: max-age=60
Expires: 20180403T192837
Any web-site written after 1998 should not use Expires anymore, and instead use max-age.
What is ETag?
ETag is similar to Last-Modified, except that it doesn't have to be a date - it just has to be a something.
If I'm pulling a list of products out of a database, the server can send the last rowversion as an ETag, rather than a date:
200 OK
ETag: "247986"
My ETag can be the SHA1 hash of a static resource (e.g. image, js, css, font), or of the cached rendered page (i.e. this is what the Mozilla MDN wiki does; they hash the final markup):
200 OK
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
And exactly like in the case of a conditional request based on Last-Modified:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
304 Not Modified
I can perform a conditional request based on the ETag:
GET / HTTP/1.1
If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4"
304 Not Modified
An ETag is superior to Last-Modified because it works for things besides files, or things that have a notion of date. It just is
RFC 2616, section 14.9.1:
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache...A private (non-shared) cache MAY cache the response.
Browsers could use this information. Of course, the current "user" may mean many things: OS user, a browser user (e.g. Chrome's profiles), etc. It's not specified.
For me, a more concrete example of Cache-Control: private is that proxy servers (which typically have many users) won't cache it. It is meant for the end user, and no one else.
FYI, the RFC makes clear that this does not provide security. It is about showing the correct content, not securing content.
This usage of the word private only controls where the response may be cached, and cannot ensure the privacy of the message content.
The Expires entity-header field gives the date/time after which the response is considered stale.The Cache-control:maxage field gives the age value (in seconds) bigger than which response is consider stale.
Althought above header field give a mechanism to client to decide whether to send request to the server. In some condition, the client send a request to sever and the age value of response is bigger then the maxage value ,dose it means server needs to send the resource to client? Maybe the resource never changed.
In order to resolve this problem, HTTP1.1 gives last-modifided head. The server gives the last modified date of the response to client. When the client need this resource, it will send If-Modified-Since head field to server. If this date is before the modified date of the resouce, the server will sends the resource to client and gives 200 code.Otherwise,it will returns 304 code to client and this means client can use the resource it cached.
I want browsers to always add (except first time) "If-Modified-Since" request header to avoid unnecessary traffic.
Response headers are:
Accept-Ranges:bytes
Cache-Control:max-age=0, must-revalidate
Connection:Keep-Alive
Content-Length:2683
Content-Type:text/html; charset=UTF-8
Date:Thu, 05 Apr 2012 13:06:19 GMT
Keep-Alive:timeout=15, max=497
Last-Modified:Thu, 05 Apr 2012 13:05:11 GMT
Server:Apache/2.2.21 (Red Hat)
FF 11 and IE 9 both send "If-Modified-Since" and get 304 in response but Chrome 18 doesn't and get 200.
Why? How to force Chrome to sent "If-Modified-Since" header?
I do not know if it important or not but all requests going through HTTPS.
I've been chasing this issue for some time, thought I'd share what I found.
"The rule is actually quite simple: any error with the certificate means the page will not be cached."
https://code.google.com/p/chromium/issues/detail?id=110649
If you are using a self-signed certificate, even if you tell Chrome to add an exception for it so that the page loads, no resources from that page will be cached, and subsequent requests will not have an If-Modified-Since header.
I just now found this question, and after puzzling over Chrome's If_Modified_Since behavior, I've found the answer.
Chrome's decision to cache files is based on the Expires header that it receives. The Expires header has two main requirements:
It must be in Greenwich Mean Time (GMT), and
It must be formatted according to RFC 1123 (which is basically RFC 822 with a four-digit year).
The format is as follows:
Expires: Sat, 07 Sep 2013 05:21:03 GMT
For example, in PHP, the following outputs a properly formatted header.
$duration = time() + 3600 // Expires in one hour.
header("Expires: " . gmdate("D, d M Y H:i:s", $duration) . " GMT");
("GMT" is appended to the string instead of the "e" timezone flag because, when used with gmdate(), the flag will output "UTC," which RFC 1123 considers invalid. Also note that the PHP constants DateTime::RFC1123 and DATE_RFC1123 will not provide the proper formatting, since they output the difference to GMT in hours [i.e. +02:00] rather than "GMT".)
See the W3C's date/time format specifications for more info.
In short, Chrome will only recognize the header if it follows this exact format. This, combined with the Cache-Control header...
header("Cache-Control: private, must-revalidate, max-age=" . $duration);
...allowed me to implement proper cache control. Once Chrome recognized those headers, it began caching the pages I sent it (even with query strings!), and it also began sending the If_Modified_Since header. I compared it to a stored "last-modified" date, sent back HTTP/1.1 304 Not Modified, and everything worked perfectly.
Hope this helps anyone else who stumbles along!
I've noticed almost the same behaviour and my findings are:
First of all the 200 status indicator in chrome is not the whole truth, you need to look at the "Size Content" column as well. If this says "(from cache)" the resource was take directly from cache without even asking if it was modified.
This caching behaviour of resources that lack any indication of expires or max-age seems to apply when requesting static files that have a last-modified header. I've noted that chrome (ver. 22):
Asks for the file the first time (obviously since it is not in cache).
Asks if it is modified the second time (since it is in cache but has no indication of freshness).
Uses it directly the third time and then on (even if it is a new browser session).
I'm a bit puzzled by this behaviour but it is fairly reasonable, if it is static, was modified a long time ago, and hasn't changed since last check you could assume that it is going to be valid for a while longer (don't know how they calculate it though).
I had the same problem, in Chrome all requests were always status code 200, in other browsers 304.
It turned out I had the disable cache (while DevTools is open) checked in on Devtools - Settings - General page..:)
Don't disable cache in Chrome Dev Tools (on "Network" tab).
Cache-Control should be Cache-Control: public. Use true as second parameter to header PHP function: header("Cache-Control: public", true);
I have also found that Chrome (recent v95+) also returns a cached 200 response if I have DevTools open. It never even sends the request on to the server! If I close DevTools, the behaviour is as it should, and the server receives the expected request.
I have these headers being sent to the client by the server:
Cache-Control:private
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/html
Date:Sun, 27 Nov 2011 11:10:38 GMT
ETag:"12341234"
Set-Cookie:connect.sid=e1u...7o; path=/; expires=Sun, 27 Nov 2011 11:40:38 GMT; httpOnly
Transfer-Encoding:chunked
last-modified:Sat, 26 Nov 2011 21:42:45 GMT
I want the client to validate that the file hasn't changed on the server and send a "200" if it has otherwise a "304".
Firefox sends:
if-modified-since: Sat, 26 Nov 2011 21:42:45 GMT
if-none-match: "12341234"
Why isn't the chrome sending the same on a refresh of the page? I'm after the behavior that .Net has running:
context.Response.Cache.SetCacheability(HttpCacheability.ServerAndPrivate)
After spending half a day on this yesterday, I tracked down what was causing the issue for me. So long as you have the Chrome object inspector/Client Debugger/Network monitor/Thing that pops up when you hit F12, Chrome will not send cache request headers. Period. (update: in newer versions of Chrome, there is a checkbox "Disable cache"). Even if you don't have the "network" tab open (ex: have the javascript console open), this checkbox still disables all cacheing.
Its sad, because debugging this from the client side obligates you to leave the network panel open to see what headers are being sent and received, and what codes are being returned. Without the network panel open, there is no way to know if your content is being cached from the client side.
If you dig into your server access logs, you will notice your server returning 304s(Cached Content) the minute you close the debug window on your Chrome client. Hope this helps.
Chrome 24.0.1312.57
I found one answer to this behaviour when using HTTPS, thought I'd share what I found. You do not specify if you are requesting via HTTP or HTTPS.
"The rule is actually quite simple: any error with the certificate means the page will not be cached."
https://code.google.com/p/chromium/issues/detail?id=110649
If you are using a self-signed certificate, even if you tell Chrome to add an exception for it so that the page loads, no resources from that page will be cached, and subsequent requests will not have an If-Modified-Since header.
In my experience you need more than just the "Private" Cache-Control header. You need either "Max-Age" or "Expires" to force Chrome to revalidate content with the server.
Remember that revalidation will only start after these time values have elapsed, so they may need to be set to a small value.
In addition (https://stackoverflow.com/a/14899869/362780):
F12 > Settings > General > Disable cache (while DevTools is open) -> uncheck this...
Browsers have a lot of counter intuitive behavior when it comes to caching. You would expect, that if the response includes a last-modified-date, that the browser would revalidate this before reusing it. But none of the major browsers actually do that.
The ideal settings for your situation depend on when you want the browser to revalidate, see link below.
Not only do browsers act counter intuitively, different browsers also behave differently in the same situation. For example when the user clicks on the refresh button.
You can read how the different browsers (Internet Explorer, Edge, Safari, FireFox, Chrome) behave with different caching directives (Etag, last-modified, must-revalidate, expires, max-age, no-cache, no-store) at https://gertjans.home.xs4all.nl/javascript/cache-control.html
I know this question is old, but still..
I noticed that chrome remembers the last refresh that you made. So if you press ctrl+shift+r (refreshing and deleting cache), and pressing ctrl+r (just refreshing), chrome keeps on deleting cache and does not show the 304 in the received response. There is a workaround for this. Press ctrl+shift+r and then go to the address bar, focus it, and hit enter. If your etags are set correctly, and your server is ready to serve a 304, you'll see a new response code in the debugger - 304. so it works.
I'm not sure whether this is a server issue, or whether I'm failing to understand how HTTP caching really works.
I have an ASP MVC application running on IIS7. There's a lot of static content as part of the site including lots of CSS, Javascript and image files.
For these files I want the browser to cache them for at least a day - our .css, .js, .gif and .png files rarely change.
My web.config goes like this:
<system.webServer>
<staticContent>
<clientCache cacheControlMode="UseMaxAge"
cacheControlMaxAge="1.00:00:00" />
</staticContent>
</system.webServer>
The problem I'm getting is that the browser (tested Chrome, IE8 and FX) doesn't seem to be caching the files as I'd expect. I've got the default settings (check for newer pages automatically in IE).
On first visit the content downloads as expected
HTTP/1.1 200 OK
Cache-Control: max-age=86400
Content-Type: image/gif
Last-Modified: Fri, 07 Aug 2009 09:55:15 GMT
Accept-Ranges: bytes
ETag: "3efeb2294517ca1:0"
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Mon, 07 Jun 2010 14:29:16 GMT
Content-Length: 918
<content>
I think that the Cache-Control: max-age=86400 should tell the browser not to request the page again for a day.
Ok, so now the page is reloaded and the browser requests the image again. This time it gets an empty response with these headers:
HTTP/1.1 304 Not Modified
Cache-Control: max-age=86400
Last-Modified: Fri, 07 Aug 2009 09:55:15 GMT
Accept-Ranges: bytes
ETag: "3efeb2294517ca1:0"
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Mon, 07 Jun 2010 14:30:32 GMT
So it looks like the browser has sent the ETag back (as a unique id for the resource), and the server's come back with a 304 Not Modified - telling the browser that it can use the previously downloaded file.
It seems to me that would be correct for many caching situations, but here I don't want the extra round trip. I don't care if the image gets out of date when the file on the server changes.
There are a lot of these files (even with sprite-maps and the like) and many of our clients have very slow networks. Each round trip to ping for that 304 status is taking about a 10th to a 5th of a second. Many also have IE6 which only has 2 HTTP connections at a time. The net result is that our application appears to be very slow for these clients with every page taking an extra couple of seconds to check that the static content hasn't changed.
What response header am I missing that would cause the browser to aggressively cache the files?
How would I set this in a .Net web.config for IIS7?
Am I misunderstanding how HTTP caching works in the first place?
You need to use the Expires directive, otherwise the browser will always check to see if the content has updated.
If a cached entry has a valid expiration date the browser can reuse the content without having to contact the server at all when a page or site is revisited. This greatly reduces the number of network round trips for frequently visited pages. For example, the Google logo is set to expire in 2038 and will only be downloaded on your first visit to google.com or if you have emptied your browser cache. If they ever want to change the image they can use a different image file name or path.
To change in IIS7 use following. This is easiest to manage if you keep static content in specific directories.
Log onto the server
Open IIS Manager (start -> adminstrative tools -> iis manager
Expand the server node
Expand the sites node
Open the site and navigate to the directory you want to change
Open the IIS HTTP Response Headers section
Click Set Common Headers on the task pane on the right
Set "Expire Web Content" as your app requires.
use expires header instead of using the cache-control.Tell your server for the first time that serve me content from my browser cache until this expiry date. There will be no cross checking for changes in file until your expiry date.
add the header in your web.config’s system.webServer section like so:
<system.webServer>
<staticContent>
<clientCache httpExpires="Sun, 29 Mar 2020 00:00:00 GMT"
cacheControlMode="UseExpires" />;
</staticContent>
</system.webServer>
Short anwser: remove Etag and use Expire header.
You should check out the 35 Yahoo Performance best practices, and more specifically:
Configure ETags on your server (removing them is a good choice)
Add Expire header for static resources
For each rule, they usually cover Apache and IIS web server configurations.
Edit: okay, it looks like there is no simple way to remove Etags in IIS, besides installing some 3rd party software...