IIS supports two types of compression: static content compression and dynamic content compression. According to applicationHost.config, they are handled by different modules: DynamicCompressionModule (compdyn.dll) and StaticCompressionModule (compstat.dll), and they are configured to compress different types of requests. Also, I’m guessing that the dynamic compression does not cache the compressed requests as opposed the static compression (by default, the compressed files are saved to %SystemDrive%\inetpub\temp\IIS Temporary Compressed Files).
However, besides those obvious differences, I suspect that there is something else. I think that they hook to the IIS pipeline to a slightly different way. Would anybody have an inside into the some more details?
The way I found out was that I was toying with a custom module for modifying CSS files on fly. When the static compression was turned on (and set to handle the default set of files, i.e. also text/css), on cached request my custom module was served the already gzipped content. When I moved text/css to the list of dynamically compressed request, it all started working. But I would like to have a more solid proof that it is really the correct way to do it. Are there some other known consequences / problems?
Update: I think I may have a theory as to why it’s happening. It may not be 100% correct, but it at least it may explain the observed behavior. I think that the static compression module registers itself to the following events (among some others):
RQ_MAP_REQUEST_HANDLER
RQ_EXECUTE_REQUEST_HANDLER
Then when a request for a static file is served, the static compression module in OnMapRequestHandler checks whether the file has been compressed before and whether the actual file has not been changed. If so, it will re-map the request to itself (returning the appropriate redirection using IMapHandlerProvider). When it later actually serves the response in OnExecuteRequestHandler, it sends the compressed file. If, on the other hand, the file has not been compressed before or if it has changed, it does not do the mapping redirect and lets the static content module serve the request and then later in OnPostExecuteRequestHandler compresses the content (and updates its cache). As mentioned above, I'm not saying that this is exactly what's happening (I don't know the source code), it may be only an approximation. Also, the dynamic compression module does not most likely do any of this. It simply compresses the outgoing responses sometimes after RQ_EXECUTE_REQUEST_HANDLER.
Your question is not quite clear, so I will answer a question and hope that it is your question.
The intent of static compression is to compress files that would otherwise be served direct from the hard drive (Css/images/javascript) and as such it compresses each file once and saves the compressed file to disk. This enables very quick very cheap serving of compressed content for static files that change infrequently. It is a pretty safe recommendation to say most website should have static compression enabled.
The intent of dynamic compression is to compress the dynamic responses from ISS modules (asp, asp.net, php, etc.). As this response can be different for each request the compressed output can not be cached. This feature is new from IIS6, though the effect has been achievable in some environments, e.g. by implementing a HttpFilter in ASP.Net. As each request needs to be compressed on the fly this is far more CPU intensive then static compression. So if a server is CPU bound this may not be a good option. Most sites are network and/or database bound so the is often a good idea.
So the dynamic and static refer to the content and effect what strategies can be used.
Some References
http://www.iis.net/overview/DynamicCachingAndCompression
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/92f627a8-4ec3-480c-b0be-59a561091597.mspx?mfr=true
Experimenting with IIS compression feature, it appeared to me that dynamic module and static module are not so tied to dynamic or static content (especially for dynamic module).
Activate compression for text/html (or text/*) mime type on dynamic module, and not on static module. Access a .html file. Checks the http response in browser: it is compressed. (Tested on IIS 7.5 on 2008R2 server.)
It appears dynamic compression module is not limited to dynamic content. It does compressed static content providing it matches its mime types list and is not already compressed. So I consider it should be understood as a dynamic "compression module", in the sense it gets triggered on each response (based on its mime type criteria, and on accept-encoding request header).
While the static compression module gets triggered a bit like a background process working on files, and starts serving compressed output only once it has them in its cache. Since static compression module does execute farer in the modules stack, it does handle the response prior to dynamic compression module, and so take precedence over dynamic one if it has compressed output to serve.
So for your specific use case, you should disable static compression module on text/css mime type (beware of removing text\* too if present) in order to avoid caching issues defeating your custom css patching module.
You may in addition activate compression of text/css in dynamic compression module for replacing static compression module on this case. But of course, it will then not leverage the static compression module caching ability.
Unfortunately, I have not found any documentation for backing up above statements.
Another option could be trying to alter IIS module execution order. You would have to remove them all in your site configuration, then re-add them, inserting your custom module maybe before the static compression one. But this could be a tricky path.
Related
This discussion around gzip, minified, and combining both for maximum compression effects made me wonder if I can get a net performance effect by placing jQuery libraries directly inside my HTML file?
I already do this for icon sprites, which I base64 encode and then load via CSS datasource, in an effort to minimize the number of HTTP requests.
Now I ask: do you know if loading the jQuery library in its raw min+gzip form, as available on the official production download link, would be more efficient compared to loading it via the Google AJAX libraries that are hosted on their CDN?
TL;DR:
You should always keep common items (like a JS library) in external files, because that allows for caching on your server, in the browser, and at many nodes along the way. Caching times dramatically outweigh compression or request times when looking at the overall speed of a page.
Longer Version:
There are a number of things to consider for this question. Here are the ones that came off the top of my head.
Minifying scripts and styles is always good for speed. I don't think there's much to discuss here.
Compression (and extraction in the browser) has an overhead. Firing up the gzip module takes time away from sending the request, so for small files or binary files (images, etc.), it's usually better to skip the gzip. Something large and consisting mostly of ascii (like a JS library), however, is a good thing to compress, so that checks out.
Reducing HTTP requests is generally a good thing to do. Note, though, that ever since HTTP 1.1 (supported in all major browsers), a new HTTP request does not mean a new socket connection. With keep-alive, the same socket connection can serve the webpage and all the affiliated files in one go. The only extra overhead on a request is the request and response headers which can be negligible unless you have a lot of small images/scripts. Just make sure these files are served from a cookie-free domain.
Related to point 3, embedding content in the page does have the benefit of reducing HTTP requests, but it also has the drawback of adding to the size of the page itself. For something large like a JS library, the size of library vastly outweighs the size of the HTTP overhead needed for an extra file.
Now here's the kicker: if the library is embedded in the page, it will almost never be cached. For starters, most (all?) major browsers will NOT cache the main HTML page by default. You can add a CACHE-CONTROL meta tag to the page header if you want, but sometimes that's not an option. Furthermore, even if that page is cached, the rest of your pages (that are probably also using the library) will now need to be cached as well. Suddenly you have your whole site cached on the user's machine including many copies of the JS library, and that's probably not what you want to do to your users. On the other hand, if you have the JS library as an external file, it can be cached by itself just once, especially (as #Barmar says) if the library is loaded from a common CDN like Google.
Finally, caching doesn't only happen in the client's browser. If you're running an enterprise-level service, you may have multiple server pools with shared and individual caches, CDN caches that have blinding speed, proxy caches on the way to your domain, and even LAN caches on some routers and other devices. So now I direct you back to the TL;DR at the top, to say that the speed-up you can gain by caching (especially in the browser) trumps any other consideration you are likely to encounter.
Hope that helps.
How can I make sure that static content (images, css, javascript) is cached? What is the best approach?
Will recommend you to go through this tutorial to understand how caching happens on web (HTTP) in general.
Simply speaking, the web server needs to generate appropriate HTTP headers while sending the content to the client in order to control client-side caching. In ASP.NET/IIS environment, its IIS that typically handles the static file contents and therefore, you must configure IIS appropriately to control caching static files as per you needs. See below links for more information about configuring IIS caching for static content:
http://www.iis.net/ConfigReference/system.webServer/staticContent/clientCache
How to configure static content cache per folder and extension in IIS7?
EDIT: As you have asked about the best approach, the most prevalent approach that I see now days is to version static content (say by appending some version identifier at the end of file or URL). Once version-ed, you can treat it as immutable and then emit cache headers for caching it for infinite duration. In ASP.NET application, you can probably append the assembly version (or product version) to each static content URL. So essentially, you will invalidating the cache for every build (or every product release).
You can also make use of the HTML5 Offline web applications manifest. It allows you to set up a manifest where you define which files will be cached locally.
It is a nice, clear to understand broadly implemented, way of avoiding having to learn about IIS and HTML Caching.
http://www.w3schools.com/html/html5_app_cache.asp
(you should totally read up about those things)
I'm working with a custom webserver on an embedded system and having some problems correctly setting my HTTP Headers for caching.
Our webserver is generating all dynamic content as XML and we're using semi-static XSL files to display it with some dynamic JSON requests thrown in for good measure along with semi-static images. I say "semi-static" because the problems occur when we need to do a firmware update which might change the XSL and image files.
Here's what needs to be done: cache the XSL and image files and do not cache the XML and JSON responses. I have full control over the HTTP response and am currently:
Using ETags with the XSL and image files, using the modified time and size to generate the ETag
Setting Cache-Control: no-cache on the XML and JSON responses
As I said, everything works dandy until a firmware update when the XSL and image files are sometimes cached. I've seen it work fine with the latest versions of Firefox and Safari but have had some problems with IE.
I know one solution to this problem would be simply rename the XSL and image files after each version (eg. logo-v1.1.png, logo-v1.2.png) and set the Expires header to a date in the future but this would be difficult with the XSL files and I'd like to avoid this.
Note: There is a clock on the unit but requires the user to set it and might not be 100% reliable which is what might be causing my caching issues when using ETags.
What's the best practice that I should employ? I'd like to avoid as many webserver requests as possible but invalidating old XSL and image files after a software update is the #1 priority.
Are we working on the same project? I went down a lot of dead ends figuring out the best way to handle this.
I set my .html and my .shtml files (dynamic JSON data) to expire immediately. ("Cache-Control: no-cache\r\nExpires: -1\r\n")
Everything else is set to expire in 10 years. ("Cache-Control: max-age=290304000\r\n")
My makefile runs a perl script over all the .html files and identifies what you call "semi-static" content (images, javascript, css.) The script then runs a md5 checksum on those files and appends the checksum to the file:
<script type="text/Javascript" src="js/all.js?7f26be24ed2d05e7d0b844351e3a49b1">
Everything after the question mark is ignored, but no browser will cache it unless everything between the quotes matches.
I use all.js and all.css because everything's combined and minified using the same script.
Out of curiosity, what embedded webserver are you using?
Try Cache-Control: no-store. no-cache tells the client that the response can be cached; it just generally isn't reused unless the cache can't contact the origin server.
BTW, setting an ETag alone won't make the response cacheable; you should also set Cache-Control: max-age=nnn.
You can check how your responses will be treated with http://redbot.org/
An architect at my work recently read Yahoo!'s Exceptional Performance Best Practices guide where it says to use a far-future Expires header for resources used by a page such as JavaScript, CSS, and images. The idea is you set a Expires header for these resources years into the future so they're always cached by the browser, and whenever we change the file and therefore need the browser to request the resource again instead of using its cache, change the filename by adding a version number.
Instead of incorporating this into our build process though, he has another idea. Instead of changing file names in source and on the server disk for each build (granted, that would be tedious), we're going to fake it. His plan is to set far-future expires on said resources, then implement two HttpModules.
One module will intercept all the Response streams of our ASPX and HTML pages before they go out, look for resource links and tack on a version parameter that is the file's last modified date. The other HttpModule will handle all requests for resources and simply ignore the version portion of the address. That way, the browser always requests a new resource file each time it has changed on disk, without ever actually having to change the name of the file on disk.
Make sense?
My concern relates to the module that rewrites the ASPX/HTML page Response stream. He's simply going to apply a bunch of Regex.Replace() on "src" attributes of <script> and <img> tags, and "href" attribute of <link> tags. This is going to happen for every single request on the server whose content type is "text/html." Potentially hundreds or thousands a minute.
I understand that HttpModules are hooked into the IIS pipeline, but this has got to add a prohibitive delay in the time it takes IIS to send out HTTP responses. No? What do you think?
A few things to be aware of:
If the idea is to add a query string to the static file names to indicate their version, unfortunately that will also prevent caching by the kernel-mode HTTP driver (http.sys)
Scanning each entire response based on a bunch of regular expressions will be slow, slow, slow. It's also likely to be unreliable, with hard-to-predict corner cases.
A few alternatives:
Use control adapters to explicitly replace certain URLs or paths with the current version. That allows you to focus specifically on images, CSS, etc.
Change folder names instead of file names when you version static files
Consider using ASP.NET skins to help centralize file names. That will help simplify maintenance.
In case it's helpful, I cover this subject in my book (Ultra-Fast ASP.NET), including code examples.
He's worried about stuff not being cached on the client - obviously this depends somewhat on how the user population has their browsers configured; if it's the default config then I doubt you'd need to worry about trying to second guess the client caching, it's too hard and the results aren't guaranteed, also it's not going to help new users.
As far as the HTTP Modules go - in principle I would say they are fine, but you'll want them to be blindingly fast and efficient if you take that track; it's probably worth trying out. I can't speak on the appropriateness of use RegEx to do what you want done inside, though.
If you're looking for high performance, I suggest you (or your architect) do some reading (and I don't mean that in a nasty way). I learnt something recently which I think will help -let me explain (and maybe you guys know this already).
Browsers only hold a limited number of simultaneous connections open to a specific hostname at any one time. e.g, IE6 will only do 6 connections to say www.foo.net.
If you call your images from say images.foo.net you get 6 new connections straight away.
The idea is to seperate out different content types into different hostnames (css.foo.net, scripts.foo.net, ajaxcalls.foo.net) that way you'll be making sure the browser is really working on your behalf.
http://code.google.com/p/talifun-web
StaticFileHandler - Serve Static Files in a cachable, resumable way.
CrusherModule - Serve compressed versioned JS and CSS in a cachable way.
You don't quite get kernel mode caching speed but serving from HttpRuntime.Cache has its advantages. Kernel Mode cache can't cache partial responses and you don't have fine grained control of the cache. The most important thing to implement is a consistent etag header and expires header. This will improve your site performance more than anything else.
Reducing the number of files served is probably one of the best ways to improve the speed of your website. The CrusherModule combines all the css on your site into one file and all the js into another file.
Memory is cheap, hard drives are slow, so use it!
I'm trying to find the best way to speed up the delivery of the static images that compose the design of an mvc site. The images are not gzipped, nor cached in the server or on the client (with content expire). Options are:
Find why images are not cached and gzipped direcly from IIS6
Write a specialized http handler
Register a special route for static images and write a bynary actionresult method
What could be the best solution in terms of performance?
Best solution is to let IIS do it.
IIS6 Compression - most likely you need to specify file types to be compressed like .jpg, .png, .gif types, etc.
Caching will come from making sure that the correct headers are being sent to the client from code, and i believe there is a setting you can set in IIS that enables it for static content, but i'm not sure on that one.
Surely the gain from gzipping most images is negligable since they're already compressed ?
Naybe you have some really badly compressed PNG files or something?
You might want to check out yahoo's performance advice site which includes some useful tips on optimizing images including links to utilities such as pngcrush.
its much better to use an image optimizing utility ONCE than to rely on IIS to compress them (possibly inefficiently) on the fly.
There's a nice library up on the MSDN Code Gallery that does this. It's called FastMVC.