I have a library of reusable partial views, scripts and images that are embedded in assembly and shared between projects.
Everything works fine, I've modified web.config to make all necessary file types to be served by System.Web.StaticFileHandler, but unfortunately, it serves all resources with Cache-Control: private.
I can write my own StaticFileHandler that would serve VPP content with Cache-Control: public and expiration date.
How do I implement caching support using VirtualPathProvider.GetCacheDependency?
I figured out why this happens. I looked at the source for the StaticFileHandler. For embedded files, it doesn't set any of the caching headers. It only does for files in the file system. Meaning this will never work right.
You have two options.
1.Find another http handler. I have never used this but it has come up in my searching: https://code.google.com/p/talifun-web/wiki/StaticFileHandler
2.Create an http module that checks to see if the static file handler was used, if so set the caching details.
Good luck.
Based on my researches, I found a hacky way to achieve caching.
In your VPP implementation, you should be implementing your own VirtualFile class, extending System.Web.Hosting.VirtualFile base class. It just expects a stream to read file if it is needed from VPP. At that phase, you can inject headers and even change cachability of resource. Because priorly, if I request a static file from VPP, it was coming with a header Cache-Control:private. Actually, server was saying that: I do not care your local caches, etags and so on. I decide whether you should cache it or not. The code below changes it to public and add required e-tag header so that it should stay at cache unless that assembly is changed:
class EmbeddedResourceVirtualFile : VirtualFile
{
readonly EmbeddedResource embedded;
public EmbeddedResourceVirtualFile(string virtualPath, EmbeddedResource embedded)
: base(virtualPath)
{
this.embedded = embedded;
}
public override Stream Open()
{
var assemblyLastModified = embedded.AssemblyLastModified;
var etag = assemblyLastModified.Ticks;
var response = HttpContext.Current.Response;
var cache = response.Cache;
cache.SetCacheability(HttpCacheability.Public);
cache.SetETag(etag.ToString());
cache.SetLastModified(assemblyLastModified);
cache.SetExpires(assemblyLastModified.AddYears(2));
return embedded.GetStream();
}
}
Special thanks for the commentor :)
Related
I want to use server-side response caching (output cache) with asp.net core 2.0 and found out about Response Caching Middleware and wanted to give it a try with a brand new asp.core mvc project.
Here is the description from the link above which makes me think this could be used like output cache.
The middleware determines when responses are cacheable, stores responses, and serves responses from cache.
Here is how my startup.cs looks like.
public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddResponseCaching();
services.AddMvc();
}
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
app.UseResponseCaching();
if (env.IsDevelopment())
{
app.UseBrowserLink();
app.UseDeveloperExceptionPage();
}
else
{
app.UseExceptionHandler("/Home/Error");
}
app.UseStaticFiles();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Home}/{action=Index}/{id?}");
});
}
}
and here is the HomeController.cs
[ResponseCache(Duration = 60)]
public class HomeController : Controller
{
public IActionResult Index()
{
return View();
}
public IActionResult About()
{
ViewData["Message"] = "Your application description page.";
return View();
}
public IActionResult Contact()
{
ViewData["Message"] = "Your contact page.";
return View();
}
public IActionResult Error()
{
return View(new ErrorViewModel { RequestId = Activity.Current?.Id ?? HttpContext.TraceIdentifier });
}
}
there is also a timestamp at the bottom of _Layout.cshtml file so i can tell when the page is rendered, like below.
<p>© 2018 - ResponseCachingMiddleware - #DateTime.UtcNow</p>
Cache-Control headers seem to be fine, this is what I get in headers when I load the page but time stamp keeps getting updated on every refresh every second.
Cache-Control:public,max-age=60
What I'm understanding from MS documentations is Response Caching Middleware is the server-side caching mechanism that takes care of caching the response while Response Caching seems to be just a filter to manipulate response headers for caching.
Can't tell if there is something wrong with my understanding or code and I wanna complain that I'm feeling this way too often since I started prototyping with ASP.Net Core. Maybe you could also suggest better resources as a side topic.
I've checked out this post before
ASP.NET Core 2.0 - Http Response Caching Middleware - Nothing cached
Also checked this out but it seems like the only difference is I'm using mvc.
https://github.com/aspnet/ResponseCaching/blob/dev/samples/ResponseCachingSample/Startup.cs
Thanks
Edit: I'm seeing the message below in the output window, cannot find anything about it on google except the few places I already checked for response caching middleware.
Microsoft.AspNetCore.ResponseCaching.ResponseCachingMiddleware:Information:
The response could not be cached for this request.
Note: I wish I could create #response-caching-middleware tag. Not sure #responsecache is relevant.
I had the same issue, I was about to pull my hairs over it, I'd set app.UseResponseCaching(); as well as services.AddResponseCaching(); and add ResponseCache on top of my action exactly like what was told in Microsoft official Docs, despite the the cache-controll header was set correctly on response returning from server but still nothing cached at server-side.
After couple of hours of sweating on this issue I figured out where the problem arises and why nothing cached at server.
Browsers by default set cache-controll value to max-age=0 for the request (if the request is not caused by back or forward) even though you set cache-controller correctly in your response by adding ResponseCache attribute on top of you action (or controller) since the cache-controller sent by request is set to max-age=0, the server is unable to cache response, I think this must be added to list of Response Caching limitation as well
Anyway you can override browser default behavior by adding few line of code right before calling app.UseResponseCaching(); on the other hand you need to add a custom middle-ware to modify request cache-control header value before calling app.UseResponseCaching();.
See code below, worked for me hope work for you too
app.Use(async (ctx, next) =>
{
ctx.Request.GetTypedHeaders().CacheControl = new Microsoft.Net.Http.Headers.CacheControlHeaderValue()
{
Public = true,
MaxAge = TimeSpan.FromSeconds(60)
};
await next();
}
);
app.UseResponseCaching();
for ensuring that ResponseCaching works as expected you can also use postman but you must set 'Send no-cache Header' to off in the setting, see image below
I had this same confusion recently.
ASP.Net Core's ResponseCaching does provide both client-side caching (through HTTP response headers) & server-side (through a memory cache'd middleware that short-circuits other middlewares if the response is in the cache). The server-side portion reads the HTTP response cache headers to determine if it should do server-side caching (similar to what an ISP or CDN might do).
Unfortunately, debugging the server-side ResponseCaching is tricky because it has weird rules & there's not adequate logging. In my case I pulled down Microsoft's source code to step through it & find the issue with my code.
The note you found in the output window "The response could not be cached for this request" is a clue.
There's 2 parts to the server-side caching of a request. The server has to prime the cache the first time the url is requested. It will serve the cached version the 2nd time it's requested. Pay attention to when the error message shows up, if it's on the 1st or 2nd request. That'll tell you if it couldn't be stored in the cache or if it couldn't be retrieved from the cache.
The rules for both storage & retrieval are in this source code file:
https://github.com/aspnet/ResponseCaching/blob/3bf5f6a1ce69b65c998d6f5c739822a9bed4a67e/src/Microsoft.AspNetCore.ResponseCaching/Internal/ResponseCachingPolicyProvider.cs
Your "Cache-Control:public,max-age=60" header should match these rules just fine.
My guess is you actually had it working, but didn't know how to test it correctly.
There is a counter-intuitive portion of ResponseCaching noted in this issue: https://github.com/aspnet/Home/issues/2607
Essentially, if the browser sends a no-cache or no-store header (when you hit CTRL+F5 or have your debugger tools open), ASP.Net Core's ResponseCaching will honor the browser's request & re-generate the response.
So, to test if your code was working you probably loaded the page, which primed the cache, then you hit CTRL+F5 to force-refresh your browser & you expected the server-side to respond with a cached entry rather than running your WebAPI code. However, it honored the no-cache request header & bypassed the cache (& wrote that message in your output log).
The way to test this would be to clear your browser cache in-between requests (or switch to incognito), rather than using CTRL+F5.
On a side note, honoring the no-cache/no-store request headers was probably a poor design choice since ASP.Net Core's ResponseCache will most likely be used by a server who owns the response, rather than an intermediary cache like a CDN/ISP. I've extended the base ResponseCache with an option to disable honoring these headers (as well as serialize the cache to disk, rather than in-memory only). It's an easy drop-in replacement for the default cache.
You can find my extension here:
https://github.com/speige/AspNetCore.ResponseCaching.Extensions
https://www.nuget.org/packages/AspNetCore.ResponseCaching.Extensions
There are also a few other other gotchas with ResponseCaching to watch out for which you may have already read about in the blog urls you posted. Authenticated requests & responses with set-cookie won't be cached. Only requests using GET or HEAD method will be cached. If the QueryString is different, it'll make a new cache entry. Also, usually you'll want a "Vary" header to prevent caching if certain conditions of a request differ from the previously-cached request (example: user-agent, accept-encoding, etc). Finally, if a Middleware handles a request it'll short-circuit later Middlewares. Make sure your app.UseResponseCaching() is registered before app.UseMVC()
If the Cache-Control header is coming through, then it's working. That's all the server can do from that perspective. The client ultimately makes the decision whether or not to actually cache the resource. Sending the header doesn't force the client to do anything; in fact, the server, in general, cannot force the client to do anything.
What is the correct way of using the following in a vNext application on an async method:
[OutputCache(NoStore = true, Duration = 0, VaryByParam = "*")]
I see it is part of System.Web.Caching, but the only place I could add that would be in the aspnet50 -> frameworkAssemblies section of my project.json file, which is incorrect.
ASP.NET Core 1.1/2.0 Answer
Add the response caching middleware like so:
public void Configure(IApplicationBuilder application)
{
application
.UseResponseCaching()
.UseMvc();
}
This middleware caches content based on the caching HTTP headers you set in your response. You can take a look at the rest of the answer to see how to use ResponseCache.
ASP.NET Core 1.0 Answer
Use the new ResponseCache attribute instead. ResponseCache is not a direct replacement of OutputCache as it only controls client and proxy caching using the Cache-Control HTTP header.
If you want to use server side caching, see this StackOverflow question discussing how to use IMemoryCache or IDistributedCache.
// Add this to your controller action.
[ResponseCache(Duration = 3600)]
Here is an example using the new cache profiles:
// Add this to your controller action.
[ResponseCache(CacheProfile="Cache1Hour")]
// Add this in Startup.cs
services.AddMvc(options =>
{
options.CacheProfiles.Add(
new CacheProfile()
{
Name = "Cache1Hour",
Duration = 3600,
VaryByHeader = "Accept"
});
});
Gotchas
The response caching middleware stops working in a variety of situations which you can learn more about in the docs. Two common ones you will probably hit are that it stops working if it sees an Authorization or Set-Cookie HTTP header.
Bonus Comment
In ASP.NET 4.6, we could represent cache profiles in the web.config and change the settings without recompiling the code. For more information about how you can move your cache profiles to the new appsettings.json, rather than hard coding it in Startup.cs see this question.
Update
As AndersNS was kind to point out, it will be available in RC1 most likely: https://github.com/aspnet/Mvc/issues/536.
To put it simply there's no OutputCache or equivalent in ASP.NET 5 currently.
However, please note that OutputCache is just an attribute with minimal logic that talks to a cache provider. You can easily implement your own such attribute, using Memory Cache for example. Or you can use third party solutions.
I am sure that when ASP.NET 5 will ship there will be plenty of solutions out on the market. And I'm quite sure that we will have an official OutputCache equivalent too.
Here's the basic MemoryCache usage in case someone finds it useful
MemoryCache cache = MemoryCache.Default;
string cacheName = "MyCache";
if (cache.Contains(cacheName) == false || cache[cacheName] == null)
{
var data = ... get data
cache.Set(cacheName, data, new CacheItemPolicy() { SlidingExpiration = DateTime.Now.AddDays(1).TimeOfDay });
}
return cache[cacheName];
I am using code like this:
bundles.Add(new ScriptBundle("~/bundles/textview")
.Include(
"~/Scripts/printarea/jquery.PrintArea.js",
"~/Scripts/pagedown/Markdown.Converter.js",
"~/Scripts/pagedown/Markdown.Sanitizer.js",
"~/Scripts/pagedown/Markdown.Editor.js"
));
This creates a file with a cache expiry date of one year in advance and this is what appears in my script HTML when I look at the source:
<script src="/bundles/textview?v=cNvP0r6Jo6hsl2Sdzhw-o3kAK7t2JdcNWiG0iIg7Lys1"></script>
So why do I in fiddler still see it going to the server to check if the file has been modified ? Is there a way that the bundle routine could be modified so that it does not add the ?v= and instead simply appends the GUID to the filename with for example a hyphen in between?
The query string v has a value token that is a unique identifier used for caching. As long as the bundle doesn't change, the ASP.NET application will request the bundle using this token. If any file in the bundle changes, the ASP.NET optimization framework will generate a new token, guaranteeing that browser requests for the bundle will get the latest bundle.
Why a server check ?
Browsers use a freshness heuristic to determine if they should validate a resource with the server or just pull it from the cache.
The browser will serve cached files without validating them with the server unless one of the following is true:
The freshness heuristic is not met (that is, the file in the cache is not considered fresh).
You have changed the expires header or other caching header.
You have set the browser to disable caching..
The URL to the resource changes or is different.
Adding a Web.config file to the Scripts folder:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<staticContent>
<clientCache cacheControlMode="UseExpires"
httpExpires="Thu, 01 Jan 2016 00:00:00 GMT" />
</staticContent>
</system.webServer>
</configuration>
This sets the Expires Header out for a year. this will allow your files to be served directly from the cache without checking with the server for the next year.
As for bundles, headers are explicitly set inside System.Web.Optimization.dll:
private static void SetHeaders(BundleResponse bundle, BundleContext context)
{
if (context.HttpContext.Response != null)
{
if (bundle.ContentType != null)
{
context.HttpContext.Response.ContentType = bundle.ContentType;
}
if (!context.EnableInstrumentation && context.HttpContext.Response.Cache != null)
{
HttpCachePolicyBase cache = context.HttpContext.Response.Cache;
cache.SetCacheability(bundle.Cacheability);
cache.SetOmitVaryStar(true);
cache.SetExpires(DateTime.Now.AddYears(1));
cache.SetValidUntilExpires(true);
cache.SetLastModified(DateTime.Now);
cache.VaryByHeaders["User-Agent"] = true;
}
}
}
so you need to check that you are not breaking any of the rules forcing the browser to check with the server!
references:
Bundling and Minification
Using CDNs and Expires to Improve Web Site Performance.
Update
If your target is to have your scripts always emitted as:
<script src="/Scripts/printarea/jquery.PrintArea.js"></script>
<script src="/Scripts/pagedown/Markdown.Converter.js"></script>
<script src="/Scripts/pagedown/Markdown.Sanitizer.js"></script>
<script src="/Scripts/pagedown/Markdown.Editor.js></script>
Rather than:
<script src="/bundles/textview?v=cNvP0r6Jo6hsl2Sdzhw-o3kAK7t2JdcNWiG0iIg7Lys1"></script>
Then add the following to the RegisterBundles method (disabling Bundling and Minification):
BundleTable.EnableOptimizations = false;
Unless EnableOptimizations is true or the debug attribute in the
compilation Element in the Web.config file is set to false, files
will not be bundled or minified. Additionally, the .min version of
files will not be used, the full debug versions will be selected.
EnableOptimizations overrides the debug attribute in the compilation
Element in the Web.config file
Probably has to do with the cache-busting querystring variable that MVC appends to the URL:
Don't include a query string in the URL for static resources.
Most proxies, most notably Squid up through version 3.0, do not cache resources with a "?" in their URL even if a Cache-control: public header is present in the response. To enable proxy caching for these resources, remove query strings from references to static resources, and instead encode the parameters into the file names themselves. "
ref: https://developers.google.com/speed/docs/best-practices/caching
When you say "still see it going to the server to check if the file has been modified" what's the return code? IE will always make a request for items it has cached, such as bundles. It includes the ETAG & LAST-MODIFIED headers in the request. IIS usually replies with 304 not modified & no content. Firefox on the other hand wont make any request at all. It's just how the browser handles caching.
The v querystring is just to make sure we're asking for the right file. Since IE will always make the request & get a 304 back, IE isn't really bothered by the v string. However Firefox with its aggressive caching will see the slightly different url & make a request for the file. If the parameter wasn't there, Firefox wouldn't no to request a new file when the bundle changed.
I'm not sure what Chrome does, but suspect it behaves similarly to Firefox.
I have a Global.asx file that needs to do custom authentication, auditing and profiling stuff. This is needed because it supports a SAML based SSO system and needs to override the normal .Net authentication (which doesn't support either SAML or mixed authentication)
I don't want to fire it for static files, such as .js, .css, .png, etc
In Cassini/WebDev and IIS7 it does.
What I want to have is some simple check, like a this.Request.IsStaticFile (which doesn't exist, unfortunately) to identify the static files.
I realise that this would be fairly simple to write, but it feels like something that must already exist - IIS has already applied caching policy stuff for the static files and so on.
I need a code solution, rather than an IIS config change one.
Update
This is my current workaround:
/// <summary>Hold all the extensions we treat as static</summary>
static HashSet<string> allowedExtensions = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
".js", ".css", ".png", ...
};
/// <summary>Is this a request for a static file?</summary>
/// <param name="request">The HTTP request instance to extend.</param>
/// <returns>True if the request is for a static file on disk, false otherwise.</returns>
public static bool IsStaticFile(this HttpRequest request)
{
string fileOnDisk = request.PhysicalPath;
if (string.IsNullOrEmpty(fileOnDisk))
{
return false;
}
string extension = Path.GetExtension(fileOnDisk);
return allowedExtensions.Contains(extension);
}
This works and is quick enough, but feels horribly clunky. In particular relying on extensions is going to be error prone if we add new static files not thought of.
Is there a better way without changing the IIS config?
You might be able to check which handler is dealing with the request.
In IIS6 only .net files, eg aspx are mapped to a handler that does stuff.
In IIS7 with the integrated pipeline, everything routes through .net, which is normally a good thing. Different handlers still deal with different file types though. In particular I believe the staticfilehandler is the one you need to check for. The httpcontext.handler property should allow you to figure it out.
You could create an extension method to add that IsStatic method...
Simon
There are a few options:
Adding authorization element and deny none for those paths that you do not need any authentication and contains your static files
You are using integrated pipeline. Turn it off on your IIS 7.
There is no doubt that you need to create a custom extension method because ASP.NET routing engine uses this code to decide whether a file exist,
if (!this.RouteExistingFiles)
{
string appRelativeCurrentExecutionFilePath = httpContext.Request.AppRelativeCurrentExecutionFilePath;
if (((appRelativeCurrentExecutionFilePath != "~/") && (this._vpp != null)) && (this._vpp.FileExists(appRelativeCurrentExecutionFilePath) || this._vpp.DirectoryExists(appRelativeCurrentExecutionFilePath)))
{
return null;
}
}
You will not able to decide whether the request is static in Application_BeginRequest using context.handler because Routing Module may change the handler and this module always execute after Application_BeginRequest. My suggestion is to use the similar code which ASP.NEt routing engine uses.
I have an asp.net .asmx webservice written to handle requests from a third party tool. The third party tool makes an http POST request to the webservice to get user information. I'm using IIS7
Running Fiddler with "Remove All Encodings" checked, I can see the webservice call and and everything functions properly. If I uncheck "Remove All Encodings", the webservice call fails with a 400 Bad Request. The difference I see is that the header "Content-Encoding: gzip" is being removed by Fiddler and the content is being decompressed.
So, when the Content-Encoding header is removed and the content is decompressed, my webservice functions perfectly. When the header is present and the content is compressed, the webservice fails.
How can I either:
Configure my webservice to tell the client that it won't accept compressed requests (and hope that the third party tool respects that)
Decompress the content early in the asp.net handling
Modify my webservice to work with compressed data
Update: To be clear, I don't need to configure gzip encoding in the Response, I need to deal with a Request TO my webservice that is gzip encoded.
Update 2: The third-party tool is the Salesforce.com Outlook plugin. So, I don't have access to modify it and it is used by many other companies without trouble. It's got to be something I'm doing (or not doing)
Update 3: I found one post here that says that IIS does not support incoming POST requests with compressed data, it only supports compressed Responses. Can this still be true?
The simplest technique is to create an HttpModule that replaces the request filter. It is more reusable and avoids having a Global.asax. There is also no need to create a new decompress stream class as the GZipStream is ready for that. Here is the full code, that also removes the Content-Encoding: gzip that is not needed any more:
public class GZipRequestDecompressingModule : IHttpModule
{
public void Init(HttpApplication context)
{
context.BeginRequest += (sender, e) =>
{
var request = (sender as HttpApplication).Request;
string contentEncoding = request.Headers["Content-Encoding"];
if (string.Equals(contentEncoding, "gzip",
StringComparison.OrdinalIgnoreCase))
{
request.Filter = new GZipStream(request.Filter,
CompressionMode.Decompress);
request.Headers.Remove("Content-Encoding");
}
};
}
public void Dispose()
{
}
}
To activate this module, add the following section into your web.config:
<system.webServer>
<modules runAllManagedModulesForAllRequests="true">
<add name="AnyUniqueName"
type="YourNamespace.GZipRequestDecompressingModule, YourAssembly"
preCondition="integratedMode" />
</modules>
</system.webServer>
Since the 3rd party service is just sending you a POST, I do not think that it is possible to tell them not to send in compressed.
You could try to override GetWebRequest and decompress it on the way in
public partial class MyWebService : System.Web.Services.Protocols.SoapHttpClientProtocol
{
protected override WebRequest GetWebRequest(Uri uri)
{
base.GetWebRequest(uri);request.AutomaticDecompression = System.Net.DecompressionMethods.GZip;
return request;
}
}
GZIP compression is a function of the server.
If you're using IIS6, consult this link.
If you're using IIS7, you could use ISAPI_Rewrite to disable gzip. See this link.
That said, because gzip is a function of IIS, you really shouldn't need to do anything "special" to get it to work with a web service (IIS should handle decompressing and compressing requests). Hopefully this info will get you further down the road to troubleshooting and resolving the issue.
I am not sure that IIS supports decompressing incoming requests, so this might have to be done further down the pipe.
Shiraz's answer has the potential of working and it would be the first thing I would try.
If that doesn't work you might consider switching your server .asmx service to WCF, which while a bit more difficult to setup it also gives more flexibility.
On the WCF side there are two things I can suggest. The first is quite easy to implement and is based on setting the WebRequest object used by WCF to automatically accept compression. You can find the details here. This one is the WCF equivalent to the solution proposed by Shiraz.
The second is more complicated, since it involves creating Custom Message Encoders, but if none of the above methods work, this should solve the problem. Creating a message compression encoder is described here. You might also want to check the answer in here which presents a sample config for the message encoder.
Please let me know if this helped or if you need more help.
I've found a partial answer here.
class DecompressStream : Stream
{
...
public override int Read(byte[] buffer, int offset, int count)
{
GZipStream test = new GZipStream(_sink, CompressionMode.Decompress);
int c = test.Read(buffer, offset, count);
return c;
}
...
}
I can then specify the filter on the request object like this:
void Application_BeginRequest(object sender, EventArgs e)
{
string contentEncoding = Request.Headers["Content-Encoding"];
Stream prevCompressedStream = Request.Filter;
if(contentEncoding == null || contentEncoding.Length == 0)
return;
contentEncoding = contentEncoding.ToLower();
if(contentEncoding.Contains("gzip"))
{
Request.Filter = new DecompressStream(Request.Filter);
}
}
I say partial answer because even though I can now process the incoming request, the response is getting a "Content-Encoding: gzip" header even though the response is not encoded. I can verify in Fiddler that the content is not encoded.
If I do encode the response, the client for the webservice fails. It seems that even though it is sending "Accept-Encoding: gzip", it does not in fact accept gzip compressed response. I can verify in Fiddler that the response is compressed and Fiddler will decompress it successfully.
So, now I'm stuck trying to get a stray "Content-Encoding: gzip" header removed from the response. I've removed all references I can find to compression from the application, the web.config, and IIS.