I am implementing page/resource compression to improve website performance.
I have tried to implement both blowery and wicked HttpCompress but end up getting the same result. This only seems to affect Firefox, I have tested on Chrome and IE.
What happens is the first time I request the page all the external resources decompress ok. The 2nd or 3rd time the page has errors because the resource doesn't seem to be decompressed. I get unicode characters like:
������í½`I%&/mÊ{JõJ×àt¡`$Ø#ìÁÍæìiG#)«*ÊeVe]f
(actually they can't be displayed properly here)
Inspecting the page via firebug displays the response header as:
Cache-Control private
Content-Type text/html; charset=utf-8
Content-Encoding gzip
Server Microsoft-IIS/7.5
X-AspNetMvc-Version 2.0
X-AspNet-Version 2.0.50727
X-Compressed-By HttpCompress
X-Powered-By ASP.NET Date Fri, 09 Jul
2010 06:51:40 GMT Content-Length 2622
This clearly states that the resource is compressed by gzip. So something seems to be going wrong on the deflate side on the client?
I have added the following sections (in the appropriate locations) in the web.config:
<sectionGroup name="blowery.web">
<section name="httpCompress" type="blowery.Web.HttpCompress.SectionHandler, blowery.Web.HttpCompress"/>
</sectionGroup>
<blowery.web>
<httpCompress preferredAlgorithm="gzip" compressionLevel="high">
<excludedMimeTypes>
<add type="image/jpeg"/>
<add type="image/png"/>
<add type="image/gif"/>
</excludedMimeTypes>
<excludedPaths>
<add path="NoCompress.aspx"/>
</excludedPaths>
</httpCompress>
</blowery.web>
<add name="CompressionModule" type="blowery.Web.HttpCompress.HttpModule, blowery.web.HttpCompress"/>
Any help?
This is an issue that I have face before and the problem is that the Content-Length is not correct. Why is not correct ? because its probably calculate before the compression.
If you set Content-Lenght by hand, just remove it and let the module set it if he can.
I note that you use the Blowery compression. Probably this is a bug/issue inside Blowery. If you can not locate it and fix it, why not use the Ms compression ?
#ptutt if you are on shared iis, then maybe there have all ready set compression, so there is one compression over the other, and you only need to remove yours. If this is the issue then for sure the content-lenght is false because after the first compression, the second is break it.
Check it out using this site https://www.giftofspeed.com/gzip-test/ if your pages is all ready compressed by default by iis.
If not compressed by default then you can do it very easy. On Global.asax
protected void Application_BeginRequest(Object sender, EventArgs e)
{
string cTheFile = HttpContext.Current.Request.Path;
string sExtentionOfThisFile = System.IO.Path.GetExtension(cTheFile);
if (sExtentionOfThisFile.Equals(".aspx", StringComparison.InvariantCultureIgnoreCase))
{
string acceptEncoding = MyCurrentContent.Request.Headers["Accept-Encoding"].ToLower();;
if (acceptEncoding.Contains("deflate") || acceptEncoding == "*")
{
// defalte
HttpContext.Current.Response.Filter = new DeflateStream(prevUncompressedStream,
CompressionMode.Compress);
HttpContext.Current.Response.AppendHeader("Content-Encoding", "deflate");
} else if (acceptEncoding.Contains("gzip"))
{
// gzip
HttpContext.Current.Response.Filter = new GZipStream(prevUncompressedStream,
CompressionMode.Compress);
HttpContext.Current.Response.AppendHeader("Content-Encoding", "gzip");
}
}
}
Please note, I just write this code and have not tested. My code is a little more complicate, so I just create a simple verion of it.
Find more examples:
http://www.google.com/search?q=Response.Filter+GZipStream
Reference:
ASP.NET site sometimes freezing up and/or showing odd text at top of the page while loading, on load balanced servers
Related
** UPDATE Jan 27, 2021 **
I have been working with Microsoft on this issue and we did a request trace and reviewed the FREB log and confirmed that the code is outputting the correct headers but in the END_REQUEST handler they are replaced with cache-control private. After building several virtual machines from scratch we learned that out of the box, this problem doesn't happen. However when we install the latest version of STACKIFY AGENT (4.29.29) once we put the agent on this behavior happens. Older versions of Stackify agent (RETRACE profiler and APM) don't do this, but so far, this is irreversible: once we install 4.29.29 of Stackify, uninstalling or installing older versions doesn't undo this issue, the environment is somehow permanently modified.
STACKIFY responded with a solution which works (but suggests something is left behind after uninstall): Set the environment variable STACKIFY_ENABLERUM = false .. we tried this and IIS returned our correct header without replacing it with cache-control: private.
I want to use CloudFront CDN to cache traffic to my API, so that requests are offloaded to the content delivery network.
I'm trying to set the Cache-Control header to return: Cache-Control: "public, max-age=10". When I run my code in Visual Studio 2019 to debug it, I get the correct header. However, when I deploy it to IIS, I always get back:
Cache-Control: private
I am running Windows Server 2019 with IIS version 10.0.17763.1. I am using ASP.NET MVC with Web API 2 to operate a REST API.
In my code I created this attribute:
Code:
public class CacheControlAttribute : System.Web.Http.Filters.ActionFilterAttribute
{
public int MaxAge { get; set; }
public CacheControlAttribute()
{
MaxAge = 0;
}
public override void OnActionExecuted(HttpActionExecutedContext context)
{
// Don't set the cache header if there was an error or if there was no content in the response
if (context.Response != null && context.Response.IsSuccessStatusCode)
{
context.Response.Headers.CacheControl = new System.Net.Http.Headers.CacheControlHeaderValue
{
Private = false,
Public = true,
MaxAge = TimeSpan.FromSeconds(MaxAge)
};
}
base.OnActionExecuted(context);
}
}
I then decorate my method with this attribute:
[CacheControl(MaxAge = 10)]
I read several articles on StackOverflow but was not able to solve this issue. I also tried adding this to web.config:
It did not help.
I have this working successfully on one IIS server, same version of IIS but older version of my code. I'm having the problem setting it up on a new IIS server and I think its an IIS configuration issue but I am not certain. Maybe its something in my web.config.
Thank you for any help with this.
Cache-control is divided into request and response directives. I guess you are talking about the cache-control in the response, because the request cache-control cannot be set in IIS.
For security reasons, IIS will set cache-control to private by default, so you will encounter this problem. This is default behaviour for .NET when there's no output cache used for a request (and you have output cache enabled). If you set the sendCacheControlHeader to false in web.config, you will not get the Cache-Control: private header.
So an easy way is set sendCacheControlHeader false and add cache-control will remove private. You also don't need to custom cache-control filter in MVC.
<system.web>
<httpRuntime sendCacheControlHeader="false" />
</system.web>
<httpProtocol>
<customHeaders>
<remove name="X-Powered-By" />
<add name="Cache-Control" value="max-age=30,public" />
</customHeaders>
</httpProtocol>
Another idea is to add cached output for each action or controller. .NET has a defined filter to control maxage, you don't need to define it yourself. You can use [OutputCache(Duration = 0)]
OutputCache
You need to implement your own on top of the existing output cache like so
public static class MyCustomCacheConfig
{
public static int Duration = 600; //whatever time you want
}
public class CdnCacheCacheAttribute : OutputCacheAttribute
{
public CdnCacheCacheAttribute()
{
this.Duration = MyCustomCacheConfig.Duration;
}
}
[CdnCacheCacheAttribute(Duration = 100, VaryByParam = "none")]
public ActionResult YourActions()
{
return View();
}
VaryByParm in output cache from Microsoft.. which controls the HttpCaching
// Get the current VaryByParam.
String varyByParamValue = outputCacheProfile.VaryByParam;
// Set the VaryByParam.outputCacheProfile.VaryByParam = string.Empty;
I'm running in the Cassini developer server inside Visual Studio 2012, and I need to redirect clients from the legacy .asp pages to .aspx pages.
Note: Ideally I would redirect clients from .asp to a friendly url, and then internally do a rewrite to .aspx
POST /ResetClock.asp
HTTP/1.1 307 Temporary Redirect
Location: //stackoverflow.us/ResetClock
And then internally:
POST /ResetClock
rewrites into /ResetClock.ashx (Yes, I changed it to .ashx; that's the virtue of url rewriting).
Like what Hanselman did
This is a lot like what Scott Hanselman did:
Request for /foo.html
gives the client a redirect to /foo
client request for /foo
is re-written into /foo.html
The attempted hack
I tried the hack solution; alter the .asp page to force a redirect to the .ashx (and live to fight with the url re-write syntax another day):
ResetClock.asp
<%
Response.Redirect("ResetClock.aspx")
Response.End
%>
Except that Cassini does not serve .asp pages at all:
This type of page is not served.
Description: The type of page you have requested is not served because it has been explicitly forbidden. The extension '.asp' may be incorrect. Please review the URL below and make sure that it is spelled correctly.
Requested URL: /WebSite/FetchTimes.asp
Which points to a related issue. The solution I end up using cannot require anything that isn't already available on the IIS7.5. And it cannot require anything that needs access to the IIS Admin tools; and must exist entirely within the web-site (e.g. the web.config).
The question
How do I re-write .asp into something more ASP.net-ish?
Edit: Changed GET to a POST to thwart nitpickers who wonder why the 307 Temporary Redirect and not 302 Found or 303 See Other.
The solution is to create an IHttpModule. HttpModules let you intercept every request, and react as you desire.
The first step is to create the plumbing of an IHttpModule:
class UrlRewriting : IHttpModule
{
public void Init(HttpApplication application)
{
application.BeginRequest += new EventHandler(this.Application_BeginRequest);
application.EndRequest += new EventHandler(this.Application_EndRequest);
}
public void Dispose()
{
//Nothing to do here
}
private void Application_BeginRequest(object sender, EventArgs e)
{
HttpApplication application = (HttpApplication)sender;
HttpContext context = application.Context;
}
private void Application_EndRequest(object sender, EventArgs e)
{
}
}
And then register our HttpHandler in the web.config file:
web.config:
<configuration>
<system.web>
<httpModules>
<add name="UrlRewriting" type="UrlRewriting"/>
</httpModules>
</system.web>
</configuration>
Now we have a method (Application_BeginRequest) that will run every time a request is made.
Issue client redirect if they ask for ASP page
The first order of business is redirect the client to a "clean" form. For example, a request for /File.asp is redirected to /File:
private void Application_BeginRequest(object sender, EventArgs e)
{
HttpApplication application = (HttpApplication)sender;
HttpContext context = application.Context;
//Redirct any requests to /File.asp into a /File
if (context.Request.Url.LocalPath == VirtualPathUtility.ToAbsolute("~/File.asp"))
{
//Be sure to issue a 307 Temporary Redirect in case the client issued a POST (i.e. a non-GET)
//If we issued 302 Found, a buggy client (e.g. Chrome, IE, Firefox) might convert the POST to a GET.
//If we issued 303 See Other, the client is required to convert a POST to a GET.
//If we issued 307 Temporary Redirect, the client is required to keep the POST method
context.Response.StatusCode = (int)HttpStatusCode.TemporaryRedirect;
context.Response.RedirectLocation = VirtualPathUtility.ToAbsolute("~/File");
context.Response.End();
}
}
And then the internal rewrite
Now that the client will be asking for /File, we have to re-write that internally to an .aspx, or in my case, an .ashx file:
private void Application_BeginRequest(object sender, EventArgs e)
{
HttpApplication application = (HttpApplication)sender;
HttpContext context = application.Context;
//Redirct any requests to /ResetClock.asp into a /File
if (context.Request.Url.LocalPath == VirtualPathUtility.ToAbsolute("~/ResetClock.asp"))
{
//Be sure to issue a 307 Temporary Redirect in case the client issued a POST (i.e. a non-GET)
//If we issued 302 Found, the buggy client might convert the POST to a GET.
//If we issued 303 See Other, the client is required to convert a POST to a GET.
//If we issued 307 Temporary Redirect, the client is required to keep the POST method
context.Response.StatusCode = (int)HttpStatusCode.TemporaryRedirect;
context.Response.RedirectLocation = VirtualPathUtility.ToAbsolute("~/ResetClock");
context.Response.End();
}
//Rewrite clean url into actual handler
if (context.Request.Url.LocalPath == VirtualPathUtility.ToAbsolute("~/ResetClock"))
{
String path = "~/ResetClock.ashx"; //no need to map the path
context.Server.Execute(path, true);
//The other page has been executed
//Do not continue or we will hit the 404 of /ResetClock not being found
context.Response.End();
}
}
IIS contains some basic url redirection
Starting with some unknown version of IIS, they added a (now mocked) form of URL Rewriting. It doesn't issuing a client redirect, only an internal re-write. But at least it could be used to solve my problem (responding to an ASP page with ASP.net content):
web.config
<configuration>
<system.web>
<urlMappings>
<add url="~/ResetClock.asp" mappedUrl="~/ResetClock.ashx"/>
</urlMappings>
</system.web>
</configuration>
The client will still appear to have found a resource at /ResetClock.asp, but the guts of the response will have come from /ResetClock.ashx.
Note: Any code is released into the public domain. No attribution required.
We have a web page that is grabs a series of strings from a url, finds some pdfs associated with those strings, zips them up using DotNetZip, and returns them to the user. The page that does this is very simple - here's the Page_Load:
protected void Page_Load(object sender, EventArgs e)
{
string[] fileNames = Request.QueryString["requests"].Split(',');
Response.Clear();
Response.ClearHeaders();
Response.ContentType = "application/zip";
string archiveName = String.Format("MsdsRequest-{0}.zip", DateTime.Now.ToString("yyyy-mm-dd-HHmmss"));
Response.AddHeader("Content-Disposition", "attachment; filename=\"" + archiveName + "\"");
using (ZipFile zip = new ZipFile())
{
foreach (string fileName in fileNames)
{
zip.AddFile(String.Format(SiteSettings.PdfPath + "{0}.pdf", msdsFileName), "");
}
zip.Save(Response.OutputStream);
}
Response.Flush();
}
(Before you ask, it would be fine if someone put other values in this url...these are not secure files.)
This works fine on my development box. However, when testing on our QA system, it downloads the zipped file, but it is corrupt. No error is thrown, and nothing is logged in the event log.
It may be possible for me to find a way to interactively debug on the QA environment, but since nothing is actually failing by throwing an error (such as if the dll wasn't found, etc.), and it's successfully generating a non-empty (but corrupt) zip file, I'm thinking I'm not going to discover much by stepping through it.
Is it possible that this is some kind of issue where the web server is "helping" me by "fixing" the file in some way?
I looked at the http response headers where it was working on my local box and not working on the qa box, but while they were slightly different I didn't see any smoking gun.
As an other idea I rejected, the content length occured to me as a possibility since if the content length value was too small I guess that would make it corrupt...but I'm not clear why that would happen and I don't think that's exactly it since if I try to zip and download 1 file I get a small zip...while downloading several files gives me a much larger zip. So that, combined with the fact that no errors are being logged, makes me think that the zip utility is correctly finding and compressing files and the problem is elsewhere.
Here are the headers, to be complete.
The response header on my development machine (working)
HTTP/1.1 200 OK
Date: Wed, 02 Jan 2013 21:59:31 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Content-Disposition: attachment; filename="MsdsRequest-2013-59-02-165931.zip"
Transfer-Encoding: chunked
Cache-Control: private
Content-Type: application/zip
The response header on the qa machine (not working)
HTTP/1.1 200 OK
Date: Wed, 02 Jan 2013 21:54:37 GMT
Server: Microsoft-IIS/6.0
P3P: CP="NON DSP LAW CUR TAI HIS OUR LEG"
SVR: 06
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Content-Disposition: attachment; filename="MsdsRequest-2013-54-02-165437.zip"
Cache-Control: private
Content-Type: application/zip
Set-Cookie: (cookie junk removed);expires=Wed, 02-Jan-2013 21:56:37 GMT;path=/;httponly
Content-Length: 16969
Not sure how to approach this since nothing is claiming a failure. I feel like this could be a web server configuration issue (since I don't have any better ideas), but am not sure where to look. Is there a tact I can take?
As it is you have miss to give an End() to the page right after the Flush() as:
...
zip.Save(Response.OutputStream);
}
Response.Flush();
Response.End();
}
But this is not the correct way, to use a page to send a zip file, probably IIS also gZip the page and this may cause also issues. The correct way is to use a handler and also avoid extra gZip compression for that handler by ether configure the IIS, ether if you make the gZip compression avoid it for that one.
a handler with a name for example download.ashx for your case will be as:
public void ProcessRequest(HttpContext context)
{
string[] fileNames = Request.QueryString["requests"].Split(',');
context.Response.ContentType = "application/zip";
string archiveName = String.Format("MsdsRequest-{0}.zip", DateTime.Now.ToString("yyyy-mm-dd-HHmmss"));
context.Response.AddHeader("Content-Disposition", "attachment; filename=\"" + archiveName + "\"");
// render direct
context.Response.BufferOutput = false;
using (ZipFile zip = new ZipFile())
{
foreach (string fileName in fileNames)
{
zip.AddFile(String.Format(SiteSettings.PdfPath + "{0}.pdf", msdsFileName), "");
}
zip.Save(context.Response.OutputStream);
}
}
I there a way to know if a request is a soap request on AuthenticateRequest event for HttpApplication? Checking ServerVariables["HTTP_SOAPACTION"] seems to not be working all the time.
public void Init(HttpApplication context) {
context.AuthenticateRequest += new EventHandler(AuthenticateRequest);
}
protected void AuthenticateRequest(object sender, EventArgs e) {
app = sender as HttpApplication;
if (app.Request.ServerVariables["HTTP_SOAPACTION"] != null) {
// a few requests do not enter here, but my webservice class still executing
// ...
}
}
I have disabled HTTP POST and HTTP GET for webservices in my web.config file.
<webServices>
<protocols>
<remove name="HttpGet" />
<remove name="HttpPost" />
<add name="AnyHttpSoap" />
</protocols>
</webServices>
Looking at ContentType for soap+xml only partially solves my problem. For example,
Cache-Control: no-cache
Connection: Keep-Alive
Content-Length: 1131
Content-Type: text/xml
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: ro
Host: localhost
mymethod: urn:http://www.wsnamespace.com/myservice
Some clients instead of having the standard header SOAPAction: "http://www.wsnamespace.com/myservice/mymethod", have someting like in example above. "mymethod" represents the method in my web service class with [WebMethod] attribute on it and "http://www.wsnamespace.com/myservice" is the namespace of the webservice. Still the service works perfectly normal.
The consumers use different frameworks (NuSOAP from PHP, .NET, Java, etc).
You could look at Request.ContentType property, which if properly set by the client should be
application/soap+xml; charset=utf-8
The utf-8 part may not be present.
Aside from that, surely you can just check the URL, and if it's a webservice one then that tells you what it is.
I always give web services their own port. That way I don't have to filter every HTTP request that comes across port 80. Or rather, I can filter port 80 for browser-oriented issues, and SOAP/SOA ports for other types of attacks.
IMAO, mixing (potentially) sensitive business data with public data just so you don't have to open another hole in the firewall is thumbing your nose at the very reason you have a firewall in the first place.
You could also go down the harder route and figure things out based on everything else that's below HTTP headers. What I mean by that is, to analyze things like below, which is the SOAP request body - part of the request...
<soap:Envelope xmlns:soap="..." soap:encodingStyle="...">
IBM
Have you tested the System.Web.HttpContext.Current.Request.CurrentExecutionFilePathExtension ??
Normally this would be .asmx for webservices (json and xml), as long as you handle the service of course.
I am using following code to identify the request type. Try this if it match your requirment. Mark as answer if it help you.
if (request.Headers["SOAPAction"] != null || request.ContentType.StartsWith("application/soap+xml"))
return ServiceRequestTypes.SoapRequest;
else if ("POST".Equals(request.RequestType, StringComparison.InvariantCultureIgnoreCase) && request.ContentType.StartsWith("application/x-www-form-urlencoded", StringComparison.InvariantCultureIgnoreCase))
return ServiceRequestTypes.HttpPostRequest;
else if ("POST".Equals(request.RequestType, StringComparison.InvariantCultureIgnoreCase) && request.ContentType.StartsWith("application/json", StringComparison.InvariantCultureIgnoreCase))
return ServiceRequestTypes.AjaxScriptServiceRequest;
return ServiceRequestTypes.Unknown;
I have implemented GZIP compression on a few of my ASP.NET pages, using a class that inherits from System.Web.UI.Page, and implementing the OnLoad method to do the compression, like so:
protected override void OnLoad(EventArgs e)
{
base.OnLoad(e);
if (Internet.Browser.IsGZIPSupported())
{
base.Response.Filter = new GZipStream(base.Response.Filter, CompressionMode.Compress, true);
base.Response.AppendHeader("Content-encoding", "gzip");
base.Response.AppendHeader("Vary", "Content-encoding");
}
else if (Internet.Browser.IsDeflateSupported())
{
base.Response.Filter = new DeflateStream(base.Response.Filter, CompressionMode.Compress, true);
base.Response.AppendHeader("Content-encoding", "deflate");
base.Response.AppendHeader("Vary", "Content-encoding");
}
}
The IsGZIPSupported method just determines whether the browser supports GZIP, looking at the Accept-encoding request header, and the browser's user agent (IE5-6 are excluded from GZIP compression). However, with this code, I am getting the web page has expired message in IE, when I postback from the page and try to use the back button. Setting the cache control to private seems to fix the problem:
base.Response.Cache.SetCacheability(HttpCacheability.Private);
But I am not sure why, or whether this will cause other problems. I haven't set any caching for any other pages in the site, and the site is running on an intranet with only a dozen concurrent users, so performance isn't a big issue at the moment.
See this article on Vary header and WinInet/MSIE
It seems you should be sending Vary: Accept-Encoding instead of Vary: Content-Encoding, as the response will vary depending on the request header.