I'm bucketing User-Agents by device using something like varnish-devicedetect and storing the result in X-UA-Device on the request and the response.
I've seen several recommendations to vary on User-Agent. Any reason not to vary instead on X-UA-Device? Seems like it'd be nicer to downstream caches.
Since X-UA-Device is not available on the client request or in any downstream proxys (it's generated inside Varnish) you have to vary on the raw User-Agent header.
Although varying on X-UA-Device is incorrect for downstream caches, Varnish itself can still benefit from that optimization if you rewrite the Vary header in vcl_deliver:
sub vcl_deliver {
if (resp.http.Vary) {
set resp.http.Vary = regsub(resp.http.Vary,
"(?i)X-UA-Device",
"User-Agent");
}
}
This way, Varnish varies its cache on X-UA-Device and downstream caches vary on User-Agent.
In your question, you mentioned you were adding X-UA-Device to the response header as well as the request header. In that case, the above suggestion will not work and you will instead need to send Vary: User-Agent unconditionally:
sub vcl_fetch {
set beresp.http.X-UA-Device = req.http.X-UA-Device;
if (!beresp.http.Vary) {
set beresp.http.Vary = "User-Agent";
} elsif (beresp.http.Vary !~ "(?i)User-Agent") {
set beresp.http.Vary = beresp.http.Vary + ", User-Agent";
}
}
(I was not sure whether you were setting the X-UA-Device response header for the benefit of client-side scripts, or in the hope that it would be recognized by downstream caches.)
Related
var epoch = time.Unix(0, 0).Format(time.RFC1123)
var headers = map[string]string{
"Expires": epoch,
"Cache-Control": "no-cache, private, max-age=0",
"Pragma": "no-cache",
"X-Accel-Expires": "0",
}
log.Errorln("no cache header")
for k, v := range headers {
rw.Header().Set(k, v)
}
http.ServeFile(rw, req, path)
I have the above code block in my server side to serve file requests. But the files served are still cached in browser.
But instead of http.ServeFile() I used http.ServeContent() with the last modified timestamp as time.Now(). This works if the request has considerable time difference. Does browser give priority to last modified time than no-cache header? Then what is the purpose of no-cache header? Am I doing anything wrong?
Some browsers cache files without paying attention to headers. Typical solution is to add some get parameter to static file path, so the browser considered it like different request. It can be hash-sum or last modified timestamp. It might look somethind like this: site.com/icon.png?q=123456.
I'm trying to add gzip compression to a HTTP server i wrote in D. here is the code that dose the gzip encoding.
if ((Info.modGzip) & (indexOf(client.getRequestHeaderFieldValue("Accept-Encoding"),"gzip") != -1)){
writeln("gzip");
auto gzip = new Compress(HeaderFormat.gzip);
client.addToResponseHeader("Content-Encoding: gzip");
client.sendHeader("200 ok");
while (0 < (filestream.readBlock(readbuffer))){
client.client.send(gzip.compress(readbuffer));
}
client.sendData(gzip.flush(Z_FINISH));
delete gzip;
} else {
writeln("no gzip");
client.sendHeader("200 ok");
while (0 < (filestream.readBlock(readbuffer))){
client.client.send(readbuffer);
}
delete filestream;
}
but when i test it Firefox, internet explorer and chrome says that the encoding or compression is bad. why? the data is compressed with gzip.
Your code isn't sending the appropriate headers. The compression portion is fine, but the stuff surrounding it has a few bugs that need to be fixed.
cross posting what I said on the D newsgroup: http://forum.dlang.org/post/rngupyejcsbzkzqwgojp#forum.dlang.org
I'm trying to add gzip compression to a HTTP server i wrote in
D. here is the code that dose the gzip encoding.
I know zlib gzip works for http, I used it in my cgi.d
if(gzipResponse && acceptsGzip && isAll) {
auto c = new Compress(HeaderFormat.gzip); // want gzip
auto data = c.compress(t);
data ~= c.flush();
t = data;
}
But your http server is buggy in a lot of ways. It doesn't reply to curl and doesn't keep the connection open to issue manual requests.
Among the bugs I see looking at it quickly:
server.d getRequestHeaderFieldValue, you don't check if epos is -1. If it is, you should return null or something instead of trying to use it - the connection will hang because of an out-of-bounds array read killing the handler.
You also wrote:
if ((Info.modGzip) & (indexOf(client.getRequestHeaderFieldValue("Accept-Encoding"),"gzip") != -1)){
Notice the & instead of &&. That's in fspipedserver.d.
Finally, you write client.client.send... which never sent the headers back to the client, so it didn't know you were gzipping! Change that to client.sendData (and change sendData in server.d to take "in void[]" instead of "void[]") and then it sends the headers and seems to work by my eyeballing.
I have set up locally Varnish and have implemented an ESI fragment on a specific area of the page; but as soon as I turn on ESI on the default.vcl, varnish stops caching and even more, that specific section of the page where the fragment is added, renders a 503 service unavailable notice instead.
The line that seems to make varnish stop caching is the else statement here:
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
unset beresp.http.set-cookie;
set beresp.ttl = 20m;
}
else {
set beresp.do_esi = true;
}
Here is my default.vcl:
http://pastebin.com/MEQF4Gbk
Now, in the RxHeaders I get a lot of "ESI_xmlerror c No ESI processing, first char not '<'"
Here a sample of one of the RxHeaders (for a JS file)
http://pastebin.com/tX7zpBN1
I guess, I'm not understanding how am suppose to tell varnish to cache only when it has esi fragment? Can someone explain what I'm doing wrong?
Your vcl is set to do esi processing of not only the page containing the esi, but the esi fragments themselves, and the latter are failing resulting in 503s instead of the snippet you want. If you do not want to recursively process fragments as esi containers themselves you can change your VCL to:
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
unset beresp.http.set-cookie;
set beresp.ttl = 20m;
}
else if (req.esi_level == 0 ) {
set beresp.do_esi = true;
}
If you do want to process those includes as esi containers themselves, make sure they start with an angle bracket < (varnish 2 defaults to only performing esi for html/xml). If that's not possible, you can configure varnish not to care by setting this param when starting varnishd:
esi_syntax=0×1
If I were you I would enable esi specifically for the resources that can contain esi:includes rather than disabling it for select content types like you are doing. This way you don't need to maintain a list in your VCL that might change, and resources that do not contain esi includes won't waste varnish's time scanning for them. You can do this based on a response header from the backend like X-Varnish-Do-Esi, and only send that response header for resources that contain includes.
Is it possible to set some http headers while http-redirect(302 or 307)?
<?
header("some-header: xxx");
header("Location: http://other.domain.com/foo.php",TRUE,307);
?>
You can basically set whatever http headers you want either as the server or the client.
If you are indicating a redirect you should supply the Location header as your example suggests. You should also ensure that your response headers refer to that response rather than the resource that the client is being redirected to. i.e. your headers here could include Content-Length: 0, omit the Content-Type header and so on.
Not sure if this is what you're after - this question could do with a bit more detail.
You can always do the redirection 301/307.
There are ways to do it
1) Do it through java code :
response.setStatus(307);
response.setHeader("Location",url);
2) THe same thing can be done in JSPs.
A tip here is: Always use the setHeader function and not the addHeader function as they behave in different ways.
I have an IHttpHandler similar to AssemblyResourceLoader. What it does is generate an image and then send it back to the browser.
In AssemblyResourceLoader there is a code block like this:
HttpCachePolicy cache = context.Response.Cache;
cache.SetCacheability(HttpCacheability.Public);
cache.VaryByParams["d"] = true;
cache.SetOmitVaryStar(true);
cache.SetExpires(DateTime.Now + TimeSpan.FromDays(365.0));
cache.SetValidUntilExpires(true);
Pair assemblyInfo = GetAssemblyInfo(assembly);
cache.SetLastModified(new DateTime((long) assemblyInfo.Second));
I have set up mine to emit the exact same headers as AssemblyResourceLoader. I set the Last-Modified header and the browser sends the If-Modified-Since header to my handler just as it does with AssemblyResourceLoader. The problem is this: My handler never returns the 304 like AssemblyResourceLoader does. I can't find anywhere in the AssemblyResourceLoader code where it deals with the If-Modified-Since header so I don't know how to deal with it myself. Does anybody know where ASP.Net does that and how I can get the same behavior out of my handler?
Thanks.
Looks like you need to do it yourself, but it's not hard: http://www.motobit.com/tips/detpg_net-last-modified/