How can I tell Varnish not to cache 404 pages for JPEG files and retry instead of caching?
Tried this but did not work:
if (beresp.status == 404) { set beresp.ttl = 0s; }
sub vcl_fetch {
if (req.url ~ "\.jpeg$" && beresp.status == 404) {
set beresp.ttl = 0s;
return(restart);
}
}
A couple of comments/notes. Please treat the above snippet as a "pseudocode" as I have not tested it.
Why retry if 404? Is there a lot of "movement" on your backend(s)?
As another side-note, you should really consider upgrading to Varnish 4.
Your version is way beyond its EOL.
Related
I'm trying to serve static contents through varnish(client<-varnish<-nginx) but I'm having an issue accessing it through public IP. Varnish is working fine when accessing it locally (where varnish is running) but throws status 404 when accessing it through public IP.
My flow looks like this:
Client--> Caching_Server[NGINX(used for SSL support only) -> Varnish] --> Origin_Server[Nginx]
My varnish server config is as simple as:
probe healthcheck {
.url = "http://10.10.10.3/healthcheck";
.timeout = 2s;
.interval = 30s;
.window = 5;
.threshold = 3;
}
backend default {
.host = "10.10.10.3";
.port = "80";
.probe = healthcheck;
}
sub vcl_recv {
if (req.url ~ ".ts$") {
unset req.http.Cookie;
}
set req.backend_hint = default;
}
I also checked the Nginx logs and I see that it's throwing 404 which explains why varnish is responding with 404. But my question is why is it working when I test it locally using localhost.
The output from varnishlog -g request -q "ReqUrl eq '/'" will probably give you the answer to your question.
Run the command while sending a request internally, and a request from the internet. Compare the output, and see where there's a difference.
If you need help, add the logs of both requests to your question and I'll help you examine the situation.
Just new to Varnish. It's getting hard, more than expected :-(
I'm trying to improve some php code, developed some time ago, using varnish.
This code uses just two cookies: PHPSESSID and LANGUAGE
All pages set PHPSESSID cookie if it's not defined. However this cookie for anonymous sessions it's only used in one page.
Let's say I have Page1, Page2, Page3 and Page4. My configuration should be as follows:
Page1, Page2 and Page3 need LANGUAGE cookie and should be cached with that cookie: one cache for each language and page.
Page4 needs PHPSESSID and LANGUAGE cookies, and shouldn't be cached as it's specific for each user.
My default.vlc is not working properly, so any orientation would be really apreciated. Maybe I've missunderstood some concepts.
sub vcl_init {
# When requests come to Varnish I need to remove PHPSESSID so it's not used for the hash in caching. Page4 doesn't need caching as it's specific for each user:
if (req.http.host ~ "Page4") {
return(pass);
}
# remove PHPSESSID so pages1, 2, and 3 get cached just once for everyuser but in all languages.
if ((req.url !~ "page4")) {
set req.http.Cookie = regsuball(req.http.Cookie, "PHPSESSID=[^;]+(; )?", "");
}
return (hash);
}
I need to cache webpages with LANGUAGE cookie so I include it in vcl_hash:
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# hash cookies for requests that have them
if (req.http.Cookie) {
hash_data(req.http.Cookie);
}
}
How can I remove just PHPSESSIONID?
sub vcl_backend_response {
# Called after the response headers has been successfully retrieved from the backend.
if (!(bereq.url ~ "Page4")) {
unset beresp.http.set-cookie;
}
return (deliver);
}
You are on the right track. If I understand your question, the thing that does not work is that you remove all cookies server response (if it's not Page4) instead of just removing PHPSESSID.
You can do a regexp in your sub vcl_backend_response to remove only phpsessionid if url is not Page4.
beresp.http.set-cookie = regsuball(beresp.http.set-cookie, "PHPSESSID=[^;]+(; )?", "")
Alternatively, if you use varnish 4 or above, you should use the vmod cookie which makes cookie handling much easier (no more need for regexp).
I wonder if I can get some help. I am running a VPS with Varnish. This VPS is predominantly WordPress but does have a Joomla site running too. For some reason my Varnish config file decided it was going to go back to default so I have been having a range of problems as I was silly enough not to save my custom config file!
Here is my current config file:
# This is a basic VCL configuration file for varnish. See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition. Set this to point to your content
# server.
#
backend default {
.host = "LIVE IP";
.port = "8080";
.max_connections = 800;
}
acl purge { "localhost"; "127.0.0.1"; }
sub vcl_recv {
set req.grace = 2m;
# Set X-Forwarded-For header for logging in nginx
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# Remove has_js and CloudFlare/Google Analytics __* cookies and statcounter is_unique
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js|is_unique)=[^;]*", "");
# Remove a ";" prefix, if present.
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
# Either the admin pages or the login
if (req.url ~ "/wp-(login|admin|cron)") {
# Don't cache, pass to backend
return (pass);
}
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie,
"wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie,
"wordpress_test_cookie=[^;]+(;)?", "");
# Static content unique to the theme can be cached (so no user uploaded images)
# The reason I don't take the wp-content/uploads is because of cache size on bigger blogs
# that would fill up with all those files getting pushed into cache
if (req.url ~ "wp-content/themes/" && req.url ~
"\.(css|js|png|gif|jp(e)?g)") {
unset req.http.cookie;
}
# Even if no cookies are present, I don't want my "uploads" to be cached due to their potential size
if (req.url ~ "/wp-content/uploads/") {
return (pass);
}
# any pages with captchas need to be excluded
if (req.url ~ "^/contact/" || req.url ~ "^/links/domains-for-sale/")
{
return(pass);
}
# Check the cookies for wordpress-specific items
if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
# A wordpress specific cookie has been set
return (pass);
}
# allow PURGE from localhost
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return (lookup);
}
# Force lookup if the request is a no-cache request from the client
if (req.http.Cache-Control ~ "no-cache") {
return (pass);
}
# Try a cache-lookup
return (lookup);
}
sub vcl_fetch {
#set obj.grace = 5m;
set beresp.grace = 2m;
}
sub vcl_hit {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}
sub vcl_miss {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}
The issue I am having is that my WP core files are moved to a subfolder called 'core' and I change my login URL to /administrator instead of wp-login.php. so to login, I'd go to either domain.com/administrator or domain.com/core/administrator. If I type the domain sans /core/ it would direct to */core/administrator.
The way that Varnish works, it doesn't allow me to log into the WP-admin side of the site which I think is due to the cookies. I added /administrator to the VCL-config text but it didn't seem to work.
Is anyone able to please help me out with this as I'd rather not revert all my installations back to wp-login.php.
I also don't seem to be able to login to SSH and purge the cache like I could before. I honestly can't remember the config I had before but I am sure I am missing something, so if anyone is able to improve on this code to make it work as best as it can (or whether I have missed important things out entirely, specifically with Joomla) then help would be greatly appreciated.
I don't make theme on the fly but I do have some users logging into various sites hosted on the VPS to add/change posts so getting the cache to purge properly would be fantastic as I'm beginning to pull my hair out!
Thanks everyone.
To make PURGE work, you can try to add your VPS hostname in "acl purge" section. I know I had to do that on my VPS and Varnish v4.
And for wp admin part, it should not be caches at all, so try changeing this part:
# Either the admin pages or the login
if (req.url ~ "/wp-(login|admin|cron)") {
# Don't cache, pass to backend
return (pass);
}
into this:
# Either the admin pages or the login
if (req.url ~ "/core/administrator" || req.url ~ "/administrator") {
# Don't cache, pass to backend
return (pass);
}
I'm attempting to get Varnish to cache two different domains with blogs, but upon adding the second one, the previous one stops working,
The basic setup is as following:
backend default {
.host = "127.0.0.1";
.port = "81";
}
backend onedomain {
.host = "127.0.0.1";
.port = "81";
}
backend newdomain {
.host = "127.0.0.1";
.port = "81";
}
acl purge {
"localhost";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
#Bypass large files
if (req.http.x-pipe-mark && req.restarts > 0) {
return(pipe);
}
# all domains in here will return a "pass" which means they won't be cached
if (req.http.host ~ "(www\.)?(domain1.com|domain2.com)") {
return (pass);
}
#else check if something we're going to cache
else if(req.http.host ~ "(www\.)?(onedomain.nu)") {
set req.http.host = "onedomain.com";
set req.backend_hint = onedomain;
}
else if(req.http.host ~ "(www\.)?(newdomain.com)") {
set req.http.host = "newdomain.com";
set req.backend_hint = newdomain;
}
else {
return (pass);
}
Newdomain loads fine while domain4 just sends me to an infinite redirect loop (according to the chrome error)
I added the full config in a pastebin: http://pastebin.com/J1Hb76dZ
I realize Varnish doesn't send any redirect commands itself, the site works on the old configuration, it's only when I try this that the redirect issue arises on one of the websites.
Is there anyone that has experience with this happening and can suggest what to do?
Old question, but try modifying the vcl_hash subroutine. This worked for me on a single site, that included multiple redirects for http -> https and domain.com -> www.domain.com. It should also configure the hash to tell your different domains apart, as that was necessary for me to store all the redirects separate from the site data that caused the dreaded "too many redirects" errors. You may need to adjust/remove the X-Forwarded-Proto header as I am behind a load balancer.
sub vcl_hash {
hash_data(req.http.host);
hash_data(req.url);
hash_data(req.http.X-Forwarded-Proto);
return(hash);
}
I'm running Varnish in front of my Wordpress site. To increase my cache hit stats I blocked cookies in my varnish configuration in default.vcl
# Drop any cookies sent to Wordpress.
sub vcl_recv {
unset req.http.cookie;
}
# Drop any cookies Wordpress tries to send back to the client.
sub vcl_fetch {
unset beresp.http.set-cookie;
}
I believe this is what is keeping me from logging in to wp-admin right now. I am continually redirected back to the login page. Any idea what kind of filter I need to pass into that cookie blocker? I'm not familiar with this configuration language.
You need to NOT drop cookies if the URL is from the admin section.
Like this:
sub vcl_recv {
if (!(req.url ~ "wp-(login|admin)")) {
unset req.http.cookie;
}
}
sub vcl_fetch {
if (!(req.url ~ "wp-(login|admin)")) {
unset beresp.http.set-cookie;
}
}
Why not:
sub vcl_recv {
if (req.http.Cookie ~ "(wordpress_|wp-)") {
return (pass); // If WP cookies exist, do not cache
} else {
unset req.http.Cookie;
}
}
?