Just new to Varnish. It's getting hard, more than expected :-(
I'm trying to improve some php code, developed some time ago, using varnish.
This code uses just two cookies: PHPSESSID and LANGUAGE
All pages set PHPSESSID cookie if it's not defined. However this cookie for anonymous sessions it's only used in one page.
Let's say I have Page1, Page2, Page3 and Page4. My configuration should be as follows:
Page1, Page2 and Page3 need LANGUAGE cookie and should be cached with that cookie: one cache for each language and page.
Page4 needs PHPSESSID and LANGUAGE cookies, and shouldn't be cached as it's specific for each user.
My default.vlc is not working properly, so any orientation would be really apreciated. Maybe I've missunderstood some concepts.
sub vcl_init {
# When requests come to Varnish I need to remove PHPSESSID so it's not used for the hash in caching. Page4 doesn't need caching as it's specific for each user:
if (req.http.host ~ "Page4") {
return(pass);
}
# remove PHPSESSID so pages1, 2, and 3 get cached just once for everyuser but in all languages.
if ((req.url !~ "page4")) {
set req.http.Cookie = regsuball(req.http.Cookie, "PHPSESSID=[^;]+(; )?", "");
}
return (hash);
}
I need to cache webpages with LANGUAGE cookie so I include it in vcl_hash:
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# hash cookies for requests that have them
if (req.http.Cookie) {
hash_data(req.http.Cookie);
}
}
How can I remove just PHPSESSIONID?
sub vcl_backend_response {
# Called after the response headers has been successfully retrieved from the backend.
if (!(bereq.url ~ "Page4")) {
unset beresp.http.set-cookie;
}
return (deliver);
}
You are on the right track. If I understand your question, the thing that does not work is that you remove all cookies server response (if it's not Page4) instead of just removing PHPSESSID.
You can do a regexp in your sub vcl_backend_response to remove only phpsessionid if url is not Page4.
beresp.http.set-cookie = regsuball(beresp.http.set-cookie, "PHPSESSID=[^;]+(; )?", "")
Alternatively, if you use varnish 4 or above, you should use the vmod cookie which makes cookie handling much easier (no more need for regexp).
Related
How can I tell Varnish not to cache 404 pages for JPEG files and retry instead of caching?
Tried this but did not work:
if (beresp.status == 404) { set beresp.ttl = 0s; }
sub vcl_fetch {
if (req.url ~ "\.jpeg$" && beresp.status == 404) {
set beresp.ttl = 0s;
return(restart);
}
}
A couple of comments/notes. Please treat the above snippet as a "pseudocode" as I have not tested it.
Why retry if 404? Is there a lot of "movement" on your backend(s)?
As another side-note, you should really consider upgrading to Varnish 4.
Your version is way beyond its EOL.
I wonder if I can get some help. I am running a VPS with Varnish. This VPS is predominantly WordPress but does have a Joomla site running too. For some reason my Varnish config file decided it was going to go back to default so I have been having a range of problems as I was silly enough not to save my custom config file!
Here is my current config file:
# This is a basic VCL configuration file for varnish. See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition. Set this to point to your content
# server.
#
backend default {
.host = "LIVE IP";
.port = "8080";
.max_connections = 800;
}
acl purge { "localhost"; "127.0.0.1"; }
sub vcl_recv {
set req.grace = 2m;
# Set X-Forwarded-For header for logging in nginx
remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;
# Remove has_js and CloudFlare/Google Analytics __* cookies and statcounter is_unique
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_[_a-z]+|has_js|is_unique)=[^;]*", "");
# Remove a ";" prefix, if present.
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
# Either the admin pages or the login
if (req.url ~ "/wp-(login|admin|cron)") {
# Don't cache, pass to backend
return (pass);
}
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie,
"wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie,
"wordpress_test_cookie=[^;]+(;)?", "");
# Static content unique to the theme can be cached (so no user uploaded images)
# The reason I don't take the wp-content/uploads is because of cache size on bigger blogs
# that would fill up with all those files getting pushed into cache
if (req.url ~ "wp-content/themes/" && req.url ~
"\.(css|js|png|gif|jp(e)?g)") {
unset req.http.cookie;
}
# Even if no cookies are present, I don't want my "uploads" to be cached due to their potential size
if (req.url ~ "/wp-content/uploads/") {
return (pass);
}
# any pages with captchas need to be excluded
if (req.url ~ "^/contact/" || req.url ~ "^/links/domains-for-sale/")
{
return(pass);
}
# Check the cookies for wordpress-specific items
if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
# A wordpress specific cookie has been set
return (pass);
}
# allow PURGE from localhost
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return (lookup);
}
# Force lookup if the request is a no-cache request from the client
if (req.http.Cache-Control ~ "no-cache") {
return (pass);
}
# Try a cache-lookup
return (lookup);
}
sub vcl_fetch {
#set obj.grace = 5m;
set beresp.grace = 2m;
}
sub vcl_hit {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}
sub vcl_miss {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
}
The issue I am having is that my WP core files are moved to a subfolder called 'core' and I change my login URL to /administrator instead of wp-login.php. so to login, I'd go to either domain.com/administrator or domain.com/core/administrator. If I type the domain sans /core/ it would direct to */core/administrator.
The way that Varnish works, it doesn't allow me to log into the WP-admin side of the site which I think is due to the cookies. I added /administrator to the VCL-config text but it didn't seem to work.
Is anyone able to please help me out with this as I'd rather not revert all my installations back to wp-login.php.
I also don't seem to be able to login to SSH and purge the cache like I could before. I honestly can't remember the config I had before but I am sure I am missing something, so if anyone is able to improve on this code to make it work as best as it can (or whether I have missed important things out entirely, specifically with Joomla) then help would be greatly appreciated.
I don't make theme on the fly but I do have some users logging into various sites hosted on the VPS to add/change posts so getting the cache to purge properly would be fantastic as I'm beginning to pull my hair out!
Thanks everyone.
To make PURGE work, you can try to add your VPS hostname in "acl purge" section. I know I had to do that on my VPS and Varnish v4.
And for wp admin part, it should not be caches at all, so try changeing this part:
# Either the admin pages or the login
if (req.url ~ "/wp-(login|admin|cron)") {
# Don't cache, pass to backend
return (pass);
}
into this:
# Either the admin pages or the login
if (req.url ~ "/core/administrator" || req.url ~ "/administrator") {
# Don't cache, pass to backend
return (pass);
}
I'm attempting to get Varnish to cache two different domains with blogs, but upon adding the second one, the previous one stops working,
The basic setup is as following:
backend default {
.host = "127.0.0.1";
.port = "81";
}
backend onedomain {
.host = "127.0.0.1";
.port = "81";
}
backend newdomain {
.host = "127.0.0.1";
.port = "81";
}
acl purge {
"localhost";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
#Bypass large files
if (req.http.x-pipe-mark && req.restarts > 0) {
return(pipe);
}
# all domains in here will return a "pass" which means they won't be cached
if (req.http.host ~ "(www\.)?(domain1.com|domain2.com)") {
return (pass);
}
#else check if something we're going to cache
else if(req.http.host ~ "(www\.)?(onedomain.nu)") {
set req.http.host = "onedomain.com";
set req.backend_hint = onedomain;
}
else if(req.http.host ~ "(www\.)?(newdomain.com)") {
set req.http.host = "newdomain.com";
set req.backend_hint = newdomain;
}
else {
return (pass);
}
Newdomain loads fine while domain4 just sends me to an infinite redirect loop (according to the chrome error)
I added the full config in a pastebin: http://pastebin.com/J1Hb76dZ
I realize Varnish doesn't send any redirect commands itself, the site works on the old configuration, it's only when I try this that the redirect issue arises on one of the websites.
Is there anyone that has experience with this happening and can suggest what to do?
Old question, but try modifying the vcl_hash subroutine. This worked for me on a single site, that included multiple redirects for http -> https and domain.com -> www.domain.com. It should also configure the hash to tell your different domains apart, as that was necessary for me to store all the redirects separate from the site data that caused the dreaded "too many redirects" errors. You may need to adjust/remove the X-Forwarded-Proto header as I am behind a load balancer.
sub vcl_hash {
hash_data(req.http.host);
hash_data(req.url);
hash_data(req.http.X-Forwarded-Proto);
return(hash);
}
I'm very lost on this problem, and I don't know where could be the problem, so, I hope that you could help me.
I have an HTTP BASIC authentification with symfony, and I'm trying to reach an url which is protected by this auth, with an tag in a Drupal page. Every requests are send to a Varnish
I give username and password in the url like that :
<esi:include src="http://admin:adminpass#api.dev:8081/app.php/next"/>
In my varnish configuration file, I have only that lines for auth.http:
if (req.http.Authorization) {
return (pass);
}
My backend for Symfony is working well without http authentification, and the http authentification is working well when there's not Varnish and esi tag.
If anyone have an idea of the problem, please, tell me, even if it's wrong =)
ESI in varnish doesn't work like an iframe or link tag in a browser in that it doesn't connect to whatever url you give it. ESI just starts a new request within varnish and goes through the workflow (vcl_recv, etc).
You are expecting varnish to act like an http client, parsing the url, setting the authorization header, setting a host header to api.dev:8081 and initiating a new http connection/request which it will not. In this case, my guess is it starts a new req with req.url set to /app.php/next inheriting the headers from the request for the parent resource (containing the esi tag) or possibly just ignores the esi tag completely.
The way to accomplish what you want to do is (in vcl_recv):
if (req.esi_level > 0 && req.url == "/app.php/next") {
set req.http.Authorization = "BASIC [base64 encoded admin:adminpass]"
return (pass);
}
and then the esi tag should look like <esi:include src="/app.php/next" />
If you need the ESI request to hit a different backend server, you need to add that server as a different named backend:
backend authorization_needed {
.host = "api.dev";
.port = "8081";
}
and in vcl_recv, tell varnish to use it for esi requests:
if (req.esi_level > 0 && req.url == "/app.php/next") {
set req.http.Authorization = "BASIC [base64 encoded admin:adminpass]"
set req.backend = authorization_needed;
return (pass);
}
you may also need to set req.http.Host in that if block if the backend responds to a different virtual host than "api.dev".
Update:
Since basic authorization is coming from the client, and you are calling return (pass) when req.http.Authorization is present, varnish will not ESI process those pages. You must explicitly enable esi in vcl_fetch() which is not called when you pass.
So to pass authorization for the ESI fragments but not for the parent page, change in vcl_rev:
if (req.http.Authorization && req.esi_level == 0) {
set req.http.X-Esi-Authorization = req.http.Authorization;
unset req.http.Authorization;
}
else if (req.http.X-Esi-Authorization && req.esi_level > 0 ) {
set req.http.Authorization = req.http.X-Esi-Authorization;
return (pass);
}
And add to vcl_fetch:
if (req.http.X-Esi-Authorization) {
set beresp.do_esi = true;
}
The net effect is the parent response is cacheable and will process esi, the esi fragments themselves will always be passed to the backend with the client's authorization header.
I'm running Varnish in front of my Wordpress site. To increase my cache hit stats I blocked cookies in my varnish configuration in default.vcl
# Drop any cookies sent to Wordpress.
sub vcl_recv {
unset req.http.cookie;
}
# Drop any cookies Wordpress tries to send back to the client.
sub vcl_fetch {
unset beresp.http.set-cookie;
}
I believe this is what is keeping me from logging in to wp-admin right now. I am continually redirected back to the login page. Any idea what kind of filter I need to pass into that cookie blocker? I'm not familiar with this configuration language.
You need to NOT drop cookies if the URL is from the admin section.
Like this:
sub vcl_recv {
if (!(req.url ~ "wp-(login|admin)")) {
unset req.http.cookie;
}
}
sub vcl_fetch {
if (!(req.url ~ "wp-(login|admin)")) {
unset beresp.http.set-cookie;
}
}
Why not:
sub vcl_recv {
if (req.http.Cookie ~ "(wordpress_|wp-)") {
return (pass); // If WP cookies exist, do not cache
} else {
unset req.http.Cookie;
}
}
?