Is it possible to remove an item from the hash which has already been added via a previous hash_data() call?
I have a varnish 3 server running in front of a standard LAMP stack and using the Maxmind geoip VMOD per the varnish documentation except there is one included VCL file that I don't have permission to edit which basically has:
sub vcl_hash {
if (req.http.X-Geo-Country) {
hash_data(req.http.X-Geo-Country);
}
}
I know I can call vcl_hash multiple times and they concatenate in order so what I'd like to do is something like the below (un_hash_data doesn't exist) so that only the homepage hashes based on req.http.X-Geo-Country.
Is there something like un_hash_data() which can remove this from the hash.
sub vcl_hash {
if (req.url == "/") {
un_hash_data(req.http.X-Geo-Country);
}
}
In vcl_recv you can unset req.http.X-Geo-Country for the pages/requests that don't match homepage. If X-Geo-Country is not set/empty - will make the same hash :)
Related
I am trying to set up Varnish to route between backends using the request body's hash. I found good examples of using body access to set up caching where the hash of the request body is used as cache key. I cannot find an example of using the body hash for routing.
I tried the following but it doesn't seem to work. It is probably because bodyaccess was not meant to be used this way. How to set up a hash based director using vcl in Varnish which is using the hashed request body?
vcl 4.1;
import directors;
import bodyaccess;
backend backend1 {
.host = "backend1.example.com";
.port = "80";
}
backend backend2 {
.host = "backend2.example.com";
.port = "80";
}
sub vcl_init {
new xhash = directors.hash();
xhash.add_backend(backend1);
xhash.add_backend(backend2);
}
sub vcl_recv {
set req.backend_hint = xhash.backend(bodyaccess.hash_req_body());
}
There is no real way to retrieve the request body as a string in Varnish Cache (open source). The bodyaccess.hash_req_body() function will actually add the request body to the caching hash in the vcl_hash subroutine. But since this function returns a void data type, this won't help you.
The only realistic way that I'm aware of is by using vmod_xbody, which is a Varnish Enterprise module. That module has an xbody.get_req_body() function that returns the request body as a string.
See https://docs.varnish-software.com/varnish-cache-plus/vmods/xbody/#get-req-body for more information.
I have a service that is hosted in an internal network that is receiving traffic in port 443 (via https) behind a custom loadbalancer both from the internet but also from the internal network.
Internal network requests are coming with an extra custom header, let's call it X-my-lb-header.
I want to block all external incoming traffic to all uris (return an http response code), except to some specific ones.
Eg, let's say that i want to allow traffic that is coming to two endpoints /endpoind1/ (preffix match) and /endpoint2 actual match.
What is the best way to achieve a behaviour like this?
If my understanding is correct I need something like (not correct syntax bellow)
location = /endpoind2 {
if ($http_x_my_lb_header not exists) {
pass
} else {
return 404
}
... the rest of the directives
}
location ~ / {
if ($http_x_my_lb_header) {
return 404;
}
... the rest of the directives
}
But since else is not supported in nginx, i cannot figure out to do it.
Any ideas?
So you need some logic like
if (header exists) {
if (request URI isn't whitelisted) {
block the request
}
}
or in another words
if ((header exists) AND (request URI isn't whitelisted)) {
block the request
}
Well, nginx don't allow nested if blocks (nor logical conditions). While some people inventing a really weird but creative solutions like this one (emulating AND) or even this one (emulating OR), a huge part of such a problems can be solved using map blocks (an extremely powerfull nginx feature).
Here is an example:
# get the $block variable using 'X-my-lb-header' value
map $http_x_my_lb_header $block {
# if 'X-my-lb-header doesn't exists, get the value from another map block
'' $endpoint;
# default value (if the 'X-my-lb-header' exists) will be an empty string
# (unless not explicitly defined using 'default' keyword)
}
# get the $endpoint variable using request URI
map $uri $endpoint {
# endpoint1 prefix matching (using regex)
~^/endpoint1 ''; don't block
# endpoint2 exact matching
/endpoint2 ''; don't block
default 1; # block everything other
}
Now you can use this check in your server block (don't put it to some location, use at the server context):
if ($block) { return 404; }
For example we have setup like this:
user -> api gateway -> (specific endpoint) varnish -> backend service
If backend returns response 500 {"message":"error"} I want to patch this response and return 200 "[]" instead.
Is it possible to do something like this using varnish or some other proxy?
It is definitely possible to intercept backend errors, and convert them into regular responses.
A very simplistic example is the following:
sub vcl_backend_error {
set beresp.http.Content-Type = "application/json";
set beresp.status = 200;
set beresp.body = "[]";
return(deliver);
}
sub vcl_backend_response {
if(beresp.status == 500) {
return(error(200,"OK"));
}
}
Whenever your backend would fail, and return an HTTP/503 error, we will send a HTTP/200 response with [] output.
This output template for backend errors is also triggered when the backend does reply, but with a HTTP/500 error.
In real world scenarios, I would a some conditional logic in vcl_backend_error to only return the JSON output template when specific criteria are matched. For example: a certain URL pattern was matched.
I would advise the same in vcl_backend_response: maybe you don't want to convert all HTTP/500 errors into regular HTTP/200 responses. Maybe you also want to add conditional logic.
I need to get the CN of a client certificate in NGINX to append it to the proxy headers.
I already found the following map code for this.
map $ssl_client_s_dn $ssl_client_s_dn_cn {
default "";
~/CN=(?<CN>[^/]+) $CN;
}
But sadly it only returns an empty string for the following $ssl_client_s_dn:
CN=testcn,O=Test Organization
I tested it with other DNs, too. But the problem is always the same.
The pattern you use requires the legacy DN, since it assumes the / to separate the RDNs. So (since nginx v1.11.6) the following works:
map $ssl_client_s_dn_legacy $ssl_client_s_dn_cn {
default "";
~/CN=(?<CN>[^/]+) $CN;
}
With $ssl_client_s_dn_legacy: /O=Test Organization/CN=testcn
As #christof-r mentioned, your regex matched with the legacy DN pattern. Please use this regex to match with the current ( > v1.11.6) pattern.
map $ssl_client_s_dn $ssl_client_s_dn_cn {
default "";
~CN=(?<CN>[^,]+) $CN;
}
I am new to varnish and I am trying to figure out a way in which varnish can reference to an external html file to serve an error page, as having the html code in the synthetic function would be too complex as the error page has too many visualizations, thanks
If you are using Varnish 4 (or more recent) you can do it with the std vmod
See the doc : https://www.varnish-cache.org/docs/4.0/reference/vmod_std.generated.html#func-fileread
I think the vcl should look like the following (not tested) :
vcl 4.0;
import std;
#other stuff
sub vcl_synth {
if (resp.status == 404) {
set resp.http.Content-Type = "text/html;";
synthetic(std.fileread("/etc/varnish/404.html"));
return (deliver);
}
}