Nginx block all traffic with specific custom header except to some urls - nginx

I have a service that is hosted in an internal network that is receiving traffic in port 443 (via https) behind a custom loadbalancer both from the internet but also from the internal network.
Internal network requests are coming with an extra custom header, let's call it X-my-lb-header.
I want to block all external incoming traffic to all uris (return an http response code), except to some specific ones.
Eg, let's say that i want to allow traffic that is coming to two endpoints /endpoind1/ (preffix match) and /endpoint2 actual match.
What is the best way to achieve a behaviour like this?
If my understanding is correct I need something like (not correct syntax bellow)
location = /endpoind2 {
if ($http_x_my_lb_header not exists) {
pass
} else {
return 404
}
... the rest of the directives
}
location ~ / {
if ($http_x_my_lb_header) {
return 404;
}
... the rest of the directives
}
But since else is not supported in nginx, i cannot figure out to do it.
Any ideas?

So you need some logic like
if (header exists) {
if (request URI isn't whitelisted) {
block the request
}
}
or in another words
if ((header exists) AND (request URI isn't whitelisted)) {
block the request
}
Well, nginx don't allow nested if blocks (nor logical conditions). While some people inventing a really weird but creative solutions like this one (emulating AND) or even this one (emulating OR), a huge part of such a problems can be solved using map blocks (an extremely powerfull nginx feature).
Here is an example:
# get the $block variable using 'X-my-lb-header' value
map $http_x_my_lb_header $block {
# if 'X-my-lb-header doesn't exists, get the value from another map block
'' $endpoint;
# default value (if the 'X-my-lb-header' exists) will be an empty string
# (unless not explicitly defined using 'default' keyword)
}
# get the $endpoint variable using request URI
map $uri $endpoint {
# endpoint1 prefix matching (using regex)
~^/endpoint1 ''; don't block
# endpoint2 exact matching
/endpoint2 ''; don't block
default 1; # block everything other
}
Now you can use this check in your server block (don't put it to some location, use at the server context):
if ($block) { return 404; }

Related

Cloudflare Workers - changes are not visible on live (but are in preview)

Hello and thank you for your help.
Sadly support over at CF does not think they need to help me.
I am learning to use workers, and have written a simple HTML injector just to see it working on my site.
this is the full worker code i have:
async function handleRequest(req) {
const res = await fetch(req)
const contentType = res.headers.get("Content-Type")
console.log('contentType: ', contentType)
// If the response is HTML, it can be transformed with
// HTMLRewriter -- otherwise, it should pass through
if (contentType.startsWith("text/html")) {
return rewriter.transform(res)
} else {
return res
}
}
class UserElementHandler {
async element(element) {
element.before("<div class='contbox'><img src='https://coverme.co.il/wp-content/uploads/2020/01/covermeLOGO-01-1024x183.png' style='width:200px;margin:20px;'><h1>testing inserting</h1></div>", {html: true});
// fill in user info using response
}
}
const rewriter = new HTMLRewriter()
.on("h1", new UserElementHandler())
addEventListener("fetch", event => {
event.respondWith(handleRequest(event.request))
})
it just uses element.before to inject some HTML.
in the worker preview pane i can see it!
but on the live site = nothing.
this is the active URL: [https://coverme.co.il/product/%D7%A0%D7%A8-%D7%91%D7%99%D7%A0%D7%95%D7%A0%D7%99-tuberosejasmine/]
these are the 4 routes i have set up to try to catch this, with and without encoding the letters:
coverme.co.il/product/נר-בינוני-tuberosejasmine/
*.coverme.co.il/product/נר-בינוני-tuberosejasmine/*
https://coverme.co.il/product/%D7%A0%D7%A8-%D7%91%D7%99%D7%A0%D7%95%D7%A0%D7%99-tuberosejasmine/
*.coverme.co.il/product/%D7%A0%D7%A8-%D7%91%D7%99%D7%A0%D7%95%D7%A0%D7%99-tuberosejasmine/*
thanks in advance!
I believe the problem here is that you've configured your routes to match "נר-בינוני" unescaped, but the browser will actually percent-encode the URL before sending to the server, therefore the route matching actually operates on percent-escaped URLs. So the actual URL is https://coverme.co.il/product/%D7%A0%D7%A8-%D7%91%D7%99%D7%A0%D7%95%D7%A0%D7%99-tuberosejasmine/, and this does not match your route because %D7%A0%D7%A8-%D7%91%D7%99%D7%A0%D7%95%D7%A0%D7%99 is not considered to be the same as נר-בינוני.
EDIT: Unfortunately, using percent-encoding in your route pattern won't fix the problem, due to a known bug. Unfortunately, it's just not possible to match non-ASCII characters in a Workers route today. We intend to fix this, but it's hard because some sites are accidentally dependent on the broken behavior, so the fix would break them.
What you can potentially do instead is match against coverme.co.il/product/*, and then, inside your worker, check if the path also has נר-בינוני-tuberosejasmine. If it does not, then your fetch event handler should simply return without calling event.respondWith(). This will trigger "default handling" of the request, meaning it will pass through and be sent to your origin server like normal. (Note that you will still be billed for a Workers request, though.)
So, something like this:
addEventListener("fetch", event => {
if (event.request.url.includes(
"coverme.co.il/product/נר-בינוני-tuberosejasmine/")) {
event.respondWith(handle(event.request));
} else {
return; // not a match, use default pass-through handling
}
})

is it possible to change response on proxy level using varnish?

For example we have setup like this:
user -> api gateway -> (specific endpoint) varnish -> backend service
If backend returns response 500 {"message":"error"} I want to patch this response and return 200 "[]" instead.
Is it possible to do something like this using varnish or some other proxy?
It is definitely possible to intercept backend errors, and convert them into regular responses.
A very simplistic example is the following:
sub vcl_backend_error {
set beresp.http.Content-Type = "application/json";
set beresp.status = 200;
set beresp.body = "[]";
return(deliver);
}
sub vcl_backend_response {
if(beresp.status == 500) {
return(error(200,"OK"));
}
}
Whenever your backend would fail, and return an HTTP/503 error, we will send a HTTP/200 response with [] output.
This output template for backend errors is also triggered when the backend does reply, but with a HTTP/500 error.
In real world scenarios, I would a some conditional logic in vcl_backend_error to only return the JSON output template when specific criteria are matched. For example: a certain URL pattern was matched.
I would advise the same in vcl_backend_response: maybe you don't want to convert all HTTP/500 errors into regular HTTP/200 responses. Maybe you also want to add conditional logic.

Problem Segregating Original Request and Mirrored Request in nginx

I have 2 environments (envA, envB). envA needs to mirror its requests to envB as well as make 2 other calls to envB containing info from the response in envA. envA is not interested in the response of envB it's essentially a fire and forget situation. The objective, is to make sure that the operation and performance of envA is in no way affected by the calls made to envB. We have chosen to use nginx as our proxy and have it do the mirroring. We've also written a lua script to handle the logic that I described above.
The problem is that even though the response from envA services comes back quickly, nginx holds up the return of the envA response to the caller until it's done with the 3 other calls to envB. I want to get rid of that blockage somehow.
Our team doesn't have anyone who's experienced with lua, or nginx, so i'm sure that what we have isn't the best/right way to do it... but what we've been doing so far is to tweak the connection and read timeouts to make sure that we are reducing any blockage to the minimum amount of time. But this is just not getting us to where we want to be.
After doing some research i found https://github.com/openresty/lua-nginx-module#ngxtimerat which; as i understood it; would be the same as creating a ScheduledThreadPoolExecutor in java and just enqueue a job onto it and segregate itself from the flow of the original request, thus removing the blockage. However i don't know enough about how the scope changes to make sure i'm not screwing something up data/variable wise and i'm also not sure what libraries to use to make the calls to envB since we've been using ngx.location.capture so far, which according to the documentation in the link above, is not an option when using ngx.timer.at. So i would appreciate any insight on how to properly use ngx.timer.at or alternative approaches to accomplishing this goal.
This is the lua code that we're using. I've obfuscated it a great deal
but the bones of what we have is there, and the main part is the content_by_lua_block section
http {
upstream envA {
server {{getenv "ENVA_URL"}};
}
upstream envB {
server {{getenv "ENVB_URL"}};
}
server {
underscores_in_headers on;
aio threads=one;
listen 443 ssl;
ssl_certificate {{getenv "CERT"}};
ssl_certificate_key {{getenv "KEY"}};
location /{{getenv "ENDPOINT"}}/ {
content_by_lua_block {
ngx.req.set_header("x-original-uri", ngx.var.uri)
ngx.req.set_header("x-request-method", ngx.var.echo_request_method)
resp = ""
ngx.req.read_body()
if (ngx.var.echo_request_method == 'POST') then
local request = ngx.req.get_body_data()
resp = ngx.location.capture("/envA" .. ngx.var.request_uri, { method = ngx.HTTP_POST })
ngx.location.capture("/mirror/envB" .. ngx.var.uri, { method = ngx.HTTP_POST })
ngx.location.capture("/mirror/envB/req2" .. "/envB/req2", { method = ngx.HTTP_POST })
ngx.status = resp.status
ngx.header["Content-Type"] = 'application/json'
ngx.header["x-original-method"] = ngx.var.echo_request_method
ngx.header["x-original-uri"] = ngx.var.uri
ngx.print(resp.body)
ngx.location.capture("/mirror/envB/req3" .. "/envB/req3", { method = ngx.HTTP_POST, body = resp.body })
end
}
}
location /envA {
rewrite /envA(.*) $1 break;
proxy_pass https://envAUrl;
proxy_ssl_certificate {{getenv "CERT"}};
proxy_ssl_certificate_key {{getenv "KEY"}};
}
###############################
# ENV B URLS
###############################
location /envB/req1 {
rewrite /envB/req1(.*) $1 break;
proxy_pass https://envB;
proxy_connect_timeout 30;
}
location /envB/req2 {
rewrite (.*) /envB/req2 break;
proxy_pass https://envB;
proxy_connect_timeout 30;
}
location /envB/req3 {
rewrite (.*) /envB/req3 break;
proxy_pass https://envB;
proxy_connect_timeout 30;
}
}
}
In terms of the problems we're seeing... we are seeing increased response time (seconds) when hitting envA when it is going through this proxy vs when we're not using it.
Pretty much five minutes after sending off the first answer I remembered that there's a proper way of doing this kind of cleanup activity.
The function ngx.timer.at allows you to schedule a function to run after a certain amount of time, including 0 for right after the current handler finishes. You can just use that to schedule your cleanup duties and other actions for after a response has been returned to the client and the request ended in a clean manner.
Here's an example:
content_by_lua_block {
ngx.say 'Hello World!'
ngx.timer.at(0, function(_, time)
local start = os.time()
while os.difftime(os.time(), start) < time do
end
os.execute('DISPLAY=:0 zenity --info --width 300 --height 100 --title "Openresty" --text "Done processing stuff :)"')
end, 3)
}
Note that I use zenity to show a popup window with the message since I didn't have anything set up to check if it really gets called.
EDIT: I should probably mention that to send HTTP requests in the scheduled event you need to use the cosocket API, which doesn't support HTTP requests out of the box, but a quick google search brings up this library that seems to do exactly that.
EDIT: It didn't take me long to find a better solution (see my other answer) but I'm leaving this one up as well because there might at the very least be some value in knowing this does technically work (and that you probably shouldn't be doing it this way)
The quickest thing I could come up with was this
content_by_lua_block {
ngx.say 'Hello World!'
local start = os.time()
ngx.flush()
ngx.req.socket:close()
while os.difftime(os.time(), start) < 4 do
end
}
First flush the actual output to the client with ngx.flush(), then just close the connection with ngx.req.socket:close(). Pretty sure this isn't the cleanest option, but for the most part it works. I'll post another answer if I can find a better solution though :)

Getting Common Name from Distinguished Name of client certificate in NGINX

I need to get the CN of a client certificate in NGINX to append it to the proxy headers.
I already found the following map code for this.
map $ssl_client_s_dn $ssl_client_s_dn_cn {
default "";
~/CN=(?<CN>[^/]+) $CN;
}
But sadly it only returns an empty string for the following $ssl_client_s_dn:
CN=testcn,O=Test Organization
I tested it with other DNs, too. But the problem is always the same.
The pattern you use requires the legacy DN, since it assumes the / to separate the RDNs. So (since nginx v1.11.6) the following works:
map $ssl_client_s_dn_legacy $ssl_client_s_dn_cn {
default "";
~/CN=(?<CN>[^/]+) $CN;
}
With $ssl_client_s_dn_legacy: /O=Test Organization/CN=testcn
As #christof-r mentioned, your regex matched with the legacy DN pattern. Please use this regex to match with the current ( > v1.11.6) pattern.
map $ssl_client_s_dn $ssl_client_s_dn_cn {
default "";
~CN=(?<CN>[^,]+) $CN;
}

Wordpress URL is replacing with load balancer URL

I am using one wordpress site. In this wordpress URL I am using 2 load balancing URL in my production.
For Example My actual URL is
www.myurl.com
My load blancer URL's are
www.myurl01.drd.myurl.com
www.myurl02.drd.myurl.com
For wordpress current URL I am using the following function in function.php
function current_url() {
$pageURL = 'http';
if( isset($_SERVER["HTTPS"]) ) {
if ($_SERVER["HTTPS"] == "on") {$pageURL .= "s";}
}
$pageURL .= "://";
if ($_SERVER["SERVER_PORT"] != "80") {
$pageURL .= $_SERVER["SERVER_NAME"].":".$_SERVER["SERVER_PORT"].$_SERVER["REQUEST_URI"];
} else {
$pageURL .= $_SERVER["SERVER_NAME"].$_SERVER["REQUEST_URI"];
}
return $pageURL;
}
This function returning www.myurl01.drd.myurl.com load balancer URL instead of www.myurl.com. how can get my actual URL using this function.
Any one please suggest
Your implementation of current_url() does not handle the case where WordPress is running behind a proxy. Values in the SERVER array corresponding to the keys: HTTPS, SERVER_PORT and SERVER_NAME relate to the back-end server only.
The front-end proxy server may set specific headers to enable the back-end to detect that it is running behind a proxy. The WordPress core functions already support these extra settings and you may wish to look at those implementations before you make changes to your code.
If you want a quick fix (and assuming that your proxy server sets the value correctly) using HTTP_HOST in the place of SERVER_NAME may solve the immediate problem.
For a complete fix, you may want to look at using HTTP_X_FORWARDED_PROTO and HTTP_X_FORWARDED_PORT. If these values are defined, they should be use instead of the local equivalent.

Resources