I am trying to block a webcrawler that uses the requested page as the http_referer, and I can't figure out what variable to compare it to.
e.g.
location / {
if ($the_variable_with_the_current_full_uri = $http_referer) {
return 403;
}
}
The variable has to match protocol, host, and URL, so that internal redirects from http to https don't get blocked.
So if someone requests "https://www.example.com/pages/1" with the $http_referer of "https://www.example.com/pages/1", it should be blocked.
As a secondary question, is it possible block requests on two conditions: where the above check matches, as well as matching a specific user agent string?
The full URL can be constructed by concatenating a number of variables together.
For example:
$scheme://$host$request_uri
The secondary condition could be handled using a map (see this document).
For example:
map $http_user_agent $my_http_referer {
default "";
blahblah $http_referer;
}
server {
...
if ($scheme://$host$request_uri = $my_http_referer) { return 403; }
...
}
Related
I have two domain names, each for different applications hosted in a single kubernetes cluster.
Is there a way to configure ingress to redirect to the different apps based on the hostname in the request it receives?
For example:
www.app1.com and www.app2.com point to the same IP address. However, I want www.app1.com to redirect to /appABC while www.app2.com redirect to /appXYZ.
I have attempted to capture the host name and use this to determine the redirect but it doesn't work.
Is what I'm trying to do possible with NGINX?
Yes,it is Possible. You must need to create two configuration files and point them to their respective paths. Please follow this link for more info and refer to this SO also to get further idea on how to use.
After some experimentation, using the NGINX Playground, I was able to come up with this solution.
...
nginx.ingress.kubernetes.io/server-snippet: |
set $is_app1_base 1;
set $is_app2_base 1;
if ($host !~ "^.*app1\.com$" ) {
set $is_app1_base 0;
}
if ($request_uri != "/") {
set $is_app1_base 0;
set $is_app2_base 0;
}
if ($is_app1_base = 1) {
return 301 $scheme://$host/appABC;
}
if ($host !~ "^.*app2\.com$" ) {
set $is_app2_base 0;
}
if ($is_app2_base = 1) {
return 301 $scheme://$host/appXYZ;
}
In case you're wondering why a number of if statements had to be used this way, NGINX is not that great with if statements and logical operations.
Another caveat worth stating here is that all ingresses associated with this NGINX controller will be affected by this server-snippet; Because nginx.ingress.kubernetes.io/server-snippet is a global annotation.
How can I deny access to nginx if the path contains /local or /local-int to all networks except the local one?
For example https://example.com/api/local/settings. I tried this, but when accessed locally, the request goes to /etc/nginx/html/api/local/settings,and not to the desired backend
location = (local|local-int) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
I have about 20 such sites, and I'm trying to come up with a solution that would not be tied to a specific location
I summarize: if I access a site from allowed ip, then it should show the page to which I am accessing, and if from a deny list, then 403
Config example:
server {
listen ip:80;
listen ip:443 ssl;
server_name test.com;
if_modified_since off;
location /api {
proxy_pass https://api.example.com;
}
location ~ (\/local) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
}
This will simply work with both of your locations, since both starts with /local
location ~ (\/local) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
Nginx takes a = location modifier as an exact match (docs are here). If you want to make a location that will catch every URI containing /local substring (obviously including /local-int), you can use a regex one:
location ~ /local {
...
}
The ^~ modifier makes the location block in #user973254 answer (original answer version, already fixed) a prefix one with the greater priority than any regex locations, so it will overtake only the URIs starting with /local (obviously not including /api/local/settings from your example).
However if your web backend requires an additional URI processing (which is a most common case nowadays), you'll need at least to replicate your main location behavior with this new location. Fortunately, there is a way to avoid such a problems, and can be easily applied to an arbitrary number of sites as you ask for in your original question. You can check required conditions to make a decision for blocking the request or not using the (very powerful) map block feature. And since we want to match address against a list of subnets, we will use a chain of map and geo blocks. To use regexes (PRCE/PCRE2 syntax) for a map block match use a ~ string prefix (~* for case-insensitive match), strings containing some special characters (e.g. curly braces) should be single- or double-qouted. Here is a generic example (you'll need only the first line of the following map block to fulfill your question requirements):
map $uri $restricted {
~/local 1; # regex to match any URI containing '/local' substring
~^/private/ 1; # regex to match any URI starting with '/private'
~*\.jpe?g$ 1; # regex to match any URI ending with '.jpg' or '.jpeg' (case-insensitive)
/some/protected/page/ 1; # exact URI match (string isn't starting with '~')
... any number of additional rules here
default 0;
}
geo $deny {
10.150.0.0/16 0;
10.160.0.0/16 0;
10.170.0.0/16 0;
default $restricted;
}
server {
...
if ($deny) { return 403; }
...
}
You can swap the logic to check the URI first (it can be some performance impact since the regex matching will be performed for every request including requests from the non-restricted networks, however if the majority of requests come from public addresses, there will be no significant difference). That way you can have a common non-restricted subnes list and per-site URI lists:
geo $restricted {
10.150.0.0/16 0;
10.160.0.0/16 0;
10.170.0.0/16 0;
default 1;
}
map $uri $deny1 {
~/local $restricted;
default 0;
}
map $uri $deny2 {
~^/admin $restricted;
default 0;
}
server {
server_name site1.com;
if ($deny1) { return 403; }
...
}
server {
server_name site2.com;
if ($deny2) { return 403; }
...
}
Of course, you are not limited to use 403 return code using this solution (which is the case when you are using allow/deny directives). It also has nothing to do with the famous "If is evil" article since this if is used in server context.
I have an Nginx config file list below. I want to send the request to different server base on Refer.
When I send a request with URL "doamin.com/capi/a/b" and refer "a.com/a/1/test", everything is good, server "be" will get "be/a/b" request.
But if I send a request with URL "doamin.com/capi/a/b" and refer "a.com/a/0/test", server "be_demo" will get "be_demo/" request, the path "a/b" is missing.
I've tried to add "/" at the end of "be_demo", it doesn't work.
map $http_referer $be_pool {
default be;
"~a\.com\/.*\/0\/.*" be_demo;
}
server {
...
location ~ ^/capi/(.*)$ {
proxy_pass http://$be_pool/$1;
}
}
Thanks.
The numeric capture $1 is set by the last regular expression to be evaluated. In the second case, the regular expression in the map statement is evaluated after the regular expression in the location statement.
The solution is to use a named capture instead.
For example:
map $http_referer $be_pool {
default be;
"~a\.com\/.*\/0\/.*" be_demo;
}
server {
...
location ~ ^/capi/(?<myuri>.*)$ {
proxy_pass http://$be_pool/$myuri;
}
}
I'm configuring nginx as reverse proxy.
I need to change (rewrite?) the URLs, example: when the request (to nginx Reverse Proxy) is "http://example.com/test/?username=test1;password=passwdtest1" it will must "modified" to the main server as "http://example.com/test/?username=production;password=passwdproduction1".
Consider that in the original request the fields "username=test1;password=passwdtest1" are not always the same (they changes), instead the "modified" to the main server are always the same.
Others example to be more clear:
"/test/?username=test1;password=passwdtest1" -> "/test/?username=production;password=passwdproduction1"
"/test/?username=test1876;password=somepasswd" -> "/test/?username=production;password=passwdproduction1"
"/test/?username=somevalues;password=somepasswdvalue" -> "/test/?username=production;password=passwdproduction1"
So, independently to what are the values of "?username=somevalues;password=somepasswdvalue" it should always become "?username=production;password=passwdproduction1".
Thanks for your help!
A little late on the answer but this should work for you:
location ~* /test/? {
if ($arg_username ~ "^$|\s+") { return 404; }
if ($arg_password ~ "^$|\s+") { return 404; }
rewrite ^ /test?username=production&password=passwdproduction1? permanent;
}
The code above checks if it is within the example.com/test path. If it is it will check if the user name or the password variable are present and not empty in the query string. In case if any isn't present or is empty it will return a 404 else it will redirect you to the preferred url.
By the way, instead of the semicolon in your example urls I would use an ampersand (&).
If I have the headers: X_HEADER1 & X_HEADER2, I want to reject all requests if either of these headers are not set or do not contain the correct values. What is the best way to do this?
Thanks
You can use two IF statements either before or in the location block to inspect the headers and then return a 403 error code if it is present. Alternatively, you can use those IF statements to rewrite to a specific location block and deny all in that location:
if ($http_x_custom_header) {
return 403;
}
Reference:
https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
https://nginx.org/en/docs/http/ngx_http_access_module.html
Adding more detail per comment/request:
if ($http_x_custom_header) {
return 405;
}
this looks to see if header exists
if you want to check to see if the correct values exist, then you first need to map the correct values to a variable.
map $http_x_header $is_ok {
default "0";
Value1 "1";
Value2 "1";
Value3 "1";
}
if ($is_ok) {
return 405;
}
this first maps the header value to whether or not its ok, then checks to see if the variable is ok.
EDIT: Removed semicolon after map block since this causes an error.
I researched a lot to solve a simple problem: Only allow proxy_pass if request have a specific token in the header. I tried all the answers here and nothing worked how I liked. My final solution is:
location /api {
proxy_http_version 1.1;
if ($http_authorization != "Bearer 1234") {
return 401;
}
proxy_pass http://app:3000/;
}
References:
NGINX not equal to
nginx - read custom header from upstream server
https://serverfault.com/questions/490760/nginx-location-exact-match-matches-beyond-arguement
https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
If you want to allow the HTTP requests only based on some valid header values which are placed inside the response header, one possible way is to use OpenResty tool to apply such restrictions.
The following example allow access to only requests having values "name1" or "name2" for header1:
header_filter_by_lua '
local val = ngx.header["header1"]
if val then
if (val ~= "name1") and (val ~= "name2") then
return ngx.exit(400)
end
end
';