NGINX
Have a locations like:
location ~ "^\/(place1|place2|...|place50)\/(service1|service2|...|service80)\-(else1|else2|...|else90)\/"
{...}
location ~ "^\/(word1|word2|...|word70)\/(place1|place2|...|place50)\-(else1|else2|...|else90)\/"
{...}
location ~ "^\/..."
The problem is there are a lot of places & services & words & else. So locations are awfully long strings. Is it possible to make them shorter? Maybe massive of places and massive of services and so on? Or something? Who has some experience?
examples of URI i want to match
/place23/service17-else87/
or
/world33/place42-else15/
and any combinations
For each location we are going to use set of rules. In order to use cache and unload our Apache
#proxy_cache start
set $do_not_cache 0;
# POST requests and urls with a query string should always go to PHP
if ($request_method = POST) {
set $do_not_cache 1;
}
if ($query_string != "") {
set $do_not_cache 1;
}
# Don't use the cache for logged in users or REBent commenters
if ($http_cookie ~* "wordpress_logged_in|bn_my_logged") {
set $do_not_cache 1;
}
if ($args ~* (show) ) {
set $do_not_cache 1;
}
ssi_types "*";
ssi on;
if ($do_not_cache = 0) {
set $memcached_key "SMREG|$request_uri";
memcached_pass memc_server;
ssi on;
}
You can use regular expressions in location blocks to make them match multiple things, rather than listing them all by hand.
e.g.
location ~* ^/world(\d{1,2})/place(\d{1,2})-else(\d{1,2})/ {
set $originalURI $uri;
fastcgi_param QUERY_STRING q=$originalURI&world=$1&place=$2&else=$3;
# or however you're passing it to your web server.
}
However as Mohammad AbuShady implied, what you're doing seems quite dumb.
You should be letting your application do the routing, and deciding whether things can be cached or not, and then using something that is actually designed to cache web pages e.g. Varnish, rather than trying to force application logic into Nginx.
Related
How can I deny access to nginx if the path contains /local or /local-int to all networks except the local one?
For example https://example.com/api/local/settings. I tried this, but when accessed locally, the request goes to /etc/nginx/html/api/local/settings,and not to the desired backend
location = (local|local-int) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
I have about 20 such sites, and I'm trying to come up with a solution that would not be tied to a specific location
I summarize: if I access a site from allowed ip, then it should show the page to which I am accessing, and if from a deny list, then 403
Config example:
server {
listen ip:80;
listen ip:443 ssl;
server_name test.com;
if_modified_since off;
location /api {
proxy_pass https://api.example.com;
}
location ~ (\/local) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
}
This will simply work with both of your locations, since both starts with /local
location ~ (\/local) {
allow 10.150.0.0/16;
allow 10.160.0.0/16;
allow 10.170.0.0/16;
deny all;
}
Nginx takes a = location modifier as an exact match (docs are here). If you want to make a location that will catch every URI containing /local substring (obviously including /local-int), you can use a regex one:
location ~ /local {
...
}
The ^~ modifier makes the location block in #user973254 answer (original answer version, already fixed) a prefix one with the greater priority than any regex locations, so it will overtake only the URIs starting with /local (obviously not including /api/local/settings from your example).
However if your web backend requires an additional URI processing (which is a most common case nowadays), you'll need at least to replicate your main location behavior with this new location. Fortunately, there is a way to avoid such a problems, and can be easily applied to an arbitrary number of sites as you ask for in your original question. You can check required conditions to make a decision for blocking the request or not using the (very powerful) map block feature. And since we want to match address against a list of subnets, we will use a chain of map and geo blocks. To use regexes (PRCE/PCRE2 syntax) for a map block match use a ~ string prefix (~* for case-insensitive match), strings containing some special characters (e.g. curly braces) should be single- or double-qouted. Here is a generic example (you'll need only the first line of the following map block to fulfill your question requirements):
map $uri $restricted {
~/local 1; # regex to match any URI containing '/local' substring
~^/private/ 1; # regex to match any URI starting with '/private'
~*\.jpe?g$ 1; # regex to match any URI ending with '.jpg' or '.jpeg' (case-insensitive)
/some/protected/page/ 1; # exact URI match (string isn't starting with '~')
... any number of additional rules here
default 0;
}
geo $deny {
10.150.0.0/16 0;
10.160.0.0/16 0;
10.170.0.0/16 0;
default $restricted;
}
server {
...
if ($deny) { return 403; }
...
}
You can swap the logic to check the URI first (it can be some performance impact since the regex matching will be performed for every request including requests from the non-restricted networks, however if the majority of requests come from public addresses, there will be no significant difference). That way you can have a common non-restricted subnes list and per-site URI lists:
geo $restricted {
10.150.0.0/16 0;
10.160.0.0/16 0;
10.170.0.0/16 0;
default 1;
}
map $uri $deny1 {
~/local $restricted;
default 0;
}
map $uri $deny2 {
~^/admin $restricted;
default 0;
}
server {
server_name site1.com;
if ($deny1) { return 403; }
...
}
server {
server_name site2.com;
if ($deny2) { return 403; }
...
}
Of course, you are not limited to use 403 return code using this solution (which is the case when you are using allow/deny directives). It also has nothing to do with the famous "If is evil" article since this if is used in server context.
I am trying to resolve proxy_pass value dynamically (through web api) in nginx.
I need something like below;
Example taken from: https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
location /proxy-pass-uri {
set $urlToProxy = CallWebAPI("http://localhost:8081/resolver?url=" + $url);
proxy_pass $urlToProxy;
}
So, my question is that, is it possible to make HTTP request or to write method such as CallWebAPI?
I know it might be a bad practice, but the website I am dealing with has thousands of web urls, which are mapped as key-value pairs, and 90% of them does not obey any specific regex rules. So I have content mapped database, and I need to fetch incoming url with content dynamically.
I am trying to use a very light web service to look up URLs from redis, and return proxy url.
Would this be a valid scenario, or is there any other built in solution in nginx like this?
I doubt this can be done with "pure" nginx, but this definitely can be done with openresty or ngx_http_lua_module with the help of ngx.location.capture method. For example:
resolver 8.8.8.8;
location ~/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("http://localhost:8081/resolver?url=" .. ngx.var.url)
ngx.var.proxy = res.body
}
proxy_pass $proxy$url;
}
There is also an ngx_http_js_module (documentation, GitHub) which have an ability to do subrequests (example), but I never used it and cannot tell if it can be used this way.
Important update
After almost a three years since this answer was written, it comes that I needed the similar functionality myself, and it turns out that the above answer is completely broken and unworkable. You can't do a subrequest via ngx.location.capture to anything else but to some other nginx location. So the correct (checked and confirmed to be workable) example for the above question is
resolver 8.8.8.8;
location /resolver {
internal;
proxy_pass http://localhost:8081;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/resolver?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
The above example assumes that the proxy resolution service is really expecting request in a /resolver?url=<uri> form. The location /resolver { ... } while being internal behaves like any other prefix location, so if the /resolver prefix for that location cannot be used for some reason, the same can be written as
resolver 8.8.8.8;
location /get_proxy {
internal;
proxy_pass http://localhost:8081/resolver;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/get_proxy?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
I'm trying to deploy two different versions of same app on the same nginx-based server. If the URL starts with /v2, then "v2" should be used, otherwise use v1. Example:
http://example.com/v2/x/y/z * runs v2 app
http://example.com/anything/else * runs v1 app
The two different versions of the app are proxied through nginx, and that piece works well.
The issue is that I have two directories of static assets, /static and /cachedassets, that are common to both versions (and both rooted from /home/v1|2/www/public. So, even though a request to http://example.com/v2/x/y/z will initially use the right app, the page that loads will contain references to /static and /cachedassets, without the /v2 prefix, that will incorrectly load from /home/v1/www/public.
I know referer is an imperfect solution. As a temporary stopgap, until I have a chance to craft a more robust solution, I'm trying to use nginx's $http_referer to point to the correct location for these assets. Here's the nginx file:
server {
listen 1.2.3.4
server_name example.com
...
location /v2 {
root /home/v2/www/public;
try_files $uri #proxyv2;
access_log off;
expires max;
}
location ^/(static|cachedassets) {
root /home/v1/www/public;
if ($http_referer ~* "/v2/") {
root /home/v2/www/public;
}
}
location / {
root /home/v1/www/public;
try_files $uri #proxyv1;
access_log off;
expires max;
}
location #proxyv1 {
include uwsgi_params;
uwsgi_pass unix:///tmp/v1-www.sock;
uwsgi_modifier1 5;
}
location #proxyv2 {
include uwsgi_params;
uwsgi_pass unix:///tmp/v2-www.sock;
uwsgi_modifier1 5;
}
...
}
Any thoughts?
Bonus points for a solution where I can easily specify several "v2" prefixes. For example, here I might specify v2, versiontwo, and vtwo, and the following URLs would all invoke the v2 app:
http://www.example.com/v2/something
http://www.example.com/versiontwo/abc
http://www.example.com/vtwo/abc/def/ghi
and of course, http://www.example.com/somethingelse would run v1.
I'm also open to other ideas that don't use http_referer to accomplish this.
Thanks!
Rather than if blocks, use a map variable to set the root. The map can contain a number of arbitrarily complex regular expressions. See this document for more.
For example:
map $http_referer $root {
default "/home/v1/www/public";
~*/v2/ "/home/v2/www/public";
}
server {
...
location ~ ^/(static|cachedassets) {
root $root;
}
...
}
I'm trying to log only java-script files request in the nginx access_log.
I tried using the following code i found on this site:
location ~* ^.+.(jpg|jpeg|gif|css|png|html|htm|ico|xml|svg)$ {
access_log off;
}
the problem is it doesn't allow the get request at all and i get a 404 error when trying to run the html file that executes the js file in the browse.
I want everything to work just the same but for the access log to log only request for js files.
How do i do that?
Put it in the server block and make sure that the "root" is correctly set up. It does work
Working example:
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires +60d;
access_log off;
}
I have this in the server block and not a location block.
Alternatively you can keep all requests within single location but use access_log with condidional if operator to disable images logging:
map $request_uri $is_loggable {
~* ^.+\.(jpg|jpeg|gif|css|png|html|htm|ico|xml|svg)$ 0;
default 1;
}
server {
location / {
access_log /path/to/log/file combined if=$is_loggable;
...
}
}
Here combined is a name of default log format.
You say that you want to log only java-script files, so actually you can use even simplier solution:
map $request_uri $is_loggable {
~* ^.+\.js$ 1;
default 0;
}
I currently have the following (hacky) re-write rule in my nginx.conf to allow dynamic sub-domains to be re-directed to one Django instance.
set $subdomain "";
set $subdomain_root "";
set $doit "";
if ($host ~* "^(.+)\.domain\.com$") {
set $subdomain $1;
set $subdomain_root "/profile/$subdomain";
set $doit TR;
}
if (!-f $request_filename) {
set $doit "${doit}UE";
}
if ($doit = TRUE) {
rewrite ^(.*)$ $subdomain_root$1;
break;
}
I'm sure there is a more efficient way to do this but I need to change this rule so that any requests to *.domain.com/media/* or *.domain.com/downloads/* go to domain.com/media/* and domain.com/downloads/*.
You can use regular expression server names (see http://nginx.org/en/docs/http/server_names.html#regex_names) and assign a matching group to a variable $subdomain directly:
server {
listen 80;
listen 443;
server_name ~^(?<subdomain>.+)\.domain\.com$
location / {
rewrite ^ /profile/$subdomain$request_uri;
}
}
Actually I think it is much easier to change the nginx re-write rules than to write middleware for django to do this. After reading up on how nginx processes it's location matching (most exact -> least exact) I created locations for /media and /download as well as a catch all location for / I then moved the rewrite rule to under the / location and simplified it - as I'm no longer worried about checking for files because this entire location is passed to django - the rule becomes :
set $subdomain "";
set $subdomain_root "";
if ($host ~* "^(.+)\.domain\.com$") {
set $subdomain $1;
set $subdomain_root "/profile/$subdomain";
rewrite ^(.*)$ $subdomain_root$1;
break;
}
and would probably be even simpler if my nginx\regex scripting was better :)
Perhaps a better idea would be to configure django to handle subdomains instead of adding a rewrite in your webserver. Here's how I did it: http://sharjeel.2scomplement.com/2008/07/24/django-subdomains/