Nginx proxy pass and url rewriting - nginx

How to trig this rule only when I have GET parameters(query string) in url,
otherwise I will match on an alias.
location ~^/static/photos/.* {
rewrite ^/static/photos/(.*)$ /DynamicPhotoQualitySwitch/photos/$1 break;
expires 7d;
proxy_pass http://foofoofoo.com;
include /etc/nginx/proxy.conf;
}

The 1st way that I know of is using a regex against the $args parameter like so:
if ($args ~ "^(\w+)=") {
Or the 2nd way is to use the convenient $is_args like so:
if ($is_args != "") {
Remember that in both styles you need to put a space between the if and the opening parenthesis; "if (" not "if(" as well as a space after the closing parenthesis and the opening brace; ") {" rather than "){".
Full example using the 1st style above, nginx.conf:
location ~^/static/photos/.* {
include /etc/nginx/proxy.conf;
if ($args ~ "^(\w+)=") {
rewrite ^/static/photos/(.*)$ /DynamicPhotoQualitySwitch/photos/$1 break;
expires 7d;
proxy_pass http://foofoofoo.com;
}
}
Full example using the 2nd style above, nginx.conf:
location ~^/static/photos/.* {
include /etc/nginx/proxy.conf;
if ($is_args != "") {
rewrite ^/static/photos/(.*)$ /DynamicPhotoQualitySwitch/photos/$1 break;
expires 7d;
proxy_pass http://foofoofoo.com;
}
}
Note that the proxy.conf include goes outside of the if statement.
Version:
[nginx#hip1 ~]$ nginx -v
nginx version: nginx/1.2.6
And some info on the $args and $is_args variables:
http://nginx.org/en/docs/http/ngx_http_core_module.html
Reading the docs is always useful, I just discovered that $query_string is the same as $args, so where I have $args above, you could also use $query_string according to the docs.
IMPORTANT
It is important to note however, that If can be Evil!
And therefore either test thoroughly or use the recommendation provided in the link above to change the URL inside location statement in a way similar to the example provided there, something like:
location ~^/static/photos/.* {
error_page 418 = #dynamicphotos;
recursive_error_pages on;
if ($is_args != "") {
return 418;
}
# Your default, if no query parameters exist:
...
}
location #dynamicphotos {
# If query parameters are present:
rewrite ^/static/photos/(.*)$ /DynamicPhotoQualitySwitch/photos/$1 break;
expires 7d;
include /etc/nginx/proxy.conf;
proxy_pass http://foofoofoo.com;
}

Related

How to forward the URL's all parameters through a proxy_pass with nginx?

How to forward the URL's all parameters through a proxy_pass with nginx?
Nginx config:
location /proxy/ {
if ($request_method = HEAD) { return 200; }
if ( $arg_address != "" ) {
proxy_pass $arg_address;
return 301 $arg_address;
}
proxy_ssl_verify off;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
this urls works:
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt ==>> https://exemple.com/transfer/file.txt
or
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1 ==>> https://exemple.com/transfer/file.tx?host-id=1
if I add multiple parameters, it will be truncated to the first "&"
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1&password=123456&date=xxxxxx
==>> https://exemple.com/transfer/file.txt?host-id=1
How can I transfer the entire url?
Before we get to the answer, may I ask what do you want to achieve? Do you want to proxy the request or to generate HTTP 301 redirect? With the following construction
if ( $arg_address != "" ) {
proxy_pass $arg_address;
return 301 $arg_address;
}
you'll always get a redirect because the directives from ngx_http_rewrite_module are executed before any others, so the proxy_pass directive is useless here. ngx_http_rewrite_module is very special and different from most of the other modules. Although nginx configuration in general is declarative, rewrite module evaluates its instructions imperatively. This is always a source of confusion for every nginx novice. You can read more about the rewrite module internal implementation here.
If you want to proxy the request instead of generating a redirect, you'll need to remove return and add a resolver directive to your configuration. Here you can read why it is required.
A "dirty hack" solution
Being that said, get back to the question. Of course, when nginx receives the request
https://localhost/proxy/?address=https://example.com/transfer/file.txt?host-id=1&password=123456&date=20210520
arg_NAME variables will be filled the following way:
arg_address => https://example.com/transfer/file.txt?host-id=1
arg_password => 123456
arg_date => 20210520
It is correct and expected behavior.
What you can do to preserve all the other query arguments? The most simple is to assume that all query arguments following the address one are subject to pass to the upstream and use a map directive to get the required string:
map $args $address {
~(?:^|&)(address=.*) $1;
}
server {
...
location /proxy/ {
...
if ($address) {
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
...
}
Here is more strict check where we take the rest of the query string only if there is a question mark after address query parameter and only $arg_address value otherwise:
map $args $address {
~(?:^|&)(address=[^&?]+\?.*) $1;
default $arg_address;
}
Reliable solution
While the answer above is generally workable, I'd rather try to design my proxy solution using URL encoding on address query argument to avoid reserved characters usage as part of the query argument value. The above request being URL-encoded would look like
https://localhost/proxy/?address=https%3A%2F%2Fexample.com%2Ftransfer%2Ffile.txt%3Fhost-id%3D1%26password%3D123456%26date%3D20210520
The bad thing is that "vanilla" nginx doesn't have an ability to URL-decode an arbitrary string. However it can be done using OpenResty/lua-nginx-module:
location /proxy/ {
...
set_by_lua_block $address { return ngx.unescape_uri(ngx.var.arg_address) }
if ($address) {
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
or set-misc-nginx-module:
location /proxy/ {
...
if ($arg_address) {
set_unescape_uri $address $arg_address;
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
Perhaps the same can be done using njs, but I didn't use it and can't give you an example.

NGINX rewrite args without an IF?

I would like to rewrite legacy links using a query parameter type of URL to a new style of URL.
Ex.
example.com/page?id=1 -> example.com/page/1
example.com/otherpage?id=1 -> example.com/otherpage/1
Currently I have the following configuration using the evil if.
if ($args ~* "id=(.*)") {
set $w1 $1;
rewrite .* $scheme://$host/page/$w1? permanent;
}
Note: I am using CloudFront, and relying on the host header above.
If the above is in a server block, with no other location block - would this qualify as a non-evil use of if in NGINX config? Also, the above only supported /page/. Any better ideas for making that portion work for otherpage and other pages?
I have seen a few other ideas discussing using a map, but I'm not quite sure how to bring it all together? I was thinking something along the lines of:
map $args_id ?? {
default ?
??
}
...
server {
...
???
}
UPDATE:
Based on the Answer from #Ivan, this was my final solution:
server {
listen 80;
root /usr/share/nginx/html;
index index.html index.htm;
# Handle legacy requests
if ($args ~* "id=(.*)") {
set $w1 $1;
rewrite ^ $scheme://$host$uri/$w1? permanent;
}
}
Your if construction isn't evil. You can use something like
rewrite ^ $scheme://$host$uri/$w1? permanent;
for any page. More complex example if you want to process both example.com/page?id=1 and example.com/page/?id=1:
map $uri $maybe_slash {
~/$ "";
default "/";
}
...
server {
...
rewrite ^ $scheme://$host$uri$maybe_slash$w1? permanent;
...
}

convert url get parameters to url segment in nginx

I have this URL https://example.com/user?param1=value1&param2=value2&param3=value3
and have it to go to https://example.com/user/value1/value2/value3 on the Nginx server.
Just FYI it is WordPress site and I have added the following rule in the Nginx config file.
location ~ /user/ {
if ($args ~* "^param1=(\d+)&param3=(\d+)&param3=(\d+)") {
set $param1 $1;
set $param2 $1;
set $param3 $1;
set $args '';
rewrite ^.*$ /user/$param1/$param2/$param3 permanent;
}
}
Your solution has two errors, 1) the location does not match /user, and 2) the rewrite is also appending the original arguments.
This could be fixed by using an exact match location and a trailing ? on the rewrite. For example:
location = /user {
...
rewrite ^ /user/$param1/$param2/$param3? permanent;
}
However, the map statement is a cleaner and extensible solution, for example:
map $request_uri $redirect {
default 0;
~*^/user?param1=(?<p1>\d+)&param2=(?<p2>\d+)&param3=(?<p3>\d+)$ /user/$p1/$p2/$p3;
}
server {
...
if ($redirect) { return 301 $redirect; }
...
}
See this document for details.

nginx proxy_pass with new URI does not work if host is varaible

I am trying to use variable to set the hostname in a proxy_pass, but once I try that, the path after the location is ignored.
If I try and get localhost:8001/dirA/x/y/z.html. The following returns the file from http://server1:8888/dirB/dirC/x/y/z.html. This is what I expect to happen.
location ^~ /dirA/ {
proxy_pass http://server1:8888/dirB/dirC/;
But if I try the following config which is just using a variable for hostname... and try and get localhost:8001/dirA/x/y/z.html
location ^~ /dirA/ {
set $endpoint server1;
proxy_pass http://$endpoint:8888/dirB/dirC/;
I get http://server1:8888/dirB/dirC/index.html returned instead.
That's just how proxy_pass works. If you use a variable in the value, you need to provide the entire URI. See this document for details;
You could use a regular expression location. For example:
location ~ ^/dirA/(.*)$ {
set $endpoint server1;
proxy_pass http://$endpoint:8888/dirB/dirC/$1;
}
Note that the order of regular expression locations is significant. See this document for details.
Alternatively, a rewrite...break should also work.
location ^~ /dirA/ {
set $endpoint server1;
rewrite ^/dirA/(.*)$ /dirB/dirC/$1 break;
proxy_pass http://$endpoint:8888;
}

Rewrite query string to path params

I have the following configuration of nginx that hosts my image service:
upstream thumbor {
server localhost:8888;
}
server {
listen 80;
server_name my.imageserver.com;
client_max_body_size 10M;
rewrite_log on;
location ~ /images {
if ($arg_width="10"){
rewrite ^/images(/.*)$ /unsafe/$1 last;
}
rewrite ^/images(/.*)$ /unsafe/$1 last;
}
location ~ /unsafe {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header HOST $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://thumbor;
proxy_redirect off;
}
location = /favicon.ico {
return 204;
access_log off;
log_not_found off;
}
}
I am trying to rewrite the following urls:
from
http://my.imageserver.com/images/Jenesis/EmbeddedImage/image/jpeg/jpeg/9f5d124d-068d-43a4-92c0-1c044584c54a.jpeg
to
http://my.imageserver.com/unsafe/Jenesis/EmbeddedImage/image/jpeg/jpeg/9f5d124d-068d-43a4-92c0-1c044584c54a.jpeg
which is quite easy, the problem begins when I want to allow query string that should go to the path of the url like so:
from
http://my.imageserver.com/images/Jenesis/EmbeddedImage/image/jpeg/jpeg/9f5d124d-068d-43a4-92c0-1c044584c54a.jpeg?width=150&height=200&mode=smart
to
http://my.imageserver.com/unsafe/150x200/smart/Jenesis/EmbeddedImage/image/jpeg/jpeg/9f5d124d-068d-43a4-92c0-1c044584c54a.jpeg
Also it will be better if the order of the query strings won't matter.
I tried using:
$arg_width but it didn't seem to work.
Using nginx 1.6.1 on ubuntu.
Help would be much much appreciated.
Working with query arguments is always hard (this is also true with Apache).
But in your example when you do:
location ~ /images {
if ($arg_width="10"){
rewrite ^/images(/.*)$ /unsafe/$1 last;
}
rewrite ^/images(/.*)$ /unsafe/$1 last;
}
I do not see any difference between the 2 rewrites... so that's maybe why it is not working.
Anyway you could maybe try something like that (based on this thread):
location ^~ /images {
# get there only if we have a query string
if ($is_args) {
set $width "";
set $height "";
if ($args ~* "(?:^|&)width=([^&]+)") {
set $width $1;
}
if ($args ~* "(?:^|&)height=([^&]+)") {
set $height $1;
}
# string concatenation using sort of bash syntax
set $dim "${width}x${height}";
# maybe we should add a control here on $dim !='x'...
# the ? here prevent query string from being appended
rewrite ^/images(/.*)$ /unsafe/$dim/$1? last;
}
rewrite ^/images(/.*)$ /unsafe/$1 last;
}
location ~ /unsafe {
(...)
You can use the arg_name parameter, and you can get rid of location block:
rewrite ^/images/(.*)$ /unsafe/$1;
# WARNING: Here we suppose that when we have a "mode" parameter, we have
# width and height paremeters too
if ($arg_mode) {
rewrite ^/unsafe/(.*)$ /images/unsafe/smart/$arg_mode/${arg_width}x${arg_height}/$1 last;
}
You need as scripting option that can evaluate the input and construct the rewrite URL. I use the third party Nginx Lua Module for such logic.
You will need to compile the module into Nginx after installing Lua or preferably, LuaJit on your machine.
Alternatively, you can install Openresty which is Nginx bundled with a few modules that completely transforms the Nginx into a full web application server. Openresty will take care of Lua/LuaJit dependencies during installation.
If you have this in place, this should do the job for you:
upstream thumbor {
server localhost:8888;
}
server {
[...]
location ~ ^/images {
rewrite_by_lua '
-- Use local variables to limit their scope to this location block
local i, j, key, val, args, dimension, modetype, tempheight, replacement
-- We only go ahead if there are request arguments to start with
if ngx.var.is_args then
-- Get the request arguments
-- These are loaded into a Lua table of the form, { a = 1, b = 2, c = 3 }
args = ngx.req.get_uri_args()
-- Loop through the args table and build our replacement string
for key, val in pairs(args) do
if key == "width" then
if tempheight then
dimension = val .. "x" .. tempheight
else
dimension = val
end
elseif key == "height" then
if dimension then
dimension = dimension .. "x" .. val
else
tempheight = val
end
elseif key == "mode" then
modetype = val
end
end
-- Construct the replacement string.
replacement = "/unsafe/" .. dimension .. "/" .. modetype .. "/"
-- Replace "/images/" in the request url with the replacement string.
local newuri, n, err = ngx.re.sub(ngx.unescape_uri(ngx.var.request_uri), "/images/", replacement, "io")
-- If there is a new string, then redirect to the new URL
if newuri then
return ngx.redirect(newuri, 301)
end
end
';
}
location ~ /unsafe {
[...]
proxy_pass http://thumbor;
}
[...]
}

Resources