In the following example:
http {
server { # simple reverse-proxy
listen 8080;
location / {
set $token $arg_token;
#return 200 $token;
add_header test "test $token";
proxy_pass http://localhost:5601;
}
} ...
}
if I leave return 200 $token I obtain the token as response + in header (which is a normal behavior) but when I delete return I obtain only "test" as test header value, what am I missing please ?
The proxy_set_header sets header that NGINX will use while communicating to the upstream/backend.
You won't see that added header in the response of NGINX back to the client.
If you want to see it, use add_header as well.
Related
I want to make sure that at least the default header value is always returned from upstream. Even if upstream is unavailable, which causes error 5xx.
Now I have tried these nginx config options:
server {
...
#add_header "Access-Control-Allow-Origin" "*"; №0
#add_header "Access-Control-Allow-Origin" "*" always; №1
#more_set_headers "Access-Control-Allow-Origin: *"; №2
#more_set_headers -s '403 500 502 504 503' "Access-Control-Allow-Origin: *"; №3
location /upstream {
proxy_pass http://localhost:1234;
}
...
}
There are problems with all the options:
№0: Duplicates the header, and in the case of 5xx will not return any.
№1: Duplicates the header
№2: Overrides the upstream header
№3: If the upstream ended with a good http code, but did not return a header, it will not add a header.
I think I'm close to the right solution, but I can't find it.
The map below uses a regex, /.+/, to check if the Access-Control-Allow-Origin header is defined. If so, it assigns its value to the $acao custom variable. Otherwise, it assigns the default value * to $acao;
To avoid duplications, use proxy_hide_header
Finally, add the header using the $acao variable content.
http {
map $upstream_http_access_control_allow_origin $acao {
~.+ $upstream_http_access_control_allow_origin;
default '*';
}
server {
#…
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin $acao always;
location /upstream {
proxy_pass http://localhost:1234;
}
}
How to forward the URL's all parameters through a proxy_pass with nginx?
Nginx config:
location /proxy/ {
if ($request_method = HEAD) { return 200; }
if ( $arg_address != "" ) {
proxy_pass $arg_address;
return 301 $arg_address;
}
proxy_ssl_verify off;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
this urls works:
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt ==>> https://exemple.com/transfer/file.txt
or
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1 ==>> https://exemple.com/transfer/file.tx?host-id=1
if I add multiple parameters, it will be truncated to the first "&"
https://localhost/proxy/?address=https://exemple.com/transfer/file.txt?host-id=1&password=123456&date=xxxxxx
==>> https://exemple.com/transfer/file.txt?host-id=1
How can I transfer the entire url?
Before we get to the answer, may I ask what do you want to achieve? Do you want to proxy the request or to generate HTTP 301 redirect? With the following construction
if ( $arg_address != "" ) {
proxy_pass $arg_address;
return 301 $arg_address;
}
you'll always get a redirect because the directives from ngx_http_rewrite_module are executed before any others, so the proxy_pass directive is useless here. ngx_http_rewrite_module is very special and different from most of the other modules. Although nginx configuration in general is declarative, rewrite module evaluates its instructions imperatively. This is always a source of confusion for every nginx novice. You can read more about the rewrite module internal implementation here.
If you want to proxy the request instead of generating a redirect, you'll need to remove return and add a resolver directive to your configuration. Here you can read why it is required.
A "dirty hack" solution
Being that said, get back to the question. Of course, when nginx receives the request
https://localhost/proxy/?address=https://example.com/transfer/file.txt?host-id=1&password=123456&date=20210520
arg_NAME variables will be filled the following way:
arg_address => https://example.com/transfer/file.txt?host-id=1
arg_password => 123456
arg_date => 20210520
It is correct and expected behavior.
What you can do to preserve all the other query arguments? The most simple is to assume that all query arguments following the address one are subject to pass to the upstream and use a map directive to get the required string:
map $args $address {
~(?:^|&)(address=.*) $1;
}
server {
...
location /proxy/ {
...
if ($address) {
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
...
}
Here is more strict check where we take the rest of the query string only if there is a question mark after address query parameter and only $arg_address value otherwise:
map $args $address {
~(?:^|&)(address=[^&?]+\?.*) $1;
default $arg_address;
}
Reliable solution
While the answer above is generally workable, I'd rather try to design my proxy solution using URL encoding on address query argument to avoid reserved characters usage as part of the query argument value. The above request being URL-encoded would look like
https://localhost/proxy/?address=https%3A%2F%2Fexample.com%2Ftransfer%2Ffile.txt%3Fhost-id%3D1%26password%3D123456%26date%3D20210520
The bad thing is that "vanilla" nginx doesn't have an ability to URL-decode an arbitrary string. However it can be done using OpenResty/lua-nginx-module:
location /proxy/ {
...
set_by_lua_block $address { return ngx.unescape_uri(ngx.var.arg_address) }
if ($address) {
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
or set-misc-nginx-module:
location /proxy/ {
...
if ($arg_address) {
set_unescape_uri $address $arg_address;
# 'proxy_pass $address' or 'return 301 $address' here
}
...
}
Perhaps the same can be done using njs, but I didn't use it and can't give you an example.
I have a requirement to do a proxy call to url delivered via a query parameter as per example:
My nginx proxy is deployed at: https://myproxy.net
if the redirect parameter is not url encoded I can do the call with this block:
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass $arg_redirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
the error intercepts and #handle_redirects then take care of othe 30X codes that might pop up at new destination.
This works for a request:
GET: https://myproxy.net/basepath?redirect=https://destination.com/somepath/uuid
What do I need to do to make it work for:
GET: https://myproxy.net/basepath?redirect=https%3A%2F%2Fdestination.com%2Fsomepath%2Fuuid
Additionally as part of spec it has to be pure nginx, not additional modules, lua etc.
Thanks!
Actually, proxy_pass does normalisation by default, but it only affects $uri part. Thus you only need to decode the beginning of the passed string to get it working:
location / {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
if ( $arg_redirect ~ (.+)%3A%2F%2F(.+) ){ # fix :// between scheme and destination
set $arg_redirect $1://$2;
}
if ( $arg_redirect ~ (.+?)%3A(.*) ){ # fix : between destination and port
set $arg_redirect $1:$2;
}
if ( $arg_redirect ~ (.+?)%2F(.*) ){ # fix / after port, the rest will be decoded by proxy_pass
set $arg_redirect $1/$2;
}
proxy_pass $arg_redirect;
}
With the above I managed to access http://localhost/?redirect=http%3A%2F%2F127.0.0.1%3A81%2Fsfoo%20something%2Fs
The solution seems dirty and the only alternative using default modules is map (even less cleaner in my opinion). I'd rather split redirect argument into pieces: scheme (http or https), destination, port, and uri. With that you would be able to construct full address without rewriting:
proxy_pass $arg_scheme://$arg_dest:$arg_port/$arg_uri
Ok, there is very weird and curious solution
server {
listen 80;
resolver x.x.x.x;
location /basepath {
if ($arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass http://127.0.0.1:80/basepath/$arg_redirect;
}
location ~ ^/basepath/(?<proto>\w+):/(?<redir>.+)$ {
proxy_pass $proto://$redir;
}
}
Nginx does not encode path with variables in proxy_pass and send it as is. So, I make $arg_* part of proxy_pass uri, send request to self and nginx will receive new request which will be decoded.
But because Nginx will clean path and replace // to / I split protocol part in regexp.
And ... I would never recommend using this solution, but it works :)
try like this and let me know if it works
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
set_unescape_uri $decodedredirect $arg_redirect;
proxy_pass $decodedredirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
I need to send some basic auth credentials (es. user:pass) to nginx in the form of query parameter (es. http://example.com?BASIC_AUTH=dXNlcjpwYXNz) and being able to forward them in the more usual Authorization: Basic dXNlcjpwYXNz header form to a target server behind the proxy.
I'm already able to retrieve the value of the encoded auth string with a regular expression. The problem is that very often that value may contain some character that need to be percent-encoded in the URL. Es. user:pass! -> ?BASIC_AUTH=dXNlcjpwYXNzIQ== becomes ?BASIC_AUTH=dXNlcjpwYXNzIQ%3D%3D
Therefore, when I forward the request to the target server, I end up specifing Authorization: Basic dXNlcjpwYXNzIQ%3D%3D which the target server will reject, giving a 401 Unauthorized.
How can I force nginx to decode the auth string before setting the Authorization header? Thanks in advance for your help.
Note: I can't send the auth string in the Authorization header in the first place due to some application-specific constraints.
"Pure" nginx solution
Unfortunately nginx does not provide a rich string operations set. I think there isn't a way to do global search-and-replace through some string (which can be a solution if we could replace all %2B with +, %2F with / and %3D with =). However there are circumstances under which nginx performs an urldecoding of some string - when this string becomes a part of an URI which will be forwarded to an upstream proxy server.
So we can add a value of a BASIC_AUTH request argument to the URI and make a proxy request to ourself:
# Main server block
server {
listen 80 default_server;
...
location / {
if ($arg_basic_auth) {
# "basic_auth" request argument is present,
# append "/decode_basic_auth/<BASE64_token>" to the URI
# and go to the next location block
rewrite ^(.*)$ /decode_basic_auth/$arg_basic_auth$1 last;
}
# No "basic_auth" request argument present,
# can do a proxy call from here without setting authorization headers
...
}
location /decode_basic_auth/ {
# This will be an internal location only
internal;
# Remove "basic_auth" request argument from the list of arguments
if ($args ~* (.*)(^|&)basic_auth=[^&]*(\2|$)&?(.*)) {
set $args $1$3$4;
}
# Some hostname for processing proxy subrequests
proxy_set_header Host internal.basic.auth.localhost;
# Do a subrequest to ourselfs, preserving other request arguments
proxy_pass http://127.0.0.1$uri$is_args$args;
}
}
# Additional server block for proxy subrequests processing
server {
listen 80;
server_name internal.basic.auth.localhost;
# Got URI in form "/decode_basic_auth/<BASE64_token>/<Original_URI>"
location ~ ^/decode_basic_auth/([^/]+)(/.*)$ {
proxy_set_header Authorization "Basic $1";
# Setup other HTTP headers here
...
proxy_pass http://<upstream_server>$2$is_args$args;
}
# Do not serve other requests
location / {
return 444;
}
}
Maybe this is not a very elegant solution, but it is tested and works.
OpenResty / ngx_http_lua_module
This can be easily solved with openresty or ngx_http_lua_module using ngx.escape_uri function:
server {
listen 80 default_server;
...
location / {
set $auth $arg_basic_auth;
if ($args ~* (.*)(^|&)basic_auth=[^&]*(\2|$)&?(.*)) {
set $args $1$3$4;
}
rewrite_by_lua_block {
ngx.var.auth = ngx.unescape_uri(ngx.var.auth)
}
proxy_set_header Authorization "Basic $auth";
# Setup other HTTP headers here
...
proxy_pass http://<upstream_server>;
}
}
I have a nginx proxy to a API server. The API sometimes sets the cache control header. If the API hasnt set the cache control I want nginx to override it.
How do I do that?
I think I want to do something like this, but it doesnt work.
location /api {
if ($sent_http_cache_control !~* "max-age=90") {
add_header Cache-Control no-store;
add_header Cache-Control no-cache;
add_header Cache-Control private;
}
proxy_pass $apiPath;
}
You cannot use if here, because if, being a part of the rewrite module, is evaluated at a very early stage of the request processing, way before proxy_pass is called and the header is returned from the upstream server.
One way to solve your problem is to use map directive. Variables defined with map are evaluated only when they are used, which is exactly what you need here. Sketchily, your configuration in this case would look like this:
# When the $custom_cache_control variable is being addressed
# look up the value of the Cache-Control header held in
# the $upstream_http_cache_control variable
map $upstream_http_cache_control $custom_cache_control {
# Set the $custom_cache_control variable with the original
# response header from the upstream server if it consists
# of at least one character (. is a regular expression)
"~." $upstream_http_cache_control;
# Otherwise set it with this value
default "no-store, no-cache, private";
}
server {
...
location /api {
proxy_pass $apiPath;
# Prevent sending the original response header to the client
# in order to avoid unnecessary duplication
proxy_hide_header Cache-Control;
# Evaluate and send the right header
add_header Cache-Control $custom_cache_control;
}
...
}
Awswer from Ivan Tsirulev is correct but you don't have to use regex.
Nginx uses the first parameter of map as default value automatically so you don't have to add that either.
# Get value from Http-Cache-Control header but override it when it's empty
map $upstream_http_cache_control $custom_cache_control {
'' "no-store, no-cache, private";
}
server {
...
location /api {
# Use the value from map
add_header Cache-Control $custom_cache_control;
}
...
}