I am trying to serve static content from S3. I am using nginx and lua script for its configuration. I am new to both nginx and lua.
The task that I want to achieve is :
Get request URL into nginx.
Authenticate query params of url.
Serve from S3 if parameters are valid.
Send error response if parameters are not valid.
My nginx.conf file is as follows :
location ~ "^/media/(.*?)/(.*?)/(.*)$" {
set $mediaUrl "$1/$2/$3";
set $key "$2/$3"
set $target http://$1.s3.amazonaws.com
rewrite_by_lua "
local uri = '/authenticate'
local res = ngx.location.capture(uri, {args = { param = '/xmedia/'.. ngx.var.mediaUrl }})
if (res.status ~= 200) then
return ngx.exit(ngx.HTTP_GONE)
end
";
rewrite .* /$key break;
proxy_pass $target;
}
location "/authenticate" {
proxy_set_header Range "";
proxy_set_header Content-Range "";
set_by_lua $param "
local params = ngx.req.get_uri_args()
return params.param
";
set $test_url http://127.0.0.1:some_port/authenticate?url=$param;
proxy_pass $test_url;
}
I my case, if authenticate returns 200, then everything works fine. But even if authenticate returns null, nginx returns correct file and does't give error report specified in if statement : return ngx.exit(ngx.HTTP_GONE).
Am I doing something wrong? How to achieve expected behavior in efficient manner?
Thanks.
As already mentioned on your question's comments, the HttpRewriteModule is always executed before rewrite_by_lua; therefore you have to put the logic on the rewrite_by_lua section using ngx.req.set_uri, like this:
location ~ "^/media/(.*?)/(.*?)/(.*)$" {
set $mediaUrl "$1/$2/$3";
set $key "$2/$3"
set $target http://$1.s3.amazonaws.com
rewrite_by_lua "
local uri = '/authenticate'
local res = ngx.location.capture(uri, {args = { param = '/xmedia/'.. ngx.var.mediaUrl }})
if (res.status ~= 200) then
ngx.exit(ngx.HTTP_GONE)
else
ngx.req.set_uri(string.format('/%s', ngx.var.key))
end
";
proxy_pass $target;
}
...
Related
I have a requirement to do a proxy call to url delivered via a query parameter as per example:
My nginx proxy is deployed at: https://myproxy.net
if the redirect parameter is not url encoded I can do the call with this block:
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass $arg_redirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
the error intercepts and #handle_redirects then take care of othe 30X codes that might pop up at new destination.
This works for a request:
GET: https://myproxy.net/basepath?redirect=https://destination.com/somepath/uuid
What do I need to do to make it work for:
GET: https://myproxy.net/basepath?redirect=https%3A%2F%2Fdestination.com%2Fsomepath%2Fuuid
Additionally as part of spec it has to be pure nginx, not additional modules, lua etc.
Thanks!
Actually, proxy_pass does normalisation by default, but it only affects $uri part. Thus you only need to decode the beginning of the passed string to get it working:
location / {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
if ( $arg_redirect ~ (.+)%3A%2F%2F(.+) ){ # fix :// between scheme and destination
set $arg_redirect $1://$2;
}
if ( $arg_redirect ~ (.+?)%3A(.*) ){ # fix : between destination and port
set $arg_redirect $1:$2;
}
if ( $arg_redirect ~ (.+?)%2F(.*) ){ # fix / after port, the rest will be decoded by proxy_pass
set $arg_redirect $1/$2;
}
proxy_pass $arg_redirect;
}
With the above I managed to access http://localhost/?redirect=http%3A%2F%2F127.0.0.1%3A81%2Fsfoo%20something%2Fs
The solution seems dirty and the only alternative using default modules is map (even less cleaner in my opinion). I'd rather split redirect argument into pieces: scheme (http or https), destination, port, and uri. With that you would be able to construct full address without rewriting:
proxy_pass $arg_scheme://$arg_dest:$arg_port/$arg_uri
Ok, there is very weird and curious solution
server {
listen 80;
resolver x.x.x.x;
location /basepath {
if ($arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
proxy_pass http://127.0.0.1:80/basepath/$arg_redirect;
}
location ~ ^/basepath/(?<proto>\w+):/(?<redir>.+)$ {
proxy_pass $proto://$redir;
}
}
Nginx does not encode path with variables in proxy_pass and send it as is. So, I make $arg_* part of proxy_pass uri, send request to self and nginx will receive new request which will be decoded.
But because Nginx will clean path and replace // to / I split protocol part in regexp.
And ... I would never recommend using this solution, but it works :)
try like this and let me know if it works
location /basepath {
if ( $arg_redirect = '') {
return 400 "Missing redirect directive in request";
}
set_unescape_uri $decodedredirect $arg_redirect;
proxy_pass $decodedredirect;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
}
I am trying to resolve proxy_pass value dynamically (through web api) in nginx.
I need something like below;
Example taken from: https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
location /proxy-pass-uri {
set $urlToProxy = CallWebAPI("http://localhost:8081/resolver?url=" + $url);
proxy_pass $urlToProxy;
}
So, my question is that, is it possible to make HTTP request or to write method such as CallWebAPI?
I know it might be a bad practice, but the website I am dealing with has thousands of web urls, which are mapped as key-value pairs, and 90% of them does not obey any specific regex rules. So I have content mapped database, and I need to fetch incoming url with content dynamically.
I am trying to use a very light web service to look up URLs from redis, and return proxy url.
Would this be a valid scenario, or is there any other built in solution in nginx like this?
I doubt this can be done with "pure" nginx, but this definitely can be done with openresty or ngx_http_lua_module with the help of ngx.location.capture method. For example:
resolver 8.8.8.8;
location ~/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("http://localhost:8081/resolver?url=" .. ngx.var.url)
ngx.var.proxy = res.body
}
proxy_pass $proxy$url;
}
There is also an ngx_http_js_module (documentation, GitHub) which have an ability to do subrequests (example), but I never used it and cannot tell if it can be used this way.
Important update
After almost a three years since this answer was written, it comes that I needed the similar functionality myself, and it turns out that the above answer is completely broken and unworkable. You can't do a subrequest via ngx.location.capture to anything else but to some other nginx location. So the correct (checked and confirmed to be workable) example for the above question is
resolver 8.8.8.8;
location /resolver {
internal;
proxy_pass http://localhost:8081;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/resolver?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
The above example assumes that the proxy resolution service is really expecting request in a /resolver?url=<uri> form. The location /resolver { ... } while being internal behaves like any other prefix location, so if the /resolver prefix for that location cannot be used for some reason, the same can be written as
resolver 8.8.8.8;
location /get_proxy {
internal;
proxy_pass http://localhost:8081/resolver;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/get_proxy?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
I am using Openresty as a server. I have the configuration file of the nginx as per the https://eclipsesource.com/blogs/2018/01/11/authenticating-reverse-proxy-with-keycloak/.
I am getting following error "openidc.lua:1053: authenticate(): request to the redirect_uri_path but there's no session state found, client"
Can someone throw some light and try to solve the problem.
Regards,
Allahbaksh
Your redirect URI must not be set to "/" but to some arbitrary path that is not supposed to return content (like /redirect_uri). It is a "vanity" URL that is handled by lua-resty-openidc
I had the same problem and was able to fix it by setting the $session_name variable in the server block. Example:
server {
...
server_name proxy.localhost;
#lua_code_cache off;
set $session_name nginx_session;
location / {
access_by_lua_block {
local opts = {
redirect_uri = "http://proxy.localhost/cb",
discovery = "http://127.0.0.1:9000/.well-known/openid-configuration",
client_id = "proxyclient-id",
client_secret = "secret",
ssl_verify = "no",
scope = "openid"
}
-- call authenticate for OpenID Connect user authentication
local res, err = require("resty.openidc").authenticate(opts)
if err then
ngx.status = 500
ngx.say(err)
ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
ngx.req.set_header("X-USER", res.id_token.sub)
}
proxy_pass http://localhost:8080/;
proxy_set_header x-forwarded-proto $scheme;
}
}
Another thing to pay attention to is the lua_code_cache off directive; It could break the session. See: https://github.com/bungle/lua-resty-session#notes-about-turning-lua-code-cache-off
I have a rental location where I want to register a variable aet with a certain value.
location /aet {
default_type 'text/plain';
content_by_lua '
if ngx.var.host:match("(.*).nexus$") ~= nil then
aet = ngx.var.host:match("(.-)%.")
ngx.say(aet)
end
';
}
And I want to use this variable in another rental location / getIp`
location /getIp {
default_type 'application/json';
rds_json on;
content_by_lua '
postgres_pass database;
postgres_query "SELECT ip FROM establishment_view WHERE aet = aet";
postgres_output rds;
';
}
I want the variable aet to initialize without needing to call path / aet
You may use Data Sharing within an Nginx Worker
But if you really need global variable for all worker processes you may use ngx.shared.DICT API
http {
lua_shared_dict my_dict 10m;
server {
location /aet {
content_by_lua_block {
local my_dict = ngx.shared.my_dict
my_dict :set("aet", ngx.var.host:match("(.-)%."))
}
}
location /getIp {
set $aet = ""
access_by_lua_block {
local my_dict = ngx.shared.my_dict
ngx.var.aet = my_dict:get("aet")
}
postgres_pass database;
postgres_query "SELECT ip FROM establishment_view WHERE aet = aet";
postgres_output rds;
}
}
}
PS: Your nginx configuration snippet is wrong - you use nginx directives postgres_* within content_by_lua.
Also all *by_lua are deprecated, use *_by_lua_block. It will save you a lot time with escaping.
I'd like to add a parameter in the URL in a proxy pass.
For example, I want to add an apiKey : &apiKey=tiger
http://mywebsite.com/oneapi?field=22 ---> https://api.somewhere.com/?field=22&apiKey=tiger
Do you know a solution ?
Thank's a lot,
Gilles.
server {
listen 80;
server_name mywebsite.com;
location /oneapi{
proxy_pass https://api.somewhere.com/;
}
}
location = /oneapi {
set $args $args&apiKey=tiger;
proxy_pass https://api.somewhere.com;
}
The other answers do not work if $args is empty.
This also works if $args is empty.
location /oneapi {
set $delimeter "";
if ($is_args) {
set $delimeter "&";
}
set $args "$args${delimeter}apiKey=tiger";
proxy_pass https://api.somewhere.com/;
}
github gist https://gist.github.com/anjia0532/da4a17f848468de5a374c860b17607e7
#set $token "?"; # deprecated
set $token ""; # declar token is ""(empty str) for original request without args,because $is_args concat any var will be `?`
if ($is_args) { # if the request has args update token to "&"
set $token "&";
}
location /test {
set $args "${args}${token}k1=v1&k2=v2"; # update original append custom params with $token
# if no args $is_args is empty str,else it's "?"
# http is scheme
# service is upstream server
#proxy_pass http://service/$uri$is_args$args; # deprecated remove `/`
proxy_pass http://service$uri$is_args$args; # proxy pass
}
#http://localhost/test?foo=bar ==> http://service/test?foo=bar&k1=v1&k2=v2
#http://localhost/test/ ==> http://service/test?k1=v1&k2=v2
Here's a way to add a paramater in nginx when it's unknown whether the original URL had arguments or not (i.e. when you have to account for both ? and &):
location /oneapi {
set $pretoken "";
set $posttoken "?";
if ($is_args) {
set $pretoken "?";
set $posttoken "&";
}
# Replace apiKey=tiger with your variable here
set $args "${pretoken}${args}${posttoken}apiKey=tiger";
# Optional: replace proxy_pass with return 302 for redirects
proxy_pass https://api.somewhere.com$uri$args;
}
For someone get here. Thanks for https://serverfault.com/questions/912090/how-to-add-parameter-to-nginx-query-string-during-redirect
The cleanest way on 2021 is:
rewrite ^ https://api.somewhere.com$uri?apiKey=tiger permanent;
If a replacement string includes the new request arguments, the previous request arguments are appended after them
proxy_pass way
upstream api {
server api.somewhere.com;
}
location /oneapi {
rewrite ^/oneapi/?(.*) /$1?apiKey=tiger break;
proxy_pass https://api$uri$is_args$args;
}