I am trying to access private S3 from an EC2 machine but using curl only.
So I am trying to create a proxy server using nginx and lua which will call metadata apis and get the authorization token and set the headers in proxy_pass.
location /download/ {
set $date '';
set $token '';
set $authorization '';
content_by_lua_block {
% some code %
ngx.var.date = date;
ngx.var.token = awsToken;
ngx.var.authorization = authorization;
}
proxy_set_header Date $date;
proxy_set_header X-AMZ-Security-Token $token;
proxy_set_header Authorization $authorization;
proxy_pass "https://nisingla-ethos.s3.amazonaws.com/";
}
However, when i check date varible outside of content_by_lua_block, its value is not set.
Can someone help me with the issue.
PS: I have tried other method of using bucket policy and vpc endpoint but due to some constraint they will not work for me.
Both content_by_lua_block and proxy_pass are processed in Content phase.
Nginx does call only one such directive.
You may use access_by_lua_block or set_by_lua_block.
Related
To start with: I am NOT an nginx expert. Very much a newbie to it.
I am attempting to protect a 3rd party piece of software with nginx doing the authentication (really - just verifying that the request has a valid OAuth2 Bearer token)
The HTTP request will have an OAuth2 bearer token in the Authentication header.
e.g. Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZ....H5w
I have an OAuth2 server (UAA) that has an api where I can call http://myuaa/check_token?token=eyJhbGciOiJSUzI1NiIsImtpZ....H5w to get back a 2XX or a 4XX if the token is valid. A complication is that this server does require basic auth to call the /check_token endpoint.
I have tried using a map to parse the token from the authorization header, but with no luck.
Just kind of at a loss.
Perhaps this isn't a good fit for Nginx?
relevant pieces of the nginx.conf
# this map isnt working as I thought it might
http {
...
map $http_authorization $token {
~Bearer(?<token>abc) $token;
}
...
# test just to see if the authorization header is being parsed and passed - no luck
location /oauth {
proxy_set_header X-my-header $token;
proxy_set_header X-another-header value;
proxy_set_header Authorization "Basic basdasdfasdf";
proxy_pass http://localhost:8080;
}
Expected request to the 3rd party server that nginx is protecting:
<GET|POST|PUT|DELETE> /anyurl HTTP1/1.1
..
Authorization: Bearer eyJhbGciOiJSUzI1NiIsImtpZ....H5w
..
Expected request forwarded to the UAA server to validate token
GET /check_token?token=eyJhbGciOiJSUzI1NiIsImtpZ....H5w
..
Authorization Basic asfasdfdf
..
Your map directive isn't working, named group token somehow interfere with the $token variable, any of these definitions would work:
map $http_authorization $token {
~^Bearer\s+([\S]+)$ $1;
}
or
map $http_authorization $token {
~^Bearer\s+(?<bearer>[\S]+)$ $bearer;
}
Full working config will be looking like this:
map $http_authorization $token {
~^Bearer\s+(?<bearer>[\S]+)$ $bearer;
}
server {
...
location / {
auth_request /uaa;
...
}
location /uaa {
internal;
proxy_pass_request_body off;
proxy_set_header Authorization "Basic your_base64_auth_string";
proxy_set_header Content-Length "";
proxy_pass http://localhost:8080/check_token?token=$token;
}
}
I am trying to resolve proxy_pass value dynamically (through web api) in nginx.
I need something like below;
Example taken from: https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
location /proxy-pass-uri {
set $urlToProxy = CallWebAPI("http://localhost:8081/resolver?url=" + $url);
proxy_pass $urlToProxy;
}
So, my question is that, is it possible to make HTTP request or to write method such as CallWebAPI?
I know it might be a bad practice, but the website I am dealing with has thousands of web urls, which are mapped as key-value pairs, and 90% of them does not obey any specific regex rules. So I have content mapped database, and I need to fetch incoming url with content dynamically.
I am trying to use a very light web service to look up URLs from redis, and return proxy url.
Would this be a valid scenario, or is there any other built in solution in nginx like this?
I doubt this can be done with "pure" nginx, but this definitely can be done with openresty or ngx_http_lua_module with the help of ngx.location.capture method. For example:
resolver 8.8.8.8;
location ~/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("http://localhost:8081/resolver?url=" .. ngx.var.url)
ngx.var.proxy = res.body
}
proxy_pass $proxy$url;
}
There is also an ngx_http_js_module (documentation, GitHub) which have an ability to do subrequests (example), but I never used it and cannot tell if it can be used this way.
Important update
After almost a three years since this answer was written, it comes that I needed the similar functionality myself, and it turns out that the above answer is completely broken and unworkable. You can't do a subrequest via ngx.location.capture to anything else but to some other nginx location. So the correct (checked and confirmed to be workable) example for the above question is
resolver 8.8.8.8;
location /resolver {
internal;
proxy_pass http://localhost:8081;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/resolver?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
The above example assumes that the proxy resolution service is really expecting request in a /resolver?url=<uri> form. The location /resolver { ... } while being internal behaves like any other prefix location, so if the /resolver prefix for that location cannot be used for some reason, the same can be written as
resolver 8.8.8.8;
location /get_proxy {
internal;
proxy_pass http://localhost:8081/resolver;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/get_proxy?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
I am using Openresty as a server. I have the configuration file of the nginx as per the https://eclipsesource.com/blogs/2018/01/11/authenticating-reverse-proxy-with-keycloak/.
I am getting following error "openidc.lua:1053: authenticate(): request to the redirect_uri_path but there's no session state found, client"
Can someone throw some light and try to solve the problem.
Regards,
Allahbaksh
Your redirect URI must not be set to "/" but to some arbitrary path that is not supposed to return content (like /redirect_uri). It is a "vanity" URL that is handled by lua-resty-openidc
I had the same problem and was able to fix it by setting the $session_name variable in the server block. Example:
server {
...
server_name proxy.localhost;
#lua_code_cache off;
set $session_name nginx_session;
location / {
access_by_lua_block {
local opts = {
redirect_uri = "http://proxy.localhost/cb",
discovery = "http://127.0.0.1:9000/.well-known/openid-configuration",
client_id = "proxyclient-id",
client_secret = "secret",
ssl_verify = "no",
scope = "openid"
}
-- call authenticate for OpenID Connect user authentication
local res, err = require("resty.openidc").authenticate(opts)
if err then
ngx.status = 500
ngx.say(err)
ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
ngx.req.set_header("X-USER", res.id_token.sub)
}
proxy_pass http://localhost:8080/;
proxy_set_header x-forwarded-proto $scheme;
}
}
Another thing to pay attention to is the lua_code_cache off directive; It could break the session. See: https://github.com/bungle/lua-resty-session#notes-about-turning-lua-code-cache-off
I'm evaluating nginx to act as rate limiter for a multi tenancy REST API system. I need to limit API calls by tenant-id.
For example i want to allow 100 r/s for tenant1 and only 50 r/s for tenant2.
It can be easily achived when there are differant urls like: "me.com/tenant1/api" and "me.com/tenant2/api" (with the location directive).
But, in my case the urls are the same for all tenants "me.com/api" (I can't change this).
To find the tenant-id I need to extract a JSON attribute from the Body of the request, and then check the DB for the real tenant-id.
Is it possible to limit_req with my requirements?
Thank for the help!
I decided to build another service getTenant for parsing the body and extracting the Tenant from the DB. This service is called internally by Nginx.
I'm not sure if that is the best nginx (/openresty) solution, but this is what i came up with:
limit_req_zone t1Limit zone=t1Zone:10m rate=200r/s;
limit_req_zone t2Limit zone=t2Zone:10m rate=90r/s;
server {
location /api{
content_by_lua_block {
ngx.req.read_body();
local reqBody = ngx.req.get_body_data()
local res = ngx.location.capture("/getTenant", {method=ngx.HTTP_POST,body=reqBody});
local tenantId= res.body;
if tenantId== "none" then
ngx.log(ngx.ERR, "Tenant not found!");
ngx.say(tenantId);
else
ngx.req.set_header("x_myTenantId", tenantId)
local res2 = ngx.location.capture("/" .. tenantId .."/doApi", {method=ngx.HTTP_POST,body=reqBody});
if res2.status == ngx.HTTP_OK then
ngx.say(res2.body);
ngx.exit(res2.status);
else
ngx.status = res2.status
ngx.exit(res2.status)
end
end;
}
}
location /getTenant {
internal; #this is not accessible from outside.
proxy_pass http://UpStream1/getCustomer;
proxy_set_header X-Original-URI $request_uri;
}
location /tenant1/doApi {
internal; #this is not accessible from outside.
# Proxy all requests to the AReqUpStream server group
proxy_pass http://UpStream2/doApi;
limit_req zone=tenant1Zone burst=25;
limit_req_log_level notice;
}
location /tenant2/doApi {
internal; #this is not accessible from outside.
# Proxy all requests to the AReqUpStream server group
proxy_pass http://UpStream2/doApi;
limit_req zone=tenant2Zone burst=10 ;#nodelay;
limit_req_status 409;
limit_req_log_level notice;
}
}
Basically, when me.com/api is called, a new subrequest is issued to service /getTenant. The response of that call is used to build another subrequest call to the /tenant[X]/doApi service. That way i can define locations per tenant and provide different rate_limis to each.
Comments on that are more than welcome!
I'm trying to get access to media files (images, videos) sitting behind an OAuth2 authentication.
In order to access the resource I need to add a custom Authorization Bearer token to the request, so I can't use a simple rewrite (well, as far as I know at least).
It cannot be done via plain HTML (say img or video tag) so I'm considering to have Nginx proxying the queries to the final server.
Each of the media resources would be loaded via a /proxy path, with a token parameter (for authentication) and url for the actual resource to load.
Sample URL:
http://myserver.com/proxy/?token=12345&url=http://protectedserver.com/custompath/asset
This is what I came up with but I am not quite sure how to configure the proxy_pass directive since I need it to proxy to the $url variable specifically. I do not need to proxy the path (which would be empty anyway).
location /proxy/ {
if ($arg_token ~ "^$") { return 404; }
if ($arg_url ~ "^$") { return 404; }
set $url $arg_url;
proxy_set_header Authorization "Bearer $arg_token";
set $args "";
#proxy_pass $url;
}
Note: this will be run in a closed environment and only specific machines (kiosks with limited interaction) will be able to access the page so I'm not concerned about a potential leak of the auth token.
I noticed a similar question on ServerFault, but no one had an answer to that:
https://serverfault.com/questions/671991/nginx-proxy-pass-url-from-get-argument
I'm looking for a config setting to make it work or a viable alternative solution.
Here is a correct configuration for my problem:
location /proxy/ {
if ($arg_token ~ "^$") { return 404; }
if ($arg_url ~ "^$") { return 404; }
set $url $arg_url;
set $token $arg_token;
set $args "";
# IMPORTANT, this is required when using dynamic proxy pass
# You can alternatively use any DNS resolver under your control
resolver 8.8.8.8;
proxy_pass $url;
proxy_set_header Authorization "Bearer $token";
proxy_redirect off;
}