I have a rental location where I want to register a variable aet with a certain value.
location /aet {
default_type 'text/plain';
content_by_lua '
if ngx.var.host:match("(.*).nexus$") ~= nil then
aet = ngx.var.host:match("(.-)%.")
ngx.say(aet)
end
';
}
And I want to use this variable in another rental location / getIp`
location /getIp {
default_type 'application/json';
rds_json on;
content_by_lua '
postgres_pass database;
postgres_query "SELECT ip FROM establishment_view WHERE aet = aet";
postgres_output rds;
';
}
I want the variable aet to initialize without needing to call path / aet
You may use Data Sharing within an Nginx Worker
But if you really need global variable for all worker processes you may use ngx.shared.DICT API
http {
lua_shared_dict my_dict 10m;
server {
location /aet {
content_by_lua_block {
local my_dict = ngx.shared.my_dict
my_dict :set("aet", ngx.var.host:match("(.-)%."))
}
}
location /getIp {
set $aet = ""
access_by_lua_block {
local my_dict = ngx.shared.my_dict
ngx.var.aet = my_dict:get("aet")
}
postgres_pass database;
postgres_query "SELECT ip FROM establishment_view WHERE aet = aet";
postgres_output rds;
}
}
}
PS: Your nginx configuration snippet is wrong - you use nginx directives postgres_* within content_by_lua.
Also all *by_lua are deprecated, use *_by_lua_block. It will save you a lot time with escaping.
Related
I am trying to resolve proxy_pass value dynamically (through web api) in nginx.
I need something like below;
Example taken from: https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/
location /proxy-pass-uri {
set $urlToProxy = CallWebAPI("http://localhost:8081/resolver?url=" + $url);
proxy_pass $urlToProxy;
}
So, my question is that, is it possible to make HTTP request or to write method such as CallWebAPI?
I know it might be a bad practice, but the website I am dealing with has thousands of web urls, which are mapped as key-value pairs, and 90% of them does not obey any specific regex rules. So I have content mapped database, and I need to fetch incoming url with content dynamically.
I am trying to use a very light web service to look up URLs from redis, and return proxy url.
Would this be a valid scenario, or is there any other built in solution in nginx like this?
I doubt this can be done with "pure" nginx, but this definitely can be done with openresty or ngx_http_lua_module with the help of ngx.location.capture method. For example:
resolver 8.8.8.8;
location ~/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("http://localhost:8081/resolver?url=" .. ngx.var.url)
ngx.var.proxy = res.body
}
proxy_pass $proxy$url;
}
There is also an ngx_http_js_module (documentation, GitHub) which have an ability to do subrequests (example), but I never used it and cannot tell if it can be used this way.
Important update
After almost a three years since this answer was written, it comes that I needed the similar functionality myself, and it turns out that the above answer is completely broken and unworkable. You can't do a subrequest via ngx.location.capture to anything else but to some other nginx location. So the correct (checked and confirmed to be workable) example for the above question is
resolver 8.8.8.8;
location /resolver {
internal;
proxy_pass http://localhost:8081;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/resolver?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
The above example assumes that the proxy resolution service is really expecting request in a /resolver?url=<uri> form. The location /resolver { ... } while being internal behaves like any other prefix location, so if the /resolver prefix for that location cannot be used for some reason, the same can be written as
resolver 8.8.8.8;
location /get_proxy {
internal;
proxy_pass http://localhost:8081/resolver;
}
location ~ ^/proxy-pass-uri(/.*)$ {
set $url $1;
set $proxy "";
access_by_lua_block {
res = ngx.location.capture("/get_proxy?url=" .. ngx.var.url)
if res.status == ngx.HTTP_OK then
ngx.var.proxy = res.body
else
ngx.exit(res.status)
end
}
proxy_pass $proxy$url$is_args$args;
}
I am trying to serve static content from S3. I am using nginx and lua script for its configuration. I am new to both nginx and lua.
The task that I want to achieve is :
Get request URL into nginx.
Authenticate query params of url.
Serve from S3 if parameters are valid.
Send error response if parameters are not valid.
My nginx.conf file is as follows :
location ~ "^/media/(.*?)/(.*?)/(.*)$" {
set $mediaUrl "$1/$2/$3";
set $key "$2/$3"
set $target http://$1.s3.amazonaws.com
rewrite_by_lua "
local uri = '/authenticate'
local res = ngx.location.capture(uri, {args = { param = '/xmedia/'.. ngx.var.mediaUrl }})
if (res.status ~= 200) then
return ngx.exit(ngx.HTTP_GONE)
end
";
rewrite .* /$key break;
proxy_pass $target;
}
location "/authenticate" {
proxy_set_header Range "";
proxy_set_header Content-Range "";
set_by_lua $param "
local params = ngx.req.get_uri_args()
return params.param
";
set $test_url http://127.0.0.1:some_port/authenticate?url=$param;
proxy_pass $test_url;
}
I my case, if authenticate returns 200, then everything works fine. But even if authenticate returns null, nginx returns correct file and does't give error report specified in if statement : return ngx.exit(ngx.HTTP_GONE).
Am I doing something wrong? How to achieve expected behavior in efficient manner?
Thanks.
As already mentioned on your question's comments, the HttpRewriteModule is always executed before rewrite_by_lua; therefore you have to put the logic on the rewrite_by_lua section using ngx.req.set_uri, like this:
location ~ "^/media/(.*?)/(.*?)/(.*)$" {
set $mediaUrl "$1/$2/$3";
set $key "$2/$3"
set $target http://$1.s3.amazonaws.com
rewrite_by_lua "
local uri = '/authenticate'
local res = ngx.location.capture(uri, {args = { param = '/xmedia/'.. ngx.var.mediaUrl }})
if (res.status ~= 200) then
ngx.exit(ngx.HTTP_GONE)
else
ngx.req.set_uri(string.format('/%s', ngx.var.key))
end
";
proxy_pass $target;
}
...
I'm trying to figure out how to do the following:
Request is coming in.
HttpLuaModule performs some action against the request. If request is valid than Lua will finish processing with ngx.exit(202). But there are some conditions that may (and will) occur during the processing and nginx might return 403 , 404, 503 Errors.
What I want to do is to write to access logs only requests that have 200 Status code.
Basically I would like to do something like this:
location /foo {
content_by_lua_file "/opt/nginx/lua/process.lua";
if (status == 200) {
access_log "/path/to/the/access_log"
}
I'm very new to both nginx and lua so for me it's a bit of a challenge to figure out where to place and if statement (ether after content_by_lua_file or in side lua file) and what this if statement should look like.
nginx 1.7.0+ allows using an if condition in access_log directive itself.
access_log path [format [buffer=size [flush=time]] [if=condition]];
The if parameter (1.7.0) enables conditional logging.
A request will not be logged if the condition evaluates to “0” or an empty string
Combined with map directive its possible to send log events to different logs based on various conditions.
http {
map $status $normal {
~^2 1;
default 0;
}
map $status $abnormal {
~^2 0;
default 1;
}
map $remote_addr $islocal {
~^127 1;
default 0;
}
server {
access_log logs/access.log combined if=$normal;
access_log logs/access_abnormal.log combined if=$abnormal;
access_log logs/access_local.log combined if=$islocal;
}
}
http://nginx.org/en/docs/http/ngx_http_log_module.html
http://nginx.org/en/docs/http/ngx_http_map_module.html
you can do it by using ngx.log and log_by_lua directives.
location /conditional_log{
log_by_lua 'if ngx.status == 200 then ngx.log(ngx.ERR, "It is 200") end';
content_by_lua 'ngx.say("I am ok") ngx.exit(200)';
}
In the above code, we use log_by_lua which is called while running in log phase. In that if ngx.status == 200, we use ngx.log to trigger the logging using ngx.log.
This will write to error_log. Not sure how to write it to access_log.
For reference
http://wiki.nginx.org/HttpLuaModule#ngx.log
http://wiki.nginx.org/HttpLuaModule#log_by_lua
In every question is a part of answer. You were very close:
if ($status != "200") {
access_log off;
}
Check info for version availability here.
http://nginx.org/en/docs/http/ngx_http_core_module.html#variables
Also, almost all access log format vars are available in "modern" versions:
http://nginx.org/en/docs/http/ngx_http_log_module.html
This is the solution I came up with:
auth.lua
-- Some logic goes here
-- ....
-- ....
ngx.var.return_status = 200
nginx.conf
http {
lua_package_path .....;
lua_package_cpath ....;
rewrite_by_lua_no_postpone on;
server {
set $return_status 1;
location /foo {
rewrite_by_lua_file "<apth_to_aut.lua";
if ($return_status = 200) {
access_log <path_to_access_log> format;
return 200;
}
}
}
}
I'd like to add a parameter in the URL in a proxy pass.
For example, I want to add an apiKey : &apiKey=tiger
http://mywebsite.com/oneapi?field=22 ---> https://api.somewhere.com/?field=22&apiKey=tiger
Do you know a solution ?
Thank's a lot,
Gilles.
server {
listen 80;
server_name mywebsite.com;
location /oneapi{
proxy_pass https://api.somewhere.com/;
}
}
location = /oneapi {
set $args $args&apiKey=tiger;
proxy_pass https://api.somewhere.com;
}
The other answers do not work if $args is empty.
This also works if $args is empty.
location /oneapi {
set $delimeter "";
if ($is_args) {
set $delimeter "&";
}
set $args "$args${delimeter}apiKey=tiger";
proxy_pass https://api.somewhere.com/;
}
github gist https://gist.github.com/anjia0532/da4a17f848468de5a374c860b17607e7
#set $token "?"; # deprecated
set $token ""; # declar token is ""(empty str) for original request without args,because $is_args concat any var will be `?`
if ($is_args) { # if the request has args update token to "&"
set $token "&";
}
location /test {
set $args "${args}${token}k1=v1&k2=v2"; # update original append custom params with $token
# if no args $is_args is empty str,else it's "?"
# http is scheme
# service is upstream server
#proxy_pass http://service/$uri$is_args$args; # deprecated remove `/`
proxy_pass http://service$uri$is_args$args; # proxy pass
}
#http://localhost/test?foo=bar ==> http://service/test?foo=bar&k1=v1&k2=v2
#http://localhost/test/ ==> http://service/test?k1=v1&k2=v2
Here's a way to add a paramater in nginx when it's unknown whether the original URL had arguments or not (i.e. when you have to account for both ? and &):
location /oneapi {
set $pretoken "";
set $posttoken "?";
if ($is_args) {
set $pretoken "?";
set $posttoken "&";
}
# Replace apiKey=tiger with your variable here
set $args "${pretoken}${args}${posttoken}apiKey=tiger";
# Optional: replace proxy_pass with return 302 for redirects
proxy_pass https://api.somewhere.com$uri$args;
}
For someone get here. Thanks for https://serverfault.com/questions/912090/how-to-add-parameter-to-nginx-query-string-during-redirect
The cleanest way on 2021 is:
rewrite ^ https://api.somewhere.com$uri?apiKey=tiger permanent;
If a replacement string includes the new request arguments, the previous request arguments are appended after them
proxy_pass way
upstream api {
server api.somewhere.com;
}
location /oneapi {
rewrite ^/oneapi/?(.*) /$1?apiKey=tiger break;
proxy_pass https://api$uri$is_args$args;
}
I am having a problem with my $memcached_keys being too long in my .conf file for nginx. I am using the memcached module but some of my urls are too long. I am in the process of trying to user MD5 hashes of the urls instead but in the meantime I was just wondering if there was a way I could check on the length of a string stored in a variable.
so:
set $memcached_key "byp-$uri";
if ($args) {
set $memcached_key "byp-$uri?$args";
}
if (len($memcache_key) < 250) {
memcached_pass 127.0.0.1:11211;
error_page 404 = #cache_miss;
error_page 502 = #cache_miss;
}
else {
pass to #cache_miss;
}
Old question but ....
To do this sort of thing, you need a scripting setup such as the Lua Module:
location / {
set_by_lua $memcached_key '
if not ngx.var.args then
return "byp-" .. ngx.var.uri
else
return "byp-" .. ngx.var.uri .. "?" .. ngx.var.args
end
';
content_by_lua '
local string = string;
if string.len($memcached_key) < 250 then
ngx.exec("/memcached");
else
ngx.exec("/cache_miss");
end
';
}
location /memcached {
internal;
memcached_pass 127.0.0.1:11211;
error_page 404 = /cache_miss;
error_page 502 = /cache_miss;
}
location /cache_miss {
internal;
...
}
Suggest "internal" locations instead of named locations due to a few quirks with the latter but named locations can be used as well.
I'm not positive, but I don't think it can be done within the nginx config language. I suspect it would be documented here if it existed, and I don't see anything.