Im trying to ignore certain urls on my host (as they get requested very often from sync clients and I'm not scared of of any exposure/volunability here)
Typical urls look like;
/cloud/remote.php/caldav/calendars/xxxx#xxxx.com/personlig/
/cloud/remote.php/caldav/calendars/xxxx#xxxx.com/contact_birthdays/
/cloud/remote.php/carddav/xxxx#xxxx.com/
/cloud/remote.php/webdav/xxxx#xxxx.com/
What I'd idealy want to achive is ignore anything after /cloud/remote.php/ or /carddav/ or /caldav/ or /webdav/ a s these belong to sync clients and gets logged seperatedly
I've played around with
location = /cloud/remote.php/ {
access_log off;
}
location = /caldav/calendars/ {
access_log off;
}
But it didnt yield what I expected so I now turn to you guys!
Any suggestions on how I can tackle this?
Use regexps, not equal expression, e.g. for /cloud/remote.php/*:
location ~ ^/cloud/remote.php/(caldav|webdav|carddav)(.*)$ {
access_log off;
}
As for regexps priority, manual says:
"The first matching expression stops the search and nginx will use this location. If no regular expression matches a request, then nginx uses the most specific prefix location found earlier."
How nginx processes a request
Related
ISSUE
I have a Vue 3 SSR application running in a docker container. I have a second docker container containing NGINX. I need to have correct location rules in my NGINX container to pass on the correct values to my Vue3 app container.
My application URLs coming into NGINX are
/my/prefix/
/my/prefix/page1/123?name=one
/my/prefix/page2/789?id=123&name=one
/my/prefix/css/styles.css and /my/prefix/js/app.js
I need those URLs to pass onto my Vue app container as the following
/
/page1/123?name=one
/page2/789?id=123&name=one
/my/prefix/css/styles.css and /my/prefix/js/app.js
I can not figure out the correct combination to get this to work.
What I have tried
The following resolves (1): /my/prefix/ becomes /. This on its own however does not resolve (2) (3) and (4), as a result the 2nd and 3rd pages in my app show a 404 and I have no styling and js loaded.
location = /my/prefix/ {
set $upstream http://newserver:8080/;
proxy_pass $upstream;
}
I tried adding the following, with this i have (4) working, styles/js are present, because the the URLs passed on are /my/prefix/css/styles.css and /my/prefix/js/app.js. However (2/3) fail as for the routes in my app for the exact reason the styles/js is working, it passes on the URLS /my/prefix/page1/123?name=one and /my/prefix/page2/789?id=123&name=one I need it to be /page1/123?name=one and /page2/789?id=123&name=one
location /my/prefix/ {
set $upstream http://newserver:8080;
proxy_pass $upstream;
}
I tried the following to resolve (2/3). This does match the (2/3) paths only, but what is passed on to the Vue container has the path AFTER page1|page2 removed, i.e. /page1/?name=one and /page2/?id=123&name=one which is wrong.
location ~ ^/my/prefix/(page1|page2) {
set $upstream http://newserver:8080;
proxy_pass $upstream/$1$is_args$args;
}
I have spent a couple of days trying to get the right combination and I am at a complete loss of what to try next.
Does anyone have any suggestions.
Finally after 2 days of trying everything, I have a working solution
# URL (1)
# match the root page URL exactly (the = does an exact match the stops searching for rules)
# - MUST have a trailing / on the location to match our app which appears to always have it
# - MUST have a trailing / on the upstream, so it removes the matched path before passing on
# the path requested and then stops searching for any more matches.
location = /my/prefix/ {
set $upstream http://newserver:8080/;
proxy_pass $upstream;
}
# URL (2) (3)
# match the page1|page2 pages
# remove the path and pass on the paths and parameter values
# $1 references what is in the first ()
# $2 references what is in the second ()
location ~ ^/my/prefix/(page1|page2)(.*) {
set $upstream http://newserver:8080;
proxy_pass $upstream/$1$2$is_args$args;
}
# URL (4)
# match any path starting with /my/prefix/,
# pass on the url exactly as is (/my/prefix/path/path etc)
# this should match the css/js folders
location /my/prefix/ {
set $upstream http://newserver:8080;
proxy_pass $upstream;
}
I am using openresty lua (https://github.com/openresty) to configure our nginx proxy. I have one main proxy.template that defines 3 locations, but want to use only one LRU cache for multiple process initialization calls (since they take so long and nothing else, that is why I am using a script). I want to define a variable that can be passed into each location, but am pretty sure I am not doing this correctly. I have:
#init_by_lua_file $lru_cache /etc/scripts/lua/process_cache.lua;
location /process {
access_by_lua_file /etc/scripts/lua/process_access.lua;
proxy_set_header Content-Type "application/json";
proxy_set_header Accept "application/json";
proxy_ssl_server_name on;
proxy_pass $target;
}
location /process/init {
set_by_lua_file $lru_cache /etc/scripts/lua/process_cache.lua;
add_header Access-Control-Expose-Headers set-cookie;
add_header Access-Control-Allow-Headers set-cookie;
access_by_lua_file /etc/scripts/lua/process_init.lua;
}
The process_cache creates the cache (one per proxy startup) and I would like it to be referenced by the process_init.lua and process_access.lua which do different things. For example, process_init is only called once for a UI initialization and establishes the specific cache entries, process_access checks to make sure the entry hasn't expired and if not uses it, otherwise creates a new entry, so that a long call to another server is not needed.
The above would require the lru_cache variable to be passed amongst the two locations. My latest attempts were in the area of trying to place process_cache.lua within the /process/init path, but then it just gets initialized each time, so starting with an empty cache each /process/init call is useless. Thoughts?
consider ngx.shared ?
You can specific exptime for your strings.
: https://github.com/openresty/lua-nginx-module#ngxshareddict ?
I want to change an existing nginx configuration in a way where I can completely "mask" the configuration and proxy everything to the upstream when a certain cookie is available (to hide a certain server).
This includes not just some location directives but basically every location directive (as opposed to set or map a variable and update n-location's try_files and more).
My basic idea was to use lua and jump into the Rewrite/Access phase like this:
access_by_lua_block {
# proceed as usual if our cookie is not detected
if ngx.var.cookie_demo ~= nil and string.len(ngx.var.cookie_demo) ~= 32 then
return
end
# proxy and return w/out further processing
ngx.exec("#ngxbackend")
return ngx.exit(ngx.HTTP_OK)
}
# proxy upstream
location #ngxbackend {
include /etc/nginx/proxy_params_demo;
proxy_pass https://demo-upstreams;
}
But this leads to an ERR with rewrite or internal redirection cycle while redirect to named location "#ngxbackend" as the named location is probably never reached because of the access_by_lua_block after it's internal redirect.
Can I solve this by use of variables and further condition checking?
I am using a nginx as a proxy for an apache server.
Here is my config:
location ~ ^/subsite/(.*)$ {
proxy_pass http://127.0.0.1/subsite/$1?$query_string;
}
the problem is that if I send a request with %20 like mywebsite.com/subsite/variable/value/title/Access%20denied/another/example
the %20 is replaced by a whitespace, and apache don't care about all the end of the request after Access /title/Access
Any Idea ?
I was able to solve a similar issue -- we have an api that requires the search terms to be part of the URL path. Passing the output directly to the proxy_pass directive caused it to throw a 502 even though the request was properly url encoded.
Here's the solution we came up with:
location ~ /api/search(/.*) {
set $query $1;
proxy_pass http://127.0.0.1:3003$query;
}
The "set" directive seems to keep the url encoding intact (or re-encodes from what the regex is passing back in $1).
I am running a Rails 3 site on Ubuntu 8.04 with Nginx 1.0.0 and Passenger 3.0.7.
In my Nginx error.log I started seeing the message X-Accel-Mapping header missing quite a lot. Googling lead me to the docs of Rack::Sendfile and to the Nginx docs.
Now, my app can be accessed through several domains and I am using send_file in my app to deliver some files specific to the domain they are requested from, e.g., if you come to domain1.com/favicon.ico I look up the favicon in at public/websites/domain1/favicon.ico.
This works fine and I don't think I need/want to get Nginx involved and create some private area where I store those files, as the samples in the Rack::Sendfile docs suggest.
How can I get rid of the error message?
this message means that Rack::Sendfile disabled X-Accel-Redirect for you, because you have missing configuration for it in nginx.conf...
I'm using Nginx + Passenger 3 + Rails 3.1.
Gathered information from this pages I've figured it out:
http://wiki.nginx.org/X-accel
http://greenlegos.wordpress.com/2011/09/12/sending-files-with-nginx-x-accel-redirect
http://code.google.com/p/substruct/source/browse/trunk/gems/rack-1.1.0/lib/rack/sendfile.rb?r=355
Serving Large Files Through Nginx via Rails 2.3 Using x-sendfile
I have controller which maps /download/1 requests to storage files which have their own directory structure, like this: storage/00/00/1, storage/01/0f/15 etc. So I need to pass this through Rails, but then I need to use send_file method which will use X-Accel-Redirect to send the final file to the browser through nginx directly.
Within the code I have this:
send_file(
'/var/www/shared/storage/00/00/01',
:disposition => :inline,
:filename => #file.name # an absolute path to the file which you want to send
)
I replaced the filename for this example purposes
Now I had to add these lines to my nginx.conf:
server {
# ...
passenger_set_cgi_param HTTP_X_ACCEL_MAPPING /var/www/shared/storage/=/storage/;
passenger_pass_header X-Accel-Redirect;
location /storage {
root /var/www/shared;
internal;
}
# ...
}
The path /storage is not visible from outside world, it is internal only.
Rack::Sendfile gets the header X-Accel-Mapping, extracts the path from it and replaces /var/www/shared/storage with /storage.... Then it spits out the modified header:
X-Accel-Redirect: /storage/00/00/01
which is then processed by nginx.
I can see this works correctly as the file is downloaded 100x faster than before and no error is shown in the logs.
Hope this helps.
We used the similar technique as NoICE described, but i replaced the "hard-coded" directory containing all the files with the regular expression describing the folder containing the folders containing the files.
Sounds hard, yeah? Just take a look on these (/etc/nginx/sites-available/my.web.site):
location /assets/(.+-[a-z0-9]+\.\w+) {
root /home/user/my.web.site/public/assets/$1;
internal;
}
location /images/(.+)(\?.*)? {
root /home/user/my.web.site/public/images/$1;
internal;
}
This should be used with this check:
location / {
# ...
if (-f $request_filename) {
expires max;
break;
}
# ...
}
to prevent the statics from Rails processing.
I did by this manual
https://mattbrictson.com/accelerated-rails-downloads
my server sends file path /private_upload/file/123/myfile.txt, the file is in /data/myapp-data/private_upload/file/123/myfile.txt
# Allow NGINX to serve any file in /data/myapp-data/private_upload
# via a special internal-only location.
location /private_upload {
internal;
alias /data/myapp-data/private_upload;
}
# ---------- BACKEND ----------
location #backend
{
limit_req zone=backend_req_limit_per_ip burst=20 nodelay;
proxy_pass http://backend;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /=/; # this header is required, it does nothing
include /etc/nginx/templates/myapp_proxy.conf;
}