Below is the relevant section of my nginx.conf file.
I only see the js|css... requests forward to my backend server when i remove the initial location block in the conf file. What im trying to accomplish is to turn off nginx access logging for files of those extensions.
Anybody know a working nginx config technique to allow me to turn off the access logs yet still forward these requests to the proxy location?
...
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
access_log off;
}
location / {
if ($ignore_ua) {
access_log off;
return 200;
}
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:7777/;
}
nginx chooses a location block to process a request. In the case of .js files, your location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ block is used. None of the directives within the location / block are involved. See this document for details.
If you need conditional logging, you could use the if= parameter to the access_log directive instead of a separate location block. See this document for an example.
In your case, it might look like this:
map $request_uri $loggable {
default 1;
\.(js|css|png|jpg|jpeg|gif|ico)(\?|$) 0;
}
access_log /path/to/access.log combined if=$loggable;
Note that the map directive goes in the http block.
Related
I could not decide the best name for the question.
Essentially what I want to achieve is to set a custom allowed body size for a specific location on the webserver.
On the other hand, I was able to achieve the necessary result already with duplicate code, so I am really looking for a way how to make the code reusable and to better understand the observed behavior.
The server reverse-proxies all API requests to the backend service.
In global nginx config /etc/nginx/nginx.conf I set the rule for max allowed body size like so client_max_body_size 50k;.
Then, in individual server config /etc/nginx/conf.d/example.com I have the following config (simplified):
server {
listen 80;
listen [::]:80;
server_name api.example.com www.api.example.com
location ~* /file/upload {
client_max_body_size 100M;
# crashes without this line
proxy_pass http://localhost:90;
#proxy_pass http://localhost:90/file/upload; # also works
}
location / {
# does not work
#location ~* /file/upload {
# client_max_body_size 100M;
#}
proxy_pass http://localhost:90;
}
}
I am trying to override the max body size for file upload endpoint. See that there is 1 proxy_pass for location /file/upload and another proxy_pass for location / pointing to the same internal service.
Question 1. If I remove the proxy_pass from the location /file/upload then error is returned by the server. (no status code in chrome debugger). Why is this happening? Shouldn't request be propagated further to location /?
Question 2. Why is it not possible to define the sublocation with body size override inside the / location as in commented section in example above? If I set it like this, then 413 error code is returned, which hints that the client_max_body_size rule is ignored..
Question 3. Finally, is it possible to tell nginx, after the request hits the /file/upload location - to apply all the rules from the / section? I guess one solution to this problem would be to move the common configuration into separate file and then import it in both sections.. I was thinking if there is any solution that does not require creating new files?
Here is the reusable config I am talking about basically:
location / {
#.s. kill cache. use in dev
sendfile off;
# kill cache
add_header Last-Modified $date_gmt;
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
if_modified_since off;
expires off;
etag off;
# don't cache it
proxy_no_cache 1;
# even if cached, don't try to use it
proxy_cache_bypass 1;
proxy_pass http://localhost:90;
client_max_body_size 100M;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
This is not the final version. If I had to copy this piece of code to 2 sections this would not be very friendly approach. So, it would be nice to hear some nginx lifehacks on how accomplish what I try to accomplish in a most friendly way and get some explanations for observed behavior.
Answer 1
If I remove the proxy_pass from the location /file/upload then error is returned by the server. (no status code in chrome debugger). Why is this happening?
Every location have a so-called content handler. If you don't specify content handler explicitly via proxy_pass (fastcgi_pass, uwsgi_pass, etc.) directive, nginx will try to serve the request locally.
Shouldn't request be propagated further to location /?
Of course not. What makes you think it should?
Answer 2
Why is it not possible to define the sublocation with body size override inside the / location as in commented section in example above? If I set it like this, then 413 error code is returned, which hints that the client_max_body_size rule is ignored..
I'd rather expect you'll get the same error as in the first case since your nested location does not have an explicitly specified content handler via the proxy_pass directive. However the following config is worth to try:
location / {
# all the common configuration
location /file/upload {
client_max_body_size 100M;
proxy_pass http://localhost:90;
}
proxy_pass http://localhost:90;
}
Answer 3
Finally, is it possible to tell nginx, after the request hits the /file/upload location - to apply all the rules from the / section?
No, unless you use a separate file via include directive in both locations. However you can try to move all the upstream related setup directives one level up to the server context:
server {
...
# all the common configuration
location / {
proxy_pass http://localhost:90;
}
location /file/upload {
client_max_body_size 100M;
proxy_pass http://localhost:90;
}
}
Note that some directives (e.g. add_header, proxy_set_header) are inherited from the previous configuration level if and only if there are no those directives defined on the current level.
Very often dynamic settings for different locations can be achieved using the map block in a following way:
map $uri $max_body_size {
~^/file/upload 100M;
default 50k;
}
server {
location / {
...
client_max_body_size $max_body_size;
...
}
}
Unfortunally not every nginx directive accepts variable as its argument. Usually when nginx documentation doesn't explicitly states that some directive can accept variables, it means it cannot, and the client_max_body_size is exactly that kind of directive, so the above configuration won't work.
I'm used to using Apache with mod_proxy_html, and am trying to achieve something similar with NGINX. The specific use case is that I have an admin UI running in Tomcat on port 8080 on a server at the root context:
http://localhost:8080/
I need to surface this on port 80, but I have other contexts on the NGINX server running on this host, so want to try and access this at:
http://localhost:80/admin/
I was hoping that the following super simple server block would do it, but it doesn't quite:
server {
listen 80;
server_name screenly.local.akana.com;
location /admin/ {
proxy_pass http://localhost:8080/;
}
}
The problem is that the returned content (html) contains URLs to scripts and style info that is all accessed at the root context, so I need to get these URLs rewritten to start with /admin/ instead of /.
How do I do this in NGINX?
We should first read the documentation on proxy_pass carefully and fully.
The URI passed to upstream server is determined based on whether "proxy_pass" directive is used with URI or not. Trailing slash in proxy_pass directive means that URI is present and equal to /. Absense of trailing slash means hat URI is absent.
Proxy_pass with URI:
location /some_dir/ {
proxy_pass http://some_server/;
}
With the above, there's the following proxy:
http:// your_server/some_dir/ some_subdir/some_file ->
http:// some_server/ some_subdir/some_file
Basically, /some_dir/ gets replaced by / to change the request path from /some_dir/some_subdir/some_file to /some_subdir/some_file.
Proxy_pass without URI:
location /some_dir/ {
proxy_pass http://some_server;
}
With the second (no trailing slash): the proxy goes like this:
http:// your_server /some_dir/some_subdir/some_file ->
http:// some_server /some_dir/some_subdir/some_file
Basically, the full original request path gets passed on without changes.
So, in your case, it seems you should just drop the trailing slash to get what you want.
Caveat
Note that automatic rewrite only works if you don't use variables in proxy_pass. If you use variables, you should do rewrite yourself:
location /some_dir/ {
rewrite /some_dir/(.*) /$1 break;
proxy_pass $upstream_server;
}
There are other cases where rewrite wouldn't work, that's why reading documentation is a must.
Edit
Reading your question again, it seems I may have missed that you just want to edit the html output.
For that, you can use the sub_filter directive. Something like ...
location /admin/ {
proxy_pass http://localhost:8080/;
sub_filter "http://your_server/" "http://your_server/admin/";
sub_filter_once off;
}
Basically, the string you want to replace and the replacement string
You may also need the following directive to be set before the first "sub_filter" for backend-servers with data compression:
proxy_set_header Accept-Encoding "";
Otherwise it may not work.
For your example it will look like:
location /admin/ {
proxy_pass http://localhost:8080/;
proxy_set_header Accept-Encoding "";
sub_filter "http://your_server/" "http://your_server/admin/";
sub_filter_once off;
}
You can use the following nginx configuration example:
upstream adminhost {
server adminhostname:8080;
}
server {
listen 80;
location ~ ^/admin/(.*)$ {
proxy_pass http://adminhost/$1$is_args$args;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
If there are no hyperlinks which need to be rewritten with sub_filter,
you might just use the proxy_redirect directive:
location /admin/ {
proxy_pass http://localhost:8080/;
proxy_redirect / /admin/
}
It changes the Location-Header of the response according to the given 'match-rewrite' rule.
Let's say we have the following quite minimal nginx.conf:
server {
listen 443 default ssl;
location /api/v1 {
proxy_pass http://127.0.0.1:8080;
}
}
Now, I'm trying to use nginx itself as an event-source. Another component in my system should be aware of any HTTP requests coming in, while ideally not blocking the traffic on this first proxy_pass directive.
Is there any possibility to have a second proxy_pass which "just" forwards the HTTP request to another component as well while completely ignoring the result of that forwarded request?
Edit: To state the requirement more clearly: What I want to achieve is that the same HTTP requests are sent to two different backend servers, only one of them really handling the connection in terms of nginx. The other should just be an "event ping" to notify the other service that there has been a request.
This can be done using echo_location directive (or similar, browse the directives) of the 3rd party Nginx Echo Module. You will need to compile Nginx with this module or use Openresty which is Nginx bundled with useful stuff such as this.
Outline code:
server {
[...]
location /main {
echo_location /sub;
proxy_pass http://main.server:PORT;
}
location /sub {
internal;
proxy_pass http://alt.server:PORT;
}
}
There is also the now undocumented post_action directive which does not require a third party module:
server {
[...]
location /main {
proxy_pass http://main.server:PORT;
post_action #sub;
}
location #sub {
proxy_pass http://alt.server:PORT;
}
}
This will fire a subrequest after the main request is completed. Here is an old answer where I recommended the use of this: NGinx - Count requests for a particular URL pattern.
However, this directive has been removed from the Nginx documentation and further usage of this is now a case of caveat emptor. Four years on from 2012 when I gave that answer, I wouldn't recommend using this.
I know this is done but I'd like to answer with the new, updated answer since it's turning up in searches 3 years later. The mirror module works wonderfully. I got this from the nginx docs so I assume it's official and available.
server {
[...]
location /main {
mirror /mirror
proxy_pass http://main.server:PORT;
}
location /mirror {
proxy_pass http://alt.server:PORT;
}
}
It's the big advantage of nginx: you can serve from multiple backend servers. You must just include one more location, referenced by another direction. Here you got a sample of my sites-available/default, that servers from fastcgi (monodevelop), glassfish (java), static content & special error threatment. Hope it helps.
#fastcgi
location ~* \.(aspx)$ {
root /home/published/;
index Default.aspx;
fastcgi_index Default.aspx;
fastcgi_pass 127.0.0.1:9000;
include /etc/nginx/fastcgi_params;
}
#Glassfish
location /GameFactoryService/ {
index index.html;
add_header Access-Control-Allow-Origin $http_origin;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-NginX-Proxy true;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:18000/GameFactoryService/;
}
#static content
location / {
root /usr/share/nginx_static_content;
}
error_page 500 501 502 503 504 505 506 507 508 509 510 511 /50x.html;
#error
location = /50x.html {
add_header Access-Control-Allow-Origin $http_origin;
internal;
}
I'm used to using Apache with mod_proxy_html, and am trying to achieve something similar with NGINX. The specific use case is that I have an admin UI running in Tomcat on port 8080 on a server at the root context:
http://localhost:8080/
I need to surface this on port 80, but I have other contexts on the NGINX server running on this host, so want to try and access this at:
http://localhost:80/admin/
I was hoping that the following super simple server block would do it, but it doesn't quite:
server {
listen 80;
server_name screenly.local.akana.com;
location /admin/ {
proxy_pass http://localhost:8080/;
}
}
The problem is that the returned content (html) contains URLs to scripts and style info that is all accessed at the root context, so I need to get these URLs rewritten to start with /admin/ instead of /.
How do I do this in NGINX?
We should first read the documentation on proxy_pass carefully and fully.
The URI passed to upstream server is determined based on whether "proxy_pass" directive is used with URI or not. Trailing slash in proxy_pass directive means that URI is present and equal to /. Absense of trailing slash means hat URI is absent.
Proxy_pass with URI:
location /some_dir/ {
proxy_pass http://some_server/;
}
With the above, there's the following proxy:
http:// your_server/some_dir/ some_subdir/some_file ->
http:// some_server/ some_subdir/some_file
Basically, /some_dir/ gets replaced by / to change the request path from /some_dir/some_subdir/some_file to /some_subdir/some_file.
Proxy_pass without URI:
location /some_dir/ {
proxy_pass http://some_server;
}
With the second (no trailing slash): the proxy goes like this:
http:// your_server /some_dir/some_subdir/some_file ->
http:// some_server /some_dir/some_subdir/some_file
Basically, the full original request path gets passed on without changes.
So, in your case, it seems you should just drop the trailing slash to get what you want.
Caveat
Note that automatic rewrite only works if you don't use variables in proxy_pass. If you use variables, you should do rewrite yourself:
location /some_dir/ {
rewrite /some_dir/(.*) /$1 break;
proxy_pass $upstream_server;
}
There are other cases where rewrite wouldn't work, that's why reading documentation is a must.
Edit
Reading your question again, it seems I may have missed that you just want to edit the html output.
For that, you can use the sub_filter directive. Something like ...
location /admin/ {
proxy_pass http://localhost:8080/;
sub_filter "http://your_server/" "http://your_server/admin/";
sub_filter_once off;
}
Basically, the string you want to replace and the replacement string
You may also need the following directive to be set before the first "sub_filter" for backend-servers with data compression:
proxy_set_header Accept-Encoding "";
Otherwise it may not work.
For your example it will look like:
location /admin/ {
proxy_pass http://localhost:8080/;
proxy_set_header Accept-Encoding "";
sub_filter "http://your_server/" "http://your_server/admin/";
sub_filter_once off;
}
You can use the following nginx configuration example:
upstream adminhost {
server adminhostname:8080;
}
server {
listen 80;
location ~ ^/admin/(.*)$ {
proxy_pass http://adminhost/$1$is_args$args;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
If there are no hyperlinks which need to be rewritten with sub_filter,
you might just use the proxy_redirect directive:
location /admin/ {
proxy_pass http://localhost:8080/;
proxy_redirect / /admin/
}
It changes the Location-Header of the response according to the given 'match-rewrite' rule.
I'm using nginx in a reverse proxy configuration with apache2. I have used other, preconfigured web servers in this way and enjoyed complete control over redirects from an .htaccess file.
My current configuration does not allow for this. First, I'll explain what happens.
Let's say I want to redirect /google to http://google.com. I add the following line to my .htaccess file.
redirect 307 /google http://google.com
I know it's working because I can test with curl from my server. This is hitting apache directly, behind the proxy.
curl -I localhost:8080/google
and I get a 307 as I would expect.
But if this request hits nginx from the outside, nginx knows there is no such file in the web root and responds 404.
Is there a configuration change I can make to remedy this?
Here's my nginx configuration file for this vhost.
server {
listen 80;
root /var/www/mywebsite.com/html;
index index.php index.html index.htm;
server_name mywebsite.com;
access_log /var/log/nginx/mywebsite.com.access.log;
error_log /var/log/nginx/mywebsite.com.error.log;
location / {
try_files $uri $uri/ /index.php;
}
location ~ \.php$ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
location ~ /\. {
deny all;
}
}
It seems that I might need to take the reverse approach and proxy everything but exclude certain file types from the proxy.
Thanks for advice folks!
If you want to forward that request to /google to apache, then just change the location ~ \.php$ pattern to include the request you desire:
location ~ (/google/?)|(\.php)$ {
On the other hand if you want nginx to handle the redirect you can add new rule:
location ~ /google {
return 307 http://google.com;
}
Just make sure to put it before the location block which contains try_files.
EDIT:
To forward all requests that don't hit a file use this:
location /
{
if (!-f $request_filename) #test if a static file does not exists
{
# forward the request if there is no file to serve:
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}
Additionally if you want for nginx to also serve entire directories then add another condition:
!-d $request_filename
Also note that if you want only specific files to be served then instead location / use the pattern to match those files. For example if you want to serve only jpg's and css's use
location ~ \.(jpg|css)$
EDIT2: Here you have a simplified version of your script, which is also more robust - lets you serve only the types of files you want:
server {
listen 80;
server_name test.lcl;
root /home/www/test;
location / {
try_files $uri $uri/ #apache;
}
location #apache
{
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}