Configure nginx proxy_pass with two parallel locations - nginx

Let's say we have the following quite minimal nginx.conf:
server {
listen 443 default ssl;
location /api/v1 {
proxy_pass http://127.0.0.1:8080;
}
}
Now, I'm trying to use nginx itself as an event-source. Another component in my system should be aware of any HTTP requests coming in, while ideally not blocking the traffic on this first proxy_pass directive.
Is there any possibility to have a second proxy_pass which "just" forwards the HTTP request to another component as well while completely ignoring the result of that forwarded request?
Edit: To state the requirement more clearly: What I want to achieve is that the same HTTP requests are sent to two different backend servers, only one of them really handling the connection in terms of nginx. The other should just be an "event ping" to notify the other service that there has been a request.

This can be done using echo_location directive (or similar, browse the directives) of the 3rd party Nginx Echo Module. You will need to compile Nginx with this module or use Openresty which is Nginx bundled with useful stuff such as this.
Outline code:
server {
[...]
location /main {
echo_location /sub;
proxy_pass http://main.server:PORT;
}
location /sub {
internal;
proxy_pass http://alt.server:PORT;
}
}
There is also the now undocumented post_action directive which does not require a third party module:
server {
[...]
location /main {
proxy_pass http://main.server:PORT;
post_action #sub;
}
location #sub {
proxy_pass http://alt.server:PORT;
}
}
This will fire a subrequest after the main request is completed. Here is an old answer where I recommended the use of this: NGinx - Count requests for a particular URL pattern.
However, this directive has been removed from the Nginx documentation and further usage of this is now a case of caveat emptor. Four years on from 2012 when I gave that answer, I wouldn't recommend using this.

I know this is done but I'd like to answer with the new, updated answer since it's turning up in searches 3 years later. The mirror module works wonderfully. I got this from the nginx docs so I assume it's official and available.
server {
[...]
location /main {
mirror /mirror
proxy_pass http://main.server:PORT;
}
location /mirror {
proxy_pass http://alt.server:PORT;
}
}

It's the big advantage of nginx: you can serve from multiple backend servers. You must just include one more location, referenced by another direction. Here you got a sample of my sites-available/default, that servers from fastcgi (monodevelop), glassfish (java), static content & special error threatment. Hope it helps.
#fastcgi
location ~* \.(aspx)$ {
root /home/published/;
index Default.aspx;
fastcgi_index Default.aspx;
fastcgi_pass 127.0.0.1:9000;
include /etc/nginx/fastcgi_params;
}
#Glassfish
location /GameFactoryService/ {
index index.html;
add_header Access-Control-Allow-Origin $http_origin;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-NginX-Proxy true;
proxy_ssl_session_reuse off;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:18000/GameFactoryService/;
}
#static content
location / {
root /usr/share/nginx_static_content;
}
error_page 500 501 502 503 504 505 506 507 508 509 510 511 /50x.html;
#error
location = /50x.html {
add_header Access-Control-Allow-Origin $http_origin;
internal;
}

Related

Sharing location configuration in nginx

I could not decide the best name for the question.
Essentially what I want to achieve is to set a custom allowed body size for a specific location on the webserver.
On the other hand, I was able to achieve the necessary result already with duplicate code, so I am really looking for a way how to make the code reusable and to better understand the observed behavior.
The server reverse-proxies all API requests to the backend service.
In global nginx config /etc/nginx/nginx.conf I set the rule for max allowed body size like so client_max_body_size 50k;.
Then, in individual server config /etc/nginx/conf.d/example.com I have the following config (simplified):
server {
listen 80;
listen [::]:80;
server_name api.example.com www.api.example.com
location ~* /file/upload {
client_max_body_size 100M;
# crashes without this line
proxy_pass http://localhost:90;
#proxy_pass http://localhost:90/file/upload; # also works
}
location / {
# does not work
#location ~* /file/upload {
# client_max_body_size 100M;
#}
proxy_pass http://localhost:90;
}
}
I am trying to override the max body size for file upload endpoint. See that there is 1 proxy_pass for location /file/upload and another proxy_pass for location / pointing to the same internal service.
Question 1. If I remove the proxy_pass from the location /file/upload then error is returned by the server. (no status code in chrome debugger). Why is this happening? Shouldn't request be propagated further to location /?
Question 2. Why is it not possible to define the sublocation with body size override inside the / location as in commented section in example above? If I set it like this, then 413 error code is returned, which hints that the client_max_body_size rule is ignored..
Question 3. Finally, is it possible to tell nginx, after the request hits the /file/upload location - to apply all the rules from the / section? I guess one solution to this problem would be to move the common configuration into separate file and then import it in both sections.. I was thinking if there is any solution that does not require creating new files?
Here is the reusable config I am talking about basically:
location / {
#.s. kill cache. use in dev
sendfile off;
# kill cache
add_header Last-Modified $date_gmt;
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
if_modified_since off;
expires off;
etag off;
# don't cache it
proxy_no_cache 1;
# even if cached, don't try to use it
proxy_cache_bypass 1;
proxy_pass http://localhost:90;
client_max_body_size 100M;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass_request_headers on;
}
This is not the final version. If I had to copy this piece of code to 2 sections this would not be very friendly approach. So, it would be nice to hear some nginx lifehacks on how accomplish what I try to accomplish in a most friendly way and get some explanations for observed behavior.
Answer 1
If I remove the proxy_pass from the location /file/upload then error is returned by the server. (no status code in chrome debugger). Why is this happening?
Every location have a so-called content handler. If you don't specify content handler explicitly via proxy_pass (fastcgi_pass, uwsgi_pass, etc.) directive, nginx will try to serve the request locally.
Shouldn't request be propagated further to location /?
Of course not. What makes you think it should?
Answer 2
Why is it not possible to define the sublocation with body size override inside the / location as in commented section in example above? If I set it like this, then 413 error code is returned, which hints that the client_max_body_size rule is ignored..
I'd rather expect you'll get the same error as in the first case since your nested location does not have an explicitly specified content handler via the proxy_pass directive. However the following config is worth to try:
location / {
# all the common configuration
location /file/upload {
client_max_body_size 100M;
proxy_pass http://localhost:90;
}
proxy_pass http://localhost:90;
}
Answer 3
Finally, is it possible to tell nginx, after the request hits the /file/upload location - to apply all the rules from the / section?
No, unless you use a separate file via include directive in both locations. However you can try to move all the upstream related setup directives one level up to the server context:
server {
...
# all the common configuration
location / {
proxy_pass http://localhost:90;
}
location /file/upload {
client_max_body_size 100M;
proxy_pass http://localhost:90;
}
}
Note that some directives (e.g. add_header, proxy_set_header) are inherited from the previous configuration level if and only if there are no those directives defined on the current level.
Very often dynamic settings for different locations can be achieved using the map block in a following way:
map $uri $max_body_size {
~^/file/upload 100M;
default 50k;
}
server {
location / {
...
client_max_body_size $max_body_size;
...
}
}
Unfortunally not every nginx directive accepts variable as its argument. Usually when nginx documentation doesn't explicitly states that some directive can accept variables, it means it cannot, and the client_max_body_size is exactly that kind of directive, so the above configuration won't work.

nginx proxy config not forwarding requests to backend server

Below is the relevant section of my nginx.conf file.
I only see the js|css... requests forward to my backend server when i remove the initial location block in the conf file. What im trying to accomplish is to turn off nginx access logging for files of those extensions.
Anybody know a working nginx config technique to allow me to turn off the access logs yet still forward these requests to the proxy location?
...
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
access_log off;
}
location / {
if ($ignore_ua) {
access_log off;
return 200;
}
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:7777/;
}
nginx chooses a location block to process a request. In the case of .js files, your location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ block is used. None of the directives within the location / block are involved. See this document for details.
If you need conditional logging, you could use the if= parameter to the access_log directive instead of a separate location block. See this document for an example.
In your case, it might look like this:
map $request_uri $loggable {
default 1;
\.(js|css|png|jpg|jpeg|gif|ico)(\?|$) 0;
}
access_log /path/to/access.log combined if=$loggable;
Note that the map directive goes in the http block.

nginx: increase timeout to prevent 404 not found error?

I have a Django server running Gunicorn, and in front of that I have nginx. I serve static files directly from nginx, and pass other things through to Gunicorn.
I have some slow-running back-end queries, and I'm finding that nginx is quite often timing out before they return - so I see a 404 page.
Is there a way I can increase the timeout level?
This is my nginx conf file:
server {
listen 443;
client_max_body_size 4G;
access_log /webapps/myapp/logs/nginx-access.log;
error_log /webapps/myapp/logs/nginx-error.log;
location /media/ {
alias /webapps/myapp/myapp/media/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://hello_app_server;
break;
}
}
I think perhaps I need proxy_read_timeout, but I'm not sure from the docs.
Try
proxy_read_timeout 120s;
Put that inside your proxy section.
The default is apparently 60s so try doubling and go from there.
Not too confident about it but i had something similar with a timeout in mysql today on a server at work and doubling that worked. Worth a try and hope it helps.

Using nginx as reverse proxy for apache and want more htaccess control

I'm using nginx in a reverse proxy configuration with apache2. I have used other, preconfigured web servers in this way and enjoyed complete control over redirects from an .htaccess file.
My current configuration does not allow for this. First, I'll explain what happens.
Let's say I want to redirect /google to http://google.com. I add the following line to my .htaccess file.
redirect 307 /google http://google.com
I know it's working because I can test with curl from my server. This is hitting apache directly, behind the proxy.
curl -I localhost:8080/google
and I get a 307 as I would expect.
But if this request hits nginx from the outside, nginx knows there is no such file in the web root and responds 404.
Is there a configuration change I can make to remedy this?
Here's my nginx configuration file for this vhost.
server {
listen 80;
root /var/www/mywebsite.com/html;
index index.php index.html index.htm;
server_name mywebsite.com;
access_log /var/log/nginx/mywebsite.com.access.log;
error_log /var/log/nginx/mywebsite.com.error.log;
location / {
try_files $uri $uri/ /index.php;
}
location ~ \.php$ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
location ~ /\. {
deny all;
}
}
It seems that I might need to take the reverse approach and proxy everything but exclude certain file types from the proxy.
Thanks for advice folks!
If you want to forward that request to /google to apache, then just change the location ~ \.php$ pattern to include the request you desire:
location ~ (/google/?)|(\.php)$ {
On the other hand if you want nginx to handle the redirect you can add new rule:
location ~ /google {
return 307 http://google.com;
}
Just make sure to put it before the location block which contains try_files.
EDIT:
To forward all requests that don't hit a file use this:
location /
{
if (!-f $request_filename) #test if a static file does not exists
{
# forward the request if there is no file to serve:
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}
Additionally if you want for nginx to also serve entire directories then add another condition:
!-d $request_filename
Also note that if you want only specific files to be served then instead location / use the pattern to match those files. For example if you want to serve only jpg's and css's use
location ~ \.(jpg|css)$
EDIT2: Here you have a simplified version of your script, which is also more robust - lets you serve only the types of files you want:
server {
listen 80;
server_name test.lcl;
root /home/www/test;
location / {
try_files $uri $uri/ #apache;
}
location #apache
{
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
}
}

Failed to make 404 page in tornado with nginx

I tried to make custom 404 page for tornado and want to deploy it with nginx but failed.
here is my domain.conf(included by nginx.conf)
server {
listen 80;
server_name vm.tuzii.me;
client_max_body_size 50M;
location ^~ /app/static/ {
root ~/dev_blog;
if ($query_string) {
expires max;
}
}
location = /favicon.ico {
rewrite (.*) /static/favicon.ico;
}
location = /robots.txt {
rewrite (.*) /static/robots.txt;
}
error_page 404 /404.html;
location /404.html {
root /home/scenk;
internal;
}
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://frontends;
}
}
But after reload nginx, nothing happen. It seems like tornado catch the 404error before nginx.
I have no idea to solve this problem.
PS. I just want to make 404error by nginx. But not rewrite 'write_error' in tornado source.
Environment: Ubtuntu 12.04 Tornado2.4.1 runsite with supervisor by Nginx 4 process.
I ran into the same problem and what you actually need is this set:
proxy_intercept_errors on;
From nginx proxy module documentation:
proxy_intercept_errors
Syntax: proxy_intercept_errors on | off
Default: off
Context: http
This directive decides if nginx will intercept responses with HTTP status codes of 400 and higher.
By default all responses will be sent as-is from the proxied server.
If you set this to on then nginx will intercept status codes that are explicitly handled by an error_page directive. Responses with status codes that do not match an error_page directive will be sent as-is from the proxied server.
Finailly solve this problem. Because
proxy_pass_header Server;
So the real TornadoServer is sent. To hide real server, simply change
proxy_pass_header User-Agent;
That's all.

Resources