nginx configuration file : server, server_name and upstream understanding - nginx

I have this nginx.conf configuration file inherited from a github project and i'd like some people to explain me what is doing what:
upstream hello_django {
server web:8000;
}
server {
listen 80;
server_name react-wagtail-api.accordbox.com;
location / {
proxy_pass http://hello_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
client_max_body_size 20M;
}
location /static/ {
alias /app/static/;
}
location /media/ {
alias /app/media/;
}
}
server {
listen 80;
server_name react-wagtail.accordbox.com;
location / {
root /usr/share/nginx/html/build;
index index.html index.htm;
try_files $uri $uri/ /index.html;
}
}
with
upstream hello_django {
server web:8000;
}
is web a service (elswhere there is a docker-compose container name which is web... is it a ref to that? ) ?
What does upstream define exactly?
with
server_name react-wagtail-api.accordbox.com;
what happens if i dont define a server_name for example in case i dont have yet a domain? is server_name the domain typed in the adress bar of the browser? can i define it as the local ip and let my domain name provider do the redirect? can i define it as the internet ip of the server and let my domain name provider do the redirect?
As there are two servers on the same port, can i define for example server_name my_internet_ip/app1 and server_name my_internet_ip/app2 to serve two servers on port 80?

Is web a service (elsewhere there is a docker-compose container name which is web... is it a ref to that?)
Generally, web here is an upstream domain name (can be also specified via IP address or UNIX socket path). However, when this is executed within the docker-compose context, at the nginx startup time it will be resolved to the web container internal IP using docker internal domain name resolving system.
What does upstream define exactly?
For this particular configuration, there will be no difference using either
upstream hello_django {
server web:8000;
}
server {
...
proxy_pass http://hello_django;
or specifying an upstream address directly in the proxy_pass directive:
server {
...
proxy_pass http://web:8000;
Really useful upstream use cases include failover (example) or load balancing (example). Read the ngx_http_upstream_module documentation to find out all the available features.
What happens if i don't define a server_name for example in case i don't have yet a domain?
To understand this part make sure you read the following two chapters from the official documentation:
How nginx processes a request
Server names
You can omit the server_name directive from the server block at all. For any HTTP request arriving at the TCP port where nginx is listening, one of the defined server blocks (usually the very first one appearing in the configuration, unless being specified explicitly using default_server parameter of listen directive) will act as the default server if more appropriate server block won't be found. The Host HTTP request header value is used to choose the most suitable server block here, being compared with the specified server_name for the server block, and it will be exactly what you typed at the browser address bar (assuming IP address/domain name being typed will actually point to the nginx server). That means there is no sense to specify the same server name for different server blocks listening on the same TCP ports - the first one will always be chosen to process such a request, and the nginx will complain with the
nginx: [warn] conflicting server name "..." on 0.0.0.0:80, ignored
warning message. For the developing purposes you can add the required domain names to the hosts system file, pointing them to your local machine, e.g.
127.0.0.1 react-wagtail.accordbox.com
127.0.0.1 react-wagtail-api.accordbox.com
This way you will be able to use those domains from your local browser, the generated HTTP requests will contain the proper Host header and will be processed with your local nginx server instance.
As there are two servers on the same port, can i define for example server_name my_internet_ip/app1 and server_name my_internet_ip/app2 to serve two servers on port 80?
No. Looks like you don't understand the internals of HTTP protocol. At the low level HTTP request will be something like
GET /app1 HTTP/1.1
Host: my_internet_ip
...
As you can see the host name and the request URL path are two completely different things. Usually this kind of tasks being solved using several location blocks:
server {
server_name example.com;
location /app1/ {
...
}
location /app2/ {
...
}
}
However it will require support from the underlying web apps. Two available options are
Referring assets using relative URIs, e.g.
<link rel="stylesheet" type="text/css" href="style.css">
or
<link rel="stylesheet" type="text/css" href="./style.css">
but not the using absolute URIs like
<link rel="stylesheet" type="text/css" href="/style.css">
Using the same URI prefix as specified in the location directive, e.g.
<link rel="stylesheet" type="text/css" href="/app1/style.css">
The reason should be obvious - every request for any first or second app assets should start with the /app1/ or /app2/ prefix for being handled with the proper location block.
As you can see, I'm using /app1/ and /app2/ suffixes here rather than /app1 and /app2. This is a very common mistake made by many people, to not understand the difference between those two. While it may seems to be an insignificant, in fact those two are drastically different in terms of browsing context which will be / in first case and /app1/ or /app2/ in second case. Unless you are proxying some kind of API endpoint but a whole web application, you probably want the second one. To make it more easy, a location directive special behavior provided by nginx:
If a location is defined by a prefix string that ends with the slash character, and requests are processed by one of proxy_pass, fastcgi_pass, uwsgi_pass, scgi_pass, memcached_pass, or grpc_pass, then the special processing is performed. In response to a request with URI equal to this string, but without the trailing slash, a permanent redirect with the code 301 will be returned to the requested URI with the slash appended. If this is not desired, an exact match of the URI and location could be defined like this:
location /user/ {
proxy_pass http://user.example.com;
}
location = /user {
proxy_pass http://login.example.com;
}
Although we can override the browsing context using the <base href="..."> HTML tag, I strongly recommend to not use this workaround and use a proper URI prefixes for your proxied web apps instead.

web service in your case resolves to some IP, same with server_name. You can read more about upstreams here, same for server_name directive.

Related

Use nginx upstreams module with a mix of SSL and non-SSL, with and without path

I want to create a reverse proxy which has 2 upstreams: a primary and a fallback one.
However the primary upstream talks http while the fallback upstream talks https.
Also, the fallback one has an api key as path component.
In theory I want something like this:
upstream backends {
server primary.mydomain.dev;
server fallback.notmydomain.com/v1/12345 ssl fallback;
}
server {
listen 80;
server_name proxy.example.com;
location / {
proxy_pass http://backends;
...
}
}
}
However my understanding is that I can't have URL paths in the upstreams module.
I could add the path to the proxy_pass directive, but then it would be applied regardless of the chosen upstream server.
Is what I want even possible with nginx?
If not, would it be possible with haproxy? How?

Nginx rewrite sends traffic to IP address, not URL

I have nginx set up as a reverse proxy for a docker microservice. There's a location block that rewrites the url from /wrong to /right:
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
server_name example.com;
location /right {
proxy_pass http://microservice_servers;
}
location /wrong {
rewrite ^/wrong/(\w+) /right/$1 redirect;
}
}
What this is intended to do is rewrite the url from https://example.com/wrong/otherstuff to https://example.com/right/otherstuff.
What actually happens though, is that it rewrites to http://<ip_address>/right/otherstuff.
(One possible complicating factor is that I don't have control of the certs for this site. Those are controlled by the client, who puts them on an app gateway in front of our server. So my nginx config only handles http traffic at port 80, no https from 443. I'm not sure if that's actually relevant, but just in case, there it is.)
I've tried an assortment of changes to the rewrite block, including adding $server_name, changing the flag to last (returns the right content but doesn't change the url), and changing the flag to break (which does not return the expected content).
Any idea what's going on here?
By default, your rewrite...redirect statement will generate a 302 response with the full URL specified in an HTTP Location response header.
You can confirm this using curl -I https://example.com/wrong/otherstuff.
Nginx fills in the protocol and domain name, based on the original request. This server block receives requests over http and we can infer from your question that the Host header uses its IP address.
You either need to specify the full URL in the rewrite statement:
rewrite ^/wrong/(\w+) https://example.com/right/$1 redirect;
Alternatively, use relative URLs:
absolute_redirect off;
See this document for details.

Nginx Reverse Proxy non listening domain catch all

Using nginx as a reverse proxy I have noted that if a site is pointed at the server but does not actually exist you get served with an existing site on the server rather than a domain cannot be found message or similar.
ie say your config block is listening for portal.test.org the user points there cname to the nginx system but with the domain header of smartportal.test.org
or say the they point a site like portal.test2.org
Could someone direct me on the following
a. How nginx decides which site to return when it does not exist as a virtual host
b. How to catch any domain that are not specified and return a default page that tells users this domain does not exist on this host
Hope someone can help I have tried googling but I think sometimes its knowing the correct terms.
Many thanks
you should use default_server parameter of listen directive and server_name directive.
your questions
How nginx decides which site to return when it does not exist as a virtual host
listen [default_server]
If the directive has the default_server parameter, then the enclosing
server {...} block will be the default server for the address:port
pair. This is useful for name-based virtual hosting where you wish to
specify the default server block for hostnames that do not match any
server_name directives. If there are no directives with the
default_server parameter, then the default server will be the first
server block in which the address:port pair appears.
server_name
This directive performs two actions:
Compares the Host header of the incoming HTTP request against the
server { ... } blocks in the Nginx configuration files and selects the
first one that matches. This is how virtual servers are defined.
Server names are processed in the following order:
1.- full, static names
2.- names with a wildcard at the start of the name — *.example.com
3.- names with a wildcard at the end of the name — www.example.*
4.- names with regular expressions
If there is no match, a server { ... } block in
the configuration file will be used based on the following order:
1.- the
server block with a matching listen directive marked as
[default|default_server]
2.- the first server block with a matching listen
directive (or implicit listen 80;)
How to catch any domain that are not specified and return a default page that tells users this domain does not exist on this host
server {
listen 1.2.3.4:80 default_server;
server_name _ "";
location / {
alias /var/www/html/domain-does-not-exists.html;
}
}
server {
listen 1.2.3.4:80 ;
server_name portal.test.org;
location / {
... your site ...
}
}

nginx variables (cname) in proxy_pass

i am trying dynamically set a the proxy_pass destination where the variable would be the cname of the original request.
what i have right now is:
server {
listen 8888;
server_name (.*).domain.com;
location / {
proxy_pass http://$1.otherdomain.com;
proxy_set_header Host $1.otherdomain.com;
but unfortunately this ends up in a 502 bad gateway.
nothing really works when using a variable in proxy_pass and proxy_set_header.
i also tried to use (?<cname>.+) or (?P<cname>.+) in the server name and $cname as the variable.
what is wrong and why does it end up in a 502?
To use regex in server name, you need to prepend the name with a tilde "~"
server_name ~(.*).domain.com;
[UPDATE]
Tried it and it successfully set the value in $1. But still get 502 and my nginx error log shows
no resolver defined to resolve xyz.otherdomain.com
even though I point that name to my localhost in my /etc/hosts file.
Find this article that explains this issue well. Basically in this special case (variable in upstream domain name), you need to use the "resolver" directive to point to a dns server (e.g., 8.8.8.8 from google dns server) that can resolve this dynamic domain name.
resolver 8.8.8.8;
It works in my test with a public upstream domain name. If you upstream domain names are local, you need to set up a local dns server for them.
The server name for proxy_pass using variables, will be a special situation.
proxy_pass http://$1.otherdomain.com;
In this case, the server name is searched among the described server groups, and, if not found, is determined using a resolver.
If you do not like to use resolver. You can use below like hosts file.
upstream www1.otherdomain.com { server 10.x.x.1; }
upstream www2.otherdomain.com { server 10.x.x.2; }

Struggling with location blocks in nginx config

I got a new slice off slicehost, for the purposes of playing around and learning nginx and more about deployment generally. I installed a ruby app on there (which i'll call app1) which uses passenger. I made it the default app to use for that server with the following server block in my nginx config:
server {
listen 80;
server_name <my server ip>;
root <path to app1 public folder>;
passenger_enabled on;
}
This works fine. However, i want to try a few different apps out on this slice, and so thought i would set it up like so:
http:///app1
http:///app2
etc. I thought i would be able to do that by adding a location block, and moving the app1 specific stuff into it like so:
server {
listen 80;
server_name <my server ip>;
location ^~ /app1 {
root <path to app1 public folder>;
passenger_enabled on;
}
}
However, on doing this (and restarting nginx of course), going to the plain ip address gives the 'welcome to nginx' message (which i'd expect). But, going to /app1 gives an error message:
404 Not Found
The requested URL /app1 was not found on this server.
This is distinct from the error message i get when i go to another path on that ip, eg /foo:
404 Not Found
nginx/0.8.53
So, it's like nginx knows about that location but i've not set it up properly. Can anyone set me straight? Should i set up different server blocks instead of using locations? I'm sure this is simple but can't work it out.
Cheers, max
What you're after is name virtual hosting. The idea is that each domain is hosted on the same IP, and nginx chooses the virtualhost to serve based on the Host: header in the HTTP request, which is sent by the browser.
To use name virtual hosting, use the domain you want to serve instead of your server's IP for the server_name directive.
server {
listen 80;
server_name app1.com;
location / {
root /srv/http/app1/public;
passenger_enabled on;
}
}
Then, to host more apps on the same box, just declare a separate server { } block for each one.
server {
listen 80;
server_name app2.com;
location / {
root /srv/http/app2/public;
passenger_enabled on;
}
}
I'm using unicorn instead of passenger, but the vhost part of the structure is the same for any backend.
The global nginx config (which on its own hosts nothing): https://github.com/benhoskings/babushka-deps/blob/master/nginx/nginx.conf.erb
The template wrapper for each virtualhost: https://github.com/benhoskings/babushka-deps/blob/master/nginx/vhost.conf.erb
The details of the unicorn virtualhost: https://github.com/benhoskings/babushka-deps/blob/master/nginx/unicorn_vhost.common.erb
I fail to see the real problem here tho,
in order for you to figure that out
you need to view the nginx log files on most systems at:
/var/log/nginx/
and open the relevant access file here(might be error.log)
in there you can see what url nginx exactly tried to access and why did it fail.
What I really think is happening, that you got the root path wrong,
maybe it should be alias instead because
if you are proxifying the connection to another app, it might get the
"app1" word in the url instead of a direct one.
so please try:
server {
listen 80;
server_name <my server ip>;
location /app1 {
alias <path to app1 public folder>;
passenger_enabled on;
}
}
and see weather it works and also try to view the logs first to really determine whats the problem.
I think its just a slight syntax problem:
location ~ ^/app1 { ...
should work, or a little more efficient:
location = /app1 { ...
One problem is that your Rails app probably wasn't designed to run from a subdirectory. Passenger has a directive that will fix this:
passenger_base_uri /app1;
However, running Rails apps in subdirectories is somewhat non-standard. If you can, a better option may be to set up subdomains using nginx's virtual hosts.
It seems that you want to host more apps on the same server with base uri. Try this:
root /srv/http/;
passenger_base_uri /app_1;
passenger_base_uri /app_2
Also under /srv/http, create 2 symlinks:
ln -s /srv/http/app_1 /srv/http/app1/public
ln -s /srv/http/app_2 /srv/http/app2/public
The app1 can be accessed under: http://domain.com/app_1.
Here is more for reading: http://www.modrails.com/documentation/Users%20guide%20Nginx.html#deploying_rack_to_sub_uri

Resources