i am trying dynamically set a the proxy_pass destination where the variable would be the cname of the original request.
what i have right now is:
server {
listen 8888;
server_name (.*).domain.com;
location / {
proxy_pass http://$1.otherdomain.com;
proxy_set_header Host $1.otherdomain.com;
but unfortunately this ends up in a 502 bad gateway.
nothing really works when using a variable in proxy_pass and proxy_set_header.
i also tried to use (?<cname>.+) or (?P<cname>.+) in the server name and $cname as the variable.
what is wrong and why does it end up in a 502?
To use regex in server name, you need to prepend the name with a tilde "~"
server_name ~(.*).domain.com;
[UPDATE]
Tried it and it successfully set the value in $1. But still get 502 and my nginx error log shows
no resolver defined to resolve xyz.otherdomain.com
even though I point that name to my localhost in my /etc/hosts file.
Find this article that explains this issue well. Basically in this special case (variable in upstream domain name), you need to use the "resolver" directive to point to a dns server (e.g., 8.8.8.8 from google dns server) that can resolve this dynamic domain name.
resolver 8.8.8.8;
It works in my test with a public upstream domain name. If you upstream domain names are local, you need to set up a local dns server for them.
The server name for proxy_pass using variables, will be a special situation.
proxy_pass http://$1.otherdomain.com;
In this case, the server name is searched among the described server groups, and, if not found, is determined using a resolver.
If you do not like to use resolver. You can use below like hosts file.
upstream www1.otherdomain.com { server 10.x.x.1; }
upstream www2.otherdomain.com { server 10.x.x.2; }
Related
I have this nginx.conf configuration file inherited from a github project and i'd like some people to explain me what is doing what:
upstream hello_django {
server web:8000;
}
server {
listen 80;
server_name react-wagtail-api.accordbox.com;
location / {
proxy_pass http://hello_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
client_max_body_size 20M;
}
location /static/ {
alias /app/static/;
}
location /media/ {
alias /app/media/;
}
}
server {
listen 80;
server_name react-wagtail.accordbox.com;
location / {
root /usr/share/nginx/html/build;
index index.html index.htm;
try_files $uri $uri/ /index.html;
}
}
with
upstream hello_django {
server web:8000;
}
is web a service (elswhere there is a docker-compose container name which is web... is it a ref to that? ) ?
What does upstream define exactly?
with
server_name react-wagtail-api.accordbox.com;
what happens if i dont define a server_name for example in case i dont have yet a domain? is server_name the domain typed in the adress bar of the browser? can i define it as the local ip and let my domain name provider do the redirect? can i define it as the internet ip of the server and let my domain name provider do the redirect?
As there are two servers on the same port, can i define for example server_name my_internet_ip/app1 and server_name my_internet_ip/app2 to serve two servers on port 80?
Is web a service (elsewhere there is a docker-compose container name which is web... is it a ref to that?)
Generally, web here is an upstream domain name (can be also specified via IP address or UNIX socket path). However, when this is executed within the docker-compose context, at the nginx startup time it will be resolved to the web container internal IP using docker internal domain name resolving system.
What does upstream define exactly?
For this particular configuration, there will be no difference using either
upstream hello_django {
server web:8000;
}
server {
...
proxy_pass http://hello_django;
or specifying an upstream address directly in the proxy_pass directive:
server {
...
proxy_pass http://web:8000;
Really useful upstream use cases include failover (example) or load balancing (example). Read the ngx_http_upstream_module documentation to find out all the available features.
What happens if i don't define a server_name for example in case i don't have yet a domain?
To understand this part make sure you read the following two chapters from the official documentation:
How nginx processes a request
Server names
You can omit the server_name directive from the server block at all. For any HTTP request arriving at the TCP port where nginx is listening, one of the defined server blocks (usually the very first one appearing in the configuration, unless being specified explicitly using default_server parameter of listen directive) will act as the default server if more appropriate server block won't be found. The Host HTTP request header value is used to choose the most suitable server block here, being compared with the specified server_name for the server block, and it will be exactly what you typed at the browser address bar (assuming IP address/domain name being typed will actually point to the nginx server). That means there is no sense to specify the same server name for different server blocks listening on the same TCP ports - the first one will always be chosen to process such a request, and the nginx will complain with the
nginx: [warn] conflicting server name "..." on 0.0.0.0:80, ignored
warning message. For the developing purposes you can add the required domain names to the hosts system file, pointing them to your local machine, e.g.
127.0.0.1 react-wagtail.accordbox.com
127.0.0.1 react-wagtail-api.accordbox.com
This way you will be able to use those domains from your local browser, the generated HTTP requests will contain the proper Host header and will be processed with your local nginx server instance.
As there are two servers on the same port, can i define for example server_name my_internet_ip/app1 and server_name my_internet_ip/app2 to serve two servers on port 80?
No. Looks like you don't understand the internals of HTTP protocol. At the low level HTTP request will be something like
GET /app1 HTTP/1.1
Host: my_internet_ip
...
As you can see the host name and the request URL path are two completely different things. Usually this kind of tasks being solved using several location blocks:
server {
server_name example.com;
location /app1/ {
...
}
location /app2/ {
...
}
}
However it will require support from the underlying web apps. Two available options are
Referring assets using relative URIs, e.g.
<link rel="stylesheet" type="text/css" href="style.css">
or
<link rel="stylesheet" type="text/css" href="./style.css">
but not the using absolute URIs like
<link rel="stylesheet" type="text/css" href="/style.css">
Using the same URI prefix as specified in the location directive, e.g.
<link rel="stylesheet" type="text/css" href="/app1/style.css">
The reason should be obvious - every request for any first or second app assets should start with the /app1/ or /app2/ prefix for being handled with the proper location block.
As you can see, I'm using /app1/ and /app2/ suffixes here rather than /app1 and /app2. This is a very common mistake made by many people, to not understand the difference between those two. While it may seems to be an insignificant, in fact those two are drastically different in terms of browsing context which will be / in first case and /app1/ or /app2/ in second case. Unless you are proxying some kind of API endpoint but a whole web application, you probably want the second one. To make it more easy, a location directive special behavior provided by nginx:
If a location is defined by a prefix string that ends with the slash character, and requests are processed by one of proxy_pass, fastcgi_pass, uwsgi_pass, scgi_pass, memcached_pass, or grpc_pass, then the special processing is performed. In response to a request with URI equal to this string, but without the trailing slash, a permanent redirect with the code 301 will be returned to the requested URI with the slash appended. If this is not desired, an exact match of the URI and location could be defined like this:
location /user/ {
proxy_pass http://user.example.com;
}
location = /user {
proxy_pass http://login.example.com;
}
Although we can override the browsing context using the <base href="..."> HTML tag, I strongly recommend to not use this workaround and use a proper URI prefixes for your proxied web apps instead.
web service in your case resolves to some IP, same with server_name. You can read more about upstreams here, same for server_name directive.
Hi Earlier I was matching location prefix uri matching to get the redirection to website www.example.com/abc/ from www.example.com/bbb/abc(requested uri).
server {
location /bbb/ {
proxy_pass http://www.example.com/;
}
}
Earlier I was requesting on www.example.com/bbb/abc and getting redirected to www.example.com/abc
as I required
However now the problem is I've resolver and I'm also using variable. This is done to ensure that nginx starts even if one of the upstream services are down and it won't show the error of
[nginx] host upstream not found
server {
location /bbb/ {
resolver 127.0.0.11 valid=some secs;
set $example www.example.com
proxy_pass http://$example/;
}
}
Now that I'm using variable, I think url location prefix match is not working as I am sending request on www.example.com/bbb/abc , and it shows the nginx 404 not found.
Can anyone guide me how redirect properly with variable and resolver included.
I am attempting to get nginx to redirect/rewrite requests to a specific subdirectory so that they are served by an S3-compatible bucket instead of the server. Here is my current server block:
{snip} (See infra.)
Despite fiddling with this for some time now, I've only been able to get it to return 404s.
Additional Information
https://omnifora.com/t/redirect-rewrite-in-nginx-requests-to-subdirectory-to-s3-compatible-bucket/402
Attempts Solutions So Far
rewrite
rewrite ^/security-now/(.*) $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
return
return 302 $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
proxy_pass
proxy_set_header Host s3.us-west-1.wasabisys.com;
proxy_pass $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
You can't use rewrite for cross-domain redirections, for this case you must use proxy_pass, for example:
location ~ ^/directory1/(.*) {
proxy_set_header Host s3.us-west-1.wasabisys.com;
proxy_pass $scheme://s3.us-west-1.wasabisys.com/target-bucket/security-now/$1;
}
Note that if you specify your server with domain name instead or IP address, you'll need to specify additional parameter resolver in your server configuration block, for example:
server {
...
resolver 8.8.8.8;
...
}
Update.
It seems I was wrong stating that you can't use rewrite for cross-domain redirections. You can, but in this case your user got HTTP 301 redirect instead of "transparent" content delivery. Maybe you got 404 error because you missed a $ sign before scheme variable?
I'm running Nginx on Kubernetes.
When I use the following proxy_pass directive it works as expected:
proxy_pass "http://service-1.default";
However the following does not work:
set $service "service-1";
proxy_pass "http://$service.default";
I get an error saying no resolver defined to resolve service-1.default
As far as I can tell proxy_pass is receiving the exact same string so why is it behaving differently?
I need to use a variable because I'm dynamically getting the service name from the URL using a regex.
I've found the reason and a solution.
Nginx detects if a variable is being used in proxy_pass (I don't know how it does that). If there is no variable it resolved the hostname at startup and caches the IP address. If there is a variable it uses a resolver (DNS server) to lookup the IP at runtime.
So the solution is to specify the Kube DNS server like this:
resolver kube-dns.kube-system.svc.cluster.local valid=5s;
set $service "service-1";
proxy_pass "http://$service.default.svc.cluster.local";
Note that the full local DNS name of the service must be used which you can get by running nslookup service-1.
we have two servers, A and B. Server A is accessed worldwide. He has nginx installed. That's what I have in conf:
location /test {
proxy_pass http://localserver.com;
}
What it should do, is translate the addreess http://globalserver.com/test (that is server A) to internal server address http://localserver.com. However, it does append the location path, that is, itries to look for http://localserver.com/test, which is not available at all. How can I make the proxy pass to the correct address, throwing out the last part in the location?
That should work. Nginx should strip the '/test' path on the upstream local server. So what I can say is that is not the cause. To make it a bit better, try this:
location /test/ {
proxy_pass http://localserver.com/;
}
The 2 slashes I added at the first 2 lines will avoid mistakenly match '/testABC' and send the wrong request to the upstream local server, for example.
Do you have a
proxy_redirect
line in the same location block? If your upstream local server has redirects, then a mistake on that line will cause an issue like you described.
[UPDATE] Found the root cause why the original config didn't work and mine works: nginx does NOT replace URI path part if the proxy_pass directive does not have a URI path itself. So my fix of adding a slash (slash is treated as a URI path) at the end triggers the URI path replacement.
Reference: http://wiki.nginx.org/HttpProxyModule#proxy_pass
If it is necessary to transmit URI in the unprocessed form then directive proxy_pass should be used without URI part
location /some/path/ {
proxy_pass http://127.0.0.1;
}
try to add as specified here http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass:
proxy_pass http://localserver.com/;
try rewrite
location /test {
rewrite ^ $scheme://$host/;
proxy_pass http://localserver.com;
}
some helpful links...
http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite
http://wiki.nginx.org/Pitfalls