I'm running Nginx on Kubernetes.
When I use the following proxy_pass directive it works as expected:
proxy_pass "http://service-1.default";
However the following does not work:
set $service "service-1";
proxy_pass "http://$service.default";
I get an error saying no resolver defined to resolve service-1.default
As far as I can tell proxy_pass is receiving the exact same string so why is it behaving differently?
I need to use a variable because I'm dynamically getting the service name from the URL using a regex.
I've found the reason and a solution.
Nginx detects if a variable is being used in proxy_pass (I don't know how it does that). If there is no variable it resolved the hostname at startup and caches the IP address. If there is a variable it uses a resolver (DNS server) to lookup the IP at runtime.
So the solution is to specify the Kube DNS server like this:
resolver kube-dns.kube-system.svc.cluster.local valid=5s;
set $service "service-1";
proxy_pass "http://$service.default.svc.cluster.local";
Note that the full local DNS name of the service must be used which you can get by running nslookup service-1.
Related
i tried to forward traffic from server 192.168.243.71 to domain that show in command "oc get routes" / "kubectl get ingress", but its not as simple as that. The fact is my Nginx Reverse Proxy in server 192.168.243.x will forward the request to the IP Address of loadbalancer instead of the real domain that i wrote in nginx.conf
the result
I was expecting it will show the same result when I access the domain via web browser that show in "oc get routes" or "kubectl get ingress"
Solved by adding set $backend mydomainname.com in server block and add dns resolver resolver 192.168.45.213; proxy_pass http://$backend; server in location block.
Result
You can actually add the set $backend mydomainname.com on the server block, and also you need to add dns resolver resolver 192.168.45.213; proxy_pass http://$backend; server in the location of block
Why does a variable not work in proxy_pass?
This works perfectly:
location /foo/ {
proxy_pass http://127.0.0.1/;
}
This doesn't work at all:
location /foo/ {
set $FOO http://127.0.0.1/;
proxy_pass $FOO;
add_header x-debug $FOO;
}
I see the x-header: http://127.0.0.1/ but the result is 404 so I don't know where it's proxying to but it's not identical to the first example.
Source where it is explained that using a variable in proxy_pass will prevent NGINX startup errors when the upstream is not available.
UPDATE: The issue is the upstream path rewriting. I expect it to rewrite /foo/blah to the upstream at /blah removing the /foo prefix. It works fine with static host/uri entries but not with a variable.
The final answer, much aided by #MSalters was more complicated than I could imagine. The reason is that NGINX works differently with variables than with statically entered hostnames - it does not even use the same DNS mechanism.
The main issue is that path handling and prefix stripping does not work the same with variables. You have to strip path prefixes yourself. In my original example:
location /foo/ {
set $FOO 127.0.0.1;
rewrite /foo/(.*) /$1 break;
proxy_pass http://$FOO/$1$is_args$args;
}
In my example I use an IP address so no resolver is required. However, if you use a host name a resolver is required so add your DNS IP there. Shrugs.
For full disclosure, we are using NGINX inside Kubernetes so it gets even more complicated. The special points of interest are:
Add a resolver directive with the IP of the cluster's DNS service (in my case 10.43.0.10). This is the ClusterIP of the kube-dns service in the kube-system namespace.
Use a FQDN even if your NGINX is in the same namespace since the DNS can only resolve FQDN apparently.
location /foo/ {
set $MYSERVICE myservice.mynamespace.svc.cluster.local;
rewrite /foo/(.*) /$1 break;
proxy_pass http://$MYSERVICE/$1$is_args$args;
resolver 10.43.0.10 valid=10s;
}
NOTE: Due to a BUG (which is unfortunately not acknowledged by NGINX maintainers) in NGINX, using $1 in URLs will break if the path contains a space. So /foo%20bar/ will be passed upstream as /foo bar/ and just break.
The idea of variables in nginx.conf is to delay evaluation. An nginx.conf configuration is parsed for two different reasons: on startup, and later when a request is made. In the second parse, request-dependent variables are filled in.
This parser is not very smart, and this behavior is not officially specified. The trick in your link is a proper hack.
The work-around I use in production is proxy-pass http://$FOO. The concatenation of the literal string http and the variable $FOO does happen when a request is made, showing that variables do work in proxy-pass. Why it doesn't work with a plain substitution, I don't know. And since it's an undocumented hack, this might change from version to version. It would be nice if nginx itself was smarter.
[EDIT]
In some cases, the part of a request URI to be replaced cannot be determined:
...
When variables are used in proxy_pass:
location /name/ { proxy_pass http://127.0.0.1$request_uri;}
In this case, if URI is specified in the directive, it is passed to the server as is, replacing the original request URI.
This different behavior if a variable is present is spelled out in the manual. You could try a rewrite directive, which modifies the URI to be sent.
I am attempting to get nginx to redirect/rewrite requests to a specific subdirectory so that they are served by an S3-compatible bucket instead of the server. Here is my current server block:
{snip} (See infra.)
Despite fiddling with this for some time now, I've only been able to get it to return 404s.
Additional Information
https://omnifora.com/t/redirect-rewrite-in-nginx-requests-to-subdirectory-to-s3-compatible-bucket/402
Attempts Solutions So Far
rewrite
rewrite ^/security-now/(.*) $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
return
return 302 $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
proxy_pass
proxy_set_header Host s3.us-west-1.wasabisys.com;
proxy_pass $scheme://s3.us-west-1.wasabisys.com/bits-podcasts/security-now/$1;
You can't use rewrite for cross-domain redirections, for this case you must use proxy_pass, for example:
location ~ ^/directory1/(.*) {
proxy_set_header Host s3.us-west-1.wasabisys.com;
proxy_pass $scheme://s3.us-west-1.wasabisys.com/target-bucket/security-now/$1;
}
Note that if you specify your server with domain name instead or IP address, you'll need to specify additional parameter resolver in your server configuration block, for example:
server {
...
resolver 8.8.8.8;
...
}
Update.
It seems I was wrong stating that you can't use rewrite for cross-domain redirections. You can, but in this case your user got HTTP 301 redirect instead of "transparent" content delivery. Maybe you got 404 error because you missed a $ sign before scheme variable?
i am trying dynamically set a the proxy_pass destination where the variable would be the cname of the original request.
what i have right now is:
server {
listen 8888;
server_name (.*).domain.com;
location / {
proxy_pass http://$1.otherdomain.com;
proxy_set_header Host $1.otherdomain.com;
but unfortunately this ends up in a 502 bad gateway.
nothing really works when using a variable in proxy_pass and proxy_set_header.
i also tried to use (?<cname>.+) or (?P<cname>.+) in the server name and $cname as the variable.
what is wrong and why does it end up in a 502?
To use regex in server name, you need to prepend the name with a tilde "~"
server_name ~(.*).domain.com;
[UPDATE]
Tried it and it successfully set the value in $1. But still get 502 and my nginx error log shows
no resolver defined to resolve xyz.otherdomain.com
even though I point that name to my localhost in my /etc/hosts file.
Find this article that explains this issue well. Basically in this special case (variable in upstream domain name), you need to use the "resolver" directive to point to a dns server (e.g., 8.8.8.8 from google dns server) that can resolve this dynamic domain name.
resolver 8.8.8.8;
It works in my test with a public upstream domain name. If you upstream domain names are local, you need to set up a local dns server for them.
The server name for proxy_pass using variables, will be a special situation.
proxy_pass http://$1.otherdomain.com;
In this case, the server name is searched among the described server groups, and, if not found, is determined using a resolver.
If you do not like to use resolver. You can use below like hosts file.
upstream www1.otherdomain.com { server 10.x.x.1; }
upstream www2.otherdomain.com { server 10.x.x.2; }
we have two servers, A and B. Server A is accessed worldwide. He has nginx installed. That's what I have in conf:
location /test {
proxy_pass http://localserver.com;
}
What it should do, is translate the addreess http://globalserver.com/test (that is server A) to internal server address http://localserver.com. However, it does append the location path, that is, itries to look for http://localserver.com/test, which is not available at all. How can I make the proxy pass to the correct address, throwing out the last part in the location?
That should work. Nginx should strip the '/test' path on the upstream local server. So what I can say is that is not the cause. To make it a bit better, try this:
location /test/ {
proxy_pass http://localserver.com/;
}
The 2 slashes I added at the first 2 lines will avoid mistakenly match '/testABC' and send the wrong request to the upstream local server, for example.
Do you have a
proxy_redirect
line in the same location block? If your upstream local server has redirects, then a mistake on that line will cause an issue like you described.
[UPDATE] Found the root cause why the original config didn't work and mine works: nginx does NOT replace URI path part if the proxy_pass directive does not have a URI path itself. So my fix of adding a slash (slash is treated as a URI path) at the end triggers the URI path replacement.
Reference: http://wiki.nginx.org/HttpProxyModule#proxy_pass
If it is necessary to transmit URI in the unprocessed form then directive proxy_pass should be used without URI part
location /some/path/ {
proxy_pass http://127.0.0.1;
}
try to add as specified here http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass:
proxy_pass http://localserver.com/;
try rewrite
location /test {
rewrite ^ $scheme://$host/;
proxy_pass http://localserver.com;
}
some helpful links...
http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite
http://wiki.nginx.org/Pitfalls