proxy_pass doesn't seem to resolve to upstream - nginx

Question

I use this configuration below but somehow it doesn't proxy to the upstream.
upstream backend {
ip_hash;
server s1.example.com;
server s2.example.com;
}
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
server_name localhost;
location / {
proxy_pass http://backend;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
However, if i replace to proxy_pass http://s1.example.com, it proxy successfully to the s1.example.com server.
How do I fix this?

Answer 1

The NGINX config says about proxy_pass:
Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.
It does NOT state how the parameter handles a hostname when it's a constant/statically defined. According to this NGINX forum post:
domain names statically configured in config are only looked
up once on startup (or configuration reload).
Using an upstream block (as in your example) won't save you from this problem ("statically defined hostname is only looked up once after start/reload") unless you are using (the pay for) NGINX Plus and then you can add the resolve argument to the end of the nested server parameter.
The good news is that even in the free NGINX you can workaround the lookup problem by using variables like so:
upstream backend { ... }
server {
...
set $backend "backend";
proxy_pass http://$backend;
...
}
but this introduces its own caveats. See https://github.com/DmitryFillo/nginx-proxy-pitfalls for a thorough discussion of workarounds and pitfalls to the "NGINX proxy repeat DNS resolution" problem.