Nginx proxy_pass with $remote_addr - nginx

I'm trying to include $remote_addr or $http_remote_addr on my proxy_pass without success.
The rewrite rule works
location ^~ /freegeoip/ {
rewrite ^ http://freegeoip.net/json/$remote_addr last;
}
The proxy_pass without the $remote_addr works, but freegeoip does not read the x-Real-IP
location ^~ /freegeoip/ {
proxy_pass http://freegeoip.net/json/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
}
Then, I'm adding the ip to the end of the request, like this:
location ^~ /freegeoip/ {
proxy_pass http://freegeoip.net/json/$remote_addr;
}
but nginx report this error: no resolver defined to resolve freegeoip.net

If the proxy_pass statement has no variables in it, then it will use the "gethostbyaddr" system call during start-up or reload and will cache that value permanently.
if there are any variables, such as using either of the following:
set $originaddr http://origin.example.com;
proxy_pass $originaddr;
# or even
proxy_pass http://origin.example.com$request_uri;
Then nginx will use a built-in resolver, and the "resolver" directive must be present. "resolver" is probably a misnomer; think of it as "what DNS server will the built-in resolver use". Since nginx 1.1.9 the built-in resolver will honour DNS TTL values. Before then it used a fixed value of 5 minutes.

It seems a bit strange that nginx is failing to resolve the domain name at runtime rather than at configuration time (since the domain name is hard coded). Adding a resolver declaration to the location block usually fixes dns issues experienced at runtime. So your location block might look like:
location ^~ /freegeoip/ {
#use google as dns
resolver 8.8.8.8;
proxy_pass http://freegeoip.net/json/$remote_addr;
}
This solution is based on an article I read a while back - Proxy pass and resolver. Would be worth a read.

If anyone is stll experiencing trouble, for me it helped to move the proxy_pass host to a seperate upstream, so I come up with something like this
upstream backend-server {
server backend.service.consul;
}
server {
listen 80;
server_name frontend.test.me;
location ~/api(.*)$ {
proxy_pass http://backend-server$1;
}
location / {
# this works mystically! backend doesn't...
proxy_pass http://frontend.service.consul/;
}
}

Another way is that you can provide your host ip address and port number instead of host URL in proxy_pass like below:
Your Code:
proxy_pass http://freegeoip.net/json/;
Updated Code
proxy_pass http://10.45.45.10:2290/json/;

So you want this to work:
location ^~ /freegeoip/ {
proxy_pass http://freegeoip.net/json/$remote_addr;
}
but as pointed out by Chris Cogdon, it fails because at run-time, nginx does not have the domain name resolved, and so proxy_pass requires a resolver (see also, proxy_pass docs).
Something that seems to work (though may be a hack!) is to do something that triggers nginx to resolve the domain name at start-up and cache this.
So, change the config to include a location (for the same host) without any variables (as well as your location with variables):
location = /freegeoip/this-location-is-just-a-hack {
# A dummy location. Use any path, but keep the hostname.
proxy_pass http://freegeoip.net/this-path-need-not-exist;
}
location ^~ /freegeoip/ {
proxy_pass http://freegeoip.net/json/$remote_addr;
}
The host will now be resolved at start-up (and its cached value will be used in the second location at run-time). You can verify this start-up resolution behavior by changing hostname (freegeoip.net) to something that doesn't exist, and when you run nginx -t or nginx -s reload you'll have an emergency error ([emerg] host not found in upstream). And of course you can verify that the cached value is used at run-time by leaving out any resolver, hitting your location block, and observing no errors in the logs.

You could also mention your nginx server port in the proxy_pass uri. This solved the issue for me.

Try to use dig / nslookup to make sure this is still nginx issue.
In my case the issue was that one of dns server between my nginx and root dns was not respecting TTL values that are as small as 1 second.
Nginx does not re-resolve DNS names in Docker

Related

How can I ensure nginx resolves variable DNS in proxy_pass?

I pass some requests to this location (this is a simplified version of the configuration for clarity). ONBOARDING_URL is set in the environment and replaced when the server is run by a URL. This URL is resolved to an IP once. It needs to be resolved for each request when the TTL has expired. Initially, the configuration looked like this:
Initial Configuration Template
server {
listen 443 ssl;
server_name localhost;
resolver ${AWS_DNS};
location /onboarding {
proxy_cache onboarding;
proxy_set_header Host $proxy_host;
proxy_cache_valid 200 10m;
proxy_cache_key $cacheKey;
proxy_set_header Host $proxy_host;
proxy_pass ${ONBOARDING_URL}/api-users;
}
}
I discovered this was only resolved one time. Changes to the IP underlying the DNS resolution had no effect; requests were routed to the IP that was resolved initially.
So following guidance in the proxy_pass documentation, and from other questions / sources (below), we modified the configuration.
https://serverfault.com/a/593003/310436
https://stackoverflow.com/a/26957754/2081835
https://forum.nginx.org/read.php?2,215830,215832#msg-215832
from: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.
The new configuration was intended to replace the template variable, populated from the environment at runtime, with a configuration variable that will always be resolved. However, it doesn't seem to be. Once an IP is resolved, it is used for the life of the process. Any ideas how to get around this? This seems a bit complex to test, too, which further complicates the fix.
New Configuration Template
server {
listen 443 ssl;
server_name localhost;
resolver ${AWS_DNS};
set $onboardingUrl ${ONBOARDING_URL};
location ~ ^/onboarding/?(.*)$ {
proxy_cache onboarding;
proxy_set_header Host $proxy_host;
proxy_cache_valid 200 10m;
proxy_cache_key $cacheKey;
proxy_set_header Host $proxy_host;
proxy_pass $onboardingUrl/api-users/$1;
}
}
Yet still, for the life of the process only the initially resolved IP is used.

NGINX proxy_pass or proxy_redirect

Need help on Nginx proxy_pass.
From outside Nginx URL will be hit like this:
http://some-IP:8080/v2/platform/general/activity/plan?.....
my downstream service looks like this:
http://another-IP:8080/activity/plan?...
I want to get rid of
/v2/platform/general
from original public url and call my downstream service like above.
In Nginx, how do I redirect public access URL to downstream service?
I tried this:
location /v2/platform/general/ {
rewrite ^/(.*) /$1 break;
proxy_redirect off;
proxy_pass http://another-IP:8080;
proxy_set_header Host $host;
But it didn't work, any help appreciated.
proxy_pass and proxy_redirect have totally different functions. The proxy_redirect directive is only involved with changing the Location response header in a 3xx status message. See the NGINX proxy_redirect docs for details.
Your rewrite statement does nothing other than prevent further modification of the URI. This line needs to be deleted otherwise it will inhibit proxy_pass from mapping the URI. See below.
The proxy_pass directive can map the URI (e.g. from /v2/platform/general/foo to /foo) by appending a URI value to the proxy_pass value, which works in conjunction with the location value. See this document for details.
For example:
location /v2/platform/general/ {
...
proxy_pass http://another-IP:8080/;
}
You may need to set the Host header only if your upstream server does not respond correctly to the value another-IP:8080.
You may need to add one or more proxy_redirect statements if your upstream server generates 3xx status responses with an incorrect value for the Location header value.

How to rewrite locations in nginx reverse proxy to load owncloud page perfectly?

I'm pretty unexperienced using reverse proxy, so my question can be really lame, sorry for that.
I'm trying to reach my owncloud server through nginx reverse-proxy, but it can't load perfectly.
I have an NGINX reverse-proxy server using multiple locations. I would like to make a new public access to my owncloud server located in another machine with apache.
I would like to use _https://my_public_url/owncloud_ to reach my owncloud server, so I made the location block like this:
Whem I'm using
location / {
proxy_pass http://my_owncloudserver_url/;
everything is fine.
But in this case:
location /owncloud/ {
proxy_pass http://my_owncloudserver_url/;
I get the index.php/login page without any formatting, as /apps, /core, etc. requests are still requested from "https://my_public_url/apps/...", "https://my_public_url/core/...", etc. instead of "https://my_public_url/owncloud/core/..." where the files are located, as these requests don't match with /owncloud/ location and aren't proxied.
I guess I should use rewrite to change the urls of these requests, putting the "/owncloud/" part into the url.
If I'm using a separate location to match with "/core/..." requests, like:
location /core/ {
rewrite ^/core/(.*)$ /owncloud/core/$1 permanent;
}
then it seems to be OK, but I won't make a lot of different locations to match with all various requests.
How could I fix this?
I'm running out if ideas, although it must be pretty easy.
Thanks,
sanglee
I'm not sure about Owncloud. But in Nextcloud you have to configure some proxy parameters in the config.php https://docs.nextcloud.com/server/16/admin_manual/configuration_server/config_sample_php_parameters.html#proxy-configurations
Please consider to use Nextcloud because it is faster than Owncloud, if fully open source, more features and is actively maintained by the community.
Update OWNCLOUD_SUB_URL” to “/owncloud” when running the container, or find the subtitute config if running not using containers
And on nginx config
location /owncloud {
proxy_pass http://my_owncloudserver_url/owncloud;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_buffering off;
}

nginx dynamic proxy_pass based on url pa

I currently have the following proxy pass definition in my nginx config file :
location /pass/ {
proxy_pass http://localhost:9999/pass/;
proxy_redirect off;
proxy_set_header Host $host;
}
This is working as expected - /pass requests are forwarded to the app running on port 9999.
Now, what I want to do is make the port forwarding part dynamic as follows :
location /pass/<input> {
{a $port variable here that is evaluated via a script (php?)}
proxy_pass http://localhost:$port/pass/;
proxy_redirect off;
proxy_set_header Host $host;
}
Requests to /pass/ABCD1234 should be forwarded to port 9898 and Requests to /pass/ABCD5678 should be forwarded to port 9797.
Note that the flow is dynamic - so, the mapping from ABCD1234 to 9898 should happen through some sort of scripting (PHP maybe?) and based on the output of the script (a port) the proxy_pass should forward the request to that port.
Please Help in this regard.
UPDATE :
Instead of getting the proxy_pass port from URI input, I would like to get this going with a cookie. So, here is the updated code block :
location /pass/ {
add_header X-debug-message $host always;
add_header X-debug-message $cookie_sdmport;
set $proxyurl http://127.0.0.1:$cookie_theport/pass/;
add_header X-debug-message $proxyurl;
proxy_pass $proxyurl;
proxy_redirect off;
proxy_set_header Host $host;
}
With this code, there is looping 301 redirect back to the browser. The moment I switch back to static port, it works again! strange! The $proxyurl in the X-debug-message looks correct on the browser. So, wondering why proxy_pass is doing a 301!
UPDATE 2:
Finally got the forwarding working with following setup:
set $targetIP 127.0.0.1;
set $targetPort $cookie_passport;
proxy_pass http://$targetIP:$targetPort$request_uri;
Not sure why the solution as posted in above keeps spinning with a 301 - I guess nginx does not like mixing dynamic and static parts in the proxy_pass param
Thank You.
You can do this using the auth_request module. It's not built by default though, you can find out if you have it by running the following:
nginx -V 2>&1 | grep -qF -- --with-http_auth_request_module && echo ":)" || echo ":("
If you see a smiley face then you are good to go.
location ~* /pass/(.*) { <- regex capture for your variable
auth_request /auth; <- location to process request
auth_request_set $proxyurl http://localhost:$upstream_http_x_port/pass/; <- set $proxyurl using value returned in x-port header of your php script
add_header x-my-variable $1; <- Pass variable from regex capture to auth location
proxy_pass $proxyurl;
}
Then a location to handle the auth subrequests:
location /auth {
internal; <- make location only accessible to internal requests from Nginx
proxy_set_header x-my-variable $http_x_my_variable; <- pass variable to php
proxy_pass_request_body off; <- No point sending body to php
proxy_set_header Content-Length "";
proxy_pass http://your-php-script/file.php;
}
This module is actually meant for access control, so if your php script returns response code 200 then client will be allowed access, if it returns 401 or 403 then access will be denied. Is you dont care about that then just set it to always return 200.
Do whatever evaluation you need and have your php return the port in the header defined earlier:
header('X-Port: 9999');
This now sets the variable for your proxy_pass directive port number and Nginx does the rest.

nginx how to proxy_pass to a proxy

I have a nginx server running. I want to issue some request to a corporate proxy and return the result to the client.
Let us say the client issues a request to http://ip:port/redirect/google.com, the server should issue the request to https://username:password#ip_proxy with the requested url as parameter.
I have found questions that are close to my problem:
http://serverfault.com/questions/732063/nginx-proxy-pass-reverse-proxying-behind-corporate-firewall
and
http://stackoverflow.com/questions/11865119/how-to-configure-nginx-behind-a-corporate-proxy
assume 205.100.100.100:80 is the proxy URI and
XXVzYTQzMjXXXDpCb25qXXyMQ== the credential for the proxy
one says we should rewrite the url and then pass it to proxy_pass directive
location ^~ /redirect/ {
rewrite ^/redirect/(.*) https://google.com;
proxy_pass_header on;
proxy_set_header Proxy-Authorization "XXVzYTQzMjXXXDpCb25qXXyMQ==";
proxy_pass https://205.100.100.100:80;
}
I think it does not work as intended cause it is showing as a redirection (http code 301)
Another one says that we should use the Host header and specify original url with it:
location ^~ /redirect/ {
proxy_pass_header on;
proxy_set_header Host "https://google.com";
proxy_set_header Proxy-Authorization "XXVzYTQzMXXXpCb25XXyMQ==";
proxy_pass https://205.100.100.100:80;
}
Doesn't work. They also say that a proxy read the url specified as an url parameter something like: http://proxy:port/url_requested so:
location /redirect {
rewrite ^/redirect/(.*)"http://205.100.100.100:80/https://google.com" break;
proxy_pass_header on;
proxy_set_header Proxy-Authorization "XXVzYTQzMXXXpCb25XXyMQ==";
proxy_pass http://corporate-proxy.mycorp.com:8080;
}
Should work ?
As you can see I do not know how to specify the username, password to the proxy. I tried a Proxy-Authorization header with "XXVzYTQzMjXXXDpCb25qXXyMQ==" with no result.
I tried the http_upstream module too, no result. I cannot use proxy_pass http://user:pwd#205.100.100.100:80 because nginx use ":" for parsing the port, it gives me error specifying port.
How should I proceed ?
Maybe it comes from the entire configuration or they way I use the proxy? Don't know.
This isn't possible, nginx isn't able to talk to forward proxy.
See also this thread in the official mailing list: http://mailman.nginx.org/pipermail/nginx-devel/2013-March/003534.html

Resources