I have HTTP cloud functions running locally and i want to hide them behind a reverse proxy to reproduce the same architecture i have in production.
When i call directly the endpoint, it's fine. But when i use my react application, it's not working. I don't manage to fix the issue.
Can you help me please?
Nginx conf :
server {
listen 9003;
#add_header 'Referrer-Policy' 'no-referrer';
add_header 'Referrer-Policy' 'unsafe-url';
location /api/callImportEvent/ {
add_header 'Referrer-Policy' 'unsafe-url';
if ($request_method = OPTIONS ) {
add_header "Access-Control-Allow-Origin" *;
add_header "Access-Control-Allow-Methods" "GET, POST, OPTIONS, HEAD";
add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
return 200;
}
proxy_pass http://192.168.99.1:8888/;
proxy_redirect off;
proxy_pass_header Authorization;
proxy_pass_header x-api-key;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Cloud function command :
$(gcloud beta emulators pubsub env-init) && functions-framework --target=callImportEventHttp --signature-type=http --source=./index.js --port=8888
Direct curl (working) :
rbarbu#DESKTOP-7EQNOFM:~$ curl -v 'http://localhost:9003/api/callImport/'
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9003 (#0)
> GET /api/callImport/ HTTP/1.1
> Host: localhost:9003
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 456 unknown
< Server: nginx/1.14.0 (Ubuntu)
< Date: Mon, 07 Sep 2020 15:37:16 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 61
< Connection: keep-alive
< X-Powered-By: Express
< ETag: W/"3d-kbNXdE/Nx2cqqnUsIdQHlgSM74w"
<
* Connection #0 to host localhost left intact
{"status":"Unrecoverable Error","message":"No API KEY given"}r
Call from React APP (not Working):
Direct call to NGINX URL from Browser (working):
Related
This question already has answers here:
CORS and non 200 statusCode
(2 answers)
Closed last year.
I know there are a million answers to "how to set Access-Control-Allow-Origin in nginx" but unfortunately if there's an answer to my specific question, it's buried with all the basic answers.
I have an Angular app pointing to an nginx server with a rest service upstream - all running on my local laptop inside a docker compose.
The nginx service is configured fairly simply:
upstream REST {
# rest-service is mapped by docker-compose to the correct container
server rest-service:8080;
}
server {
listen 80;
listen [::]:80;
# This is mapped in /etc/hosts on my laptop
server_name rest-api.mylocal.com;
location / {
proxy_hide_header 'Access-Control-Allow-Origin';
# web-ui is also mapped in /etc/hosts on my laptop
add_header "Access-Control-Allow-Origin" "http://web-ui.mylocal.com";
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Max-Age' 1728000; # 20 days
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Headers' 'Content-Type, Authorization';
add_header Reverse-Proxy true;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://REST;
}
}
If I call a valid URL on the rest service, e.g. http://rest-api.mylocal.com/api/v1/auth/config (this is an unauthenticated endpoint), everything works as expected:
curl -v 'http://rest-api.mylocal.com/api/v1/auth/config'
...
< Access-Control-Allow-Origin: http://web-ui.mylocal.com
< Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
< Access-Control-Max-Age: 1728000
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Headers: Content-Type, Authorization
< Reverse-Proxy: true
<
* Connection #0 to host rest-api.mylocal.com left intact
{"status":"OK","name":"foo",...}* Closing connection 0
However, if I call an invalid URL, say http://rest-api.mylocal.com/api/v1/accounts (which is an authenticated endpoint, and I provide no credentials), I get back the expected error, but there are no CORS headers. In a browser this causes everything to flake out so I don't even get the error message in Angular to display to the user:
curl -v 'http://rest-api.mylocal.com/api/v1/accounts'
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to rest-api.mylocal.com (127.0.0.1) port 80 (#0)
> GET /api/v1/accounts HTTP/1.1
> Host: rest-api.mylocal.com
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 403
< Server: nginx/1.21.6
< Date: Wed, 09 Feb 2022 20:57:49 GMT
< Content-Type: application/json
< Transfer-Encoding: chunked
< Connection: keep-alive
< Set-Cookie: JSESSIONID=F3B21A55841C8E5A6F838F2427A0E22B; Path=/; HttpOnly
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< X-Frame-Options: DENY
<
* Connection #0 to host rest-api.mylocal.com left intact
{"timestamp":"2022-02-09T20:57:49.526+00:00","status":403,"error":"Forbidden","message":"Access Denied","path":"/api/v1/accounts"}* Closing connection 0
Browser console:
Access to XMLHttpRequest at 'http://rest-api.mylocal.com/api/v1/accounts' from origin 'http://web-ui.mylocal.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
So... Is there a way to make sure the CORS headers are still added to the response by nginx even if the response from the upstream server is an error (401/403/404/etc)?
You need to add always to each header you want added to error responses. It was added in 1.7.5: https://nginx.org/r/add_header
I am currently running a Grafana server with an NGINX server in front of it to server up from my grafana.olddomain.uk:
location / {
proxy_pass http://127.0.0.1:3000;
}
I would like to proxy from my new domain (https://my.newdomain/grafana) to the old domain but have been hitting problems with the config:
location /grafana {
max_ranges 0;
rewrite ^/grafana/(.*)$ /$1 break;
proxy_pass https://grafana.olddomain.uk/;
proxy_ssl_server_name on;
proxy_ssl_trusted_certificate /etc/nginx/conf.d/trusted_ca_cert.pem;
proxy_set_header Host $host;
}
This is throwing an error cleared being served by Grafana:
If you're seeing this Grafana has failed to load its application files
1. This could be caused by your reverse proxy settings.
2. If you host grafana under subpath make sure your grafana.ini root_url setting includes subpath
3. If you have a local dev build make sure you build frontend using: yarn start, yarn start:hot, or yarn build
4. Sometimes restarting grafana-server can help
So that seems pretty clear, I am using a sub folder now and need to change grafana.ini but that will break grafana.olddomain.uk.
What do I need in my newdomain location to get this working?
EDIT: curl response (following redirects) from https://my.newdomain.uk/grafana
> GET /grafana HTTP/1.1
> Host: my.newdomain.uk
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: nginx
< Date: Fri, 16 Apr 2021 17:39:13 GMT
< Content-Type: text/html
< Content-Length: 162
< Location: https://my.newdomain.uk/grafana/
< Connection: keep-alive
<
> GET /grafana/ HTTP/1.1
> Host: my.newdomain.uk
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: nginx
< Date: Fri, 16 Apr 2021 17:39:13 GMT
< Content-Type: text/html; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: no-cache
< Expires: -1
< Pragma: no-cache
< X-Frame-Options: deny
<
So after a lot of attempts at getting a sub path working I had to give up and set this up on root. This required splitting all the configurations up into separate sites.
location / {
max_ranges 0;
proxy_pass https://grafana.olddomain.uk/;
proxy_ssl_server_name on;
proxy_ssl_trusted_certificate /etc/nginx/conf.d/trusted_ca_cert.pem;
proxy_set_header Host $host;
}
I have the following nginx configuration:
location ~* ^/path/(.*) {
set $query $1;
proxy_pass http://backend_app/$query;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
}
I expect nginx to proxy the request from http://frontend/path/subpath name to http://backend_app/subpath name. But I am seeing some weird behavior:
http://frontend/path/subpath --> http://backend_app/path/subpath works fine.
BUT
http://frontend/path/subpath name --> returns a 400 bad request with the following error message:
< HTTP/1.1 400 Bad Request
< Server: nginx/1.16.1
< Date: Sat, 06 Feb 2021 01:13:15 GMT
< Content-Type: text/html
< Content-Length: 198
< Connection: keep-alive
< Strict-Transport-Security: max-age=31536000; includeSubDomains
<
<html>
<head>
<title>Bad Request</title>
</head>
<body>
<h1><p>Bad Request</p></h1>
Invalid HTTP Version 'Invalid HTTP Version: 'name HTTP/1.0''
</body>
</html>
* Connection #0 to host frontend left intact
* Closing connection 0
Notice the error message in the response says Invalid HTTP Version 'Invalid HTTP Version: 'name HTTP/1.0''
So nginx is interpreting the word after the space in the path as part of the http version name.
I tried this with URL encoding as well but still no luck.
Just learned about URI normalization and I was able to fix this problem by following this config:
location /path {
rewrite /path/(.*) /$1 break;
proxy_pass http://backend_app;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
}
And things are working as expected.
Receive 404 error while calling URL - http://10.240.0.133/swagger. Below is the snippet of nginx.conf file, I need to append index.html at end of the URI, so I placed a rewrite rule.
server {
listen 80;
listen [::]:80;
server_name localhost;
server_name 10.240.0.133;
server_name 127.0.0.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
access_log /var/log/nginx/resources-reverse-access.log;
error_log /var/log/nginx/resources-reverse-error.log;
location /swagger {
rewrite ^/swagger/index.html break;
proxy_pass http://52.177.131.103:8082/;
}
}
When I visited the URL - curl -v http://10.240.0.133/swagger
404 is thrown:-
* Trying 10.240.0.133...
* TCP_NODELAY set
* Connected to 10.240.0.133 (10.240.0.133) port 80 (#0)
> GET /swagger HTTP/1.1
> Host: 10.240.0.133
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: nginx/1.14.0 (Ubuntu)
< Date: Wed, 18 Mar 2020 14:41:50 GMT
< Content-Length: 0
< Connection: keep-alive
<
* Connection #0 to host 10.240.0.133 left intact
I believe your rewrite rule is incorrect. It should look more like this.
location /swagger {
rewrite ^\/swagger\/?.*?$ /swagger/index.html break;
proxy_pass http://52.177.131.103:8082/;
}
but I believe this still not correct since you have not a set a root directive for this server.
I am trying to set up nginx with this config. To access backend.mygreat.server.com I have to go through my corporate proxy, which is myproxy.server.com:80.
Hence, I have added this in /etc/environment
https_proxy=myproxy.server.com:80
Yet, nginx is unable to reach https://backend.mygreat.server.com:443. I'm seeing 504 as HTTP status in nginx logs.
I could use wget or curl to load the page (goes via corporate proxy)
server {
listen 443;
server_name mygreat.server.com;
ssl on;
ssl_protocols TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:EDH+aRSA:!aNULL:!eNULL:!LOW:!RC4:!3DES:!MD5:!EXP:!PSK:!SRP:!SEED:!DSS:!CAMELLIA;
ssl_certificate /etc/nginx/ssl/mygreat.server.com.pem;
ssl_certificate_key /etc/nginx/ssl/mygreat.server.com.key;
access_log /var/log/nginx/access.ssl.log;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host-Real-IP $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-Pcol http;
proxy_intercept_errors on;
error_page 301 302 307 = #handle_redirects;
proxy_pass https://backend.mygreat.server.com:443;
}
location #handle_redirects {
set $saved_redirect_location '$upstream_http_location';
proxy_pass $saved_redirect_location;
}
}
Any help is greatly appreciated.
Thanks
Update :
Here is the sample error log from nginx
2017/10/18 06:55:51 [warn] 34604#34604: *1 upstream server temporarily disabled while connecting to upstream, client: <ip-address>, server: mygreat.server.com, request: "GET / HTTP/1.1", upstream: "https://<ip-of-backend>:443/", host: "mygreat.server.com"
If I run curl -v https://backend.mygreat.server.com/ below is the response
* About to connect() to proxy corp-proxy.server.com port 80 (#0)
* Trying <some-ip-address>...
* Connected to corp-proxy.server.com (<ip-of-proxy>) port 80 (#0)
* Establish HTTP proxy tunnel to backend.mygreat.server.com:443
> CONNECT backend.mygreat.server.com:443 HTTP/1.1
> Host: backend.mygreat.server.com:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=backend.mygreat.server.com,OU=Technology Operations,O=MyCompany.,L=San Diego,ST=California,C=US
* start date: Mar 15 00:00:00 2017 GMT
* expire date: Mar 15 23:59:59 2020 GMT
* common name: backend.mygreat.server.com
* issuer: CN=Symantec Class 3 Secure Server CA - G4,OU=Symantec Trust Network,O=Symantec Corporation,C=US
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: backend.mygreat.server.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: openresty/1.11.2.5
< Date: Wed, 18 Oct 2017 14:03:10 GMT
< Content-Type: text/html;charset=UTF-8
< Content-Length: 5642
< Connection: keep-alive
< X-XSS-Protection: 1; mode=block
< Cache-Control: max-age=0, no-cache, no-store, must-revalidate, private
< Expires: 0
< P3P: policyref="http://backend.mygreat.server.com/w3c/p3p.xml" CP="CURa OUR STP UNI INT"
< Content-Language: en
< Set-Cookie: qboeuid=127.0.0.1.1508335390550307; path=/; expires=Thu, 18-Oct-18 14:03:10 GMT; domain=.server.com
< Set-Cookie: JSESSIONID=784529AA39C10C3DB4B0ED0D61CC8F31.c23-pe2ec23uw2apu012031; Path=/; Secure; HttpOnly
< Set-Cookie: something.blah_blah=testme; Domain=.server.com; Path=/; Secure
< Vary: Accept-Encoding
<
<!DOCTYPE html>
<html>
....
</html>
So first of all I am not sure if Nginx is suppose to respect http_proxy and https_proxy variables. I didn't find any documentation on the same. So I assume your issues is related to nginx not using proxy at a all
So now you have an option to use something which actually uses proxy. This is where socat comes to rescue.
Running socat forwarder
If you have a transparent proxy then run
socat TCP4-LISTEN:8443,reuseaddr,fork TCP:<proxysever>:<proxyport>
And if you have CONNECT proxy then use below
socat TCP4-LISTEN:8443,reuseaddr,fork PROXY:yourproxy:backendserver:443,proxyport=<yourproxyport>
Then in your nginx config use
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host-Real-IP $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-Pcol http;
proxy_intercept_errors on;
proxy_set_header Host backend.mygreat.server.com;
proxy_pass https://127.0.0.1:8443;
proxy_redirect https://backend.mygreat.server.com https://mygreat.server.com;
}
You probably want to use Systemd service to launch the socat, so it runs on startup and is handled as a service
Nginx's proxy_pass does not support https proxy.
http proxy can be supported, but the request url only supports http.
this is a example:
server {
listen 8880;
server_name localhost;
location / {
rewrite ^(.*)$ "://developer.android.com$1";
rewrite ^(.*)$ "http$1" break;
proxy_set_header Proxy-Connection Keep-Alive;
proxy_set_header Host developer.android.com;
proxy_pass http://127.0.0.1:1080;
proxy_redirect ~^https?://developer\.android\.com(.*)$ http://$host:8080$1;
}
}
see: https://serverfault.com/a/683955/418613