What is wrong with my FastAPI Reverse Proxy? - nginx

I have a web application which is setup in the following way: the frontend is being served by nginx. The backend is handled by FastAPI. I setup a rule in nginx to proxy all requests with url /api to the backend directly. Now, I also proxy to Grafana. The way I did this was to build a reverse proxy within my FastAPI server. So in nginx, I have a rule to proxy all requests with url /grafana to FastAPI. FastAPI then does some user authentication before proxying to the grafana server. The exception is that any Grafana websocket connection gets proxied directly from nginx to the grafana server.
Here's my nginx conf file
server {
server_name example.com www.example.com;
root /var/www/web-app/html;
index index.html index.htm index.nginx-debian.html;
location / {
try_files $uri $uri/ =404;
}
location /api {
proxy_pass http://localhost:5000;
}
# grafana reverse proxy
location /grafana {
proxy_set_header Origin http://localhost:3000;
# proxy_set_header Origin https://example.com;
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin $http_origin;
proxy_pass http://localhost:5000;
}
location /grafana/api/live {
rewrite ^/(.*) /$1 break;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $http_host;
proxy_pass http://localhost:3030;
}
listen 443 ssl http2; # managed by Certbot
ssl_certificate /etc/letsencrypt/example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name example.com www.example.com;
return 404; # managed by Certbot
}
Here's my FastAPI proxy
async def _reverse_proxy(request: Request):
url = httpx.URL(path=request.url.path,
query=request.url.query.encode("utf-8"))
new_header = MutableHeaders(request.headers)
if "grafana_session" not in request.cookies:
if "authorization" not in request.headers:
return RedirectResponse("/")
jwt_token = request.headers["authorization"].replace('Bearer ', '')
# user authentication stuff...
current_user = get_current_user_from_token(token=jwt_token, db=some_session)
org = get_org_by_id(current_user.org_id, some_session)
if org is None:
raise HTTPException(
status_code=404,
detail="Organization not found",
)
del new_header['authorization']
new_header['X-WEBAUTH-USER'] = current_user.username
new_header['X-Grafana-Org-Id'] = f"{org.grafana_org_id}"
if "authorization" in new_header:
del new_header['authorization']
rp_req = client.build_request(request.method, url,
headers=new_header.raw,
content=await request.body())
rp_resp = await client.send(rp_req, stream=True)
return StreamingResponse(
rp_resp.aiter_raw(),
status_code=rp_resp.status_code,
headers=rp_resp.headers,
background=BackgroundTask(rp_resp.aclose),
)
app = FastAPI(title=settings.PROJECT_NAME, version=settings.PROJECT_VERSION)
origins = ["http://localhost:3000", "https://example.com"]
app.add_middleware(
CORSMiddleware,
allow_origins=origins,
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(api_router, prefix=settings.API_V1_STR)
app.add_route("/grafana/{path:path}", _reverse_proxy, ["GET", "POST", "DELETE"])
If I run this locally on localhost:3000 (with some modifications to nginx to be listening on 3000 instead of 443 ssl), everything works perfectly. If I try in production at example.com, I have to include this hacky bit proxy_set_header Origin http://localhost:3000; to change the origin in a request to my FastAPI proxy to make it work. Why is this happening?
NOTE: I've checked the grafana server logs to make sure that it wasn't the issue. None of the requests make it past the FastAPI server, it is the one returning error 403, origin not allowed.

Related

NGINX reverse proxy relative links, issue with routed location

So I have 2 docker containers running different Flask apps, port forwarded to the host. The host has a NGINX server redirecting requests to the server to the respective containers. The "sites available" file for the NGINX server is as shown:
server {
root /var/www/myserver/html;
index index.html index.htm index.nginx-debian.html;
# Don't forget to include .com below!
server_name myserver.com www.myserver.com;
location / {
proxy_pass http://127.0.0.1:6789/;
} # Here, we reverse proxy the port 80 to port 6789, where the website is served by Docker
location /smallblog/ {
rewrite ^/smallblog/(.*) /$1 break;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_pass http://127.0.0.1:8000/;
proxy_pass_request_headers on;
} # Here, we reverse proxy the port 80 with /smallblog route to port 8000, where the smallblog is also served by another Docker container
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/myserver.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myserver.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.myserver.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = myserver.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
listen [::]:80;
server_name myserver.com www.myserver.com;
return 404; # managed by Certbot
}
With this configuration, accessing www.myserver.com/smallblog indeed gives the right page. But, all links inside that page point to the myserver.com, for example a link to login page is www.myserver.com/login instead of www.myserver.com/smallblog/login. And all internal relative links (e.g. javascript links) don't work. Is there a way to solve this?
Buit if I use the / location in the script above to serve smallblog instead, this issue doesn't exist. So I figure there must be a way I don't see.

Nginx: Redirecting server names in one paragraph

I using the Nginx configuration below and it works fine.
However, I'm using three paragraphs {...} for redirections, can this be optimized to a single paragraph or set directly in the main paragraph ?
upstream mywebsite_upstream {
server 127.0.0.1:3003;
keepalive 64;
}
server {
server_name www.mywebsite.com;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_pass http://mywebsite_upstream;
proxy_redirect off;
proxy_read_timeout 240s;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/www.mywebsite.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/www.mywebsite.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.mywebsite.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name www.mywebsite.com;
return 404; # managed by Certbot
}
server {
listen 80;
server_name mywebsite.com;
return 301 https://www.mywebsite.com;
}
server {
listen 443;
server_name mywebsite.com;
return 301 https://www.mywebsite.com;
}
Yes, sure you can:
server {
listen 80;
server_name mywebsite.com www.mywebsite.com;
return 301 https://www.mywebsite.com$request_uri;
}
server {
listen 443;
server_name mywebsite.com;
# SSL config here
return 301 https://www.mywebsite.com$request_uri;
}
I think you'll need to copy all SSL-related configuration from your main server block to the second one.
I think (and I'm not alone, check an answer under the link) that nginx config produced by certbot is a crap and it is better to do nginx config changes manually a leave for certbot only a certificate receiving/renewing:
certbot certonly --webroot -w /var/www -d hostname -d hostname ...
Update
When your server hosts other domains (and even if ins't) it is a good practice to have an additional server block serving requests than does not contain a valid domain name (or does not have Host HTTP header at all - those are typically port scanners, vulnerability searchers etc.) To close suspicious connections on port 443 you'll need minimal SSL config within that block. It's best to use a self-signed key and certificate for that purpose. For generating a pair of self-signed key/cart in one line you can use the following command:
openssl req -nodes -new -x509 -subj "/CN=localhost" -keyout /etc/nginx/server.key -out /etc/nginx/server.crt
You can use special nginx 444 code (close connection without any response) for the suspicious connections:
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/nginx/server.crt;
ssl_certificate_key /etc/nginx/server.key;
return 444;
}
To simplify certbot key renewing process with the command like given above, you can add an additional section to all your server blocks for hosted domains (including that ones used for redirection too):
location /.well-known/acme-challenge/ {
root /var/www;
}

Nginx + GraphQL Connection Refused

My Client (next.js) app running at port 3000
My Server (graphql) app running at port 4000
My website is https://example.com, nginx will pass proxy port 3000.
If the user access the site, the page is loaded successfully.
But behind the scene, In my webpage some api requests are sended to graphql server. (http://localhost:4000)
This api requests are failed.
I don't know why, but when I access http://example.com:4000/graphql the graphql playground (graphiql?) loaded successfully and I can send some query and result showed well. But request from webpage is failed.
nginx/sites-enabled/example.com
server {
listen 80;
listen [::]:80;
server_name www.example.com example.com;
return 301 https://example.com$request_uri;
}
server {
listen 80;
listen [::]:80;
server_name example.com;
location / {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
client app's graphql part
export default function createApolloClient(initialState, ctx) {
return new ApolloClient({
ssrMode: Boolean(ctx),
link: authLink.concat(new HttpLink({
uri: 'http://localhost:4000/graphql', // Server URL (must be absolute)
credentials: 'same-origin', // Additional fetch() options like `credentials` or `headers`
fetch,
})),
cache: new InMemoryCache({ fragmentMatcher }).restore(initialState),
credentials: 'include',
})
}
** What I tried...**
I added below snippets to nginx conf (above listen [::]443 part) and restart the nginx service, but nothing changed.
location /graphql {
proxy_pass http://localhost:4000/graphql;
}
I think I miss something in nginx conf. How do I fix it?
I resolved like below.
createApolloClient
... createApolloClient(initialState, ctx) {
return new ApolloClient({
...
link: createHttpLink({ uri: '/graphql' })
nginx
location /graphql {
proxy_pass http://localhost:4000/graphql;
}

Nginx server name issue

In my Nginx configuration, I would like to keep one service to be accessible with http, while all the others should be accessed through https, and forced to ssl when trying to connect with http. This is my config:
server{
server_name localhost;
listen 80;
proxy_http_version 1.1;
location /services/ {
proxy_pass http://localhost:47440/;
}
listen / {
rewrite ^ https://$server_name$request_uri? permanent;
}
server{
server_name localhost_ssl;
listen 443 ssl;
ssl_certificate /etc/nginx/ssl/mycert.crt;
ssl_certificate_key /etc/nginx/ssl/mycert.key;
proxy_http_version 1.1;
location /db/ {
proxy_pass http://localhost_ssl:8084/;
}
}
My problem is that when trying to reload I get this error:
host not found in upstream "localhost_ssl" in /etc/nginx/nginx.conf:46
Any idea of why this happens?
It seems your DNS resolver is failing for some reason.
Try adding:
options single-request
to /etc/resolv.conf
This causes IPv6/v4 lookups to be done sequentially.
You got this error because nginx can't find the host "localhost_ssl". Indeed it doesn't exist unless you specify it with upstream directive (or in the hosts file I think).
You should set it to proxy_pass http://localhost:8084/; assuming your service is really listening on 127.0.0.1:8084.
Furthermore you may want to replace listen / { with location / {.
UPDATE : If you access your server with your IP (you don't have a domain name), then you can remove server_name directive :
server {
listen 80;
proxy_http_version 1.1;
location /services {
proxy_pass http://localhost:47440/;
proxy_set_header Host $host;
}
location / {
return 301 https://$host$request_uri?; # Replace $server_name by $host
}
server {
listen 443 ssl;
ssl_certificate /etc/nginx/ssl/mycert.crt;
ssl_certificate_key /etc/nginx/ssl/mycert.key;
proxy_http_version 1.1;
location /db {
proxy_pass http://localhost:8084/;
proxy_set_header Host $host;
}
}
That config redirects requests received on port 80 to 443 if they don't match location /services. Requests received on port 443 are proxied if they match location /db.
But is this what you really want to achieve ? I mean a request on port 443 for /test would not match any location as there is only /db.

'conflicting server name' with two default/catchall locations for nginx - one with default_server, one without?

I'm at my wit's end here. I've been fighting an nginx configuration for hours. Here are the two blocks I'm trying to use:
server {
listen 80 default_server;
location /health-check {
default_type 'text/plain';
access_log off;
return 200;
}
}
server {
listen 80;
location / {
return 301 https://$http_host$request_uri;
}
}
# other application servers/upstreams follow -- one is provided here for completeness,
# although the issue is almost certainly above
upstream quinoa-icehouse {
server 172.17.8.100:49153;
}
server {
server_name ~^quinoa-icehouse\.(?<domain>.+)$;
server_name_in_redirect off;
port_in_redirect off;
listen 443 ssl spdy;
listen 80;
ssl_certificate /etc/ssl/deis.cert;
ssl_certificate_key /etc/ssl/deis.key;
location / {
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_connect_timeout 30s;
proxy_send_timeout 1200s;
proxy_read_timeout 1200s;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_next_upstream error timeout http_502 http_503 http_504;
add_header X-Deis-Upstream $upstream_addr;
proxy_pass http://quinoa-icehouse;
}
}
Note that I want the /health-check endpoint to work only when other server names aren't matched, but I want the 301 to occur whenever a server name is matched.
I seem to have tried every combination of these directives, only to get:
[INFO] - 2014/12/30 01:26:34 [warn] 39#0: conflicting server name "" on 0.0.0.0:80, ignored
Is there a way for me to accomplish what I seek? Thank you!!
Essentially, you are going outside the Nginx' defined parameters. You cannot have two default server blocks sitting on each other so to speak.
You can however achieve what you need by defining:
A catch all block listening on Port 80 to redirect all requests to Port 443
A catch all block listening on Port 443 for all unmatched domains
You then need to ensure the following:
Drop the "default_server" directive and rely on the position of the server blocks as in the example answer
Ensure other application servers/upstreams only listen on Port 443.
So your config should be something along these lines:
http {
[ ... ]
# Default to redirect from Port 80 to Port 443
server {
listen 80;
return 301 https://$host$request_uri;
}
# Default for unmatched domains on Port 443
server {
listen 443 ssl spdy;
ssl_certificate /etc/ssl/someCert.cert;
ssl_certificate_key /etc/ssl/someKey.key;
# Return 403, 404 or 444
return 403;
}
# Other servers.
# 1. These must be below this for this configuration to work.
# 2. None should listen on Port 80
server {
server_name ABC
listen 443 ssl spdy;
ssl_certificate /etc/ssl/someCert.cert;
ssl_certificate_key /etc/ssl/someKey.key;
[ ... ]
}
server {
server_name XYZ
listen 443 ssl spdy;
ssl_certificate /etc/ssl/someCert.cert;
ssl_certificate_key /etc/ssl/someKey.key;
[ ... ]
}
}
Refer to: Why is nginx responding to any domain name?
Note also that for simple server blocks that will just return simple responses, you don't need to have location blocks.

Resources