I'm trying to set a server-side event correctly with Traefik to no avail.
I have a Django server that has a URL that sends sse, I can proxy easily with nginx with this simple conf:
server {
listen 80;
server_name _ ;
charset utf-8;
client_max_body_size 20M;
location / {
proxy_pass http://django:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
}
}
In traefik using docker labels I set:
labels:
- "traefik.enable=true"
- "traefik.http.routers.dj.rule=Host(`$HOST_ADK`)"
- "traefik.http.routers.dj.entrypoints=web"
- "traefik.http.routers.dj.service=dj"
- "traefik.http.services.dj.loadbalancer.server.port=8000"
#- "traefik.http.routers.dj.middlewares=compress-no-stream"
#- "traefik.http.middlewares.compress-no-stream.compress=true"
#- "traefik.http.middlewares.compress-no-stream.compress.excludedcontenttypes=text/event-stream"
This resolve to internal server error. When proxying to django run with runserver (ie non Daphne that is used in the docker) I get a "Temporary failure in name resolution" and I cannot understand what it refers to.
Any hint on what's the correct configuration to make Server-Sent event work?
It turns out that Django's view that responded with the sse was setting 2 identical headers and that makes Traefik bristle.
Complete explanation on the issue.
Once fixed the headers everything worked.
Related
I am having trouble setting up an nginx reverse proxy to serve content from different containers depending on the URI.
I have three docker images: (1) a flask app that serves a website (2) a bokeh app that serves a dashboard, (3) nginx to direct external requests to either (1) or (2). I build and run the stack using docker-compose. After running the stack (docker-compose -f docker-compose-local.yaml up):
I can, as expected, access (1) flask app via (3) nginx at localhost:1337.
I can, as expected, access (2) bokeh app directly at localhost:8002
I cannot access (2) the bokeh app via (3) nginx at localhost:1337/dboard
I would like the user to experience the dashboard as though it is part of the website, when on the backend the dashboard is in one container and all the other content is in a different container.
I have read a great many stackoverflow questions (e.g. here, here), serverfault questions (e.g. here, here), and blog posts, tried all manner of permutations of trailing slashes in the dashboard nginx location line and its proxy_pass, a number of proxy_set_header directives. I've gotten all sorts of "404 not found" and "502 bad gateway" responses, but so far no luck accessing the dashboard through nginx.
Any help much appreciated.
docker-compose-local.yaml:
version: '3.10'
services:
website:
image: website_image_name
build: .
command: gunicorn -b 0.0.0.0:8000 app_name:app
expose:
- 8000
env_file:
- ./app.env
dashboard:
image: timh-urban-vprm-dashboard:latest
command: mamba run --no-capture-output -n venv-urbanVPRM python urban_vprm_dashboard.py
expose:
- 8002
ports:
- 8002:8002
nginx:
image: nginx_image_name
build: ./nginx
ports:
- 1337:80
depends_on:
- website
- dashboard
links:
- website
- dashboard
nginx.conf:
upstream site {
server website:8000;
}
upstream dboard {
server dashboard:8002;
}
server {
listen 80;
server_name 0.0.0.0;
location / {
proxy_pass http://site;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /dboard/ {
proxy_pass http://dboard;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
}
Through Jelastic's dashboard, I created this:
I just clicked "New environment", then I selected nodejs. I added a docker image (of mailhog).
Now, I would like that port 80 of my environment serves the nodejs application. This is by default so. Therefore nothing to do.
In addition to this, I would like port 8080 (or any other port than 80, like port 5000 for example) of my environment serves mailhog, hosted on the docker image. To do that, I added the following lines to the nginx-jelastic.conf (right after the first server serving the nodejs app):
server {
listen *:8080;
listen [::]:8080;
server_name _;
location / {
proxy_pass http://mailhog_upstream;
}
}
where I have also defined mailhog_upstream like this:
upstream mailhog_upstream{
server 10.102.8.215; ### DEFUPPROTO for common ###
sticky path=/; keepalive 100;
}
If I now browse my environment's 8080 port, then I see ... the nodejs app. If I try any other port than 80 or 8080, I see nothing. Putting another server_name doesn't help. I tried several things but nothing seems to work. Why is that? What am I doing wrong here?
Then I tried to get rid of the above mailhog_upstream and instead write
server {
listen *:5000;
listen [::]:5000;
server_name _;
location / {
proxy_pass http://10.102.8.215;
}
}
Browsing the environment's port 5000 doesn't work either.
If I replace the IP of the nodejs' app with that of my mailhog service, then mailhog runs on port 80. I don't understand how I can make the nodejs app run on port 80 and the mailhog service on port 5000 (or any other port than 80).
Could someone enlighten me please?
After all those failures, I tried another ansatz. Assume the path my env is example.com/. What I've tried above is to get mailhog to work upon calling example.com:5000, which I failed doing. Then I tried to make mailhog available through a call to example.com/mailhog. In order to do that, I got rid of all my modifications above and completed the current server in nginx-jelastic.conf with
location /mailhog {
proxy_pass http://10.102.8.96:8025/;
add_header Set-Cookie "SRVGROUP=$group; path=/";
}
That works in the sense that if I know browse example.com/mailhog, then I get something on the page, but not exactly what I want: it's the mailhog's page without any styling. Also, when I call mailhog's API through example.com/mailhog/api/v2/messages, I get a successful response without body, when I should've received
{"total":0,"count":0,"start":0,"items":[]}
What am I doing wrong this time?
Edit
To be more explicit, I put the following manifest that exhibits the second problem with the nginx location.
Full locations list for your case is a following:
(please pay attention to URIs in upstreams, they are different)
location /mailhog { proxy_pass http://172.25.2.128:8025/; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection " upgrade"; }
location /mailhog/api { proxy_pass http://172.25.2.128:8025/api; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection " upgrade"; }
location /css { proxy_pass http://172.25.2.128:8025; }
location /js { proxy_pass http://172.25.2.128:8025; }
location /images { proxy_pass http://172.25.2.128:8025; }
that works for me with your application
# curl 172.25.2.127/mailhog/api/v2/messages
{"total":0,"count":0,"start":0,"items":[]}
The following ports are opened by default: 80, 8080, 8686, 8443, 4848, 4949, 7979.
Additional ports can be opened using:
endpoints - maps the container internal port to random external
via Jelastic Shared LB
Public IP - provides a direct access to all ports of your
container
Read more in the following article: "Container configuration - Ports". This one may also be useful:"Public IP vs Shared Load Balancer"
I was having trouble configuring an nginx reverse proxy within my development environment when I stumbled on a behaviour that I do not quite get.
So nginx is listening on port 8080. When I make a request to my development-server, I can access my development server on
localhost:8080
With the following directives:
server {
listen 8080;
server_name site.com;
location / {
proxy_pass http://localhost:3000/;
proxy_redirect off;
}
But when I put a known website in the proxy pass_directive like google or apple the behaviour is different. I can not access e. g. apple.com as localhost:8080 with the following directives - I am immediately pushed to the real website and not the localhost:
server {
listen 8080;
server_name site.com;
location / {
proxy_pass http://apple.com/;
proxy_redirect off;
}
How is that behaviour called and how is it achieved? Can you guys put me in the right direction to understanding this? Thanks.
This is the correct behavior for the proxy service, you can find docs here https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/
Some information regarding proxies here https://en.wikipedia.org/wiki/Proxy_server
Example: if you want to go to http://apple.com/apple-card/, you can point out to localhost:8080/apple-card and you will be redirected to /requested_path
I'm using proxies with docker containers just to route the requests to the correct application using different ports.
Short description:
Nginx running on docker, how to configure nginx so that it forwards calls to host.
Long description:
We have one web application which communicates to couple of backends (lets says rest1, rest2 and rest3). We are responsible for rest1.
Lets consider that I started rest1 manually on my pc and running on 2345 port. I want nginx (which is running in docker) to redirect all call torest1 to my own running instance(note, the instance is running on host, not any container and not in docker). And for rest2 and rest3 to some other docker node or may be some other server (who cares).
What I am looking for is:
docker-compose.yml configurations (if needed).
nginx configuration.
Thanks in advance.
Configure nginx like the following (make sure you replace IP of Docker Host) and save it as default.conf:
server {
listen 80;
server_name _;
location / {
proxy_pass http://<IP of Docker Host>;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Now bring up the container:
docker run -d --name nginx -p 80:80 -v /path/to/nginx/config/default.conf:/etc/nginx/conf.d/default.conf nginx
If you are using Docker Compose file version 3 you don't need any special config for docker-compose.yml file at all, just use the special DNS name host.docker.internal to reach a host service, as on the following nginx.conf example:
events {
worker_connections 1024;
}
http {
upstream host_service {
server host.docker.internal:2345;
}
server {
listen 80;
access_log /var/log/nginx/http_access.log combined;
error_log /var/log/nginx/http_error.log;
location / {
proxy_pass http://host_service;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $realip_remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}
}
Solution 1
Use network_mode: host, this will bind your nginx instance to host's network interface.
This could result in conflicts when running multiple nginx containers: every exposed port is binded to host's interface.
Solution 2
I'm running more nginx instances for every service I would like expose to outside world.
To keep the nginx configurations simple and avoid binding every nginx to host use the container structure:
dockerhost - a dummy container with network_mode: host
proxy - nginx container used as a proxy to host service,
link dockerhost to proxy, this will add an /etc/hosts entry in proxy contianer - we can use 'dockerhost' as a hostname in nginx configuration.
docker-compose.yaml
version: '3'
services:
dockerhost:
image: alpine
entrypoint: /bin/sh -c "tail -f /dev/null"
network_mode: host
proxy:
image: nginx:alpine
links:
- dockerhost:dockerhost
ports:
- "18080:80"
volumes:
- /share/Container/nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
default.conf
location / {
proxy_pass http://dockerhost:8080;
This method allows us to have have automated let's encrtypt certificates generated for every service running on my server. If interested I can post a gist about the solution.
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://host.docker.internal:3000;
}
}
Docker expose host address is host.docker.internal in Mac os
There a couple of things you have to keep in mind:
Docker compose (from version 3) by default uses the service name as hostname for inter container networking
Nginx need to know the upstream first
I strongly recommend mounting the default.conf directly into your docker-compose.yml.
Lastly you have to dockerize your backend to make use of docker internal networking.
An example repo where I use nginx and docker-compose in a full-stack project: https://gitlab.com/datails/api.
The following example have some prerequisites:
you have a folder structure like:
- backend/
- frontend/
- default.conf
- docker-compose.yml
Secondly the backend and front-end dit have a Dockerfile that exposes an application on port 3000.
Example default.conf:
upstream backend {
server backend:3000;
}
upstream frontend {
server frontend:3000;
}
server {
listen 80;
location /api {
proxy_pass http://backend;
}
location / {
proxy_pass http://frontend/;
}
}
Example docker-compose.yml:
version: '3.8'
services:
nginx:
image: nginx:1.19.4
depends_on:
- server
- frontend
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- '8080:80'
Then make sure you have your backend dockerized and called (in this case) backend as a service and a front-end (if needed) called frontend as a service in your docker-compose:
version: '3.8'
services:
nginx:
image: nginx:1.19.4
depends_on:
- server
- frontend
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf
ports:
- '8080:80'
frontend:
build: ./frontend
backend:
build: ./backend
This is a bare minimum example to get started. Hope this will help future developers.
lets say that now I have different app running on the same server on different path:
10.200.200.210/app1
10.200.200.210/app2
10.200.200.210/app3
I want to run each app on a different Docker container using nginx as a proxy.
I tried jwilder/nginx-proxy and works great if I use different domain names (app1.domain.com, app2.domain.com, etc), but I'm not able to use domains, I need to use the same IP.
also I can't use different ports like:
10.200.200.210:81/app1
10.200.200.210:82/app2
10.200.200.210:83/app3
all must work on port 80.
Is there a way to configure jwilder/nginx-proxy to do this?
Is there another Docker image like jwilder/nginx-proxy that make it.
or pls could you give me some hint to build an nginx docker container by myself?
In case if somebody is still looking for the answer.
jwilder/nginx-proxy allows you to use custom Nginx configuration either a proxy-wide or per-VIRTUAL_HOST basis.
Here's how can you do it with Per-VIRTUAL_HOST location configuration.
Inside your poject folder create another folder - "vhost.d".
Create file "whoami.local" with custom nginx configuration inside "vhost.d" folder. This file must have the same name as VIRTUAL_HOST!
./vhost.d/whoami.local
location /app1 {
proxy_pass http://app1:8000;
}
location /app2 {
proxy_pass http://app2:8000;
}
Create docker-compose.yml file.
./docker-compose.yml
version: '3'
services:
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- "8080:80"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /path/to/vhost.d:/etc/nginx/vhost.d:ro
gateway:
image: jwilder/whoami
environment:
- VIRTUAL_HOST=whoami.local
app1:
image: jwilder/whoami
app2:
image: jwilder/whoami
Run docker-compose up
Check configuration
In bash run:
$ curl -H "Host: whoami.local" localhost:8080
I'm 1ae273bce7a4
$ curl -H "Host: whoami.local" localhost:8080/app1
I'm 52b1a7b1992a
$ curl -H "Host: whoami.local" localhost:8080/app2
I'm 4adbd3f9e7a0
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6a659a4d4b0a jwilder/nginx-proxy "/app/docker-entrypo…" 54 seconds ago Up 53 seconds 0.0.0.0:8080->80/tcp nginxreverseproxy_nginx-proxy_1
4adbd3f9e7a0 jwilder/whoami "/app/http" 54 seconds ago Up 53 seconds 8000/tcp nginxreverseproxy_app2_1
52b1a7b1992a jwilder/whoami "/app/http" 54 seconds ago Up 53 seconds 8000/tcp nginxreverseproxy_app1_1
1ae273bce7a4 jwilder/whoami "/app/http" 54 seconds ago Up 53 seconds 8000/tcp nginxreverseproxy_gateway_1
You can also add "whoami.local" domain to /etc/hosts file and make calls to this domain directly.
/etc/hosts
...
127.0.0.1 whoami.local
...
Result:
$ curl whoami.local:8080
I'm 52ed6da1e86c
$ curl whoami.local:8080/app1
I'm 4116f51020da
$ curl whoami.local:8080/app2
I'm c4db24012582
Just use nginx image to create container,**do remember set net "host" **which will make your container share same address and port with host machine.mount nginx.conf file and config proxy table.for example:
docker command:
docker run --name http-proxy -v /host/nginx.conf:/etc/nginx/nginx.conf --net host -itd --restart always nginx
nginx.conf:
server {
listen 80;
location /app1 {
proxy_pass YOUR_APP1_URL;
}
location /app2 {
proxy_pass YOUR_APP2_URL;
}
}
Here is a full nginx.conf
It redirects all to root, and only /api to a different container.
Source and an example container using it
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log;
events {
worker_connections 1024;
}
http {
server {
listen 80;
location / {
proxy_pass http://frontend:3000/;
}
location /api {
proxy_pass http://backend/api;
}
}
}
just put this inside /etc/nginx/nginx.conf
worker_processes 1;
error_log /var/log/nginx/error.log;
events {
worker_connections 1024;
}
http {
server {
listen 80;
location /api {
proxy_pass http://awesome-api;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
}
Default bridge network has gateway on 172.17.0.1. You can use this IP address in your nginx.conf
server {
listen 80;
server_name example.com;
location /app1 {
proxy_pass http://172.17.0.1:81;
}
location /app2 {
proxy_pass http://172.17.0.1:82;
}
}
They will be accessible using port 80 from outside
You can check your bridge gateway IP address by running command docker network inspect bridge
My situation is a little bit different. I'm working on a project where Django and a couple of other apps sit behind nginx (acting as a reverse proxy)
The accepted solution did not work for me, and I think it is because the various apps do not serve files (i.e. /app1-uri/bla/blo/bli/ and /app1-uri/bla/blo/bli are exactly equivalent). All static files are gathered for nginx to serve.
The 'problematic' behavior is explained in the docs here. In essence, nginx picks up the uris without trailing slash and try to resolve them as resources, if it can't, it then redirects to /bla/blo/bli/ instead of /app1-uri/bla/blo/bli/
This is what finally worked for me. In this example app1-uri is repairapp.
server {
listen 80;
listen 443 ssl http2;
...
# This is the line that fixes the issue.
rewrite ^/repairapp/([^static].*[^/])$ /repairapp/$1/ permanent;
# This is nginx serving my static files
location /repairapp/static/ {
alias /var/www/repairapp/static/;
}
# This is the uri that maps to my app (no file serving here)
location /repairapp/ {
proxy_pass http://repairappcontainer:8000/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
...
Note the line rewrite ^/repairapp/([^static].*[^/])$ /repairapp/$1/ permanent;
It rewrites and adds a trailing forward slash to any (of my app) uris that misses it; except for those that start with /repairapp/static. Those uris map to resources that nginx will then serve.
To debug, open a shell in the nginx container and run curl -IL http://[server-name]/repairapp/[string-of-uris-without-trailing-slash] to see exactly what happens.