Nginx connectivity with vultr loadbalancer - nginx

We had many applications on single vultr cloud instance, but it has only one default healthcheck for a single https loadBalancer with SSL certificate.
so we used nginx to configure mutliple /backend URL with http specified and running using docker-compose to make applications running on a single network.
server {
listen 80;
listen [::]:80;
server_name *.example.com;
access_log /var/log/nginx/host.access.log main;
location / {
proxy_pass http://strapi-container:1337/;
}
location /chat {
proxy_pass http://rocketchat-container:3000;
}
location /auth {
proxy_pass http://keycloak-container:8080;
proxy_set_header Host $host;
}
}
}
The backend url is http://instance-ip/, http://instance-ip/chat, http://instance-ip/auth respectively
nginx:
image: nginx:1.20
container_name: nginx
ports:
- 80:80
restart: unless-stopped
volumes:
- ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
depends_on:
- strapi-cms
- rocketchat
- keycloak
networks:
- test-network
Everything works fine and we are able to access the applications through nginx default port 80 with the above backend URL's.
But our intentions is to somehow connect the nginx with HTTPS LoadBalancer in vultr, it should works as
For example: https://qa.example.com/, https://qa.example.com/chat, https://qa.example.com/auth

What you will want to do is setup a single forwarding rule on the Vultr Load Balancer.
Forwarding rule 443->443 for TLS on the instance
Forwarding rule 443->80 for TLS on the LB.
This will have the LB forward all incoming traffic on the defined LB port to your NGINX defined port. Then your nginx instance should route the the location to the appropriate proxy_pass you have defined.
As for the health check...Vultr Load Balancers only have a single health check as they were designed to work with single applications behind the LB. However, you could have a /health endpoint that when hit would check the status of all of your other applications and return 200 ok if they are all running.
We have some detailed docs available at https://www.vultr.com/docs/vultr-load-balancers
Full disclosure I am the Technical Lead on load balancers for Vultr.

Related

Trying to configure Let's Encrypt with Nginx but reverse proxy only works within local network?

I had a matrix synapse server set up in a docker container with a Caddy container being the reverse proxy and handling let's encrypt. It has been working just fine.
I am trying to move to Nginx but I am having a little trouble. I set up the reverse proxy and i can access my synapse server from port 80 with 192.168.0.23:80 and it goes to my synapse server even from machines that the synapse server isn't running on.
Here is my default file located in sites-available and linked to sites-enabled:
server {
listen 80;
server_name MY_DOMAIN.org;
location / {
proxy_pass http://127.0.0.1:8008;
}
}
And my nginx.conf is as follows:
http {
include mime.types;
include /etc/nginx/sites-enabled/*;
}
events {}
This is my docker-compose.yaml:
version: '3'
services:
synapse:
image: matrixdotorg/syanpse:latest
container_name: synapse
restart: unless-stopped
ports:
- 127.0.0.1:8008:8008
volumes:
- /home/me/Docker/Synapse/data:/data
environment:
- SYNAPSE_SERVER_NAME=my-server-name
- SYNAPSE_REPORT_STATS=no
- VIRTUAL_HOST=MY_DOMAIN.org
if I go into my browser and type: http://192.168.0.23 it brings up my matrix server. Yet if I go to http://MY_DOMAIN.org it doesn't load.
Does anyone see something I'm doing wrong?
Both ports 80 and 443 are allowed in ufw. I have read through many posts on reddit and stack exchange and none of the solutions are working. I am using No-IP's dynamic update client to get a DNS record and the same exact client has been working just fine with my original Caddy server so I do not believe it is an issue with No-IP.
I have tried disabling UFW but it still fails. I have checked if SELinux is enabled but it is disabled. I have tried updating the default file in sites-available to listen on MY_PUBLIC_IP:80. I have ports 80 and 443 forwarded in my router and know that the forwarding is working because if I turn the Caddy server back on it has HTTPS enabled.
I have added an entry to my hosts file which lets me access it without need a NAT hairpin but doesn't help accessing it from outside my LAN.

nginx + cloudflare + digitalocean = 521

I'm trying host a website with multiple subdomains (created with Cloudflare, which also provides SSL) hosted on DigitalOcean with Nginx serving as a reverse proxy.
My Cloudflare Configs
DNS setup:
Type ~ Name ~ Value
A ~ api ~ MyDigitalOceanIPv4
A ~ example.com ~ MyDigitalOceanIPv4
A ~ www ~ MyDigitalOceanIPv4
Crypto setup:
SSL: Full (strict)
Always use HTTPS: On
Automatic HTTPS Rewrites: On
I've also used Cloudflare to Create Certificate (and followed their instructions to set it up with Nginx)
My Nginx config:
server {
listen 443;
server_name example.com www.example.com;
ssl on;
ssl_certificate /srv/example.com/cloudflare.pem;
ssl_certificate_key /srv/example.com/cloudflare.key;
location / {
proxy_pass http://localhost:8000;
}
}
server {
listen 443;
server_name api.example.com;
ssl on;
ssl_certificate /srv/example.com/cloudflare.pem;
ssl_certificate_key /srv/example.com/cloudflare.key;
location / {
proxy_pass http://localhost:8080;
}
}
I have opened for all TCP ports on DigitalOcean, and if I try to open MyDigitalOceanIPv4:8000 in my browser then my website (hosted in a Docker container) successfully loads. However, if I try to open my website "example.com" then I get Cloudflare's 521 web server is down message.
I have also verified that the Cloudflare SSL key paths and content are correct, nginx -t shows no errors, and I've made sure to restart nginx after making changes.
I have also tried to whitelist Cloudflare's IPs using my Nginx config file but it didn't work.
If I try to telnet MyDigitalOceanIPv4 443 or 80 then I get telnet: Unable to connect to remote host: Connection refused.
Inside my DigitalOcean instance I have tried to curl http://localhost:8000 which successfully prints my website content.
I suspect there's some DigitalOcean setting I need to configure, or there's something wrong with my Nginx file (even though I've successfully used same Nginx config on a different cloud provider), but feel like I've tried everything..

Issue with setting up Nginx with multiple Docker containers

Currently I want to setup one server that has a Docker WordPress and Nginx that serves as a proxy in front. I would like in future to be able have multiple WordPress, NodeJS, ROR, etc, sitting behind this Nginx proxy.
When ever I try to connect to my server on port 80 I get a 403 forbidden.
I am able to build a Docker WordPress image and can connect to it on port 8080 on a remote PC.
Here is the compose.yml for my Docker WordPress:
version: "3.1"
services:
my_wordpress:
image: wordpress
ports:
- 8080:80
environment:
WORDPRESS_DB_PASSWORD: password
WORDPRESS_DB_HOST: my_mysql_wordpress
my_mysql_wordpress:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: password
This is the part when I try to build a Nginx container I am getting a 403 forbidden.
Nginx DockerFile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf
Nginx compose.yml:
version: "2"
services:
web:
restart: always
image: nginx
ports:
- "80:80"
volumes:
- /path/in/vm/www:/usr/share/nginx/html
external_links:
- mywordpress_wordpress_1:mywordpress
networks:
default:
external:
name: mywordpress_default
Nginx nginx.conf:
http {
#...
upstream wordpress {
server mywordpress:8080;
}
#...
server {
listen 80;
server_name 192.168.1.124 test.me;
location / {
proxy_pass http://wordpress/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cache_bypass $http_upgrade;
}
}
}
Now for me it would seem that this most likely has to do with my nginx.conf as I am still able to connect to my WordPress site on port 8080. As well as I stated I am also able to connect to my Nginx proxy and I don't see any errors when it launches.
Is what I'm trying to doing even possible or do I need to have the Nginx application sitting on the OS and not inside a docker container?
You are putting Nginx and Wordpress in 2 different compose files. If you are running then on same machines then
external_links:
- mywordpress_wordpress_1:mywordpress
Above would not work if you are on different machines. Also make sure the external link you are using the correct name by checking docker ps.
Also check the logs of your nginx container to see if it is showing in any error. Because the error log will give a pointer as to why a 403 is being thrown, and it could be that the proxy_pass is not able to connect to your wordpress server because of the way you have configured it.
If you are running these compose files on different machines then instead of external_links use extra_hosts
extra_hosts:
- "mywordpress:<IP of the wordpress machine>"
If I am right, You want to run multiple wordpress docker images and use nginx to reverse proxy to the wordpress instances. In that Use-case, The nginx should sit on your OS and not inside a docker image. That way, the nginx will have the ability to proxy to ports on your OS which are tied to the wordpress containers.

How to use Nginx to connect to my app in Docker image?

My Nginx is not in docker image. My app is in docker image. They both live on the same server.
I don't want Nginx in a docker image, since it looks awful complex for me to configure. But my app is running in a docker container.
How to configure Nginx to use the docker image which my app is running in?
Here is my Nginx config file:
server {
listen 80;
server_name my.domain.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl;
server_name www.nicolasxu.space nicolasxu.space;
# add Strict-Transport-Security to prevent man in the middle attacks
add_header Strict-Transport-Security "max-age=31536000";
ssl_certificate /root/.ssh/nicolasxu.space.cert;
ssl_certificate_key /root/nicolasxu.space.key;
[....]
}
To easily setup nginx (in docker host) as a reverse proxy in front of a dockerized webapp you could just --publish the port of your webapp and route the trafic to this port:
Run your docker container with --publish argument to bind host port with container's webapp port, for instance with a jenkins container I would do:
docker run --publish 127.0.0.1:8080:8080 --name jenkins jenkins
This binds port 8080 of the container to port 80 on localhost's 127.0.0.1 of the host machine (this avoids port 8080 to be opened to anyone if you don't use any firewall). The Docker User Guide explains in detail how to manipulate ports in Docker.
Forward all incoming trafic as a reverse proxy to the local container your port (8080 in my example)
server {
...
listen 443 ssl;
server_name www.nicolasxu.space nicolasxu.space;
...
ssl_certificate ...
location / {
# forward all the trafic to docker container's published port
proxy_pass http://localhost:8080;
}
}
Setting SSL on nginx and routing the trafic as HTTP to dockerized webapp is a good practice and will work like a charm.
Edit
For maximum performances, you can also use :
docker run --network=host ...
When using --network=host, docker will instruct the container to use the hosts networking stack. You won't have to --publish ports on host as it is the same network stack, and web application will be available on it's native port.

Docker nginx proxy to host

Short description:
Nginx running on docker, how to configure nginx so that it forwards calls to host.
Long description:
We have one web application which communicates to couple of backends (lets says rest1, rest2 and rest3). We are responsible for rest1.
Lets consider that I started rest1 manually on my pc and running on 2345 port. I want nginx (which is running in docker) to redirect all call torest1 to my own running instance(note, the instance is running on host, not any container and not in docker). And for rest2 and rest3 to some other docker node or may be some other server (who cares).
What I am looking for is:
docker-compose.yml configurations (if needed).
nginx configuration.
Thanks in advance.
Configure nginx like the following (make sure you replace IP of Docker Host) and save it as default.conf:
server {
listen 80;
server_name _;
location / {
proxy_pass http://<IP of Docker Host>;
index index.html index.htm;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Now bring up the container:
docker run -d --name nginx -p 80:80 -v /path/to/nginx/config/default.conf:/etc/nginx/conf.d/default.conf nginx
If you are using Docker Compose file version 3 you don't need any special config for docker-compose.yml file at all, just use the special DNS name host.docker.internal to reach a host service, as on the following nginx.conf example:
events {
worker_connections 1024;
}
http {
upstream host_service {
server host.docker.internal:2345;
}
server {
listen 80;
access_log /var/log/nginx/http_access.log combined;
error_log /var/log/nginx/http_error.log;
location / {
proxy_pass http://host_service;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $realip_remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}
}
Solution 1
Use network_mode: host, this will bind your nginx instance to host's network interface.
This could result in conflicts when running multiple nginx containers: every exposed port is binded to host's interface.
Solution 2
I'm running more nginx instances for every service I would like expose to outside world.
To keep the nginx configurations simple and avoid binding every nginx to host use the container structure:
dockerhost - a dummy container with network_mode: host
proxy - nginx container used as a proxy to host service,
link dockerhost to proxy, this will add an /etc/hosts entry in proxy contianer - we can use 'dockerhost' as a hostname in nginx configuration.
docker-compose.yaml
version: '3'
services:
dockerhost:
image: alpine
entrypoint: /bin/sh -c "tail -f /dev/null"
network_mode: host
proxy:
image: nginx:alpine
links:
- dockerhost:dockerhost
ports:
- "18080:80"
volumes:
- /share/Container/nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
default.conf
location / {
proxy_pass http://dockerhost:8080;
This method allows us to have have automated let's encrtypt certificates generated for every service running on my server. If interested I can post a gist about the solution.
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://host.docker.internal:3000;
}
}
Docker expose host address is host.docker.internal in Mac os
There a couple of things you have to keep in mind:
Docker compose (from version 3) by default uses the service name as hostname for inter container networking
Nginx need to know the upstream first
I strongly recommend mounting the default.conf directly into your docker-compose.yml.
Lastly you have to dockerize your backend to make use of docker internal networking.
An example repo where I use nginx and docker-compose in a full-stack project: https://gitlab.com/datails/api.
The following example have some prerequisites:
you have a folder structure like:
- backend/
- frontend/
- default.conf
- docker-compose.yml
Secondly the backend and front-end dit have a Dockerfile that exposes an application on port 3000.
Example default.conf:
upstream backend {
server backend:3000;
}
upstream frontend {
server frontend:3000;
}
server {
listen 80;
location /api {
proxy_pass http://backend;
}
location / {
proxy_pass http://frontend/;
}
}
Example docker-compose.yml:
version: '3.8'
services:
nginx:
image: nginx:1.19.4
depends_on:
- server
- frontend
volumes:
- ./nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- '8080:80'
Then make sure you have your backend dockerized and called (in this case) backend as a service and a front-end (if needed) called frontend as a service in your docker-compose:
version: '3.8'
services:
nginx:
image: nginx:1.19.4
depends_on:
- server
- frontend
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf
ports:
- '8080:80'
frontend:
build: ./frontend
backend:
build: ./backend
This is a bare minimum example to get started. Hope this will help future developers.

Resources