Unable to connect docker nginx with docker ubuntu - nginx

I have noticed an issue with docker nginx which is not the case when nginx is running on the host machine (apt-get install). Here is how to reproduce my issue:
solution A: 'nc' on container 1, 'nginx' on container 2, 'curl' on host
docker stop $(docker ps -aq)
docker rm $(docker ps -aq)
'nc' on container 1
docker run -ti --name agitated_stallman ubuntu:14.04 bash
nc -l 4545
'nginx' on container 2
LOLPATH=$HOME/testdocker
echo $LOLPATH
mkdir -p $LOLPATH
cd $LOLPATH
subl mple.conf
.
server {
listen 80;
root /var/www/html;
location /roz {
proxy_pass http://neocontainer:4545;
proxy_set_header Host $host;
}
}
.
docker run --link agitated_stallman:neocontainer -v $LOLPATH/mple.conf:/etc/nginx/sites-available/default -p 12345:80 nginx:1.9
'curl' on host
sudo apt-get install curl
curl http://localhost:12345/roz
ERROR response from 'nginx':
2016/03/04 19:59:18 [error] 8#8: *3 open() "/usr/share/nginx/html/roz" failed (2: No such file or directory), client: 172.17.0.1, server: localhost, request: "GET /roz HTTP/1.1", host: "localhost:12345"
172.17.0.1 - - [04/Mar/2016:19:59:18 +0000] "GET /roz HTTP/1.1" 404 169 "-" "curl/7.45.0" "-"
solution B: 'nginx' on host, 'nc' on host, 'curl' on host
'nginx' on host
sudo apt-get install nginx
sudo subl /etc/nginx/sites-available/default
.
server {
listen 80;
root /var/www/html;
location /roz {
proxy_pass http://localhost:4646;
proxy_set_header Host $host;
}
}
.
sudo service nginx restart
'nc' on host
nc -l 4646
'curl' on host
sudo apt-get install curl
curl http://localhost:80/roz
SUCCESS response from 'nc':
GET /roz HTTP/1.0
Host: localhost
Connection: close
User-Agent: curl/7.45.0
Accept: */*

In short: run nginx container with -v $LOLPATH/mple.conf:/etc/nginx/conf.d/default.conf
nginx:1.9 docker image currently uses nginx package from nginx's own repository, not from official debian repository. If you examine that package, you'll find that /etc/nginx/nginx.conf does include only from /etc/nginx/conf.d/*.conf, and that package ships with pre-included /etc/nginx/conf.d/default.conf:
server {
listen 80;
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
# other not important stuff
# ...
}
So your config is not used at all, which explains the open() "/usr/share/nginx/html/roz" failed error.
When you install nginx directly on host, you probably use official debian repository, which has different main config file, which in turn does include /etc/nginx/sites-available/*, and your config is actually used.

Related

Wildlfy with Nginx not working properly

We had installed wildfly for a couple of time working correctly. We configured right now Nginx as reverse proxy for wildfly.
We're getting on OPTIONS method 405 Method Not Allowed. Here is the configuration of nginx.
/etc/nginx/conf.d/wildfly.conf
upstream wildfly {
server 127.0.0.1:8081;
}
server {
listen 8080;
server_name guest1;
location/ {
proxy_pass http://wildfly;
}
}
Error obtained after installing nginx:
This is the error got by nginx:
2017/06/23 08:16:54 [crit] 1386#0: *9 connect() to 127.0.0.1:8081 failed (13: Permission denied) while connecting to upstream, client: 172.28.128.1, server: guest1, request: "OPTIONS /commty/cmng/users HTTP/1.1", upstream: "http://127.0.0.1:8081/commty/cmng/users", host: "guest1:8080"
What I'm missing?
I've done the following to finally make it work on CentOS7 + Wildfly.
Vagrant up
Install NGINX
yum install epel-release
yum install nginx
Configure /etc/nginx/nginx.conf (default configuration)
Configure /etc/nginx/conf.d/wildfly.conf (using port 80 for nginx and 8080 for wildfly)
upstream wildfly {
server 127.0.0.1:8080;
}
server {
listen 80;
server_name guest1;
location / {
proxy_pass http://wildfly;
}
}
Also set SELinux permissive for let nginx work.
$ setenforce permissive
After that wildfly is working properly through nginx.

Nginx as reverse proxy for docker containers

I'm trying to get Nginx to reverse proxy connections within a lan to several web applications including ones inside docker containers.
Both webapps are reachable with the proxy_pass url
I'm using the following dockerfile:
# Set the base image to Ubuntu
FROM ubuntu
RUN apt-get update
RUN apt-get install -y nginx
RUN rm -v /etc/nginx/nginx.conf
RUN echo "daemon off; \n\
\n\
worker_processes 1; \n\
events { worker_connections 1024; } \n\
\n\
http { \n\
\n\
server { \n\
listen 99; \n\
\n\
server_name dashboard; \n\
location / { \n\
proxy_pass http://dashboard:80; \n\
} \n\
location /app1 { \n\
proxy_pass http://otherhostname:9000/app1; \n\
} \n\
} \n\
} \n\
" >> /etc/nginx/nginx.conf
EXPOSE 99
CMD service nginx start
When running this as a service (container) I can reach app1, but not the dashboard.
The weird thing is that I had this working before, and I'm pretty sure I did not change anything fundamental to the dockerfile. Am I missing something?
EDIT: (I have currently exposed the dashboard on port 80, and am testing on 99 with nginx)
I run the nginx container with:
docker service create \
--replicas 1 \
--name nginx \
-p 99:99 \
nginx_image
the dashboard also has the correct port exposed.
docker service create \
--replicas 1 \
--name dashboard \
-p 80:8080 \
dashboard_image
Looking in the nginx error.log I found:
2016/11/08 08:46:41 [error] 25#25: *42 upstream timed out (110: Connection timed out) while connecting to upstream, client: 10.255.0.3, server: dashboard, request: "GET / HTTP/1.1", upstream: "http://dockerhostip:80/", host: "dashboard:99"
Nginx is working as intended. I found when changing the proxy pass to example.com it works fine. It must be something that changed in the dashboard that messes things up.

nginx docker container: 502 bad gateway response

I've a service listening to 8080 port. This one is not a container.
Then, I've created a nginx container using official image:
docker run --name nginx -d -v /root/nginx/conf:/etc/nginx/conf.d -p 443:443 -p 80:80 nginx
After all:
# netstat -tupln | grep 443
tcp6 0 0 :::443 :::* LISTEN 3482/docker-proxy
# netstat -tupln | grep 80
tcp6 0 0 :::80 :::* LISTEN 3489/docker-proxy
tcp6 0 0 :::8080 :::* LISTEN 1009/java
Nginx configuration is:
upstream eighty {
server 127.0.0.1:8080;
}
server {
listen 80;
server_name eighty.domain.com;
location / {
proxy_pass http://eighty;
}
}
I've checked I'm able to connect with with this server with # curl http://127.0.0.1:8080
<html><head><meta http-equiv='refresh'
content='1;url=/login?from=%2F'/><script>window.location.replace('/login?from=%2F');</script></head><body
style='background-color:white; color:white;'>
...
It seems running well, however, when I'm trying to access using my browser, nginx tells bt a 502 bad gateway response.
I'm figuring out it can be a problem related with the visibility between a open by a non-containerized process and a container. Can I container stablish connection to a port open by other non-container process?
EDIT
Logs where upstream { server 127.0.0.1:8080; }:
2016/07/13 09:06:53 [error] 5#5: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 62.57.217.25, server: eighty.domain.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "eighty.domain.com"
62.57.217.25 - - [13/Jul/2016:09:06:53 +0000] "GET / HTTP/1.1" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-"
Logs where upstream { server 0.0.0.0:8080; }:
62.57.217.25 - - [13/Jul/2016:09:00:30 +0000] "GET / HTTP/1.1" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-" 2016/07/13 09:00:30 [error] 5#5: *1 connect() failed (111: Connection refused) while connecting to upstream, client:
62.57.217.25, server: eighty.domain.com, request: "GET / HTTP/1.1", upstream: "http://0.0.0.0:8080/", host: "eighty.domain.com" 2016/07/13 09:00:32 [error] 5#5: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 62.57.217.25, server: eighty.domain.com, request: "GET / HTTP/1.1", upstream: "http://0.0.0.0:8080/", host: "eighty.domain.com"
62.57.217.25 - - [13/Jul/2016:09:00:32 +0000] "GET / HTTP/1.1" 502 173 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0" "-"
Any ideas?
The Problem
Localhost is a bit tricky when it comes to containers. Within a docker container, localhost points to the container itself.
This means, with an upstream like this:
upstream foo{
server 127.0.0.1:8080;
}
or
upstream foo{
server 0.0.0.0:8080;
}
you are telling nginx to pass your request to the local host.
But in the context of a docker-container, localhost (and the corresponding ip addresses) are pointing to the container itself:
by addressing 127.0.0.1 you will never reach your host machine, if your container is not on the host network.
Solutions
Host Networking
You can choose to run nginx on the same network as your host:
docker run --name nginx -d -v /root/nginx/conf:/etc/nginx/conf.d --net=host nginx
Note that you do not need to expose any ports in this case.
This works though you lose the benefit of docker networking. If you have multiple containers that should communicate through the docker network, this approach can be a problem. If you just want to deploy nginx with docker and do not want to use any advanced docker network features, this approach is fine.
Access the hosts remote IP Address
Another approach is to reconfigure your nginx upstream directive to directly connect to your host machine by adding its remote IP address:
upstream foo{
//insert your hosts ip here
server 192.168.99.100:8080;
}
The container will now go through the network stack and resolve your host correctly:
You can also use your DNS name if you have one. Make sure docker knows about your DNS server.
For me helped this line of code proxy_set_header Host $http_host;
server {
listen 80;
server_name localhost;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_redirect off;
proxy_pass http://myserver;
}
Just to complete other answers, I'm using mac for development and using host.docker.internal directly on upstream worked for me and no need to pass the host remote IP address. Here is config of the proxy nginx:
events { worker_connections 1024; }
http {
upstream app1 {
server host.docker.internal:81;
}
upstream app1 {
server host.docker.internal:82;
}
server {
listen 80;
server_name app1.com;
location / {
proxy_pass http://app1;
}
}
server {
listen 80;
server_name app2.com;
location / {
proxy_pass http://app2;
}
}
}
As you can see, I used different ports for different apps behind the nginx proxy. I used port 81 for the app1 and port 82 for the app2 and both app1 and app2 have their own nginx containers:
For app1:
docker run --name nginx -d -p 81:80 nginx
For app2:
docker run --name nginx -d -p 82:80 nginx
Also, please refer to this link for more details:
docker doc for mac
What you can do is configure proxy_pass that from container perspective the adress will be pointing to your real host.
To get host address from container perspective you can do as following on Windows with docker 18.03 (or more recent):
Run bash on container from host where image name is nginx (works on Alpine Linux distribution):
docker run -it nginx /bin/ash
Then run inside container
/ # nslookup host.docker.internal
Name: host.docker.internal
Address 1: 192.168.65.2
192.168.65.2 is the host's IP - not the bridge IP like in spinus accepted answer.
I am using here host.docker.internal:
The host has a changing IP address (or none if you have no network access). From 18.03 onwards our recommendation is to connect to the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. This is for development purpose and will not work in a production environment outside of Docker for Windows.
Then you can change nginx config to:
proxy_pass http://192.168.65.2:{your_app_port};
and it should work fine.
Remember to provide the same port as your local application runs with.
# the upstream component nginx needs to connect to
upstream django {
# server unix:///path/to/your/mysite/mysite.sock; # for a file socket
server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
location / {
uwsgi_pass django;
include /path/to/your/mysite/uwsgi_params; # the uwsgi_params file you installed
}
complete reference: https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html
nginx.sh
ip=$(ifconfig | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*' | grep -v '127.0.0.1' | head -n 1)
docker run --name nginx --add-host="host:${ip}" -p 80:80 -d nginx
nginx.conf
location / {
...
proxy_pass http://host:8080/;
}
It‘s works for me
I had this issue and it turned out to be an issue with the docker container not starting up due to a permissions issue.
In my case running
docker-compose ps
showed that the container had not started and exited with status 1. Turns out the permissions had been lost in migrating to a new machine. Adjusting the permissions to a know staff user on the parent directory fixed the problem for me and I was then able to start docker service where as previously I was getting
nginx_1_c18a7f6f7d6d | chown: /var/www/html: Operation not permitted

Dockerized Nginx upstream error serving separate Docker container with Flask/uWSGI app

I am experiencing the following error with my multi-container Docker setup after running docker-compose build && docker-compose up and attempting to hit my index page:
[error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: localhostz, request: "GET / HTTP/1.1", upstream: "uwsgi://172.17.0.39:8000", host: "192.168.99.100"
Here is my docker-compose.yml:
web:
restart: always
build: ./web-app
expose:
- "8000"
command: /usr/local/bin/uwsgi --ini sample-uwsgi.ini
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
links:
- web:web
nginx/Dockerfile
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
ADD sample-nginx.conf /etc/nginx/conf.d/
nginx/sample-nginx.conf
upstream flask {
server web:8000;
}
server {
listen 80;
server_name localhostz;
charset utf-8;
client_max_body_size 75M;
location / {
uwsgi_pass flask;
include uwsgi_params;
}
}
web-app/Dockerfile
FROM ansible/ubuntu14.04-ansible:stable
WORKDIR /root
ADD application.py application.py
ADD requirements.txt requirements.txt
ADD sample-uwsgi.ini sample-uwsgi.ini
ADD ansible /srv/ansible
WORKDIR /srv/ansible
RUN ansible-playbook container-bootstrap.yml -c local
web-app/sample-uswgi.ini
[uwsgi]
module = application
callable = app
master = true
processes = 5
socket = web:8000
chown-socket = www-data:www-data
vacuum = true
enable-threads=True
die-on-term = true
Please do not post suggestions regarding a single container setup. I am doing as an exercise in being able to scale Docker app containers served under a single nginx container.
Secret sauce was changing the socket line in sample-uwsgi.ini to:
socket = 0.0.0.0:8000

Docker containers experiencing socket issue (separate Flask + Nginx containers)

I am experiencing the following error with my multi-container Docker setup after running docker-compose build && docker-compose up and attempting to hit my index page:
[crit] 8#8: *1 connect() to unix:/tmp/uwsgi.sock failed (2: No such file or directory) while connecting to upstream, client: 192.168.99.1, server: localhost, request: "GET / HTTP/1.1", upstream: "uwsgi://unix:/tmp/uwsgi.sock:", host: "192.168.99.100"
Here is my docker-compose.yml:
web:
restart: always
build: ./web-app
expose:
- "8000"
command: /usr/local/bin/uwsgi --ini sample-uwsgi.ini
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
links:
- web:web
nginx/Dockerfile
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
ADD sample-nginx.conf /etc/nginx/conf.d/
nginx/sample-nginx.conf
server {
listen 80;
server_name localhost;
charset utf-8;
client_max_body_size 75M;
location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/uwsgi.sock;
}
}
web-app/Dockerfile
FROM ansible/ubuntu14.04-ansible:stable
WORKDIR /root
ADD application.py application.py
ADD requirements.txt requirements.txt
ADD sample-uwsgi.ini sample-uwsgi.ini
ADD ansible /srv/ansible
WORKDIR /srv/ansible
RUN ansible-playbook container-bootstrap.yml -c local
web-app/sample-uswgi.ini
[uwsgi]
module = application
callable = app
master = true
processes = 5
socket = /tmp/uwsgi.sock
chown-socket = www-data:www-data
vacuum = true
enable-threads=True
die-on-term = true
UPDATE
On advice from #kryten I will be using TCP/IP
Updated nginx.conf:
server {
listen 80;
server_name localhost;
charset utf-8;
client_max_body_size 75M;
location / {
uwsgi_pass localhost:8000;
include uwsgi_params;
}
}
Updated uwsgi.ini:
[uwsgi]
module = application
callable = app
master = true
processes = 5
socket = localhost:8000
chown-socket = www-data:www-data
vacuum = true
enable-threads=True
die-on-term = true
and am now pursuing the following error:
[error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 192.168.99.1, server: localhost, request: "GET / HTTP/1.1", upstream: "uwsgi://127.0.0.1:8000", host: "192.168.99.100"
Since web and nginx are separate containers, nginx needs to connect to another computer over TCP. Linking the containers does most of the work already, you just have to point the upstream to web:8000 instead of localhost.
It looks like one of your applications is attempting to connect via a Unix socket instead of over TCP/IP.
This will not work from different containers, because the filesystem in one container (where the socket lives) is not accessible in the other container.
The solution is to reconfigure your application to connect over TCP/IP instead of a Unix socket.
It might be possible to connect by exposing the location in the filesystem where the socket resides to the other container, but I've never tried this & don't know if it would work.

Resources