I am using Nginx docker image on port 80 in my VM (address x.y.z.w). So when I try http://x.y.z.w in my browser, it is showing me Nginx index file.
Now I am configuring (basically proxy passing) it for my two docker images running on same VM on ports 8081 and 8082. What I want:
when I type http://x.y.z.w/a it should go to http://x.y.z.w:8081
when I type http://x.y.z.w/b it should go to http://x.y.z.w:8082
For this, I changed a portion in my conf file:
location /a {
rewrite ^/a(.*) /$1 break;
proxy_pass http://x.y.z.w:8081 ;
}
location /b {
rewrite ^/b(.*) /$1 break;
proxy_pass http://x.y.z.w:8082 ;
}
It is working as expected. But as all the images are in same machine (have same IP), I want to use localhost instead of x.y.z.w. But it is not working with localhost.
Basically, I don't want to use the hardcoded IP (x.y.z.w) in links, as the IP can change in the future.
Is there any way, Nginx can know the variable IP on which it is running and I may use that IP. Or how can it work with localhost with some modification?
You can't use localhost to address another docker container, since it refers to the loopback of the container and not of the host. As #Tuan suggested, you can link (https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/) containers, so they can communicate via their names. E.g.:
docker run --rm -ti --name backend nginx:latest
docker run --rm -ti --name proxy --link backend:backend nginx:latest
Now proxy knows about backend.
I just replaced every occurrence of x.y.z.w in my nginx.conf with $host and it is working fine.
For example, instead of proxy_pass http://x.y.z.w:8081 ;,
wrote it proxy_pass http://$host:8081 ;
Now if my IP (x.y.z.w) changes, it does not effect to my applications.
Related
Is there a way to disable the default EXPOSE 80 443 instruction in the nginx docker file without creating my own image?
I'm using Docker Nginx image and trying to expose only port 443 in the following way:
docker run -itd --name=nginx-test --publish=443:443 nginx
But I can see using docker ps -a that the container exposes port 80 as well:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ddc0bca08acc nginx "nginx -g 'daemon off" 17 seconds ago Up 16 seconds 80/tcp, 0.0.0.0:443->443/tcp nginx-test
How can I disable it?
The expose instruction is in the docker file which the image is built from.
You need to create your own customized Image for that.
To get the job done:
First locate the dockerfile for the official nginx (library)
Then Edit the dockerfile's expose instruction to 443 only.
Now build your own image modified image using official(customized) dockerfile.
To answer your edited question:
Docker uses iptables, While you could manually update the firewall rules to make the service unavailable at a certain port, you would not be able to unbind the Docker proxy. So port 80 will still be consumed on the docker host and docker proxy.
according to nginx docker image configuration , you can set this before container starts passing an environment var like :
docker run -itd -e NGINX_PORT=443 --name=nginx-test nginx
see :
using environment variables in nginx configuration
then in your nginx you can set :
listen ${NGINX_PORT};
There is a workaround to free the port (but not to unexpose it). I tried avoiding to publish the port but it didn't work and I got errors about the por being already in use anyway. Until I found that the trick is to publish the exposed port but mapped to a different one.
Let me explain with an example.
This will still try to use port 80:
docker up -p 443:443
But this will use 443 and some other random port you pick
docker up -p 443:443 -p<some free port>:80
You can do this in your commands, docker-compose or ansible playbooks to be able to start more than one instance on the same machine. (ie: nginx, which exposes port 80 by default)
I do this from docker-compose and ansible too.
I have 5 lamp containers (tutum/lamp) with mounted ports as follows:
127.0.0.1:81:80
127.0.0.1:82:80
127.0.0.1:83:80
127.0.0.1:84:80
127.0.0.1:85:80
What I would like to do is put an NGINX in front of them so that it redirects to the appropriate containers depending on the URL. For example, let's assume that the host IP is 12.45.5.113. Then, when I visit 12.45.5.113/c1/ I want to get redirected to home page of container 127.0.0.1:81:80, when I visit 12.45.5.113/c2/ I want to get redirected to home page of container 127.0.0.1:82:80 and so on so forth.
How should the NGINX configuration look like? Should I installed NGINX on the host with apt-get install or it could be possible to install it as a additional container too?
I think easiest approach is to launch nginx in container.
docker run --port 80:80 --link c1 ... --link cn ... nginx
with config like (can be mounted from host by --volume argument to docker run):
{
listen 80;
location /c1/ {
proxy_pass http://c1;
}
...
location /cn/ {
proxy_pass http://cn;
}
}
this way it will redirect all request as you wish using Docker container linking mechanism (all request will be routed through bridge network).
For more information check Docker documentation: https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/#/connect-with-the-linking-system
I am trying to get rid of deprecated Docker links in my configuration. What's left is getting rid of those Bad Gateway nginx reverse proxy errors when I recreated a container.
Note: I am using Docker networks in bridge mode. (docker network create nettest)
I am using the following configuration snippet inside nginx:
location / {
resolver 127.0.0.1 valid=30s;
set $backend "http://confluence:8090";
proxy_pass $backend;
I started a container with hostname confluence on my Docker network with name nettest.
Then I started the nginx container on network nettest.
I can ping confluence from inside the nginx container
confluence is listed inside the nginx container's /etc/hosts file
nginx log says send() failed (111: Connection refused) while resolving, resolver: 127.0.0.1:53
I tried the docker network default dns resolver 127.0.0.11 from /etc/resol.conf
nginx log says confluence could not be resolved (3: Host not found)
Anybody knows how to configure nginx resolver with Docker Networks or an alternative on how to force Nginx to correctly resolve the Docker network hostname?
First off, you should be using the Docker embedded DNS server at 127.0.0.11.
Your problem could be caused by 1 of the following:
nginx is trying to use IPv6 (AAAA record) for the DNS queries.
See https://stackoverflow.com/a/35516395/1529493 for the solution.
Basically something like:
http {
resolver 127.0.0.11 ipv6=off;
}
This is probably no longer a problem with Docker 1.11:
Fix to not forward docker domain IPv6 queries to external servers
(#21396)
Take care that you don't accidentally override the resolver configuration directive. In my case I had in the server block resolver 8.8.8.8 8.8.4.4; from Mozilla's SSL Configuration Generator, which was overriding the resolver 127.0.0.11; in the http block. That had me scratching my head for a long time...
Maybe you should check your container's /etc/resolv.conf
It shows your container's correct DNS config and then use that DNS server IP for resolver.
127.0.0.11 does not works in Rancher
I was running "node:12.18-alpine" with angular frontend and hit the same problem with proxy_pass.
Locally it was working with:
resolver 127.0.0.11;
As simple as that! Just execute:
$ cat /etc/resolv.conf | grep nameserver
In your container to get this ip address.
However, when deploying to kubernetes (AWS EKS) I got the very same error:
failed (111: Connection refused) while resolving, resolver: 127.0.0.11:53
Solution:
First solution was to find out the IP of the kube-dns service like below:
$ kubectl get service kube-dns -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 172.20.0.10 <none> 53/UDP,53/TCP 178d
Simple replacing IP for CLUSTER-IP worked like a charm.
Later, after some more doc digging, I find out that I could reference the service by name (which is little bit more elegant and resilient):
resolver kube-dns.kube-system valid=10s;
My problem was $request_uri at the end. After adding it at the end of uri and changing the 127.0.0.1 to 127.0.0.11 solved my issue. I hope it will help people to not spend hours on this.
location /products {
resolver 127.0.0.11;
proxy_pass http://products:3000$request_uri;
}
In several cases where I had this error, adding resolver_timeout 1s; to the Nginx config solved the issue. Most of the time I don't have a resolver entry.
Edit: what also worked for containers where I could explicitly define a nameserver: resolver DNS-IP valid=1s;
We hit this with docker containers on windows trying to lookup host.docker.internal using the docker internal resolver at 127.0.0.11. All queries would resolve correctly except host.docker.internal. Fix was to add the ipv6=off flag to the resolver line in nginx.conf.
I solved this problem with the following way:
docker run --rm -d --network host --name "my_domain" nginx
https://docs.docker.com/network/network-tutorial-host/
You need a local dns server like dnsmasq to resolve using 127.0.0.1. Try installing it using apk add --update dnsmasq and set it up if you're using an alpine (nginx:alpine) variant.
I have an nginx docker container and a webapp container successfully running and talking to eachother.
The nginx container listens on port 80, and uses proxy_pass to direct traffic to the IP of the webapp container.
upstream app_humansio {
server humansio:8080 max_fails=3 fail_timeout=30s;
}
"humansio" is set in the /etc/hosts file by docker because I've started nginx with --link humansio:humansio. The webapp container (humansio) is always exposing 8080.
The problem is, when I reload the webapp container, the link to the nginx container breaks and I need to restart that as well. Is there any way I can do this differently so I don't need to restart the nginx container when the webapp container reloads?
--
I've tried to do something like connecting them manually by using a common port (8001 on both), but since they actually reserve that port, the 2nd container cannot use it as well.
Thanks!
I prefer to run the proxy (nginx of haproxy) directly on the host for this reason.
But an option is to "Link via an Ambassador Container" https://docs.docker.com/articles/ambassador_pattern_linking/
https://www.digitalocean.com/community/tutorials/how-to-use-the-ambassador-pattern-to-dynamically-configure-services-on-coreos
If you don't want to restart your proxy container whenever you have to restart one of the proxied ones (e.g. fig), you could take a look at the autoupdated proxy configuration approach: http://jasonwilder.com/blog/2014/03/25/automated-nginx-reverse-proxy-for-docker/
if u use some modern version of docker the links in nginx container to your web service probably get updated (u can check it with docker exec -ti nginx bash - then cat /etc/hosts) - problem is nginx doesnt' use /etc/hosts every time - it caches the ip and when it changes - he gets lost. 'docker kill -s HUP nginx' which makes nginx reload its config without restart helps too.
I have the same problem. I used to start my services with systemd unit files - and when u make one service (nginx) dependant on other (webapp) and then restart the webapp - systemd is smart enough to restart the nginx as well. Now I'm trying my luck with docker-compose and restarting webapp container confuses nginx.
I have a wildcard DNS set up so that all web requests to a custom domain (*.foo) map to the IP address of the Docker host. If I have multiple containers running Apache (or Nginx) instances, each container maps the Apache port (80) to some external inbound port.
What I would like to do is make a request to container-1.foo, which is already mapped to the correct IP address (of the Docker host) via my custom DNS server, but proxy the default port 80 request to the correct Docker external port such that the correct Apache instance from the specified container is able to respond based on the custom domain. Likewise, container-2.foo would proxy to a second container's apache, and so on.
Is there a pre-built solution for this, is my best bet to run an Nginx proxy on the Docker host, or should I write up a node.js proxy with the potential to manage Docker containers (start/stop/reuild via the web), or...? What options do I have that would make using the Docker containers more like a natural event and not something with extraneous ports and container juggling?
This answer might be a bit late, but what you need is an automatic reverse proxy. I have used two solutions for that:
jwilder/nginx-proxy
Traefik
With time, my preference is to use Traefik. Mostly because it is well documented and maintained, and comes with more features (load balancing with different strategies and priorities, healthchecks, circuit breakers, automatic SSL certificates with ACME/Let's Encrypt, ...).
Using jwilder/nginx-proxy
When running a Docker container Jason Wilder's nginx-proxy Docker image, you get a nginx server set up as a reverse proxy for your other containers with no config to maintain.
Just run your other containers with the VIRTUAL_HOST environment variable and nginx-proxy will discover their ip:port and update the nginx config for you.
Let say your DNS is set up so that *.test.local maps to the IP address of your Docker host, then just start the following containers to get a quick demo running:
# start the reverse proxy
docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock jwilder/nginx-proxy
# start a first container for http://tutum.test.local
docker run -d -e "VIRTUAL_HOST=tutum.test.local" tutum/hello-world
# start a second container for http://deis.test.local
docker run -d -e "VIRTUAL_HOST=deis.test.local" deis/helloworld
Using Traefik
When running a Traefik container, you get a reverse proxy server set up which will reconfigure its forwarding rules given docker labels found on your containers.
Let say your DNS is set up so that *.test.local maps to the IP address of your Docker host, then just start the following containers to get a quick demo running:
# start the reverse proxy
docker run --rm -it -p 80:80 -v /var/run/docker.sock:/var/run/docker.sock traefik:1.7 --docker
# start a first container for http://tutum.test.local
docker run -d -l "traefik.frontend.rule=Host:tutum.test.local" tutum/hello-world
# start a second container for http://deis.test.local
docker run -d -l "traefik.frontend.rule=Host:deis.test.local" deis/helloworld
Here are two possible answers: (1) setup ports directly with Docker and use Nginx/Apache to proxy the vhosts, or (2) use Dokku to manage ports and vhosts for you (which is how I learned to do Method 1).
Method 1a (directly assign ports with docker)
Step 1: Setup nginx.conf or Apache on the host, with the desired port number assignments. This web server, running on the host, will do the vhost proxying. There's nothing special about this with regard to Docker - it is normal vhost hosting. The special part comes next, in Step 2, to make Docker use the correct host port number.
Step 2: Force port number assignments in Docker with "-p" to set Docker's port mappings, and "-e" to set custom environment variables within Docker, as follows:
port=12345 # <-- the vhost port setting used in nginx/apache
IMAGE=myapps/container-1
id=$(docker run -d -p :$port -e PORT=$port $IMAGE)
# -p :$port will establish a mapping of 12345->12345 from outside docker to
# inside of docker.
# Then, the application must observe the PORT environment variable
# to launch itself on that port; This is set by -e PORT=$port.
# Additional goodies:
echo $id # <-- the running id of your container
echo $id > /app/files/CONTAINER # <-- remember Docker id for this instance
docker ps # <-- check that the app is running
docker logs $id # <-- look at the output of the running instance
docker kill $id # <-- to kill the app
Method 1b Hard-coded application port
...if you're application uses a hardcoded port, for example port 5000 (i.e. cannot be configured via PORT environment variable, as in Method 1a), then it can be hardcoded through Docker like this:
publicPort=12345
id=$(docker run -d -p $publicPort:5000 $IMAGE)
# -p $publicPort:5000 will map port 12345 outside of Docker to port 5000 inside
# of Docker. Therefore, nginx/apache must be configured to vhost proxy to 12345,
# and the application within Docker must be listening on 5000.
Method 2 (let Dokku figure out the ports)
At the moment, a pretty good option for managing Docker vhosts is Dokku. An upcoming option may be to use Flynn, but as of right now Flynn is just getting started and not quite ready. Therefore we go with Dokku for now: After following the Dokku install instructions, for a single domain, enable vhosts by creating the "VHOST" file:
echo yourdomain.com > /home/git/VHOST
# in your case: echo foo > /home/git/VHOST
Now, when an app is pushed via SSH to Dokku (see Dokku docs for how to do this), Dokku will look at the VHOST file and for the particular app pushed (let's say you pushed "container-1"), it will generate the following file:
/home/git/container-1/nginx.conf
And it will have the following contents:
upstream container-1 { server 127.0.0.1:49162; }
server {
listen 80;
server_name container-1.yourdomain.com;
location / {
proxy_pass http://container-1;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
When the server is rebooted, Dokku will ensure that Docker starts the application with the port mapped to its initially deployed port (49162 here), rather than getting assigned randomly another port. To achieve this deterministic assignment, Dokku saves the initially assigned port into /home/git/container-1/PORT and on the next launch it sets the PORT environment to this value, and also maps Docker's port assignments to be this port on both the host-side and the app-side. This is opposed to the first launch, when Dokku will set PORT=5000 and then figure out whatever random port Dokku maps on the VPS side to 5000 on the app side. It's round about (and might even change in the future), but it works!
The way VHOST works, under the hood, is: upon doing a git push of the app via SSH, Dokku will execute hooks that live in /var/lib/dokku/plugins/nginx-vhosts. These hooks are also located in the Dokku source code here and are responsible for writing the nginx.conf files with the correct vhost settings. If you don't have this directory under /var/lib/dokku, then try running dokku plugins-install.
With docker, you want the internal ips to remain normal (e.g. 80) and figure out how to wire up the random ports.
One way to handle them, is with a reverse proxy like hipache. Point your dns at it, and then you can reconfigure the proxy as your containers come up and down. Take a look at http://txt.fliglio.com/2013/09/protyping-web-stuff-with-docker/ to see how this could work.
If you're looking for something more robust, you may want to take a look at "service discovery." (a look at service discovery with docker: http://txt.fliglio.com/2013/12/service-discovery-with-docker-docker-links-and-beyond/)