I cannot connect to external mongodb server from my docker swarm cluster.
As I understand this is because of cluster uses overlay network driver. Am I right?
If not, how does docker overlay driver works and how can I connect to external mongodb server from cluster?
Q. How does the docker overlay driver work?
I would recommend this good reference for understanding docker swarm network overlay, and more globally, Docker's architecture.
This states that:
Docker uses embedded DNS to provide service discovery for containers running on a single Docker Engine and tasks running in a Docker Swarm. Docker Engine has an internal DNS server that provides name resolution to all of the containers on the host in user-defined bridge, overlay, and MACVLAN networks.
Each Docker container ( or task in Swarm mode) has a DNS resolver that forwards DNS queries to Docker Engine, which acts as a DNS server.
So, in multi-host docker swarm mode, with this example setup :
In this example there is a service of two containers called myservice. A second service (client) exists on the same network. The client executes two curl operations for docker.com and myservice.
These are the resulting actions:
DNS queries are initiated by client for docker.com and myservice.
The container's built-in resolver intercepts the DNS queries on 127.0.0.11:53 and sends them to Docker Engine's DNS server.
myservice resolves to the Virtual IP (VIP) of that service which is internally load balanced to the individual task IP addresses. Container names resolve as well, albeit directly to their IP addresses.
docker.com does not exist as a service name in the mynet network and so the request is forwarded to the configured default DNS server.
Back to your question:
How can I connect to an external mongodb server form cluster?
For your external mongodb (let's say you have a DNS for that mongodb.mydomain.com), you are in the same situation as the client in above architecture, wanting to connect to docker.com, except that you certainly don't wan't to expose that mongodb.mydomain.com to the entire web, so you may have declared it in your internal cluster DNS server.
Then, how to tell docker engine to use this internal DNS server to resolve mongodb.mydomain.com?
You have to indicate in your docker service task that you want to use an internal DNS server, like so:
docker service create \
--name myservice \
--network my-overlay-network \
--dns=10.0.0.2 \
myservice:latest
The important thing here is --dns=10.0.0.2. This will tell the Docker engine to use the DNS server at 10.0.0.2:53 as default if it can not resolve the DNS name in the VIP.
Finally, when you say :
I cannot connect to external mongodb server from my docker swarm cluster. As I understand this is because of cluster uses overlay network driver. Am I right?
I would say no, as there is a built in method in docker engine to forward unknown DNS name coming from overlay network to the DNS server you want.
Hope this helps!
Related
I am new to Dockers and containers. I was going through the tutorials for docker and came across this information.
https://docs.docker.com/get-started/part3/#docker-composeyml
networks:
- webnet
networks:
webnet:
What is webnet? The document says
Instruct web’s containers to share port 80 via a load-balanced network called webnet. (Internally, the containers themselves will publish to web’s port 80 at an ephemeral port.)
So, by default, the overlay network is load balanced in docker cluster? What is load balancing algo used?
Actually, it is not clear to me why do we have load balancing on the overlay network.
Not sure I can be clearer than the docs, but maybe rephrasing will help.
First, the doc you're following here uses what is called the swarm mode of docker.
What is swarm mode?
A swarm is a cluster of Docker engines, or nodes, where you deploy services. The Docker Engine CLI and API include commands to manage swarm nodes (e.g., add or remove nodes), and deploy and orchestrate services across the swarm.
From SO Documentation:
A swarm is a number of Docker Engines (or nodes) that deploy services collectively. Swarm is used to distribute processing across many physical, virtual or cloud machines.
So, with swarm mode you have a multi host (vms and/or physical) cluster a machines that communicate with each other through their docker engine.
Q1. What is webnet?
webnet is the name of an overlay network that is created when your stack is launched.
Overlay networks manage communications among the Docker daemons participating in the swarm
In your cluster of machines, a virtual network is the created, where each service has an ip - mapped to an internal DNS entry (which is service name), and allowing docker to route incoming packets to the right container, everywhere in the swarm (cluster).
Q2. So, by default, overlay network is load balanced in docker cluster ?
Yes, if you use the overlay network, but you could also remove the service networks configuration to bypass that. Then you would have to publish the port of the service you want to expose.
Q3. What is load balancing algo used ?
From this SO question answered by swarm master bmitch ;):
The algorithm is currently round-robin and I've seen no indication that it's pluginable yet. A higher level load balancer would allow swarm nodes to be taken down for maintenance, but any sticky sessions or other routing features will be undone by the round-robin algorithm in swarm mode.
Q4. Actually it is not clear to me why do we have load balancing on overlay network
Purpose of docker swarm mode / services is to allow orchestration of replicated services, meaning that we can scale up / down containers deployed in the swarm.
From the docs again:
Swarm mode has an internal DNS component that automatically assigns each service in the swarm a DNS entry. The swarm manager uses internal load balancing to distribute requests among services within the cluster based upon the DNS name of the service.
So you can have deployed like 10 exact same container (let's say nginx with you app html/js), without dealing with private network DNS entries, port configuration, etc... Any incoming request will be automatically load balanced to hosts participating in the swarm.
Hope this helps!
I have two servers on the same LAN. Their IP addresses are 10.0.0.1 (Server A) and 10.0.0.2 (Server B).
The MySQL server runs on Server B.
The docker container runs on Server A. It's IP address is 172.17.0.2, and the eth0 of the host is 172.17.0.1.
My question is, how to connect to Server B in the docker container inside Server A?
Thanks.
Something very easy to setup is the new Docker swarm mode (if you have Docker 1.12.2) https://docs.docker.com/engine/swarm/
With this all you have to do is connect your two servers by following the doc. You can then create an overlay network. Then to create your containers you will have to use the command docker service create instead of docker run.
You may also want to use some constraints to specify where the services should run.
So my application server (instance with swarm installed) and many clients (docker containers on a different physical nodes) require that clients should connect to server by hostname and I don't know how container can get node IP where swarm runs to be able to connect.
I know that I can pass hosts rules to each docker container with hostname and IP of the node where swarm runs but I don't know how to get that IP because there is a lot of interfaces with different IPs.
So generally I'm looking for something like docker client can ask swarm what is your IP to be able to connect back.
Being new to Docker and VM's, I have run into a blocker. I have a node app that needs to send a POST request from a Docker container to a Virtual Machine or to my local machine.
I have read through the Docker documentation, but still don't understand what I need to do in order to accomplish this.
So how can I send an http request from my node app running in a Docker Container to my Vagrant Box?
By default, Docker creates a virtual interface (docker0) in your host machine with IP 172.17.42.1. Each of the container launched will have an IP of the network 172.17.42.1/16, and they will be able to connect to host machine connecting to IP 172.17.42.1.
If you want to connect a docker container with another service running in a virtual machine running with other provider (e.g.: virtualbox, vmware), the easiest way is forwarding the ports needed by the service to you host machine and then, from your docker container, connecting to IP 172.17.42.1. You should check your virtual machine provider documentation to see details about this. And if you are using libvirt/KVM (or with any other provider), you can use iptables to enable port forwarding.
On a VPS with a static, publicly routable IP, I have a simple web server running (on port 8080) in a container that exports port 8080 (-p 0.0.0.0:8080:8080).
If I spin up another container on the same box and try to curl <public ip of host>:8080 it resolves the address, tries to connect but fails when making the request (it just hangs).
From the host's shell (outside containers), curl <public ip of host>:8080 succeeds.
Why is this happening? My feeling is that, somehow, the virtual network cards fail to communicate with each other. Is there a workaround (besides using docker links)?
According to Docker's advanced networking docs (http://docs.docker.io/use/networking/): "Docker uses iptables under the hood to either accept or drop communication between containers."
As such, I believe you would need to setup inbound and outbound routing with iptables. This article gives a solid description of how to do so: http://blog.codeaholics.org/2013/giving-dockerlxc-containers-a-routable-ip-address/