Connecting Docker Containers - networking

Hello Helpful Developers,
I'm having issues connecting docker containers. I have built a subversion docker container and a mongo docker container.
docker run -d -p 3343:3343 -p 4434:4434 -p 18080:18080 --name svn-server mamohr/subversion-edge
docker run -p 27017:27017 --name my-mongo -d mongo
I'm able to hit http://x.x.x.x:18080/ from a browser, but unable to curl from the my-mongo instance. I can talk to each container from my development machine, but unable to talk from container to container.
I see things like --net=bridge, host, ????, but I'm getting confused.
Please help.....

Borrowing this schema from SDN hub, imagine that C1 is your SVN container and C2 is your Mongo container:
Both containers are connected to docker0 bridge and NATed to external 192.168.50.16 network.
To connect from your Mongo container, check the bridge0 IP address of the SVN container:
# docker inspect <svn-container-name>
"Networks": {
"bridge0": {
"IPAddress": "172.17.0.19",
}
then CURL directly to it's bridge0 IP address:
curl http://172.17.0.19:18080/

To get you immediately going, you can start your hosts with --net=host and then both containers and host will be able to communicate.
Or you can use link( --link ) between from mongo to the other container.
There is lot to explain about docker networking and the docker documentation will be good point to start.
Read the documentation at https://docs.docker.com/engine/userguide/networking/dockernetworks/

I would advice you to take a look at docker compose. I think it's the best way to manage a system, which is composed of many containers.
Here is the official guide: https://docs.docker.com/compose/

Docker containers by default start attached to a bridge network called default. You can do docker network ls and see the networks you have available. You can also create networks with different attributes etc...
So in your case, both your containers are being started on the same default network, which means they should be able to communicate with each other just fine. In fact, if you only want your SVN server to be able to talk to Mongo (and don't need to connect to mongo from your host) you don't even need to expose ports on the Mongo container. Containers on the same network as each can communicate with each other just fine without ports being exposed. Exposing ports is to allow host > container connectivity.
So, what hostname / port are you using when you try to curl from the mongo instance to your SVN instance? You should be using svn-server as that will resolve to the SVN container (using Docker's built-in DNS resolution).

Direct container to container networking via container name can be achieved with a user defined network.
docker network create mynet
docker run -d --net=mynet --name svn-server mamohr/subversion-edge
docker run -d --net=mynet --name my-mongo mongo
docker exec <svn-id> ping my-mongo
docker exec <mongo-id> ping svn-server
You should always be able to connect to mapped ports though, even in your current setup. The hosts runs a process that listens on that port so any host IP should do.
$ docker run -d -p 8080:80 --net=mynet --name sleep busybox nc -lp 80 -e echo here!
63115ef88664f1186ea012e41138747725790383c741c12ca8675c3058383e68
$ ss -lntp | grep 8080
LISTEN 0 128 :::8080 :::* users:(("exe",pid=6287,fd=4))
$ docker run busybox nc <any_host_ip> 8080
here!

Please remember, container is not available by default to the ourside world.
When you running the svn-server container, you published the container's 18080 port and mapped it from the host's 18080 port. So you can access it by http://your_host_IP:18080.
From your two docker run commands, both svn-server container and my-mongo container are on the default bridge network. These two containers are connected by docker0, so they can communicate each other directly by localhost.
But if you tried to access http://your_host_IP:18080 from within your my-mongo container, that means your request would first be send to docker0, but docker0 will drop your request because you're trying to access the host, not the svn-server container.
So try this curl http://localhost:18080 or curl http://svn-server_IP:18080 from my-mongo container to access svn-server container.

Related

Can't expose port with podman

I am trying to walk through a tutorial that brings up an application in a docker/podman container instance.
I have attempted to use -p port:port and --expose port but neither seems to work.
I've ensured that I see the port in a listen state with ss -an.
I've made sure there isn't anything else trying to bind to that port.
No matter what I do, I can never hit localhost:port or ip_address:port.
I feel like I am fundamentally missing something but don't know where to look next.
Any suggestions for things to try or documentation to review would be appreciated.
Thanks,
Shawn
Expose: (Reference)
Expose tells Podman that the container requests that port be open, but
does not forward it. All exposed ports will be forwarded to random
ports on the host if and only if --publish-all is also specified
As per Redhat documentation for Containerfile,
EXPOSE indicates that the container listens on the specified network
port at runtime. The EXPOSE instruction defines metadata only; it does
not make ports accessible from the host. The -p option in the podman
run command exposes container ports from the host.
To specify Port Number,
The -p option in the podman run command exposes container ports from
the host.
Example:
podman run -d -p 8080:80 --name httpd-basic quay.io/httpd-parent:2.4
In above example, Port # 80 is the port number which Container listens/exposes and we can access this from outside the container via Port # 8080

Docker with multiple exposed ports

I have a container with say, 3 ports, 1000 (nodejs-express), 1001 (python-flask) and 1002 (angular2-client) exposed. When I use
docker run --name test -d -p 1000:1000 -p 1001:1001 -p 1002:1002 docker_image
Only the Express server is working fine on the host computer. However, when I log into the container and do curl, all three servers are responding just fine.
Any ideas what is going on with multiple port bindings with docker/host?
Once you do the following:
EXPOSE ports on the DockerFile
set -p flag for each port to expose externally
You just need to make sure that your services allows external connections.
i.e. for python flask: http://dixu.me/2015/10/26/How_to_Allow_Remote_Connections_to_Flask_Web_Service/ the default listen is localhost. Make sure it's listening on 0.0.0.0

docker nginx container not receiving request from outside, connection refused

I have a running nginx container: # docker run --name mynginx1 -P -d nginx;
And got its PORT info by docker ps: 0.0.0.0:32769->80/tcp, 0.0.0.0:32768->443/tcp
Then I could get response from within the container(id: c30991a04b2f):
docker exec -i -t c3099 bash
curl http://localhost => which return the default index.html page content, it works
However, when I make the curl http://localhost:32769 outside of the container, I got this:
curl: (7) failed to connect to localhost port 32769: Connection refused
I am running on a mac with docker version 1.9.0; nginx latest
Does anyone know what cause this? Any help? thank you
If you are On OSX, you are probably using a VirtualBox VM for your docker environment.
Make sure you have forwarded your port 32769 to your actual host (the mac), in order for that port to be visible from localhost.
This is valid for the old boot2docker, or the new docker machine.
VBoxManage controlvm "boot2docker-vm" --natpf1 "tcp-port32769 ,tcp,,32769,,32769"
VBoxManage controlvm "boot2docker-vm" --natpf1 "udp-port32769 ,udp,,32769,,$32769
(controlvm if the VM is running, modifyvm is the VM is stopped)
(replace "boot2docker-vm" b ythe name of your vm: see docker-machine ls)
I would recommend to not use -P, but a static port mapping -p xxx:80 -p yyy:443.
That way, you can do that port forwarding once, using fixed values.
Of course, you can access the VM directly through docker-machine ip vmname
curl http://$(docker-machine ip vmname):32769
Solved.. I misunderstood how docker port mapping works.
Since I'm using mac, the host for nginx container is a VM, 0.0.0.0:32769->80/tcp maps the port 80 of the container to the port 32769 of the VM.
solution:
docker-machine ip vm-name => 192.168.99.xx
curl http://192.168.99.xx:32769
Not exactly answers for your question but spend some time trying to figure out similar thing in context of "why is my docker container not connecting to elastic search localhost:9200" and this was the first S.O. question that pops up, so I hope it helps some other googling person
if you are linking containers together (e.g. docker run --rm --name web2 --link db:db training/webapp env)
... then Dockers adds enviroment variables:
DB_NAME=/web2/db
DB_PORT=tcp://172.17.0.5:5432
DB_PORT_5432_TCP=tcp://172.17.0.5:5432
DB_PORT_5432_TCP_PROTO=tcp
DB_PORT_5432_TCP_PORT=5432
DB_PORT_5432_TCP_ADDR=172.17.0.5
... and also updates your /etc/hosts
# /etc/hosts
#...
172.17.0.9 db
so you can technically connect to ping db
https://docs.docker.com/v1.8/userguide/dockerlinks/
so for elastic search is
# /etc/hosts
# ...
172.17.0.28 elasticsearch f9db83d0dfb5 ecs-awseb-qa-3Pobblecom-env-f7yq6jhmpm-10-elasticsearch-fcbfe5e2b685d0984a00
so wget elasticseach:9200 will work

How to communicate between Docker containers via "hostname"

I plan to split my monolthic server up into many small docker containers but haven't found a good solution for "inter-container communication" yet. This is my target scenario:
I know how to link containers together and how to expose ports, but none of these solutions are satisfying to me.
Is there any solution to communicate via hostnames (container names) between the containers like in a traditional server network?
The new networking feature allows you to connect to containers by
their name, so if you create a new network, any container connected to
that network can reach other containers by their name. Example:
1) Create new network
$ docker network create <network-name>
2) Connect containers to network
$ docker run --net=<network-name> ...
or
$ docker network connect <network-name> <container-name>
3) Ping container by name
docker exec -ti <container-name-A> ping <container-name-B>
64 bytes from c1 (172.18.0.4): icmp_seq=1 ttl=64 time=0.137 ms
64 bytes from c1 (172.18.0.4): icmp_seq=2 ttl=64 time=0.073 ms
64 bytes from c1 (172.18.0.4): icmp_seq=3 ttl=64 time=0.074 ms
64 bytes from c1 (172.18.0.4): icmp_seq=4 ttl=64 time=0.074 ms
See this section of the documentation;
Note: Unlike legacy links the new networking will not create environment variables, nor share environment variables with other containers.
This feature currently doesn't support aliases
Edit: After Docker 1.9, the docker network command (see below https://stackoverflow.com/a/35184695/977939) is the recommended way to achieve this.
My solution is to set up a dnsmasq on the host to have DNS record automatically updated: "A" records have the names of containers and point to the IP addresses of the containers automatically (every 10 sec). The automatic updating script is pasted here:
#!/bin/bash
# 10 seconds interval time by default
INTERVAL=${INTERVAL:-10}
# dnsmasq config directory
DNSMASQ_CONFIG=${DNSMASQ_CONFIG:-.}
# commands used in this script
DOCKER=${DOCKER:-docker}
SLEEP=${SLEEP:-sleep}
TAIL=${TAIL:-tail}
declare -A service_map
while true
do
changed=false
while read line
do
name=${line##* }
ip=$(${DOCKER} inspect --format '{{.NetworkSettings.IPAddress}}' $name)
if [ -z ${service_map[$name]} ] || [ ${service_map[$name]} != $ip ] # IP addr changed
then
service_map[$name]=$ip
# write to file
echo $name has a new IP Address $ip >&2
echo "host-record=$name,$ip" > "${DNSMASQ_CONFIG}/docker-$name"
changed=true
fi
done < <(${DOCKER} ps | ${TAIL} -n +2)
# a change of IP address occured, restart dnsmasq
if [ $changed = true ]
then
systemctl restart dnsmasq
fi
${SLEEP} $INTERVAL
done
Make sure your dnsmasq service is available on docker0. Then, start your container with --dns HOST_ADDRESS to use this mini dns service.
Reference: http://docs.blowb.org/setup-host/dnsmasq.html
That should be what --link is for, at least for the hostname part.
With docker 1.10, and PR 19242, that would be:
docker network create --net-alias=[]: Add network-scoped alias for the container
(see last section below)
That is what Updating the /etc/hosts file details
In addition to the environment variables, Docker adds a host entry for the source container to the /etc/hosts file.
For instance, launch an LDAP server:
docker run -t --name openldap -d -p 389:389 larrycai/openldap
And define an image to test that LDAP server:
FROM ubuntu
RUN apt-get -y install ldap-utils
RUN touch /root/.bash_aliases
RUN echo "alias lds='ldapsearch -H ldap://internalopenldap -LL -b
ou=Users,dc=openstack,dc=org -D cn=admin,dc=openstack,dc=org -w
password'" > /root/.bash_aliases
ENTRYPOINT bash
You can expose the 'openldap' container as 'internalopenldap' within the test image with --link:
docker run -it --rm --name ldp --link openldap:internalopenldap ldaptest
Then, if you type 'lds', that alias will work:
ldapsearch -H ldap://internalopenldap ...
That would return people. Meaning internalopenldap is correctly reached from the ldaptest image.
Of course, docker 1.7 will add libnetwork, which provides a native Go implementation for connecting containers. See the blog post.
It introduced a more complete architecture, with the Container Network Model (CNM)
That will Update the Docker CLI with new “network” commands, and document how the “-net” flag is used to assign containers to networks.
docker 1.10 has a new section Network-scoped alias, now officially documented in network connect:
While links provide private name resolution that is localized within a container, the network-scoped alias provides a way for a container to be discovered by an alternate name by any other container within the scope of a particular network.
Unlike the link alias, which is defined by the consumer of a service, the network-scoped alias is defined by the container that is offering the service to the network.
Continuing with the above example, create another container in isolated_nw with a network alias.
$ docker run --net=isolated_nw -itd --name=container6 -alias app busybox
8ebe6767c1e0361f27433090060b33200aac054a68476c3be87ef4005eb1df17
--alias=[]
Add network-scoped alias for the container
You can use --link option to link another container with a preferred alias
You can pause, restart, and stop containers that are connected to a network. Paused containers remain connected and can be revealed by a network inspect. When the container is stopped, it does not appear on the network until you restart it.
If specified, the container's IP address(es) is reapplied when a stopped container is restarted. If the IP address is no longer available, the container fails to start.
One way to guarantee that the IP address is available is to specify an --ip-range when creating the network, and choose the static IP address(es) from outside that range. This ensures that the IP address is not given to another container while this container is not on the network.
$ docker network create --subnet 172.20.0.0/16 --ip-range 172.20.240.0/20 multi-host-network
$ docker network connect --ip 172.20.128.2 multi-host-network container2
$ docker network connect --link container1:c1 multi-host-network container2
EDIT : It is not bleeding edge anymore : http://blog.docker.com/2016/02/docker-1-10/
Original Answer
I battled with it the whole night.
If you're not afraid of bleeding edge, the latest version of Docker engine and Docker compose both implement libnetwork.
With the right config file (that need to be put in version 2), you will create services that will all see each other. And, bonus, you can scale them with docker-compose as well (you can scale any service you want that doesn't bind port on the host)
Here is an example file
version: "2"
services:
router:
build: services/router/
ports:
- "8080:8080"
auth:
build: services/auth/
todo:
build: services/todo/
data:
build: services/data/
And the reference for this new version of compose file:
https://github.com/docker/compose/blob/1.6.0-rc1/docs/networking.md
As far as I know, by using only Docker this is not possible. You need some DNS to map container ip:s to hostnames.
If you want out of the box solution. One solution is to use for example Kontena. It comes with network overlay technology from Weave and this technology is used to create virtual private LAN networks for each service and every service can be reached by service_name.kontena.local-address.
Here is simple example of Wordpress application's YAML file where Wordpress service connects to MySQL server with wordpress-mysql.kontena.local address:
wordpress:
image: wordpress:4.1
stateful: true
ports:
- 80:80
links:
- mysql:wordpress-mysql
environment:
- WORDPRESS_DB_HOST=wordpress-mysql.kontena.local
- WORDPRESS_DB_PASSWORD=secret
mysql:
image: mariadb:5.5
stateful: true
environment:
- MYSQL_ROOT_PASSWORD=secret

Expose Docker container to public ip

I started an Ubuntu Docker container, installed ssh, run ssh with port 22 attached to it.
$ docker ps
CONTAINER ID IMAGE COMMAND PORTS
f580e36aa7f0 martin/ssh2 /usr/sbin/sshd -D 0.0.0.0:49154->22/tcp
From my server I can now ssh my container. It work fine!
ssh root#172.17.42.1 -p 49154
But how can I ssh my container from the outside word?
(my server is running in my local network on 192.168.1.8/24)
Should I install a load balancer who redirect 192.168.1.8:2222 to 172.17.42.1:49154?
Should I need pipework for that? How?
Can someone point me in the right decision?
You should be able to connect to your container at 192.168.1.8:49154 already.
Your ssh container is bind to 0.0.0.0 (=any interfaces) and port 49154 so it means container port 22 is accessible on any interface on your host at port 49154.

Resources