I'm having some experiments to do with Docker container technology.
I need for a specific reason to connect two veth container interfaces together without using a bridge, Docker creates a bridge by default, so I do not want to use it.
I'm confused and want to know if it is right to do that way. Anyone can give advices and point me out some links or methods ? I will appreciate.
Thank you so much.
+--------------+ +--------------+
| | | |
| Container X | | Container Y |
| | | |
+--------------+ +--------------+
^ veth ^ veth
| |
+--------------------+
Sure, it's possible, although you won't be able to get Docker to do it for you automatically. Begin by creating your two containers with no networking:
# docker run --net=none --name container_x ...
# docker run --net=none --name container_y ...
Now create a veth pair:
# ip link add c_x_eth0 type veth peer name c_y_eth0
Assign each side of the veth pair to a container. You will need to know the PID of the container to do this, which you can get with, for example:
docker inspect --format '{{.State.Pid}}' container_x
I'm going to assume you've stuck this in a shell script named docker-pid. Set the name space on the first veth link:
# ip link set netns $(docker-pid container_x) dev c_x_eth0
And on the second:
# ip link set netns $(docker-pid container_y) dev c_y_eth0
Now you will need to configure the link inside each container. If you haven't started your containers with --privileged, you will need to do this using nsenter:
# nsenter -t $(docker-pid container_x) -n ip link set c_x_eth0 up
# nsenter -t $(docker-pid container_y) -n ip link set c_y_eth0 up
And then assign them ip addresses:
# nsenter -t $(docker-pid container_x) -n ip addr add 10.10.10.1/24 dev c_x_eth0
# nsenter -t $(docker-pid container_y) -n ip addr add 10.10.10.2/24 dev c_y_eth0
And you should be all set.
Update
If nsenter is unavailable...
The easiest solution really is just to install nsenter on your system; if you are able to create new veth interfaces and start Docker containers you should have all the privileges you need.
You could accomplish the above without nsenter if you run your containers in privileged mode (docker run --privileged...). This will allow your containers to do things -- such as run network configuration commands -- that are normally prohibited. In this case, you would just run the ip link and ip addr commands in the container, either from a shell you started with docker run or using something like docker exec. You should be aware that running a container in privileged mode removes many of the restrictions normally placed on containers, and so it is not something you want to do if anyone else has access to those containers.
Related
I'm a little bit newbie with Docker. The problem is my server provider changed the public IP recently. When I ran my wordpress container I used the following:
docker run -e WORDPRESS_DB_PASSWORD=xxx --name wordpress-xx --link wordpressdb-xx -p 185.166.xx.xx:8081:80 -v "$PWD/docker/data/wordpress/xx":/var/www/html -d wordpress
How can I change the old IP in order to assign the new one in a container that is already running?
Is it possible to run this containers with localhost IP? For example:
docker run -e WORDPRESS_DB_PASSWORD=xxx --name wordpress-xx --link wordpressdb-xx -p 127.0.0.1:8081:80 -v "$PWD/docker/data/wordpress/xx":/var/www/html -d wordpress
You can also try to save the current container as an image using docker commit and then run the image as a new container with the new IP.
If you have only a single network interface just pass the port only. You can also use the 127.0.0.1 address
See https://docs.docker.com/engine/reference/commandline/run/#publish-or-expose-port--p-expose
I try to expose a docker container to the outside world (well, to be specific just in my internal network - 10.0.0.0/24) with a static ip adress. In my example the container should have the IP address 10.0.0.200.
Docker version is 1.10.3.
Therefore i created a userdefined network:
docker network create --subnet 10.0.0.0/24 --gateway 10.0.0.254 dn in bridge mode.
Then i created a container and attached it to the container.
docker run -d \
--name testhost \
-it \
-h testhost \
--net dn \
--ip 10.0.0.200 \
-p 8080:8080 \
some image
The container has the correct ip and gw assigned (10.0.0.200, 10.0.0.254 - which is also the ip from the docker created bridge interface) but no communication from the container to the outside world nor from the outside to the container is possible. only thing that works is nslookup but tbh i dont know why this is working.
From another host in the network i can ping the bridge interface which was created through the docker network create command.
A second container connected the the dn network can ping my first container. so communication inside the network seems fine.
As in the docker [network documentation][1]
[1]: https://docs.docker.com/engine/userguide/networking/#a-bridge-network "docker network docu" (second picture in bridge network) it should be possible
It seems that im missing some step or config. Any advice is appreciated, thank you.
Docker creates these virtual ethernet interfaces veth[UNIQUE ID] listed in ifconfig. How can I find out which interface belongs to a specific docker container?
I want to listen to the tcp traffic.
To locate interface
In my case getting value from container was like (check eth0 to):
$ docker exec -it my-container cat /sys/class/net/eth1/iflink
123
And then:
$ ip ad | grep 123
123: vethd3234u4#if122: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
Check with tcpdump -i vethd3234u4
Reference about mysterious iflink from http://lxr.free-electrons.com/source/Documentation/ABI/testing/sysfs-class-net:
150 What: /sys/class/net/<iface>/iflink
151 Date: April 2005
152 KernelVersion: 2.6.12
153 Contact: netdev#vger.kernel.org
154 Description:
155 Indicates the system-wide interface unique index identifier a
156 the interface is linked to. Format is decimal. This attribute is
157 used to resolve interfaces chaining, linking and stacking.
158 Physical interfaces have the same 'ifindex' and 'iflink' values.
Based on the provided answer (which worked for me), I made this simple bash script:
#!/bin/bash
export containers=$(sudo docker ps --format "{{.ID}}|{{.Names}}")
export interfaces=$(sudo ip ad);
for x in $containers
do
export name=$(echo "$x" |cut -d '|' -f 2);
export id=$(echo "$x"|cut -d '|' -f 1)
export ifaceNum="$(echo $(sudo docker exec -it "$id" cat /sys/class/net/eth0/iflink) | sed s/[^0-9]*//g):"
export ifaceStr=$( echo "$interfaces" | grep $ifaceNum | cut -d ':' -f 2 | cut -d '#' -f 1);
echo -e "$name: $ifaceStr";
done
My answer more like improvement on that important topic because it didn't help to "Find out which network interface belongs to docker container", but, as author noticed, he "want to listen to the tcp traffic" inside docker container - I'll try to help on that one during your troubleshooting of network.
Considering that veth network devices are about network namespaces, it is useful to know that we can execute program in another namespace via nsenter tool as follow (remember - you need a privileged permission (sudo/root) for doing that):
Get ID of any container you are interested in capture the traffic, for example it will be 78334270b8f8
Then we need to take PID of that containerized application (I assume you are running only 1 network-related process inside container and want to capture its traffic. Otherwise, that approach is hard to be suitable):
sudo docker inspect 78334270b8f8 | grep -i pid
For example, output for pid will be 111380 - that's ID of your containerized app, you can check also it via ps command: ps aux | grep 111380 just in curiosity.
Next step is to check what network interfaces you have inside your container:
sudo nsenter -t 111380 -n ifconfig
This command will return you list of network devices in network namespace of the containerized app (you should not have ifconfig tool on board of your container, only on your node/machine)
For example, you need to capture traffic on interface eth2 and filter it to tcp destination port 80 (it may vary of course) with this command:
sudo nsenter -t 111380 -n tcpdump -nni eth2 -w nginx_tcpdump_test.pcap 'tcp dst port 80'
Remember, that in this case you do not need tcpdump tool to be installed inside your container.
Then, after capturing packets, .pcap file will be available on your machine/node and to read it use any tool you prefer tcpdump -r nginx_tcpdump_test.pcap
approach's pros:
no need to have network tools inside container, only on docker node
no need to search for map between network devices in container and node
cons:
you need to have privileged user on node/machine to run nsenter tool
One-liner of the solution from #pbaranski
num=$(docker exec -i my-container cat /sys/class/net/eth0/iflink | tr -d '\r'); ip ad | grep -oE "^${num}: veth[^#]+" | awk '{print $2}'
If you need to find out on a container that does not include cat then try this tool: https://github.com/micahculpepper/dockerveth
You can also read the interface names via /proc/PID/net/igmp like (container name as argument 1):
#!/bin/bash
NAME=$1
PID=$(docker inspect $NAME --format "{{.State.Pid}}")
while read iface id; do
[[ "$iface" == lo ]] && continue
veth=$(ip -br addr | sed -nre "s/(veth.*)#if$id.*/\1/p")
echo -e "$NAME\t$iface\t$veth"
done < <(</proc/$PID/net/igmp awk '/^[0-9]+/{print $2 " " $1;}')
I have created a Docker multi-host network using Docker Overlay network with 4 nodes: node0, node1, node2, and node3. Node0 act as key-value store which shares the information of nodes while node1, node2, and node3 are bound to the key-value store.
Here are node1 networks:
user#node1$ docker network ls
NETWORK ID NAME DRIVER
04adb1ab4833 RED overlay
[ . . ]
As for node2 networks:
user#node2$ docker network ls
NETWORK ID NAME DRIVER
04adb1ab4833 RED overlay
[ . . ]
container1 is running on node1, that hosts the RED-named network.
user#node1$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f9bacac3c01d ubuntu "/bin/bash" 3 hours ago Up 2 hours container1
Docker added an entry to /etc/hosts for each container that belongs to the RED overlay network.
user#node1$ docker exec container1 cat /etc/hosts
10.10.10.2 d82c36bc2659
127.0.0.1 localhost
[ . . ]
10.10.10.3 container2
10.10.10.3 container2.RED
From node2, I'm trying to access the container1 running on node1. I tried to run container1 using command below but it returns error.
`user#node2$ docker docker exec -i -t container1 bash`
Error response from daemon: no such id: container1
Any suggestion?
Thanks.
The network is shared only for the containers.
While the network is shared among the containers across the multi-hosts overlay, the docker daemons cannot communicate between them as is.
The user#_node2_$ docker exec -i -t container1 bash doest not work because, indeed, no such id: container1 are running from node2.
Accessing remote Docker daemon
Docker daemons communicate through socket. UNIX socket by default, but it is possible to add an option, --host to specify other sockets the daemon should bind to.
See the docker daemon man page:
-H, --host=[unix:///var/run/docker.sock]: tcp://[host:port] to bind or unix://[/path/to/socket] to use.
The socket(s) to bind to in daemon mode specified using one or more
tcp://host:port, unix:///path/to/socket, fd://* or fd://socketfd.
Thus, it is possible to access from any node a docker daemon bind to a tcp socket.
The command user#node2$ docker -H tcp://node1:port exec -i -t container1 bash would work well.
Docker and Docker cluster (Swarm)
I do not know what you are trying to deploy, maybe just playing around with the tutorials, and that's great! You may be interested to look into Swarm that deploys a cluster of docker. In short: you can use several nodes as it they were one powerful docker daemon access through a single node with the whole Docker API.
I'm trying to set up a docker dnsmasq container so that I can have all my docker containers look up the domain names rather than having hard-coded IPs (if they are on the same host). This fixes an issue with the fact that one cannot alter the /etc/hosts file in docker containers, and this allows me to easily update all my containers in one go, by altering a single file that the dnsmasq container references.
It looks like someone has already done the hard work for me and created a dnsmasq container. Unfortunately, it is not "working" for me. I wrote a bash script to start the container as shown below:
name="dnsmasq_"
timenow=$(date +%s)
name="$name$timenow"
sudo docker run \
-v="$(pwd)/dnsmasq.hosts:/dnsmasq.hosts" \
--name=$name \
-p='127.0.0.1:53:5353/udp' \
-d sroegner/dnsmasq
Before running that, I created the dnsmasq.hosts directory and inserted a single file within it called hosts.txt with the following contents:
192.168.1.3 database.mydomain.com
Unfortunately whenever I try to ping that domain from within:
the host
The dnsmasq container
another container on the same host
I always receive the ping: unknown host error message.
I tried starting the dnsmasq container without daemon mode so I could debug its output, which is below:
dnsmasq: started, version 2.59 cachesize 150
dnsmasq: compile time options: IPv6 GNU-getopt DBus i18n DHCP TFTP conntrack IDN
dnsmasq: reading /etc/resolv.dnsmasq.conf
dnsmasq: using nameserver 8.8.8.8#53
dnsmasq: read /etc/hosts - 7 addresses
dnsmasq: read /dnsmasq.hosts//hosts.txt - 1 addresses
I am guessing that I have not specified the -p parameter correctly when starting the container. Can somebody tell me what it should be for other docker containers to lookup the DNS, or whether what I am trying to do is actually impossible?
The build script for the docker dnsmasq service needs to be changed in order to bind to your server's public IP, which in this case is 192.168.1.12 on my eth0 interface
#!/bin/bash
NIC="eth0"
name="dnsmasq_"
timenow=$(date +%s)
name="$name$timenow"
MY_IP=$(ifconfig $NIC | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}')
sudo docker run \
-v="$(pwd)/dnsmasq.hosts:/dnsmasq.hosts" \
--name=$name \
-p=$MY_IP:53:5353/udp \
-d sroegner/dnsmasq
On the host (in this case ubuntu 12), you need to update the resolv.conf or /etc/network/interfaces file so that you have registered your public IP (eth0 or eth1 device) as the nameserver.
You may want to set a secondary nameserver to be google for whenever the container is not running, by changing the line to be dns-nameservers xxx.xxx.xxx.xxx 8.8.8.8 E.g. there is no comma or another line.
You then need to restart your networking service sudo /etc/init.d/networking restart if you updated the /etc/network/interfaces file so that this auto updates the /etc/resolve.conf file that docker will copy to the container during the build.
Now restart all of your containers
sudo docker stop $CONTAINER_ID
sudo docker start $CONTAINER_ID
This causes their /etc/resolv.conf files update so they point to the new nameserver settings.
DNS lookups in all your docker containers (that you built since making the changes) should now work using your dnsmasq container!
As a side note, this means that docker containers on other hosts can also take advantage of your dnsmasq service on this host as long as their host's nameserver settings is set to using this server's public IP.