Find out which network interface belongs to docker container - networking

Docker creates these virtual ethernet interfaces veth[UNIQUE ID] listed in ifconfig. How can I find out which interface belongs to a specific docker container?
I want to listen to the tcp traffic.

To locate interface
In my case getting value from container was like (check eth0 to):
$ docker exec -it my-container cat /sys/class/net/eth1/iflink
123
And then:
$ ip ad | grep 123
123: vethd3234u4#if122: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker_gwbridge state UP group default
Check with tcpdump -i vethd3234u4
Reference about mysterious iflink from http://lxr.free-electrons.com/source/Documentation/ABI/testing/sysfs-class-net:
150 What: /sys/class/net/<iface>/iflink
151 Date: April 2005
152 KernelVersion: 2.6.12
153 Contact: netdev#vger.kernel.org
154 Description:
155 Indicates the system-wide interface unique index identifier a
156 the interface is linked to. Format is decimal. This attribute is
157 used to resolve interfaces chaining, linking and stacking.
158 Physical interfaces have the same 'ifindex' and 'iflink' values.

Based on the provided answer (which worked for me), I made this simple bash script:
#!/bin/bash
export containers=$(sudo docker ps --format "{{.ID}}|{{.Names}}")
export interfaces=$(sudo ip ad);
for x in $containers
do
export name=$(echo "$x" |cut -d '|' -f 2);
export id=$(echo "$x"|cut -d '|' -f 1)
export ifaceNum="$(echo $(sudo docker exec -it "$id" cat /sys/class/net/eth0/iflink) | sed s/[^0-9]*//g):"
export ifaceStr=$( echo "$interfaces" | grep $ifaceNum | cut -d ':' -f 2 | cut -d '#' -f 1);
echo -e "$name: $ifaceStr";
done

My answer more like improvement on that important topic because it didn't help to "Find out which network interface belongs to docker container", but, as author noticed, he "want to listen to the tcp traffic" inside docker container - I'll try to help on that one during your troubleshooting of network.
Considering that veth network devices are about network namespaces, it is useful to know that we can execute program in another namespace via nsenter tool as follow (remember - you need a privileged permission (sudo/root) for doing that):
Get ID of any container you are interested in capture the traffic, for example it will be 78334270b8f8
Then we need to take PID of that containerized application (I assume you are running only 1 network-related process inside container and want to capture its traffic. Otherwise, that approach is hard to be suitable):
sudo docker inspect 78334270b8f8 | grep -i pid
For example, output for pid will be 111380 - that's ID of your containerized app, you can check also it via ps command: ps aux | grep 111380 just in curiosity.
Next step is to check what network interfaces you have inside your container:
sudo nsenter -t 111380 -n ifconfig
This command will return you list of network devices in network namespace of the containerized app (you should not have ifconfig tool on board of your container, only on your node/machine)
For example, you need to capture traffic on interface eth2 and filter it to tcp destination port 80 (it may vary of course) with this command:
sudo nsenter -t 111380 -n tcpdump -nni eth2 -w nginx_tcpdump_test.pcap 'tcp dst port 80'
Remember, that in this case you do not need tcpdump tool to be installed inside your container.
Then, after capturing packets, .pcap file will be available on your machine/node and to read it use any tool you prefer tcpdump -r nginx_tcpdump_test.pcap
approach's pros:
no need to have network tools inside container, only on docker node
no need to search for map between network devices in container and node
cons:
you need to have privileged user on node/machine to run nsenter tool

One-liner of the solution from #pbaranski
num=$(docker exec -i my-container cat /sys/class/net/eth0/iflink | tr -d '\r'); ip ad | grep -oE "^${num}: veth[^#]+" | awk '{print $2}'
If you need to find out on a container that does not include cat then try this tool: https://github.com/micahculpepper/dockerveth

You can also read the interface names via /proc/PID/net/igmp like (container name as argument 1):
#!/bin/bash
NAME=$1
PID=$(docker inspect $NAME --format "{{.State.Pid}}")
while read iface id; do
[[ "$iface" == lo ]] && continue
veth=$(ip -br addr | sed -nre "s/(veth.*)#if$id.*/\1/p")
echo -e "$NAME\t$iface\t$veth"
done < <(</proc/$PID/net/igmp awk '/^[0-9]+/{print $2 " " $1;}')

Related

How to capture packets for single docker container

There have many container running on the host. And I want to capture packets for the one container of these. Is there any way to do this?
You can bind to the network namespace of one container to another:
docker run -it --rm --net container:<container_name> \
nicolaka/netshoot tcpdump ...
To see more about the netshoot image used above, see: https://github.com/nicolaka/netshoot
From and for a workstation with Wireshark:
docker exec -ti <container id> cat /sys/class/net/eth0/iflink
28
ip link | grep 28
28: veth11b0a6c#if27: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
First, get pid of the container you are interested in
docker inspect --format "{{ .State.Pid }}" "$CONTAINER_ID"
Then enter the same network namespace
nsenter -n -t "$PID"
Finally, run tcpdump
In rare cases (or maybe when you create your own container images) the container may have tcpdump installed. In that case, you can issue the following command to get a 10 second capture saved to the host computer (outside of the container):
# Set CONATINER_ID to the process you want to perform a dump from
# Consider docker ps for finding the container id
# for example, CONTAINER_ID=$( docker ps | grep $IMAGE | awk '{print $1}' )
docker exec $CONTAINER_ID bash -c 'timeout 10 tcpdump -i eth0 -w /dev/stdout' > capture.pcap

Ucarp update switch's arp cache

I'm using ucarp over linux bonding for high availability and automatic failover of two servers.
Here are the commands I used on each server for starting ucarp :
Server 1 :
ucarp -i bond0 -v 2 -p secret -a 10.110.0.243 -s 10.110.0.229 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh -b 1 -k 1 -r 2 -z
Server 2 :
ucarp -i bond0 -v 2 -p secret -a 10.110.0.243 -s 10.110.0.242 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh -b 1 -k 1 -r 2 -z
and the content of the scripts :
vip-up.sh :
#!/bin/sh
exec 2> /dev/null
/sbin/ip addr add "$2"/24 dev "$1"
vip-down.sh :
#!/bin/sh
exec 2> /dev/null
/sbin/ip addr del "$2"/24 dev "$1"
Everything works well and the servers switch from one to another correctly when the master becomes unavailable.
The problem is when I unplug both servers from the switch for a too long time (approximatively 30 min). As they are unplugged they both think they are master,
and when I replug them, the one with the lowest ip address tries to stay master by sending gratuitous arps. The other one switches to backup as expected, but I'm unable to access the master through its virtual ip.
If I unplug the master, the second server goes from backup to master and is accessible through its virtual ip.
My guess is that the switch "forgets" about my servers when they are disconnected from too long, and when I reconnect them, it is needed to go from backup to master to update correctly switch's arp cache, eventhough the gratuitous arps send by master should do the work. Note that restarting ucarp on the master does fix the problem, but I need to restart it each time it was disconnected from too long...
Any idea why it does not work as I expected and how I could solve the problem ?
Thanks.

how to connect Docker containers without a bridge?

I'm having some experiments to do with Docker container technology.
I need for a specific reason to connect two veth container interfaces together without using a bridge, Docker creates a bridge by default, so I do not want to use it.
I'm confused and want to know if it is right to do that way. Anyone can give advices and point me out some links or methods ? I will appreciate.
Thank you so much.
+--------------+ +--------------+
| | | |
| Container X | | Container Y |
| | | |
+--------------+ +--------------+
^ veth   ^ veth  
| |
+--------------------+
Sure, it's possible, although you won't be able to get Docker to do it for you automatically. Begin by creating your two containers with no networking:
# docker run --net=none --name container_x ...
# docker run --net=none --name container_y ...
Now create a veth pair:
# ip link add c_x_eth0 type veth peer name c_y_eth0
Assign each side of the veth pair to a container. You will need to know the PID of the container to do this, which you can get with, for example:
docker inspect --format '{{.State.Pid}}' container_x
I'm going to assume you've stuck this in a shell script named docker-pid. Set the name space on the first veth link:
# ip link set netns $(docker-pid container_x) dev c_x_eth0
And on the second:
# ip link set netns $(docker-pid container_y) dev c_y_eth0
Now you will need to configure the link inside each container. If you haven't started your containers with --privileged, you will need to do this using nsenter:
# nsenter -t $(docker-pid container_x) -n ip link set c_x_eth0 up
# nsenter -t $(docker-pid container_y) -n ip link set c_y_eth0 up
And then assign them ip addresses:
# nsenter -t $(docker-pid container_x) -n ip addr add 10.10.10.1/24 dev c_x_eth0
# nsenter -t $(docker-pid container_y) -n ip addr add 10.10.10.2/24 dev c_y_eth0
And you should be all set.
Update
If nsenter is unavailable...
The easiest solution really is just to install nsenter on your system; if you are able to create new veth interfaces and start Docker containers you should have all the privileges you need.
You could accomplish the above without nsenter if you run your containers in privileged mode (docker run --privileged...). This will allow your containers to do things -- such as run network configuration commands -- that are normally prohibited. In this case, you would just run the ip link and ip addr commands in the container, either from a shell you started with docker run or using something like docker exec. You should be aware that running a container in privileged mode removes many of the restrictions normally placed on containers, and so it is not something you want to do if anyone else has access to those containers.

Setting Up Docker Dnsmasq

I'm trying to set up a docker dnsmasq container so that I can have all my docker containers look up the domain names rather than having hard-coded IPs (if they are on the same host). This fixes an issue with the fact that one cannot alter the /etc/hosts file in docker containers, and this allows me to easily update all my containers in one go, by altering a single file that the dnsmasq container references.
It looks like someone has already done the hard work for me and created a dnsmasq container. Unfortunately, it is not "working" for me. I wrote a bash script to start the container as shown below:
name="dnsmasq_"
timenow=$(date +%s)
name="$name$timenow"
sudo docker run \
-v="$(pwd)/dnsmasq.hosts:/dnsmasq.hosts" \
--name=$name \
-p='127.0.0.1:53:5353/udp' \
-d sroegner/dnsmasq
Before running that, I created the dnsmasq.hosts directory and inserted a single file within it called hosts.txt with the following contents:
192.168.1.3 database.mydomain.com
Unfortunately whenever I try to ping that domain from within:
the host
The dnsmasq container
another container on the same host
I always receive the ping: unknown host error message.
I tried starting the dnsmasq container without daemon mode so I could debug its output, which is below:
dnsmasq: started, version 2.59 cachesize 150
dnsmasq: compile time options: IPv6 GNU-getopt DBus i18n DHCP TFTP conntrack IDN
dnsmasq: reading /etc/resolv.dnsmasq.conf
dnsmasq: using nameserver 8.8.8.8#53
dnsmasq: read /etc/hosts - 7 addresses
dnsmasq: read /dnsmasq.hosts//hosts.txt - 1 addresses
I am guessing that I have not specified the -p parameter correctly when starting the container. Can somebody tell me what it should be for other docker containers to lookup the DNS, or whether what I am trying to do is actually impossible?
The build script for the docker dnsmasq service needs to be changed in order to bind to your server's public IP, which in this case is 192.168.1.12 on my eth0 interface
#!/bin/bash
NIC="eth0"
name="dnsmasq_"
timenow=$(date +%s)
name="$name$timenow"
MY_IP=$(ifconfig $NIC | grep 'inet addr:'| grep -v '127.0.0.1' | cut -d: -f2 | awk '{ print $1}')
sudo docker run \
-v="$(pwd)/dnsmasq.hosts:/dnsmasq.hosts" \
--name=$name \
-p=$MY_IP:53:5353/udp \
-d sroegner/dnsmasq
On the host (in this case ubuntu 12), you need to update the resolv.conf or /etc/network/interfaces file so that you have registered your public IP (eth0 or eth1 device) as the nameserver.
You may want to set a secondary nameserver to be google for whenever the container is not running, by changing the line to be dns-nameservers xxx.xxx.xxx.xxx 8.8.8.8 E.g. there is no comma or another line.
You then need to restart your networking service sudo /etc/init.d/networking restart if you updated the /etc/network/interfaces file so that this auto updates the /etc/resolve.conf file that docker will copy to the container during the build.
Now restart all of your containers
sudo docker stop $CONTAINER_ID
sudo docker start $CONTAINER_ID
This causes their /etc/resolv.conf files update so they point to the new nameserver settings.
DNS lookups in all your docker containers (that you built since making the changes) should now work using your dnsmasq container!
As a side note, this means that docker containers on other hosts can also take advantage of your dnsmasq service on this host as long as their host's nameserver settings is set to using this server's public IP.

How to bind rsync to a specific interface?

I have a lot of interfaces configured in my server, each of which connect to a specific nic card and have a separate routing table. These interfaces can be identified by "netstat -a" command.
Now, I want to execute the rsync command connecting only to specific interface. I have this requirement because each of the interfaces will go through a separate tunnel/path and I want a particular rsync command to sync files through a specified tunnel.
Specifically, I want a way to specify the interface name.
Thanks,
Mohan.
You can specify the address of the interface using --address=x.x.x.x on the command-line.
I don't think there is any way to specify the interface directly, but the ip command can tell you the address for an interface, so you could use something like this:
IP=$(ip -4 -br addr show eth0 | awk '{split($3,a,"/"); print a[1]}')
rsyncd ... --address=$IP
Edit For systems with the "real" iproute2 (anything not busybox-based, essentially), ip can produce JSON output which can be parsed a bit more sanely:
IP=$(ip -j -4 addr show wlo1 | jq .[0].addr_info[0].local)
rsyncd ... --address=$IP
I've written this little perl script to turn interface names to addresses, save it as iftoip (or similar)
#!/usr/bin/perl
use strict;
use warnings;
use IO::Interface::Simple;
use feature qw(say);
my $iface = shift;
my $if = IO::Interface::Simple->new($iface) or die "$!: $iface";
say $if->address;
exit 0;
You can do something similar with bash:
iftoip() {
ip addr show $1 | grep inet | awk '{print $2}' | cut -d'/' -f1
}
just add the above 3 lines to ~/.bashrc and start a new shell or source ~/.bashrc
Running it produces:
v#juno:~$ iftoip ens33
10.251.17.94
v#juno:~$ iftoip ens34
192.168.78.128
v#juno:~$ echo "IP=$(iftoip ens33)"
IP=10.251.17.94
v#juno:~$ iftoip ens35 #perl
No such device: ens35 at /home/v/bin/iftoip line 10.
or
v#juno:~$ iftoip ens35 #bash
Device "ens35" does not exist.
This has been tried using 2 interfaces, with different subnets and worked.
rsync -avzP -e 'ssh -b 10.100.16.X' /var/tmp/ent1 10.100.16.X:/var/tmp/;
rsync -avzP -e 'ssh -b 10.100.20.X' /var/tmp/ent2 10.100.20.X:/var/tmp/ ;
From client to server, over ssh use:
rsync -avP -e 'ssh -b x.x.x.x' tmp/ server:tmp/

Resources