How to use Docker (or Linux Containers) for Network Emulation? - networking

Question

Edit: As of March 2019, although I have not tested it, I believe Docker now has the ability to do real network emulation.
Edit: As of May 2015, SocketPlane (see website and repo) has joined the Docker team and they're in the process of integrating their OVS solution into Docker core. It appears as if theirs will be the winner of the various Docker networking solutions.
So I've been using Mininet to run tests on my networking software. It seems to have hit its limits though as Mininet containers are essentially linux containers with only a networking stack. I'd like each container to have its own networking stack, file system AND set of processes - basically I'd like a container as close to a VM as possible. Which brings me to Docker, as I understand, Docker is opposite of Mininet, its containers have a file system and their own processes but not their own networking stack. I'm leaning towards Docker as it has a nice API for forking containers, using the disk space of only the diff. My question is, is it possible to create a set of linux containers (with Docker or similar) with the following container layout + network interface setup?

Answer 1

You can use Pipework for that purpose. It is specifically one of the scenarios it implements (private networks between containers, in addition to the standard Docker network).

Answer 2

I am aware of two open-source network emulators that use linux containers:
The CORE Network Emulator uses containers and each container has its own filesystem (or partial filesystem, because it only creates mount namespaces for the directories required by the services running on each node).
The VNX network emulator is another option. It uses either KVM or LXC to create virtual nodes (but I have not tried the LXC option, yet).

Answer 3

CORE Network Emulator does have a Docker Service that I contributed and wrote an article about. The initial version that is in 4.8 is mostly broken but I have fixed and improved it. A pull request is on GitHub.
The service allows you to tag Docker Images with 'core' and then they appear as an option in the services settings. You must select the Docker image which starts the docker service in the container. You then select the container or containers that you want to run in that node. It scales quite well and I have had over 2000 nodes on my 16Gb machine.
You mentioned OVS as well. This is not yet built in to CORE but can be used manually. I just answered a question on the CORE mailing list on this. It gives a brief overview of switching out a standard CORE switch(bridge) with OVS. Text reproduced below if it is useful:
Not really used openvswitch before but had a quick look.
I installed openvswitch via my package manager (Ubuntu 15.04):
sudo apt-get install openvswitch-switch
I then built a very simple network in CORE 4.8. 2 PCs connected to a switch. I started the emulation in CORE. Then on the host I looked at the bridges that had been set up:
sudo brctl show
bridge name bridge id STP enabled interfaces
b.3.76 8000.42c789ce95e9 no veth1.0.76
veth2.0.76
docker0 8000.56847afe9799 no
lxcbr0 8000.000000000000 no
I can see the bridge that represents the switch is called b.3.76 and has interfaces veth1.0.76 and veth2.0.76 attached to it. I delete the bridge:
sudo ip link set b.3.76 down
sudo brctl delbr b.3.76
I then set up the openvswitch bridge:
sudo ovs-vsctl add-br b.3.76
sudo ovs-vsctl add-port b.3.76 veth1.0.76
sudo ovs-vsctl add-port b.3.76 veth2.0.76
I can now ping between the nodes so the switch seems to be working. I have not tried to do any further configuration of openvswitch.
When you stop the CORE emulation it does not obviously delete the openvswitch bridge or ports so you have to do that by hand:
sudo ovs-vsctl del-port veth2.0.76
sudo ovs-vsctl del-port veth1.0.76
sudo ovs-vsctl del-br b.3.76
This would be relatively simple to automate with a script or with a little bit of work could be integrated in to docker.
Hope this helps

Answer 4

#jpillora IMUNES network emulator uses Docker for their L3 (PC, Router, Host) and Open vSwitch for the L2 (Hub, Switch) nodes. For example, the Router node is actually a Docker container with a Debian Jessie system that runs Quagga automatically configured so you just have to draw the nodes inside the GUI. You can then access those containers by double-clicking on them and do whatever you would do on a Linux system. It uses a "special" Docker image available on Hub called imunes/vroot that uses a dummy init process so it doesn't terminate. Technically, with a bit of tweaking you can replace it with whatever you want. Its source code is available on Github.
I think it would be appropriate for you use case.

Answer 5

I tried CORE and a few others but found them hard to set up and run (especially in AWS or on Mac). They are probably powerful but overkill if you just want to simulate simple networks.
Hence I wrote YANS (Yet Another Network Simulator). YANS is based on Docker. Even I myself am surprised to see how fast it runs. Give it a shot!