I need help understanding my networking logs due to docker-compose networking.
I'm ssh'd into a VM, and I have two projects with docker-compose. The first is launched simply with docker-compose up. When I try to launch the second, my ssh session freezes, and I can no longer ssh into the VM. After lots of trial and error, and after reading this I tried to append to my 2nd project's docker-compose.yml file the following:
networks:
default:
external:
name: abcdef_default
where abcdef_default is the name of the network created by docker-compose up of the 1st project. With this, the docker-compose up on the 2nd project doesn't kick me out of the ssh session.
I tailed the logs in /var/log/*.log, and here's the output with the networks section in the docker-compose.yml file (without the timestamp prefix: Jan 19 09:13:42 hostname kernel: [420096.305357]):
aufs au_opts_verify:1597:dockerd[13813]: dirperm1 breaks the protection by the permission bits on the lower branch
device veth6a84537 entered promiscuous mode
IPv6: ADDRCONF(NETDEV_UP): veth6a84537: link is not ready
eth0: renamed from veth2480623
IPv6: ADDRCONF(NETDEV_CHANGE): veth6a84537: link becomes ready
br-fe0deb0149df: port 18(veth6a84537) entered forwarding state
br-fe0deb0149df: port 18(veth6a84537) entered forwarding state
aufs au_opts_verify:1597:dockerd[25317]: dirperm1 breaks the protection by the permission bits on the lower branch
device veth1a3c1e3 entered promiscuous mode
IPv6: ADDRCONF(NETDEV_UP): veth1a3c1e3: link is not ready
br-fe0deb0149df: port 22(veth1a3c1e3) entered forwarding state
br-fe0deb0149df: port 22(veth1a3c1e3) entered forwarding state
eth0: renamed from veth54e576d
IPv6: ADDRCONF(NETDEV_CHANGE): veth1a3c1e3: link becomes ready
br-fe0deb0149df: port 22(veth1a3c1e3) entered disabled state
veth54e576d: renamed from eth0
br-fe0deb0149df: port 22(veth1a3c1e3) entered disabled state
device veth1a3c1e3 left promiscuous mode
br-fe0deb0149df: port 22(veth1a3c1e3) entered disabled state
br-fe0deb0149df: port 18(veth6a84537) entered forwarding state
and here's the output without the networks section (i.e. when I get kicked out of the ssh session):
IPv6: ADDRCONF(NETDEV_UP): br-55349b03453a: link is not ready
aufs au_opts_verify:1597:dockerd[26982]: dirperm1 breaks the protection by the permission bits on the lower branch
aufs au_opts_verify:1597:dockerd[26982]: dirperm1 breaks the protection by the permission bits on the lower branch
aufs au_opts_verify:1597:dockerd[3051]: dirperm1 breaks the protection by the permission bits on the lower branch
device veth7a1bcde entered promiscuous mode
IPv6: ADDRCONF(NETDEV_UP): veth7a1bcde: link is not ready
br-55349b03453a: port 1(veth7a1bcde) entered forwarding state
br-55349b03453a: port 1(veth7a1bcde) entered forwarding state
br-55349b03453a: port 1(veth7a1bcde) entered disabled state
eth0: renamed from veth5d8a2ea
IPv6: ADDRCONF(NETDEV_CHANGE): veth7a1bcde: link becomes ready
br-55349b03453a: port 1(veth7a1bcde) entered forwarding state
br-55349b03453a: port 1(veth7a1bcde) entered forwarding state
IPv6: ADDRCONF(NETDEV_CHANGE): br-55349b03453a: link becomes ready
aufs au_opts_verify:1597:dockerd[13814]: dirperm1 breaks the protection by the permission bits on the lower branch
aufs au_opts_verify:1597:dockerd[13814]: dirperm1 breaks the protection by the permission bits on the lower branch
aufs au_opts_verify:1597:dockerd[13922]: dirperm1 breaks the protection by the permission bits on the lower branch
device veth3253bd4 entered promiscuous mode
IPv6: ADDRCONF(NETDEV_UP): veth3253bd4: link is not ready
br-55349b03453a: port 2(veth3253bd4) entered forwarding state
br-55349b03453a: port 2(veth3253bd4) entered forwarding state
br-55349b03453a: port 2(veth3253bd4) entered disabled state
eth0: renamed from veth9c8aaa3
IPv6: ADDRCONF(NETDEV_CHANGE): veth3253bd4: link becomes ready
br-55349b03453a: port 2(veth3253bd4) entered forwarding state
br-55349b03453a: port 2(veth3253bd4) entered forwarding state
br-55349b03453a: port 2(veth3253bd4) entered disabled state
veth9c8aaa3: renamed from eth0
br-55349b03453a: port 2(veth3253bd4) entered disabled state
device veth3253bd4 left promiscuous mode
br-55349b03453a: port 2(veth3253bd4) entered disabled state
br-55349b03453a: port 1(veth7a1bcde) entered forwarding state
br-55349b03453a: port 1(veth7a1bcde) entered disabled state
veth5d8a2ea: renamed from eth0
br-55349b03453a: port 1(veth7a1bcde) entered disabled state
device veth7a1bcde left promiscuous mode
br-55349b03453a: port 1(veth7a1bcde) entered disabled state
I don't really understand how to read these logs.
Here is my ifconfig also.
Can someone help me read the logs and figure out what the problem is?
Diagnosis
Our team is using AWS EC2 instances running Ubuntu 18.04 as devservers. We recently got reports that docker-compose broke SSH connections. Even after restarting, the devservers are still inaccessible. So I started investigation.
I was able to exclude the cause of docker-compose by reproducing using docker only.
ubuntu#ip-172-31-115-116:~$ docker network create -d bridge my-bridge-network
aca5884d60f146cef81ac55c8cccd231a43f40927d645168642d9b28c5e009a6
ubuntu#ip-172-31-115-116:~$ docker network prune
WARNING! This will remove all custom networks not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Networks:
my-bridge-network
ubuntu#ip-172-31-115-116:~$ docker network create -d bridge my-bridge-network
f0a7a06a9627bc2de00eb60091a92010451690626d95e077f622f3058cc3a07c
ubuntu#ip-172-31-115-116:~$ docker network prune
WARNING! This will remove all custom networks not used by at least one container.
Are you sure you want to continue? [y/N] y
Deleted Networks:
my-bridge-network
ubuntu#ip-172-31-115-116:~$ docker network create -d bridge my-bridge-network
Connection reset by 172.31.115.116 port 22
Then the root cause occurred to me.
Root cause
Our docker-compose files are using the bridge network mode which will create a new bridge network by default. When docker-compose down or docker network prune is run, the bridge network will be torn down. And the next docker-compose run or docker network create will create a new bridge network.
The default IP range for the docker0 bridge adapter is 172.17.0.0/16.
When I first ran the docker network create -d bridge my-bridge-network command, it created a new bridge adapter for 172.18.0.0/16.
The second bridge adapter was created for 172.19.0.0/16.
Naturally, the 3rd bridge adapter is created for 172.20.0.0/16. However, that is our Engineering VPN IP range. Therefore the overlap caused the server unable to communicate with our laptop.
Solutions
The solution is to make sure new docker bridge networks will skip our VPN IP range.
Temp solution
If we add the skipped IP ranges to system route table, docker will automatically skip them. Therefore, we can run the below script whenever the devserver got rebooted.
sudo route add -net [our VPN IP range] netmask 255.255.0.0 gw [our gateway]
This solution is imperfect that the new routes will be discarded after restarting the machine.
Main solution
We should permanently apply the route changes to all devservers.
echo " routes:" | sudo tee -a /etc/netplan/50-cloud-init.yaml
echo " - to: [our VPN IP range]" | sudo tee -a /etc/netplan/50-cloud-init.yaml
echo " via: [our gateway]" | sudo tee -a /etc/netplan/50-cloud-init.yaml
sudo netplan apply
Docker IP changes
We also plan to modify the docker default-address-pools to redefine docker IP ranges. Refer to https://github.com/docker/compose/issues/4336#issuecomment-457326123. I would say modifying /etc/docker/daemon.json is better.
br-xxxxxxx are the bridge interfaces of Docker and vethxxxxxxx are the virtual interfaces of your containers, Docker use those veth interfaces but you do not directly interact on it, they use an IPv6 address and don't have IPv4. Docker can't create NAT interfaces, it can only create bridge and veth with IPv6 for containers. You can link your bridge to any physical or virtual interface of your host.
So it work like that:
eth0 (your interface or v-interface if you want) ↔ brxxxxx(docker bridge) ↔ vethxxxxx (v-interface of your container)
It's all I can say, I'm not sure that someone else will answer, there is not a lot of Docker experts, so I give you all informations I can to help you to understand your logs.
I finally ended up running a docker network ls. The output was a list of more than 15 networks which were very old. I ran a docker ps to make sure that nothing related to these networks was still running. One container was indeed still running (redis) and it was on a network called bridge. I stopped the container. Then I started going through all the networks with docker network rm <network name> until I was left with 4 networks: bridge, host, none, and the only network that was still working. Then I could start new networks with docker-compose up again as usual
I had the same issue, I solved it by setting the network_mode option on docker compose (see the docs here. The solution came from this thread ).
services:
my_service:
image: ...
network_mode: "host"
Related
Background
I have a strange use-case where my VPN cannot be on any of the private subnets, but, also cannot use a TAP interface. The machine will be moving through different subnets, and requires access to the entire private address space by design. A single blocked IP would be considered a failure of design.
So, these are all off limits:
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
169.254.0.0/16
In searching for a solution, I came across RFC 5735, which defines:
192.0.2.0/24 TEST-NET-1
198.51.100.0/24 TEST-NET-2
203.0.113.0/24 TEST-NET-3
As:
For use in documentation and example code. It is often used in conjunction with domain names
example.com or example.net in vendor and protocol documentation. As described in [RFC5737], addresses within this block do not legitimately appear on the public Internet and can be used without any coordination with IANA or an Internet registry.
Which, was a "Jackpot" moment for me and my use case.
Config
I configured an OpenVPN server as such:
local 0.0.0.0
port 443
proto tcp
dev tun
topology subnet
server 203.0.113.0 255.255.255.0 # TEST-NET-3 RFC 5735
push "route 203.0.113.0 255.255.255.0"
...[Snip]...
With Client:
client
nobind
dev tun
proto tcp
...[Snip]...
And ufw rules:
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 203.0.113.0/24 -o ens160 -j MASQUERADE
COMMIT
However, upon running I get /sbin/ip route add 203.0.113.0/24 via 203.0.113.1 RTNETLINK answers: File exists in the error logs. While the VPN completes the rest of its connection successfully.
No connection
Running the following commands:
Server: sudo python3 -m http.server 80
Client: curl -X GET / 203.0.113.1
Results in:
curl: (28) Failed to connect to 203.0.113.1 port 80: Connection timed out
I have tried:
/sbin/ip route replace 203.0.113.0/24 dev tun 0 on client and server.
/sbin/ip route change 203.0.113.0/24 dev tun 0 on client and server.
Adding route 203.0.113.0 255.255.255.0 to the server.
Adding push "route 203.0.113.0 255.255.255.0 127.0.0.1" to server
And none of it seems to work.
Does anyone have any idea how I can force the client to push this traffic over the VPN to my server, instead of to the public IP?
This does actually work!
Just dont forget to allow connections within your firewall. I fixed my config with:
sudo ufw allow in on tun0
However, 198.18.0.0/15 and 100.64.0.0/10 defined as Benchmarking and Shared address space respectively, may be more appropriate choices, since being able to forward TEST-NET addresses may be considered a bug.
Since Docker 1.10 (and libnetwork update) we can manually give an IP to a container inside a user-defined network, and that's cool!
I want to give a container an IP address in my LAN (like we can do with Virtual Machines in "bridge" mode). My LAN is 192.168.1.0/24, all my computers have IP addresses inside it. And I want my containers having IPs in this range, in order to reach them from anywhere in my LAN (without NAT/PAT/etc...).
I obviously read Jessie Frazelle's blog post and a lot of others post here and everywhere like :
How to set a docker container's iP?
How to assign specific IP to container and make that accessible outside of VM host?
and so much more, but nothing came out; my containers still have IP addresses "inside" my docker host, and are not reachable for others computers on my LAN.
Reading Jessie Frazelle's blog post, I thought (since she uses public IP) we can do what I want to do?
Edit: Indeed, if I do something like :
network create --subnet 192.168.1.0/24 --gateway 192.168.1.1 homenet
docker run --rm -it --net homenet --ip 192.168.1.100 nginx
The new interface on the docker host (br-[a-z0-9]+) take the '--gateway' IP, which is my router IP. And the same IP on two computers on the network... BOOM
Thanks in advance.
EDIT : This solution is now useless. Since version 1.12, Docker provides two network drivers : macvlan and ipvlan. They allow assigning static IP from the LAN network. See the answer below.
After looking for people who have the same problem, we went to a workaround :
Sum up :
(V)LAN is 192.168.1.0/24
Default Gateway (= router) is 192.168.1.1
Multiple Docker Hosts
Note : We have two NIC : eth0 and eth1 (which is dedicated to Docker)
What do we want :
We want to have containers with ip in the 192.168.1.0/24 network (like computers) without any NAT/PAT/translation/port-forwarding/etc...
Problem
When doing this :
network create --subnet 192.168.1.0/24 --gateway 192.168.1.1 homenet
we are able to give containers the IP we want to, but the bridge created by docker (br-[a-z0-9]+) will have the IP 192.168.1.1, which is our router.
Solution
1. Setup the Docker Network
Use the DefaultGatewayIPv4 parameter :
docker network create --subnet 192.168.1.0/24 --aux-address "DefaultGatewayIPv4=192.168.1.1" homenet
By default, Docker will give to the bridge interface (br-[a-z0-9]+) the first IP, which might be already taken by another machine. The solution is to use the --gateway parameter to tell docker to assign a arbitrary IP (which is available) :
docker network create --subnet 192.168.1.0/24 --aux-address "DefaultGatewayIPv4=192.168.1.1" --gateway=192.168.1.200 homenet
We can specify the bridge name by adding -o com.docker.network.bridge.name=br-home-net to the previous command.
2. Bridge the bridge !
Now we have a bridge (br-[a-z0-9]+) created by Docker. We need to bridge it to a physical interface (in my case I have to NIC, so I'm using eth1 for that):
brctl addif br-home-net eth1
3. Delete the bridge IP
We can now delete the IP address from the bridge, since we don't need one :
ip a del 192.168.1.200/24 dev br-home-net
The IP 192.168.1.200 can be used as bridge on multiple docker host, since we don't use it, and we remove it.
Docker now supports Macvlan and IPvlan network drivers. The Docker documentation for both network drivers can be found here.
With both drivers you can implement your desired scenario (configure a container to behave like a virtual machine in bridge mode):
Macvlan: Allows a single physical network interface (master device) to have an arbitrary number of slave devices, each with it's own MAC adresses.
Requires Linux kernel v3.9–3.19 or 4.0+.
IPvlan: Allows you to create an arbitrary number of slave devices for your master device which all share the same MAC address.
Requires Linux kernel v4.2+ (support for earlier kernels exists but is buggy).
See the kernel.org IPVLAN Driver HOWTO for further information.
Container connectivity is achieved by putting one of the slave devices into the network namespace of the container to be configured. The master devices remains on the host operating system (default namespace).
As a rule of thumb you should use the IPvlan driver if the Linux host that is connected to the external switch / router has a policy configured that allows only one MAC per port. That's often the case in VMWare ESXi environments!
Another important thing to remember (Macvlan and IPvlan): Traffic to and from the master device cannot be sent to and from slave devices. If you need to enable master to slave communication see section "Communication with the host (default-ns)" in the "IPVLAN – The beginning" paper published by one of the IPvlan authors (Mahesh Bandewar).
Use the official Docker driver:
As of Docker v1.12.0-rc2, the new MACVLAN driver is now available in an official Docker release:
MacVlan driver is out of experimental #23524
These new drivers have been well documented by the author(s), with usage examples.
End of the day it should provide similar functionality, be easier to setup, and with fewer bugs / other quirks.
Seeing Containers on the Docker host:
Only caveat with the new official macvlan driver is that the docker host machine cannot see / communicate with its own containers. Which might be desirable or not, depending on your specific situation.
This issue can be worked-around if you have more than 1 NIC on your docker host machine. And both NICs are connected to your LAN. Then can either A) dedicate 1 of your docker hosts's 2 nics to be for docker exclusively. And be using the remaining nic for the host to access the LAN.
Or B) by adding specific routes to only those containers you need to access via the 2nd NIC. For example:
sudo route add -host $container_ip gw $lan_router_ip $if_device_nic2
Method A) is useful if you want to access all your containers from the docker host and you have multiple hardwired links.
Wheras method B) is useful if you only require access to a few specific containers from the docker host. Or if your 2nd NIC is a wifi card and would be much slower for handling all of your LAN traffic. For example on a laptop computer.
Installation:
If cannot see the pre-release -rc2 candidate on ubuntu 16.04, temporarily add or modify this line to your /etc/apt/sources.list to say:
deb https://apt.dockerproject.org/repo ubuntu-xenial testing
instead of main (which is stable releases).
I no longer recommended this solution. So it's been removed. It was using bridge driver and brctrl
.
There is a better and official driver now. See other answer on this page: https://stackoverflow.com/a/36470828/287510
Here is an example of using macvlan. It starts a web server at http://10.0.2.1/.
These commands and Docker Compose file work on QNAP and QNAP's Container Station. Notice that QNAP's network interface is qvs0.
Commands:
The blog post "Using Docker macvlan networks"[1][2] by Lars Kellogg-Stedman explains what the commands mean.
docker network create -d macvlan -o parent=qvs0 --subnet 10.0.0.0/8 --gateway 10.0.0.1 --ip-range 10.0.2.0/24 --aux-address "host=10.0.2.254" macvlan0
ip link del macvlan0-shim link qvs0 type macvlan mode bridge
ip link add macvlan0-shim link qvs0 type macvlan mode bridge
ip addr add 10.0.2.254/32 dev macvlan0-shim
ip link set macvlan0-shim up
ip route add 10.0.2.0/24 dev macvlan0-shim
docker run --network="macvlan0" --ip=10.0.2.1 -p 80:80 nginx
Docker Compose
Use version 2 because version 3 does not support the other network configs, such as gateway, ip_range, and aux_address.
version: "2.3"
services:
HTTPd:
image: nginx:latest
ports:
- "80:80/tcp"
- "80:80/udp"
networks:
macvlan0:
ipv4_address: "10.0.2.1"
networks:
macvlan0:
driver: macvlan
driver_opts:
parent: qvs0
ipam:
config:
- subnet: "10.0.0.0/8"
gateway: "10.0.0.1"
ip_range: "10.0.2.0/24"
aux_address: "host=10.0.2.254"
It's possible map a physical interface into a container via pipework.
Connect a container to a local physical interface
pipework eth2 $(docker run -d hipache /usr/sbin/hipache) 50.19.169.157/24
pipework eth3 $(docker run -d hipache /usr/sbin/hipache) 107.22.140.5/24
There may be a native way now but I haven't looked into that for the 1.10 release.
I have two hosts running opensuse 42.1 connected to a dlink router via eth0, accessible on 192.168.0.1 and using NetworkManager:
- vboard/eth0 is assigned via router DHCP ip 192.168.0.199
- rihana/eth0 192.168.0.198
Using KVM on both hosts, I have two opensuse VMs ( vmvboard, vmrihana) one on each host.
I configured on both hosts a virbr0 network identically, in the range 192.168.100.0/24 and DHCP range 192.168.100.128-254 and NAT on any physical device.
Vm can ping its KVM host on both side, but VM's cannot talk to each other across router network. This config used to work on opensuse 13.2, but not using network manager...
What am I doing wrong?
Is there anyone to help me with that configuration: networking with 2 hosts, a router and 2 VM's, one on each host ?
Thanks a lot in advance for your ideas.
Bridged Network for hosts and vms in a few clicks : using wicked.
Set the Dlink router settings via
Firefox/Chrome url: 192.168.0.1
User:Admin
Pass: Blank
IP on 192.168.0.1 subnet mask 255.255.255.224 ( 30 usable IPs)
Enable DHCP: unchecked
Note: it might be easier to first reset router to standard setting and connect through Network Manager
Setting up the Bridged Network with Libvirt/virt-manager and Wicked opensuse network
service.
1.1 Clean host1 and host2 previous bridge definitions
with YAST/NetworkManager/Network Settings/
Global Options Tab> Select in Dropdown: Wicked
> Overview Tab > Delete all Interfaces to make them appear as "not configured"
> HostName/DNS > Note that your hostname , domain remains there
> Routing Tab> Enable IPv4 Forwarding is off (no routing features for host1 and host2)
Click Ok.
This has now cleaned all interfaces/bridges and activated Wicked Network Service, instead of Network Manager:GNOME/Right Upper Corner has no Wired/Wifi Settings Menu Options.
1.2 Check the cleanup in Gnome Terminal as root: # su root
Password:
# cd /etc/sysconfig/network
ls ifcfg
returns only
ifcfg-lo ifcfg.template
ls .ifcfg*
ls: cannot access .ifcfg*: No such file or directory
but in case there are still .ifcfg-br0 .ifcfg-eth0,
rm .ifcfg-br0 # rm .ifcfg-eth0
ifconfig
shows only lo interface:
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 [...]
Check if wicked is active:
# systemctl status wicked
wicked.service - wicked managed network interfaces
Loaded: loaded (/usr/lib/systemd/system/wicked.service; enabled) Active: active (exited) since Fri 2016-01-08 15:37:56 AZOT; 34min ago
2. Setting up the Bridge with LIBVIRT: Gnome Terminal command line:
# virt-manager
or
GNOME YAST/Virtualization/Create Vm,followed by Cancel Creation
2.1 Click on your hypervisor QEMU/KVM to connect ( in my case QEMU/KVM ) 2.2 Menu Edit/Connections Details or Right-Click Details
2.3 Goto Network Interfaces Tab > Click + (Add) > Bridge > Forward :
Name : br0
Start Mode: none
Activate Now: checked on
IP Settings: Leave DHCP
or Configure, Mode: Static ( to continue on previous example and because VMs IP are already statically defined)
Address: 192.168.0.2/27 (equivalent to Subnet Mask set to 255.255.255.224) Gateway 192.168.0.1
Bridge Settings: turn STP off ( no complex networks )
Choose Interface(s)to Bridge
eth0 is checked
2.4 Finish - This will take sometimes to set up.
"The virtual interface is now being created." Processing... And br0 or brx shows as active.
2.5 Adjust your VMs Network NIC settings while they are still down
2.5.0 Remove Old NIC from VM:
Virt-Manager > Select VM > Open > Click Lamp Icon > Select NIC > Click Remove (right- down corner).
Note: if Lamp Icon does not appear after Open, Goto View and Select Toolbar checkbox.
2.5.1 Add new NIC to VM:
Virt-Manager > Select VM > Open > Click Lamp Icon > Add Hardware > Network >
Network source: Bridge br0: host device eth0
MAC Address: checked, leave the suggested one
Device Model: Hypervisor Default or the one you know the vm-guest has the driver for.
Finish
2.5.2 Run the VM and test in VM's Gnome Terminal
ping vm
# ping host1 etc...
2.6 Repeat the steps in host2 bare metal, with Step 1 and subsequent 2.3 with Name : br0
Start Mode: none
Activate Now: checked on
IP Settings: Leave DHCP
or Configure, Mode: Static
Address: 192.168.0.10/27
Bridge Settings: turn STP off
Choose Interface(s)to Bridge eth0 is checked
2.7 Adjust host2 VMs Network NIC settings according to 2.5
Conclusion: With a clean starting situation,
virt-manager could set up the bridge and network connection to the router successfully,
in just a few clicks.
When I create my container, I want to set a specific container's IP address in the same LAN.
Is that possible? If not, after the creation can I edit the DHCP IP address?
Considering the conclusion of the (now old October 2013) article "How to configure Docker to start containers on a specific IP address range", this doesn't seem to be possible (or at least "done automatically for you by Docker") yet.
Update Nov 2015: a similar problem is discussed in docker/machine issue 1709, which include the recent workaround (Nov 2015)proposed by Tobias Munk (schmunk42) for docker machine
(for container see the next section):
A workaround for some use-cases could be to create machines like so:
192.168.98.100
docker-machine create -d virtualbox --virtualbox-hostonly-cidr "192.168.98.1/24" m98
192.168.97.100
docker-machine create -d virtualbox --virtualbox-hostonly-cidr "192.168.97.1/24" m97
192.168.96.100
docker-machine create -d virtualbox --virtualbox-hostonly-cidr "192.168.96.1/24" m96
If there's no other machine with the same cidr (Classless Inter-Domain Routing), the machine should always get the .100 IP upon start.
Another workaround:
(see my script in "How do I create a docker machine with a specific URL using docker-machine and VirtualBox?")
My virtualbox has dhcp range 192.168.99.100 - 255 and I want to set an IP before 100.
I've found a simple trick to set a static IP: after create a machine I run this command and restart the machine:
echo "ifconfig eth1 192.168.99.50 netmask 255.255.255.0 broadcast 192.168.99.255 up" \
| docker-machine ssh prova-discovery sudo tee /var/lib/boot2docker/bootsync.sh > /dev/null
This command create a file bootsync.sh that is searched by boot2docker startup scripts and executed.
Now during machine boot the command is executed and set static IP.
docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM
test-1 - virtualbox Running tcp://192.168.99.50:2376 test-1 (mast
Michele Tedeschi (micheletedeschi) adds
I've updated the commands with:
echo "kill `more /var/run/udhcpc.eth1.pid`\nifconfig eth1 192.168.99.50 netmask 255.255.255.0 broadcast 192.168.99.255 up" | docker-machine ssh prova-discovery sudo tee /var/lib/boot2docker/bootsync.sh > /dev/null
then run command (only the first time)
docker-machine regenerate-certs prova-discovery
now the IP will not be changed by the DHCP
(replace prova-discovery by the name of your docker-machine)
April 2015:
The article mentions the possibility to create your own bridge (but that doesn't assign one of those IP addresses to a container though):
create your own bridge, configure it with a fixed address, tell Docker to use it. Done.
If you do it manually, it will look like this (on Ubuntu):
stop docker
ip link add br0 type bridge
ip addr add 172.30.1.1/20 dev br0
ip link set br0 up
docker -d -b br0
To assign a static IP within the range of an existing bridge IP range, you can try "How can I set a static IP address in a Docker container?", using a static script which creates the bridge and a pair of peer interfaces.
Update July 2015:
The idea mention above is also detailed in "How can I set a static IP address in a Docker container?" using:
Building your own bridge
The result should be that the Docker server starts successfully and is now prepared to bind containers to the new bridge.
After pausing to verify the bridge’s configuration, try creating a container — you will see that its IP address is in your new IP address range, which Docker will have auto-detected.
you can use the brctl show command to see Docker add and remove interfaces from the bridge as you start and stop containers, and can run ip addr and ip route inside a container to see that it has been given an address in the bridge’s IP address range and has been told to use the Docker host’s IP address on the bridge as its default gateway to the rest of the Internet.
Start docker with: -b=br0 (that is also what the echo 'DOCKER_OPTS="-b=bridge0"' >> /etc/default/docker can set for you by default)
Use pipework (192.168.1.1 below being the default gateway ip address):
pipework br0 container-name 192.168.1.10/24#192.168.1.1
I am using the latest version of boot2docker version 1.3.2, 495c19a on a windows 7 (SP1) 64 bit machine.
My docker container is running a celery process which attempts to connect to a rabbitMQ service running on the same machine that boot2docker is running on.
The Celery process running within the docker container cannot connect to RabbitMQ and reports the following :
[2014-12-02 10:28:41,141: ERROR/MainProcess] consumer: Cannot connect
to amqp:// guest:**#127.0.0.1:5672//: [Errno 111] Connection refused.
Trying again in 2.00 seconds...
I have reason to believe this is a network related issue, associated with routing from the container, to the VirtualBox host, and from the host to the RabbitMQ service running on the local machine; I do not know how to configure this and I was wondering if anyone can advise me how to proceed?
I tried setting up port 5672 in port forwarding but it didn't work (but I believe this is for incoming traffic to the VM, like boot2docker ssh).
I am running the container as docker run -i -t tagname
I am not specifying a host with -h when I run the container.
I'm sorry if this question appears rather clueless or if the answer appears obvious ... I appreciate any help!
Some additional information :
The routing table of the host VM is what boot2docker configured during installation as follows :
docker0 IP Address is 172.17.42.1
eth0 IP Address is 10.0.2.15
eth1 IP Address is 192.168.59.103
eth0 is attached to NAT (Adapter 1) in the VirtualBox VM network configuration.
Adapter 1 has port forwarding setup for ssh; default setting of host IP 127.0.0.1, host port 2022, guest port 22.
eth1 is attached to Host-only adapter (Adapter 2).
Both adapters are set to promiscuous mode (allow all).
The IP Address of the docker container is 172.17.0.33.
[2014-12-02 10:28:41,141: ERROR/MainProcess] consumer: Cannot connect to amqp:// guest:**#127.0.0.1:5672//: [Errno 111] Connection refused. Trying again in 2.00 seconds...
127.0.0.1 is a special IP address that means "me", and inside the container it means "me the container", so this is why it is not connecting to the outer host. So the first thing to do is change the IP address where you are trying to connect to Rabbit to that of the outer host where it is running.
Then you probably have to do something about routing, but let's take one step at a time.
as your RabbitMQ server is running on your Windows host, you need to tell your container that it should talk to that IP - which would probably be 192.168.59.3
most importantly, your container's 127.0.0.1 is only a loopback device to that container's services - not even the boot2docker vm's ports.
You could set up an ambassador container that has --expose=80 and uses something like socat to forward all traffic from that container to your host (see svendowideit/ambassador). Then you'd --link that ambassador container to your current image
but personally, I'd avoid that initially, and just configure your containerised app to talk to the real host's IP
You have to specifc explicitely ports for port redirection separately for boot2docker and docker.
Please try this:
c:\>boot2docker init
c:\>boot2docker up
c:\>boot2docker ssh -L 0.0.0.0:5672:localhost:5672
docker#boot2docker:~$ docker run -it -p 5672:5672 tagname