I'm trying to create default NAT and bridge interface in my QEMU machine. Naturally, I created bridge interface in separate file /etc/network/intefaces.d/virbr2. Here is virbr2 file configuration:
# Configuring network virtual interface
# to be a virt switch
auto virbr2
iface virbr2 inet static
bridge_ports enp1s0
address 192.168.1.3
netmask 255.255.255.0
broadcast 192.168.1.255
up ip route add 192.168.1.2 via 192.168.1.1 via enp1s0
brdige_stp off
bridge_waitport 0
bridge_fd 0
My general interface configuration file is pretty simple
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# auto launch enp1s0 interface after the host os is booted
# since we want create a bridge interface, let's attach
# it to bridge interface br0
auto enp1s0
iface enp1s0 inet manual
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
Thus, interface virbr2 was created with proper IP address
$ ip a | grep -A 5 virbr2
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr2 state UP group default qlen 1000
link/ether e8:d8:d1:51:15:c2 brd ff:ff:ff:ff:ff:ff
3: wlp0s20f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 04:ea:56:59:cf:a4 brd ff:ff:ff:ff:ff:ff
inet 192.168.31.69/24 brd 192.168.31.255 scope global dynamic noprefixroute wlp0s20f3
valid_lft 41947sec preferred_lft 41947sec
--
4: virbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether d6:71:34:e1:fa:9b brd ff:ff:ff:ff:ff:ff
inet 192.168.1.3/24 brd 192.168.1.255 scope global virbr2
valid_lft forever preferred_lft forever
inet6 fdf7:2246:8eb:0:d471:34ff:fee1:fa9b/64 scope global dynamic mngtmpaddr
valid_lft forever preferred_lft forever
inet6 fe80::d471:34ff:fee1:fa9b/64 scope link
valid_lft forever preferred_lft forever
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242daa58f02 no
virbr0 8000.525400d87725 yes
virbr2 8000.d67134e1fa9b no enp1s0
As bridge interface created, I'm trying to launch my VM with next command:
qemu-system-x86_64 \
-m 4096 \
-smp 4 \
-drive 'file=debian-opkg-server.qcow2,if=virtio,format=qcow2' \
-net 'user,hostfwd=tcp::2200-:22' \
-net nic \
-netdev 'tap,id=br1,ifname=virbr2,script=no,downscript=no' \
-device 'virtio-net-pci,netdev=br1'
After the script launch i get next error message
Unable to init server: Could not connect: Connection refused
qemu-system-x86_64: could not configure /dev/net/tun (virbr2): Invalid argument
How it's possible that argument is invalid? Interface name is correct, so I have no idea about the reason it's not working.
Related
I have a Raspberry PI 4 running Ubuntu 21.10 with a static ip-address on eth0. Despite that, I keep getting a secondary 'dynamic' DHCP address on on it.
netplan
network:
version: 2
renderer: networkd
ethernets:
eth0:
dhcp4: no
addresses:
- 192.168.0.10/24
routes:
- to: default
via: 192.168.0.1
nameservers:
search: [lan]
addresses: [192.168.0.12]
ip addr show
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether dc:a6:32:da:df:55 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.10/23 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.0.225/23 brd 192.168.1.255 scope global secondary dynamic eth0
valid_lft 68727sec preferred_lft 68727sec
inet6 fe80::dea6:32ff:feda:df55/64 scope link
valid_lft forever preferred_lft forever
Even if I delete that interface, it keeps coming back after a few minutes. I have another PI with the "same" configuration and it doesn't have this problem.
I also have the /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg per instructions.
Have you tried using the word false instead of no on your dhcp4 entry to netplan?
I have a host machine with Debian 10 & QEMU-KVM. I installed packages and rebooted:
sudo apt install qemu-kvm virt-manager
sudo reboot
So now I want to create a bridge that will enable my virtual servers to (a) connect to network and (b) to be seen to a host machine and other computers on the network.
I read dozen of tutorials on how to do this and failed miserably every time. I had some sucess setting up bridge with (a) package iproute2 and (b) package virt-manager (ran as super user).
Trying as a root:
By folowing archwiki I set up my bridge using these commands:
sudo ip link add virtual_bridge type bridge
sudo ip link set dev virtual_bridge up
I then reset the ethernet card and connect it to the bridge as it's slave:
sudo ip link set dev enx24f5a2f17b27 down
sudo ip addr flush dev enx24f5a2f17b27
sudo ip link set dev enx24f5a2f17b27 up
sudo ip link set dev enx24f5a2f17b27 master virtual_bridge
And then I open the GUI application:
sudo virt-manager
I right click the QEMU/KVM session (qemu:///system) and I choose connect:
When session is connected I start creating a new virtual machine. During it's creation I come to a window asking to choose type of virtual network. There are two options. First one has suboptions while second one enables manual input of the device:
Host device enx24f5a2f17b27: macvtap
Bridge
VEPA
Private
Passthrough
Specify shared device name
I tried choosing suboptions offered by the first option, but when selected they, issue a warning:
In most configurations macvtap does not work for host to guest network communication
This is not an option for me because my virtual servers will need two-way communication. This is why I choose the second option and I manualy specify my bridge virtual_bridge:
Then I start the virtual machine which can browse the internet and also can the host machine. Both are assigned the IP in the same network. But when I try to ping them ping doesn't work in any direction. Host, can't ping virtual machine and vice versa.
I can't explain this, because archwiki states that bridge should be transparent like a switch and devices should therefore be able to ping eachother:
A bridge is a piece of software used to unite two or more network
segments. A bridge behaves like a virtual network switch, working
transparently (the other machines do not need to know or care about
its existence).
If I check internet settings on a host:
ziga#ziga-laptop:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c4:85:08:3c:1a:59 brd ff:ff:ff:ff:ff:ff
3: enx24f5a2f17b27: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virtual_bridge state UP group default qlen 1000
link/ether 24:f5:a2:f1:7b:27 brd ff:ff:ff:ff:ff:ff
inet 192.168.64.100/24 brd 192.168.64.255 scope global enx24f5a2f17b27
valid_lft forever preferred_lft forever
inet6 fe80::26f5:a2ff:fef1:7b27/64 scope link
valid_lft forever preferred_lft forever
32: virtual_bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 24:f5:a2:f1:7b:27 brd ff:ff:ff:ff:ff:ff
inet6 fe80::26f5:a2ff:fef1:7b27/64 scope link
valid_lft forever preferred_lft forever
34: vnet0: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virtual_bridge state UNKNOWN group default qlen 1000
link/ether fe:54:00:c4:3e:62 brd ff:ff:ff:ff:ff:ff
inet 169.254.82.75/16 brd 169.254.255.255 scope global vnet0
valid_lft forever preferred_lft forever
inet6 fe80::2c93:eff:fea5:c52b/64 scope link
valid_lft forever preferred_lft forever
From the above, I can confirm that my ethernet interface enx24f5a2f17b27 and vnet0 (which was automaticaly created by virtual machine) are both slaves to virtual_bridge *(note the keywords master virtual_bridge)*.
If I am honest I was expecting GUI application to also create TAP device as well but it only created vnet0... Is this actually a TAP device?
How can I make connection two-way?
Trying as a normal user (without bridge):
I deleted virtual_bridge and virtual_tap so that everything was back to normal.
ziga#ziga-laptop:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c4:85:08:3c:1a:59 brd ff:ff:ff:ff:ff:ff
3: enx24f5a2f17b27: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 24:f5:a2:f1:7b:27 brd ff:ff:ff:ff:ff:ff
inet 192.168.64.100/24 brd 192.168.64.255 scope global enx24f5a2f17b27
valid_lft forever preferred_lft forever
inet6 fe80::26f5:a2ff:fef1:7b27/64 scope link
valid_lft forever preferred_lft forever
I noticed that if I start virt-manager with sudo and use qcow2 image that image will become owned by root and it will become part of group root. This was part of my problem why I avoided using virt-manager as a normal user. So I fixed this and started virt-manager as a normal user.
I created the identical virtual machine but when a network window pops up it had different (!) options:
Userspace networking
Specify shared device name
I was unable to specify my interface enx24f5a2f17b27 manualy with the second option so I chose a userspace networking.
Then I started the virtual machine which can browse the internet and also can the host machine. Both are assigned the IP which is totaly different. When I try to ping them ping doesn't work in any direction. Host, can't ping virtual machine and vice versa.
Trying as a normal user (with bridge)
So now I first set up my my bridge precisely like I did in my first attempt as a sudo user:
sudo ip link add virtual_bridge type bridge
sudo ip link set dev virtual_bridge up
sudo ip link set dev enx24f5a2f17b27 down
sudo ip addr flush dev enx24f5a2f17b27
sudo ip link set dev enx24f5a2f17b27 up
sudo ip link set dev enx24f5a2f17b27 master virtual_bridge
so that I have:
ziga#ziga-laptop:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: wlp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether c4:85:08:3c:1a:59 brd ff:ff:ff:ff:ff:ff
3: enx24f5a2f17b27: <BROADCAST,MULTICAST,DYNAMIC,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virtual_bridge state UP group default qlen 1000
link/ether 24:f5:a2:f1:7b:27 brd ff:ff:ff:ff:ff:ff
inet 192.168.64.100/24 brd 192.168.64.255 scope global enx24f5a2f17b27
valid_lft forever preferred_lft forever
inet6 fe80::26f5:a2ff:fef1:7b27/64 scope link
valid_lft forever preferred_lft forever
11: virtual_bridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 24:f5:a2:f1:7b:27 brd ff:ff:ff:ff:ff:ff
inet6 fe80::805f:cfff:feb6:ec91/64 scope link
valid_lft forever preferred_lft forever
I started the virt-manager as a normal user and created the identical virtual machine. When a network window pops up it has same options than before:
Userspace networking
Specify shared device name
I was unable to specify my bridge virtual_bridge manualy with the second option because Qemu reports an internal eror:
I am trying to understand how Octavia is put together. I created a loadbalancer on a vlan network. It was assigned an address of 10.40.0.7. When I do openstack loadbalancer list, I see a vip_address of 10.40.0.7 which is not assigned to any amphorae.
I want to understand where the loadbalancer address is mapped. It is not a host. I can't ssh to that address. Perhaps it is the amphora driver but what exactly is that? I can't see that address find it in any namespace. I can't see it assigned to any bridge. What is it assigned to?
Thanks
Ranga
It is not a host.
It is a host! An amphora is just a nova server -- the same thing you get when you run openstack server create. The difference is that the amphora is owned by the service project, so you'll only see it if you were to run (as admin) openstack server list --all-projects. For example:
$ openstack --os-cloud as_me loadbalancer list
+--------------------------------------+---------+----------------------------------+-------------+---------------------+----------+
| id | name | project_id | vip_address | provisioning_status | provider |
+--------------------------------------+---------+----------------------------------+-------------+---------------------+----------+
| 64a6a56d-beeb-4ee2-b495-1cdc26ffd399 | test_lb | 0ac1e30189da48b387cf3c2f5582b2a3 | 10.254.0.6 | ACTIVE | octavia |
+--------------------------------------+---------+----------------------------------+-------------+---------------------+----------+
$ openstack --os-cloud as_admin server list --all-projects | grep amphora
| f6cd75fe-8513-4aae-bee9-af9362525703 | amphora-50dddb41-decf-4b3b-bb7a-f35a751d74af | ACTIVE | lb-mgmt-net=172.24.0.16; test_lb_net=10.254.0.11; test_net1=10.0.1.5; test_net0=10.0.0.4 | octavia-amphora-13.0-20181107.1.x86_64 | octavia_65 |
If you look at that server, you'll see it has several ip addresses:
The one you assigned to it when created the loadbalancer, and
A management network address
Addresses on any subnets to which it is attached
You can ssh into the amphora using the management network address. You should be able to reach it from your controllers. You'll need the appropriate ssh key; where to find that probably depends a lot on how you installed things. I'm using tripleo, and it looks as if the install uses ~/.ssh/id_rsa from the stack user for the amphora ssh key.
[controller ~]$ ssh -i amphora_private_key cloud-user#172.24.0.7
Last login: Thu Nov 15 22:01:16 2018 from 172.24.0.6
[cloud-user#amphora-7d48e10b-5ba4-42c9-bcd5-941d224b2a46 ~]$
Update
The loadbalancer VIP is assigned to an interface inside a namespace on
the amphora. Given the above configuration, I see:
[root#amphora-50dddb41-decf-4b3b-bb7a-f35a751d74af ~]# ip netns
amphora-haproxy (id: 0)
[root#amphora-50dddb41-decf-4b3b-bb7a-f35a751d74af ~]# ip netns exec amphora-haproxy ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 1000
link/ether fa:16:3e:07:d2:26 brd ff:ff:ff:ff:ff:ff
inet 10.254.0.11/24 brd 10.254.0.255 scope global eth1
valid_lft forever preferred_lft forever
inet 10.254.0.6/24 brd 10.254.0.255 scope global secondary eth1:0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe07:d226/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 1000
link/ether fa:16:3e:21:9a:d1 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.4/24 brd 10.0.0.255 scope global eth2
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe21:9ad1/64 scope link
valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 1000
link/ether fa:16:3e:2a:63:58 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.5/24 brd 10.0.1.255 scope global eth3
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe2a:6358/64 scope link
valid_lft forever preferred_lft forever
I have my local Machine (10.0.0.2/16) directly connected to the eth4 network interface of my server.
The connection works as expected and I can traceroute the ip of eth4, namely 10.0.0.1.
However, I can also traceroute the ip 10.1.0.23 of the other interface (eth5), even though it is on the wrong subnet!
In the following you see the settings of my local machine and my server.
On my local Machine (Arch Linux)
Output of ip addr:
....
2: enp0s25: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 3c:97:0e:8a:a1:5a brd ff:ff:ff:ff:ff:ff
inet 10.0.0.2/16 brd 10.0.255.255 scope global enp0s25
valid_lft forever preferred_lft forever
inet6 fe80::7a0b:adb3:2eef:a3a8/64 scope link
valid_lft forever preferred_lft forever
....
Traceroutes
% sudo traceroute -I 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
1 10.0.0.1 (10.0.0.1) 0.184 ms 0.170 ms 0.163 ms
% sudo traceroute -I 10.1.0.23
traceroute to 10.1.0.23 (10.1.0.23), 30 hops max, 60 byte packets
1 10.1.0.23 (10.1.0.23) 0.240 ms 0.169 ms 0.166 ms
On Server (Debian)
My /etc/network/interfaces.
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
#iface eth5 inet dhcp
auto eth5
allow-hotplug eth5
iface eth5 inet static
address 10.1.0.23
netmask 255.255.0.0
gateway 10.1.0.1
## Automatically load eth4 interface at boot
auto eth4
allow-hotplug eth4
# Configure network interface at eth4
iface eth4 inet static
address 10.0.0.1
netmask 255.255.0.0
gateway 10.0.0.1
Output of ip addr:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
...
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:08:a2:0a:e8:86 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/16 brd 10.0.255.255 scope global eth4
valid_lft forever preferred_lft forever
inet6 fe80::208:a2ff:fe0a:e886/64 scope link
valid_lft forever preferred_lft forever
7: eth5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 00:08:a2:0a:e8:87 brd ff:ff:ff:ff:ff:ff
inet 10.1.0.23/16 brd 10.1.255.255 scope global eth5
valid_lft forever preferred_lft forever
Output of ip route:
default via 10.1.0.1 dev eth5
10.0.0.0/16 dev eth4 proto kernel scope link src 10.0.0.1
10.1.0.0/16 dev eth5 proto kernel scope link src 10.1.0.23
Why wouldn't you expect this behavior. As you can see from your Debian server's routing tables, it knows how to route packets to your arch linux machine, so it can respond if it wants to.
I can see two likely questions you might be having:
Why does it choose to respond?
You haven't given us your firewall rules, or told us whether your server has ip_forwarding enabled. Even without IP forwarding enabled, Linux will see a locally received packet for any of its local addresses as an INPUT packet (in terms of iptables and access control decisions), not a forwarded packet. So it will respond even if forwarding is disabled.
If you don't want this behavior you could add an iptables rule to the INPUT chain to drop the packet being received on the server.
Why is there only one hop in the traceroute
You might expect that in order to respond the packet would need to traverse (be forwarded) and so you would get two hops in your traceroute one for eth4 and one for eth5. However, as mentioned above, any locally received ppacket will be treated as input if it matches one of the local IPs. Your arch linux box presumably uses the Debian server as its default route. So, it sends a packet with the Debian server's MAC address, hoping the Debian server will forward it. That makes it a locally received packet at the ethernet level on the Debian serevr. The server then cehcks teh IP address, finds it is local, doesn't care it's for another ethernet and locally receives it at the IP layer.
If you don't want that behavior, fix in firewall rules.
When my server reboots the ip address for eth0 is 192.168.1.2 when it should be 192.168.1.100 per the static ip address settings in /etc/network/interfaces. After boot if I run service networking restart it will assigning 192.168.1.100 to eth0. Also I don't know if this matters but the hostname displayed in my router is different the the hostname displayed in /etc/hosts.
/etc/network/interfaces
auto lo eth0
iface lo inet loopback
# IPv4 address
auto eth0
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
broadcast 192.168.1.255
network 192.168.1.0
ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether a4:1f:72:7c:61:8d brd ff:ff:ff:ff:ff:ff
inet 192.168.1.100/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet 192.168.1.2/24 brd 192.168.1.255 scope global secondary dynamic eth0
valid_lft 85312sec preferred_lft 85312sec
ip route show
10.8.0.0/24 dev tun0 proto kernel scope link src 10.8.0.1
169.254.0.0/16 dev eth0 scope link metric 1000
192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.100
192.168.1.1 dev eth0 proto dhcp scope link src 192.168.1.2 metric 1024
I don't know about 16.04 but in previous versions you have Network Manager daemon setting the IPs. Use the applet 'nm-applet' to set up your static address. Right click it and go for 'edit connections'.
https://help.ubuntu.com/community/NetworkManager
I fixed part of the problem it was as simple as changing the file to the parameters outline below.
/etc/network/interfaces
# IPv4 address
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8
Now when the server boots up it will auto assigning 192.168.1.100 although it will also still assigning 192.168.1.2. If I find a way to have it stop assigning the 2nd ip address I will update my answer. Thanks