Openstack Cirros instance cannot ping or resolve Internet hosts - openstack

Question

I have deployed RDO Openstack Xena on Virtual Box. There were no errors in the installation. Created external network another network named blue and attached it to a router. I defined 8.8.8.8 as DNS. Everything looks fine but when I create Cirros instance, this instance cannot ping outside the Internet. Floating IP has been defined. The second Cirros instance has the same issue as well.
Any help is much appreciated.

Answer 1

Most probably, you need to revise the Security Group assigned to the instance.
Ensure you have an Egress rule like this.
If two cirros instances can ping the internal ip (not the floating) one another but can not ping the public (floating) ip, then the problem is that you need a router between the internal network and the public network.

Answer 2

happy you brought this issue to see the light of day.
This is something that still intrigues me to date.
My OpenStack proof of concept is an OSA openstack ansible using three nodes a infra, compute and a storage all VM's on top of my proxmox as per OSA deployment documentation.
The deployment goes on without a hitch, a bit lengthy though must admint ~ 1 to 2 hours give or take.
I created a public (external) network using a provider flat that using the same nic assigned to get an external ip from my isp router
Then to cut it short, I assinged the new cirros instance this same external network that has a limited dhcp from a range not conflicting with my main router's dhcp to avoid messing up.
I get my cirros successfully deployed and I can even ssh it from my external network all works fine but I can't somehow make it connect to the external world, kind of DNS is lost somehow.
root#infra2:~# ssh cirros#10.171.101.28 The authenticity of host '10.171.101.28 (10.171.101.28)' can't be established. ECDSA key fingerprint is SHA256:IGTpW0rXV44lIMJVmT+hRyxUqTuj0DZU8rqMe2Te3rU. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '10.171.101.28' (ECDSA) to the list of known hosts. cirros#10.171.101.28's password: $ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:01:dc:8d brd ff:ff:ff:ff:ff:ff
inet 10.171.101.28/24 brd 10.171.101.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe01:dc8d/64 scope link
valid_lft forever preferred_lft forever $ netstat -rn Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.171.101.1 0.0.0.0 UG 0 0 0 eth0
10.171.101.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.169.254 10.171.101.10 255.255.255.255 UGH 0 0 0 eth0 $ nslookup www.google.com ;; connection timed out; no servers could be reached
And yes my cirros instance that got an ip from the same subnet as my router isp can ping my router main gateway.
$ ping 10.171.101.1
PING 10.171.101.1 (10.171.101.1): 56 data bytes
64 bytes from 10.171.101.1: seq=0 ttl=64 time=1.601 ms
64 bytes from 10.171.101.1: seq=1 ttl=64 time=0.748 ms
64 bytes from 10.171.101.1: seq=2 ttl=64 time=0.869 ms
64 bytes from 10.171.101.1: seq=3 ttl=64 time=1.549 ms
64 bytes from 10.171.101.1: seq=4 ttl=64 time=0.953 ms
Os network topology
And I tweaked the neutron network agent to make sure the provider flat is using the same nic the node is receiving an ip from my router isp.
root#infra1-utility-container-f9cbd806:~# openstack network agent list
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+
| 33455545-6dc5-4c08-b169-71f13aa3abb1 | L3 agent | infra2 | nova | :-) | UP | neutron-l3-agent |
| 6cdb7d41-f14e-49c4-b302-4878d82ff9cc | Metering agent | infra2 | None | :-) | UP | neutron-metering-agent |
| 78dfafac-d347-48b6-a0e1-83b55890f989 | DHCP agent | infra2 | nova | :-) | UP | neutron-dhcp-agent |
| d1ca6f9c-aee8-4ab3-8b51-942c8c9df05b | Linux bridge agent | infra2 | None | :-) | UP | neutron-linuxbridge-agent |
| e912fbc7-f066-4fc7-838c-77013ad30239 | Metadata agent | infra2 | None | :-) | UP | neutron-metadata-agent |
| ed12f36e-8f59-43b0-9786-b6f5d77880b0 | Linux bridge agent | compute2 | None | :-) | UP | neutron-linuxbridge-agent |
+--------------------------------------+--------------------+----------+-------------------+-------+-------+---------------------------+
root#infra1-utility-container-f9cbd806:~# openstack network agent show d1ca6f9c-aee8-4ab3-8b51-942c8c9df05b
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| agent_type | Linux bridge agent |
| alive | :-) |
| availability_zone | None |
| binary | neutron-linuxbridge-agent |
| configuration | {'bridge_mappings': {}, 'devices': 2, 'extensions': [], 'interface_mappings': {'vlan': 'br-vlan', 'physnet1': 'ens18'}, 'l2_population': False, 'tunnel_types': ['vxlan'], 'tunneling_ip': '172.29.240.11'} |
| created_at | 2022-06-30 21:31:57 |
| description | None |
| ha_state | None |
| host | infra2 |
| id | d1ca6f9c-aee8-4ab3-8b51-942c8c9df05b |
| last_heartbeat_at | 2022-07-01 23:34:39 |
| name | None |
| resources_synced | None |
| started_at | 2022-07-01 22:34:40 |
| topic | N/A |
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
root#infra1-utility-container-f9cbd806:~# openstack network agent show ed12f36e-8f59-43b0-9786-b6f5d77880b0
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| agent_type | Linux bridge agent |
| alive | :-) |
| availability_zone | None |
| binary | neutron-linuxbridge-agent |
| configuration | {'bridge_mappings': {}, 'devices': 1, 'extensions': [], 'interface_mappings': {'vlan': 'br-vlan', 'physnet1': 'ens18'}, 'l2_population': False, 'tunnel_types': ['vxlan'], 'tunneling_ip': '172.29.240.12'} |
| created_at | 2022-06-30 21:33:58 |
| description | None |
| ha_state | None |
| host | compute2 |
| id | ed12f36e-8f59-43b0-9786-b6f5d77880b0 |
| last_heartbeat_at | 2022-07-01 23:35:19 |
| name | None |
| resources_synced | None |
| started_at | 2022-07-01 22:35:19 |
| topic | N/A |
+-------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
And depite being really close to getting it work the cirros instance still can't get to my routers dns somehow to resolve names.
$ nslookup www.google.com
;; connection timed out; no servers could be reached
Anyone that had this issue solved please chime in, really appreciated to hear from you how you solved this!
As I zeron on in this issue it seems to be related to the linux-bridge somehow.
As we can see here the my os router linux-bridge took over my ens18 physical network with brq270399cc-48
root#infra2:~# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.171.101.1 0.0.0.0 UG 0 0 0 brq270399cc-48
10.0.3.0 0.0.0.0 255.255.255.0 U 0 0 0 lxcbr0
10.171.101.0 0.0.0.0 255.255.255.0 U 0 0 0 brq270399cc-48
172.29.236.0 0.0.0.0 255.255.252.0 U 0 0 0 br-mgmt
172.29.240.0 0.0.0.0 255.255.252.0 U 0 0 0 br-vxlan
As my from my linuxbridge namespace the dns also doesn't work.
root#infra2:~# ip netns list
qrouter-1d2b49cf-87db-4a13-8dce-ab29817974a7 (id: 17)
qdhcp-270399cc-4830-45a7-97df-e9e0c0929706 (id: 16)
root#infra2:~# ip netns exec qrouter-1d2b49cf-87db-4a13-8dce-ab29817974a7 ip route s
default via 10.171.101.1 dev qg-f9fd9de9-98 proto static
10.171.101.0/24 dev qg-f9fd9de9-98 proto kernel scope link src 10.171.101.22
root#infra2:~# ip netns exec qrouter-1d2b49cf-87db-4a13-8dce-ab29817974a7 dig www.google.com +short
;; communications error to 127.0.0.53#53: connection refused
And it is a sure thing a DNS issue cause I can ping google's public ip address from my cirros intance
$ ping 142.251.32.68
PING 142.251.32.68 (142.251.32.68): 56 data bytes
64 bytes from 142.251.32.68: seq=0 ttl=116 time=11.621 ms
64 bytes from 142.251.32.68: seq=1 ttl=116 time=11.090 ms
64 bytes from 142.251.32.68: seq=2 ttl=116 time=25.061 ms
Tried to change the /etc/resolve.conf but still no luck
$ cat /etc/resolv.conf
search openstacklocal
nameserver 8.8.8.8
#nameserver 10.171.101.1