Ping failed to second ip in openstack instance - openstack

Question

I have RDO openstack environment in a machine for testing. The RDO was installed with packstack --allinone command. Using HOT I have created two instances. One with cirros image and another with Fedora. The Fedora instance have two interfaces that are connected to same network while cirros have only one interface and connected to same network. The template looks like this -
heat_template_version: 2015-10-15
description: Simple template to deploy two compute instances
resources:
local_net:
type: OS::Neutron::Net
local_signalling_subnet:
type: OS::Neutron::Subnet
properties:
network_id: { get_resource: local_net }
cidr: "50.0.0.0/24"
ip_version: 4
fed:
type: OS::Nova::Server
properties:
image: fedora
flavor: m1.small
key_name: heat_key
networks:
- network: local_net
networks:
- port: { get_resource: fed_port1 }
- port: { get_resource: fed_port2 }
fed_port1:
type: OS::Neutron::Port
properties:
network_id: { get_resource: local_net }
fed_port2:
type: OS::Neutron::Port
properties:
network_id: { get_resource: local_net }
cirr:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.tiny
key_name: heat_key
networks:
- network: local_net
networks:
- port: { get_resource: cirr_port }
cirr_port:
type: OS::Neutron::Port
properties:
network_id: { get_resource: local_net }
The Fedora instance got two ips (50.0.0.3 and 50.0.0.4). Cirros got ip 50.0.0.5. I can ping 50.0.0.3 from cirros instance but not the ip 50.0.0.4. If I manually down the interface with ip 50.0.0.3 in the Fedora instance, then only I can ping 50.0.0.4 from cirros instance. Is there a restriction in the configuration of neutron that prohibits ping to both the ips of Fedora instance at same time. Please help.

Answer 1

This happens because of the default firewall-ing done by OpenStack networking (neutron) -- it simply drops any packets received on a port if the source address of the packet does not match the IP address assigned to the port.
When cirros instance sends ping packet to 50.0.0.4, fedora instance receives it on the interface with IP address 50.0.0.4. However, when it is responding back to cirros's IP address 50.0.0.5, the linux networking stack on your fedora machine has two interfaces to choose from to send out the response (because both those interfaces are connected to the same network). In your case, fedora choose to respond back on on 50.0.0.3. However, the source IP address in the packet is still 50.0.0.4, and thus the OpenStack networking layer simply drops it.
General recommendation is to not have multiple interfaces on the same network. If you want multiple IP addresses from the same network for your VM, you can use "fixed_ips" option in your heat template:
fed_port1:
type: OS::Neutron::Port
properties:
network_id: { get_resource: local_net }
fixed_ips:
- ip_address: "50.0.0.4"
- ip_address: "50.0.0.3"
Since DHCP server would offer only IP address, fedora would be configured with only one IP. You can add another IP to your interface using "ip addr add" command (see http://www.unixwerk.eu/linux/redhat/ipalias.html):
ip addr add 50.0.0.3/24 brd + dev eth0 label eth0:0