GCE + OpenVPN + subnetwork does not work the routing - networking

I want to create a Private network in google compute platform where I will be able to enter only using a vpn.
So, I create a machine in GCE and I install openvpn. This machine has an static IP, the ssh port open and the default network configuration from GCE.
Then, I create a second machine (call it MachineA) , in the same network, but without external IP.
Then I create the route rule in order to redirect traffic from vpn-machine to another internal instances.
I'm able to connect from my machine to the vpn.
I'm able to ping to vpn machine.
I'm able to ping to MachineA.
I'm able to ssh to vpn machine.
I'm able to ssh to MachineA.
but...
When I connect to ssh vpn machine and run gsutil it works, also ping to 8.8.8.8
When I connect to ssh MachineA and run gstult or ping 8.8.8.8 does not work.
Any Idea what Im doing wrong ?
Some information
from VPN-machine
xxx#dev-vpn:~$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.240.10.1 0.0.0.0 UG 0 0 0 eth0
10.16.0.0 10.16.0.2 255.255.255.0 UG 0 0 0 tun0
10.16.0.2 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
10.240.10.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
xxx#dev-vpn:~$ traceroute 10.240.10.3
traceroute to 10.240.10.3 (10.240.10.3), 30 hops max, 60 byte packets
1 * * instance-1.c.project.internal (10.240.10.3) 1.188 ms
from MachineA
traceroute to 10.240.10.2 (10.240.10.2), 30 hops max, 60 byte packets
1 * * *
2 * * dev-vpn.c.project.internal (10.240.10.2) 0.899 ms
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.240.10.1 0.0.0.0 UG 0 0 0 eth0
10.240.10.1 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
In google networking I add this rule
vpn-routing 10.16.0.0/24 1001 None dev-vpn (Zone us-central1-a)

I have a working system that does this. The innovation I made was to add an alias IP range and make the OpenVPN server use that Google IP range.
So for a router instance, that has an external IP address and will run the OpenVPN server on GCE, you need to create the instance with just one interface and a small IP Alias range. This would then permit IP forwarding from the main interface to the small Alias range. Let's say you are on default network 10.156.0.10/20 and you have added an Alias range of 10.156.1.0/28, you then add as the server line in your OpenVPN server configuration: server 10.156.1.1.
So the tun0 interface of OpenVPN (server-side) will come up on 10.156.1.1 and the tunnel endpoint on 10.156.1.2.
You have to push the routes to the OpenVPN clients (so push 10.156.0.0/20 in the server configuration). You will also need iroute statements in the server's ccd/client.
Here's an excerpt from the OpenVPN server's configuration file:
server 10.156.1.0 255.255.255.240
push "route 10.156.0.0 255.255.240.0"
push "route 10.164.0.0 255.255.240.0"
push "route 10.132.0.0 255.255.240.0"
route 192.168.127.0 255.255.255.0
If your site network is 192.168.127.0/24 and you use 3 Google networks. The ccd/client file has this
route 192.168.127.0 255.255.255.0
iroute 192.168.127.0 255.255.255.0
And you may need to add others, if you have other routes. (There's a section in the OpenVPN manuals about ccd/ and iroute.)
Back on the Google cloud, you will need to add a Google route via the OpenVPN gateway on 10.156.0.10 to get back to 192.168.127.0/24.
And there's lots of firewalling that you should do to make your hosts safe, but you must at least open the 1194 port for OpenVPN.
On your site, if you want to access the Google Cloud private networks, you will need to use RIPd from Quagga. That's a relatively easy configuration:
router rip
network eth0
passive-interface tun0
no default-information originate
redistribute kernel route-map GMAP
access-list GCE permit 10.156.0.0/20
access-list GCE permit 10.164.0.0/20
access-list GCE permit 10.132.0.0/20
access-list GCE deny any
route-map GMAP permit 10
match ip address GCE
This is the RIPd configuration for the OpenVPN client gateway, which is a router. This configuration propagates any routes for the three Google Cloud networks 10.{156,164,132}.0.0/20.
The RIPd configuration on the other hosts in your site network doesn't require any special configuration, just name the "router rip" and the "network eth0", your host's network interface and start RIPd. (The configuration on the VPN client gateway should be easier than this, but I found that "no default-information originate" didn't work for me, so I had to just propagate the Google routes.)

Related

Can an OpenVPN Route over TEST-NET-1 (RFC 5735)

Background
I have a strange use-case where my VPN cannot be on any of the private subnets, but, also cannot use a TAP interface. The machine will be moving through different subnets, and requires access to the entire private address space by design. A single blocked IP would be considered a failure of design.
So, these are all off limits:
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
169.254.0.0/16
In searching for a solution, I came across RFC 5735, which defines:
192.0.2.0/24 TEST-NET-1
198.51.100.0/24 TEST-NET-2
203.0.113.0/24 TEST-NET-3
As:
For use in documentation and example code. It is often used in conjunction with domain names
example.com or example.net in vendor and protocol documentation. As described in [RFC5737], addresses within this block do not legitimately appear on the public Internet and can be used without any coordination with IANA or an Internet registry.
Which, was a "Jackpot" moment for me and my use case.
Config
I configured an OpenVPN server as such:
local 0.0.0.0
port 443
proto tcp
dev tun
topology subnet
server 203.0.113.0 255.255.255.0 # TEST-NET-3 RFC 5735
push "route 203.0.113.0 255.255.255.0"
...[Snip]...
With Client:
client
nobind
dev tun
proto tcp
...[Snip]...
And ufw rules:
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 203.0.113.0/24 -o ens160 -j MASQUERADE
COMMIT
However, upon running I get /sbin/ip route add 203.0.113.0/24 via 203.0.113.1 RTNETLINK answers: File exists in the error logs. While the VPN completes the rest of its connection successfully.
No connection
Running the following commands:
Server: sudo python3 -m http.server 80
Client: curl -X GET / 203.0.113.1
Results in:
curl: (28) Failed to connect to 203.0.113.1 port 80: Connection timed out
I have tried:
/sbin/ip route replace 203.0.113.0/24 dev tun 0 on client and server.
/sbin/ip route change 203.0.113.0/24 dev tun 0 on client and server.
Adding route 203.0.113.0 255.255.255.0 to the server.
Adding push "route 203.0.113.0 255.255.255.0 127.0.0.1" to server
And none of it seems to work.
Does anyone have any idea how I can force the client to push this traffic over the VPN to my server, instead of to the public IP?
This does actually work!
Just dont forget to allow connections within your firewall. I fixed my config with:
sudo ufw allow in on tun0
However, 198.18.0.0/15 and 100.64.0.0/10 defined as Benchmarking and Shared address space respectively, may be more appropriate choices, since being able to forward TEST-NET addresses may be considered a bug.

Configure 2 NIC on the same subnet

For debugging purpose I need to set up a MITM proxy between 2 devices.
All devices have static IP (example) and are directly connected each other:
Device 1 : 192.168.10.50
Device 2 : 192.168.10.60
Proxy computer have 2 nic on the same subnet and is between other devices :
D1 .50 <=> [.60 PROXY .50] <=> D2 .60
My problem is, from the proxy if one of the 2 nic is disabled, D1 or D2 can reach the proxy.
Once I bring up the 2 nic, no one can see any other device.
D1 and D2 ip can't be changed.
Proxy is linux centos 8.
Already tested :
adding log on iptables : if 2 nic are up no more input / output log
Drop all input / output then accept only right ip from right interface => no result
Changing arp_filter = 1 and arp_announce = 2 => no result
Testing multiple SO post about 2 nic on same subnet
If anyone can help.
Thanks .
I've found a solution which consist in creating a transparent proxy and intercept some packets.
1- Create a bridge with the 2 NIC :
nmcli connection add type bridge autoconnect yes con-name "br0" ifname "br0"
nmcli connection modify "br0" ipv4.addresses "192.168.10.10/24" ipv4.method manual
nmcli connection delete enp0s3
nmcli connection delete enp0s8
nmcli connection add type bridge-slave autoconnect yes con-name enp0s3 ifname enp0s3 master br0
nmcli connection add type bridge-slave autoconnect yes con-name enp0s8 ifname enp0s8 master br0
2 Add correct rules to intercept specific traffic
nft add table bridge mitm
nft add chain bridge mitm filter { type filter hook prerouting priority 0\; }
nft add rule bridge mitm filter tcp dport 10000 ip saddr 192.168.10.50 meta pkttype set host ether daddr set xx:xx:xx:xx:xx:xx # br0 mac address
nft add rule ip nat PREROUTING tcp dport 10000 ip saddr 192.168.10.50 dnat to 192.168.10.10
It worked for me.

docker-machine: No route to host to default docker machine (using host-only vbox network) on some wifi networks

I'm running docker-machine on OS X and it works great on my home wifi network (wifi-one).
On another wifi network (wifi-two), I cannot access the default machine, either through docker-machine or with pings. I've tried removing all the docker and virtual box software and caches and reinstalling while connected wifi-two, to no avail.
docker-machine uses a small vm in virtual box that uses a host-only vbox network (vboxnet0) to communicate between the host and vm.
On wifi-one, I can ping the gateway to the vboxnet0 (192.168.99.1), and I can ping the vm (192.168.99.100). In the routing table, I can see routes to the vm using the right interface:
Internet:
Destination Gateway Flags Refs Use Netif Expire
default dlinkrouter UGSc 60 2 en0
...
dlinkrouter c0:a0:bb:xx:xx:xx UHLWIir 66 155 en0 1180
192.168.99 link#10 UC 2 0 vboxnet
192.168.99.100 8:0:27:db:16:21 UHLWIi 2 42 vboxnet 1109
...
On wifi-two, I can ping the vboxnet0 gateway, but not the vm. Ping reports request timeout, but docker-machine reports "no route to host". The routing table, while connected to wifi-two, shows the route to the vboxnet0 host-only network on the wifi interface (en0), not the vboxnet0 interface. Also curious is the default route is the route to the vboxnet0 gateway, but no sign of the wifi router in the routing table. Despite this, internet connectivity works fine on wifi-two.
Internet:
Destination Gateway Flags Refs Use Netif Expire
default 192.168.99.1 UGSc 46 0 en0
...
192.168.99 link#4 UCS 9 0 en0
192.168.99.1/32 link#4 UCS 2 0 en0
192.168.99.1 dc:9f:db:2b:89:39 UHLWIir 46 61 en0 1197
192.168.99.100 link#4 UHLWIi 1 3 en0
So, I don't really know where the problem is here - the router, mac os, virtual box or docker-machine. I suspect there is something funny about the configuration of the router, but I don't see why the wifi router would have anything to do with the routes to a virtualbox host-only network.
edit: manually deleting the en0 route to 192.168.99 via en0 and running sudo route -nv add -net 192.168.99 -interface vboxnet0 does seem to resolve this on wifi-two.
But I'd like to understand why this is happening - why there is no route to vboxnet0 automatically when I join wifi-two and I'd like to find a better solution than running route add when I'm on wifi-two.
I figured it out. it's kinda stupid:
wifitwo's subnet is 192.168.99. The same subnet docker uses for the host-only network it creates for the default VM. So the conflict was causing all sorts of cryptic errors.

Hitting resources in a private network from within a Docker container using VPN

I'm running Docker 1.9.1 on OSX, and I'm connected to my private work network with Cisco AnyConnect VPN. A service that I'm running in a Docker container connects to a DB within the work network, and is unreachable from within the container, but reachable from outside the container in OSX. It's also reachable from within the container if I'm connected directly to the work network, not through VPN. I suspect I may have to do some network configuration with the docker-machine VM, but I'm not sure where to go from here.
If you are using Virtualbox as your hypervisor for the docker-machines, I suggest you set your network mode as Bridged Adapter. This way your VM will be connected to the network individually just like your own machine. Also to gather more information for troubleshooting try pinging the db host machine from the container machine command line. use docker exec -it <container-name> /bin/bash
I ran into this problem today and got AnyConnect to work without the need for split tunneling or a different VPN client like OpenConnect. All it took was a bit of port forwarding.
My Setup
MacOS Sierra 10.12
VirtualBox 5.0.26
Docker ToolBox 1.12.2
docker-vpn-helper script located at https://gist.github.com/philpodlevsky/040b44b2f8cee750ecc308271cb8d1ab
Instructions
The above software configuration was utilized when tested.
Make sure you don't have any VMs running and you are disconnected from the VPN.
Modify line 47 to either specify your insecure registry or delete the "--engine-insecure-registry :5000" parameter.
Execute the following in a shell on your Mac:
sudo launchctl unload /System/Library/LaunchDaemons/org.ntp.ntpd.plist
Workaround for MacOS Sierra. For some reason having NTP enabled causes the docker engine to hang. See:
https://forums.docker.com/t/docker-beta-for-mac-does-not-work-and-hangs-frequently-on-macos-10-12/18109/7
./docker-vpn-helper
Sets up the port forwarding, regenerates TLS certificates.
Pay attention to the following lines emitted by the script you will need to cut and paste them into your shell.
export DOCKER_HOST=tcp://localhost:2376
export DOCKER_CERT_PATH=/Users/<username>/.docker/machine/machines/default
export DOCKER_MACHINE_NAME=default
Connect to your AnyConnect VPN and test out docker:
docker run hello-world
Check your routing inside the Docker Machine VM with
docker-machine ssh default
$ route -n
which looks like this on a fresh machine:
docker#default:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.2.2 0.0.0.0 UG 1 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 lo
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
192.168.99.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
If you've created a lot of networks, i.e. by using docker-compose it might have created routes to stacks, which conflict with your VPN or local network routes.
docker#dev15:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.2.2 0.0.0.0 UG 1 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.1 0.0.0.0 255.255.255.255 UH 0 0 0 lo
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-7400365dbd39
172.25.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-4db568a601b4
[...]
192.168.80.0 0.0.0.0 255.255.240.0 U 0 0 0 br-97690a1b4313
192.168.105.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
TL;dr
It should be safe to remove all networks, with
docker network rm $(docker network ls -q)
since active networks are not removed by default ... but nonetheless be careful when running rm commands :)

openstack instance getting ip and not getting ip

I am new to openstack and I followed the installation guide of icehouse for ubuntu 12.04/14.04
I chose 3 node architecture. Controller, Nova, Neutron.
The 3 nodes are installed in VM's. I used nested KVM. Inside VM's kvm is supported so nova will use virt_type=kvm. In controller I created 2 nics. eth0 is a NAT interface with ip 203.0.113.94 and eth1 a host only interface with ip 10.0.0.11.
In nova there are 3 nics. eth0 NAT - 203.0.113.23, eth1 host only 10.0.0.31 and eth2 another host only 10.0.1.31
In neutron 3 nics. eth0 NAT 203.0.113.234, eth1 host only 10.0.0.21 and eth2 another hosty only 10.0.1.21 (during installation guide in neutron node i created a br-ex (and a port to eth0) which took the settings of eth0 and eth0 settings are:
auto eth0 iface eth0 inet manual up ifconfig $IFACE 0.0.0.0 up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down)
Everything seemed fine. I can create networks, routers etc, boot instances but I have this error.
When I launch an instance it takes a fixed ip but when I log in into instance (cirros) can't ping anything. ifconfig with no ip.
I noticed that in demo-net (tenant network) properties under subnet in the ports field it has 3 ports. 172.16.1.1 network:router_interface active 172.16.1.3 network:dhcp active 172.16.1.6 compute:nova down
I searched for solutions over the net but couldn't find anything!
Any help?
Ask me if you want specific logs because I don't know which ones to post!
Thanks anyway!
Looks like you are using Fixed IP to ping..If so please assign floating IP to your instance, and then try to ping..
If you have already assigned floating IP and you are pinging using that IP..please upload log of your instance

Resources