We run K8S cluster in private network behind firewall with nasty reverse NAT rules that prevent our hosts to connect to themselves over public IPs.
Problem
From inside a pod I need to connect to the service that is resolved to public IP 203.X.X.X that is not possible. However I can connect to this service over private IP 192.Y.Y.Y
I can setup iptables rule on the node running pod as follows:
iptables -A OUTPUT -t nat -p tcp -d 203.X.X.X --dport 80 -j DNAT --to-destination 192.Y.Y.Y:80
Then I can successfully connect to the service from node, but not from pod running on this node.
Solution with hostAliases would not work as we:
do not know the host names in advance
would like to reroute all traffic to known public (unreachable) IPs to the respective private IPs
PS:
We use calico as networking plugin
Related
How do I allow a Cloud Composer Airflow DAG to connect to a REST API via VPN gateway? The cluster is connected to the according VPC.
The kube-proxy is able to reach the API, yet the containers can not.
I have SSH'd into the kube-proxy and containers and tried a traceroute. The containers' traceroute ends with the kube-proxy. The kube-proxy has 4 hops before reaching destination.
I have dumped the iptables from the kube-proxy, they do not specify anything in regards to NATing the VPCs subnet with the containers.
The VPC subnet also does not show up in the containers, which is by design.
Some reading material:
https://www.stackrox.com/post/2020/01/kubernetes-networking-demystified/
EDIT1: More info:
Let's assume the VPN connects the VPC to the remote 10.200.0.0 network.
The VPC has multiple subnets. The primary range is e.g. 10.10.0.0/20. For each Kubernetes cluster we have two more subnet, one for each pod (10.16.0.0/14) and another for services (10.20.0.0/20). The gateway is 10.10.0.1.
Each pod again has it's own range, where pod_1 is 10.16.0.0/14, pod_2 is 10.16.1.0/14, pod_3 10.16.3.0/14 and so on.
One of the kube-proxies has multiple network adapters. It resides in the 10.10.0.0 network with eth0 and has a cbr0 bridge to 10.16.0.0. Through said kube-proxy via the bridge the workers for Airflow are connecting to the network. One worker e.g. 10.16.0.1 has only one network adapter.
The kube-proxy can reach the 10.200.0.0 network. The Airflow workers can not.
How do we get the worker to access the 10.200.0.0 network? Do we need to change the iptables of the kube-proxy?
One of the possible solutions would be to forward the packages from the kube virtual interface to the node's real one. E.g. adding the following rules to ip tables:
iptables -A FORWARD -i cbr0 -o eth0 -d 10.200.0.0/25 -j ACCEPT
iptables -A FORWARD -i eth0 -o cbr0 -s 10.200.0.0/25 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
I am trying to setup a public squid proxy that routes it's traffic via a VPN server elsewhere in the world. It's running inside a docker container on a VPS host.
Using the default settings with push gateway, I can access the squidproxy on the VPS itself and it does route it's traffic via the vpn.
However, no external IPs can access the squid proxy.
I do have docker forwarding the port 3128:3128.
It is something to do with the OpenVPN routes that are created (as the Squid proxy is accessible until OpenVPN starts)
I found it is this route that seems to "block" my external traffic.
128.0.0.0/1 via 10.91.10.5 dev tun0
(10.91.10.5 is the gateway of the VPN)
If I remove it I can access squid again but then outgoing requests don't use the VPN.
I can make my external IP work by explicitly adding it like so
ip route add 203.X.X.X via 172.18.0.1 dev eth0
(172.18.0.1 is the docker gateway)
But I need it to work with any external IPs.
I have tried ip route add 0.0.0.0 via 172.18.0.1 dev eth0.
But this doesn't work as 128.0.0.0/1 is more specific so matches first.
In conclusion
1) Need any IP to access the SquidProxy (port 3128)
2) Need all outgoing SquidProxy requests (80,443) to go via the VPN
Any help would be greatly appreciated!
UPDATE:
So I have this working
1) Start OpenVPN with the below command
openvpn --route-nopull --script-security 2 --up /etc/openvpn/up.sh
This disables it from setting up the VPN routes. So all traffic in and out is using the default route not via VPN
2) In the up.sh, I run the below commands
#!/bin/sh
/sbin/ip route add 0.0.0.0/0 dev $1 table 100
/sbin/ip rule add from all fwmark 1 table 100
/sbin/iptables -A OUTPUT -t mangle -p tcp -m multiport --dports 80,443 -j MARK --set-mark 1
/sbin/iptables -t nat -A POSTROUTING -o $1 -j MASQUERADE
I have then setup Squid to only allow ports 80 & 443. Docker has port 3128 open for access to the container.
I also needed to use --sysctl net.ipv4.conf.all.rp_filter=0 in the docker run command.
It is common to access a container from out of host with simple iptable rules which forward a port on host to a port on container with container internal IP in hand
-A PREROUTING -d 1.2.3.4/32 -i eth0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.0.4.10:80
So public IP 1.2.3.4 will routed to the container. with a proper dns record one can point a FQDN (eg: test.example.com) to this container also. and all things is working good til this point.
BUT in the host or in neighbor containers on same host, the FQDN is useless and not routed to that container.
Currently the dirty way is to point the FQDN to container private IP in /etc/hosts for host and all neighbor containers. but I think there is cleaner way to do that.
Can any one help?
UPDATE 1:
after alot of try and error, got this for host only. by setting this beside above rule it routes packets from host to that FQDN to container
iptables -A OUTPUT -d 1.2.3.4/32 -j DNAT --to-destination 10.0.4.10 -t nat
but nieghbor containers still cannot use that FQDN.
I don't know if the question is related to the topics on stackoverflow so, if not sorry in advance.
I have the following problem that I'm trying to solve :
I am connected to my university network that requires me to configure my browser to use a proxy. I want to set up a router in my house ( D-Link Dir-300 with Firmware: DD-WRT v24-sp2) and because I am using multiple Android devices that are not rooted ( I can't use global proxy apps ) I was thinking about the following solution :
Question Can I make my router redirect all the traffic from my wireless network to the proxy server that I'm using ( allowing the devices not to configure a proxy server) ?
I have a HTTP redirect tool available on the router configuration page
I looked up the destination IP address from the proxy URL that I was using and I get 4 IP addresses for that node, I tried to use two of them with the source network set to 192.168.1.0 but it didn't work -my browser on the android devices doesn't work without setting up the proxy.
____________________________________ EDIT ____________________________
After looking some more I found this link
#!/bin/sh
PROXY_IP=[IP]
PROXY_PORT=[PORT]
LAN_IP=`nvram get lan_ipaddr`
LAN_NET=$LAN_IP/`nvram get lan_netmask`
iptables -t nat -A PREROUTING -i br0 -s $LAN_NET -d ! $LAN_IP -p tcp --dport 80 -j DNAT --to $PROXY_IP:$PROXY_PORT
iptables -t nat -A POSTROUTING -o br0 -s $PROXY_IP -p tcp -d $LAN_NET -j SNAT --to $PROXY_IP
iptables -A FORWARD -i vlan1 -o br0 -s $LAN_NET -d $PROXY_IP -p tcp --dport $PROXY_PORT -j ACCEPT
How can I modify it to redirect all the traffic udp/tcp etc coming from all the ports?
Easy option is to configure your proxy on your android device. When setting up the wireless connection select advanced options and enter your proxy settings, on older android devices it's somewhat hidden but I managed to do it on a 2.3 one.
For iOS you can add it later to your wifi settings.
If you router is not the main entry point for internet (if it's secondary) you can configure it to route to your PC/laptop. Configure a static IP op your PC/laptop, share internet over there, and configure that IP as gateway on your router.
You can install a proxy tool (charles proxy for example) on your PC and configure it to forward your traffic to the internet-proxy. On your devices you still need to setup the proxy, but this time it's your PC's proxy (gateway+8080).
I have X-Wrt based on OpenWrt 8.09 on my router
I have home LAN of few computers on which I have some network servers (SVN, web, etc). For each of service I made forwarding on my router (Linksys wrt54gl) to access it from the Internet (<my_external_ip>:<external_port> -> <some_internal_ip>:<internal_port>)
But within my local network this resources by above request is unreachable (so I need make some reconfiguration <some_internal_ip>:<internal_port> to access).
I added some line to my /etc/hosts
<my_external_ip> localhost
So now all requests from local network to <my_external_ip> forwards to my router but further redirection to appropriate port not works.
Advise proper redirection please.
You need to install an IP redirect for calls going out of the internal network and directed to the public IP. Normally these packets get discarded. You want to reroute them, DNATting to the destination server, but also masqueraded so that the server, seeing as you, its client, are in its same network, doesn't respond directly to you with its internal IP (which you, the client, not having sent the packet there, would discard).
I found this on OpenWRT groups:
iptables -t nat -A prerouting_rule -d YOURPUBLICIP -p tcp --dport PORT -j DNAT --to YOURSERVER
iptables -A forwarding_rule -p tcp --dport PORT -d YOURSERVER -j ACCEPT
iptables -t nat -A postrouting_rule -s YOURNETWORK -p tcp --dport PORT -d YOURSERVER -j MASQUERADE
https://forum.openwrt.org/viewtopic.php?id=4030
If I remember correctly OpenWrt allows you to define custom DNS entries. So maybe simply give a proper local names to your sources (ie. svnserver.local) and map them to specific local IPs. This way you do not even need to go through router to access local resources from local network.