Everything network related seems to be down Ubuntu 16.04 - networking

Yesterday I was developing a dashboard on my server using PHP and MySQL. I came in today to finish said dashboard and am met with the following output:
git clone https://github.com/tbg/tcd
Cloning into 'tcd'...
fatal: unable to access 'https://github.com/tbg/tcd/': Could not resolve host: github.com
Looking deeper into this issue I cam across the following issues as well with apt-get:
apt-get update
Err:1 http://ppa.launchpad.net/certbot/certbot/ubuntu xenial InRelease
Could not resolve 'ppa.launchpad.net'
Err:2 http://security.ubuntu.com/ubuntu xenial-security InRelease
Could not resolve 'security.ubuntu.com'
Err:3 http://ppa.launchpad.net/ondrej/php/ubuntu xenial InRelease
Could not resolve 'ppa.launchpad.net'
Err:4 http://us.archive.ubuntu.com/ubuntu xenial InRelease
Could not resolve 'us.archive.ubuntu.com'
Err:5 http://us.archive.ubuntu.com/ubuntu xenial-updates InRelease
Could not resolve 'us.archive.ubuntu.com'
Err:6 http://us.archive.ubuntu.com/ubuntu xenial-backports InRelease
Could not resolve 'us.archive.ubuntu.com'
Err:7 https://download.docker.com/linux/ubuntu xenial InRelease
Could not resolve host: download.docker.com
Err:8 https://deb.nodesource.com/node_8.x xenial InRelease
Could not resolve host: deb.nodesource.com
Err:9 http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 InRelease
Could not resolve 'repo.mongodb.org'
Reading package lists... Done
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial/InRelease Could not resolve 'us.archive.ubuntu.com'
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease Could not resolve 'us.archive.ubuntu.com'
W: Failed to fetch http://us.archive.ubuntu.com/ubuntu/dists/xenial-backports/InRelease Could not resolve 'us.archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease Could not resolve 'security.ubuntu.com'
W: Failed to fetch https://download.docker.com/linux/ubuntu/dists/xenial/InRelease Could not resolve host: download.docker.com
W: Failed to fetch http://ppa.launchpad.net/certbot/certbot/ubuntu/dists/xenial/InRelease Could not resolve 'ppa.launchpad.net'
W: Failed to fetch http://repo.mongodb.org/apt/ubuntu/dists/xenial/mongodb-org/3.2/InRelease Could not resolve 'repo.mongodb.org'
W: Failed to fetch https://deb.nodesource.com/node_8.x/dists/xenial/InRelease Could not resolve host: deb.nodesource.com
W: Failed to fetch http://ppa.launchpad.net/ondrej/php/ubuntu/dists/xenial/InRelease Could not resolve 'ppa.launchpad.net'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Trying to resolve the apt-get host I get the following:
host -v us.archive.ubuntu.com
Trying "us.archive.ubuntu.com"
;; connection timed out; no servers could be reached
I figured this would be a DNS issue so I checked my resolv.conf file and am met with the following output:
cat /etc/resolv.conf
cat: /etc/resolv.conf: No such file or directory
So now I'm lost, what's going on and how can I fix this issue successfully?
EDIT:
ifconfig shows that I have an IP address as expected:
eno1 Link encap:Ethernet HWaddr d0:94:67:41:8b:b3
inet addr:10.1.1.78 Bcast:10.1.1.255 Mask:255.255.255.0
inet6 addr: fe80::d294:66ff:fe51:5bb3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:13800 errors:0 dropped:1 overruns:0 frame:0
TX packets:2420 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4767680 (4.7 MB) TX bytes:559473 (559.4 KB)
Interrupt:17

I had the same issue on Ubuntu:
Run:
systemctl enable systemd-resolved --now
ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf
nano /etc/systemd/resolved.conf
and add the relevant DNS, for example:
DNS=8.8.8.8

I was able to fix this issue completely with the following:
$ sudo dpkg-reconfigure resolvconf
$ sudo cat<<EOF > /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 8.8.8.8
nameserver 8.8.4.4
EOF
$ /etc/init.d/networking restart
$ sudo ifdown <INTERFACE>
$ sudo ifup <INTERFACE>

Related

Mac M1: Dnsmasq not resolving the local host on new apple m1 chip

I'm trying to setup my M1 laptop for dev env. But some dnsmasq is not working. I executed the following commands for dnsmasq:
brew install dnsmasq
echo 'address=/.local/127.0.0.1' > /opt/homebrew/etc/dnsmasq.conf
sudo brew services start dnsmasq
sudo brew services list
But when I try to ping to my local it's not responding.
$ ping test90.local
ping: cannot resolve test90.local: Unknown host
What DNS is ping using to resolve the hostname?
I'm not a mac user, but on BSD and Linux my next step would be to check /etc/resolv.conf to see if 127.0.0.1 was in the list of domains.
Also, .local is reserved for mdns. See https://en.wikipedia.org/wiki/.local

Netplan Error: Cannot bind to port 2893, Address is already in use

I want to configure static IP address in Ubuntu.
Here you can see my configuration file for static IP addressing:
network:
version: 2
renderer: NetworkManager
ethernets:
wlp1s0:
dhcp4: no
addresses: [192.168.0.103/24]
Gateway: 192.168.0.1
Nameservers:
Addresses: [127.0.0.53]
While testing the configuration through $ sudo netplan try, I get the following response:
bind: Address already in use
netplan: fatal error: cannot bind to port 2983, is another daemon running?, exiting.
Netstat shows the port in use by Netplan.
netstat -pnlt | grep ':2983'
tcp 0 0 0.0.0.0:2983 0.0.0.0:* LISTEN 1524/netplan
So can someone please give me a way to solve this issue??
Got same problem today with one of my servers. The reason was because 2 packages exists for Ubuntu with same binary name: netplan and netplan.io. First is "Calendar Service" and the second is for networking. My server have netplan package installed. I just removed it and now netplan for networking works fine. May be it will helps to you.
Same here ...
installed debian server
apt-get update
apt-get upgrade
reboot and it broke networking so I had to go on site and W%%#$ing fix it .. DIAF 'netplan'
apt-get remove --purge netplan* -y
apt-get install netplan.io -y

Change default docker0 bridge ip address

After starting docker-ce (18.06.1-ce, build e68fc7a) on Ubuntu Server 16.04, the following is created:
$ ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:fe:36:81:72
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Unfortunately, my PC is on this network. Meaning that as soon as the docker starts, I lose ssh connectivity to the server.
Attempt 1
Several forums are saying to add the following to /etc/docker/daemon.json:
{
"bip": "192.168.1.1/24"
}
Attempt 2, using this SO accepted answer
One possible solution that is working is running:
sudo ip addr add dev docker0 192.168.1.1/24
sudo ip addr del dev docker0 172.17.0.1/16
Source: forums.docker.com
Although this is a possible solution, I have to do it after the docker service starts. Something I can't do because I lose connectivity meanwhile.
Attempt 3, following #Light.G answer
After adding the -bip to ExecStart line, trying to start docker gives (journalctl -xe):
-- Unit docker.socket has begun starting up.
Sep 11 11:13:19 PTLISLABHLC01 systemd[1]: Listening on Docker Socket for the API.
-- Subject: Unit docker.socket has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.socket has finished starting up.
--
-- The start-up result is done.
Sep 11 11:13:19 PTLISLABHLC01 systemd[1]: docker.service: Start request repeated too quickly.
Sep 11 11:13:19 PTLISLABHLC01 systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Sep 11 11:13:19 PTLISLABHLC01 systemd[1]: docker.socket: Unit entered failed state.
Sep 11 11:13:19 PTLISLABHLC01 polkitd(authority=local)[1062]: Unregistered Authentication Agent for unix-process:15666:32644036 (system bus name :1.56, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale en_US.UTF-8) (disconnected from bus)
Only error here is that you should use --bip and not -bip. Check the accepted answer!
No need extra entity:
Edit /lib/systemd/system/docker.service before start Docker. Add --bip "192.168.1.1/24" at the end of line ExecStart=/usr/bin/dockerd.
systemctl daemon-reload
systemctl start docker
Tested on Ubuntu 16.04 with Docker 17.03-ce.
Edit on 2018-09-13:
Since we might still need user-defined bridge networks, there is still a potential issue.
By default bridge is assigned one subnet from the ranges 172.[17-31].0.0/16 or 192.168.[0-240].20/20 which does not overlap with any existing interface. Unlike the default bridge network, user-defined networks supports manual IP address and subnet assignment. If an assignment is not given, then Docker’s default IPAM driver assigns the next subnet available in the private space.
Thougt they say it would not overlap with any existing interfaces on host, you still suffered such an issue. So if you need user-defined bridge networks, you’d better assign specific subnet for them. As I know, there is no parameters for customizing IPAM driver default pool.
My solution was to do it with a service that runs a shell command.
I created docker-network-setup.sh:
sudo ip addr add dev docker0 192.168.1.1/24
sudo ip addr del dev docker0 172.17.0.1/16
chmod u+x docker-network-setup.sh to ensure you can execute it.
And docker-network-setup.service:
[Unit]
Description=Change docker0 default IP
# When systemd stops or restarts the docker.service, the action is propagated to this unit
PartOf=docker.service
# Start this unit after the docker.service start
After=docker.service
[Service]
# The program will exit after running the script
Type=oneshot
# Execute the shell script
ExecStart=/bin/bash /home/user01/docker-network-setup.sh start
# This service shall be considered active after start
RemainAfterExit=yes
[Install]
# This unit should start when docker.service is starting
WantedBy=docker.service
sudo cp docker-network-setup.service /etc/systemd/system and the service is enabled!
Don't forget sudo systemctl daemon-reload if you want to test the service right away.
I struggled with AWS VPC peering connection that didn't work with my Docker containers until I found this thread.
In case you're using an AWS 172.17.x.x subnet, it might conflict with the default Docker subnet (172.17.0.1/16), so nothing will work between those subnets...
One way to fix it will be just moving to a different AWS subnet (which is not always possible due to legacy systems/configurations);
Another way will be to change the default Docker subnet with bip setup in /etc/docker/daemon.json (as described above) - something like:
{
"bip": "192.168.1.1/24"
}
Best approach will be just to avoid using 172.17.x.x subnet in AWS...

Kubectl gives "Unable to connect to the server: dial tcp i/o timeout

I have installed minikube and kubectl on Ubuntu 16.04LTS
However when i try any command with kubectl it give the below error:
Unable to connect to the server: dial tcp x.x.x.x:x i/o timeout
kubectl version only gives client version. The server version is not dispalyed
Is there any workaround to fix this?
I had to ensure the interface was up and running.
So a sudo ifconfig vboxnet0 up resolved the issue.

Docker - Not able to telnet Linked Container

I have a container A with Dockerfile as follows:
...
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
EXPOSE 8080
and another container B with Dockerfile as follows:
...
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
RUN apt-get install -q -y mysql-server
EXPOSE 3306
Then I started the container B using command:
docker run -P -it --name db B /bin/bash
And running the command: docker run --rm --name web --link db A env
gives me below output:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=a5dd66b72ca8
DB_PORT=tcp://172.17.0.2:3306
DB_PORT_3306_TCP=tcp://172.17.0.2:3306
DB_PORT_3306_TCP_ADDR=172.17.0.2
DB_PORT_3306_TCP_PORT=3306
DB_PORT_3306_TCP_PROTO=tcp
DB_NAME=/web/db
HOME=/root
Then I logged into container A and run the command: ping db
which gives me output:
PING db (172.17.0.2) 56(84) bytes of data.
64 bytes from db (172.17.0.2): icmp_seq=1 ttl=64 time=0.082 ms
64 bytes from db (172.17.0.2): icmp_seq=2 ttl=64 time=0.063 ms
64 bytes from db (172.17.0.2): icmp_seq=3 ttl=64 time=0.065 ms
64 bytes from db (172.17.0.2): icmp_seq=4 ttl=64 time=0.061 ms
64 bytes from db (172.17.0.2): icmp_seq=5 ttl=64 time=0.066 ms
And when I tried telnet command with port 3306, as follows:
telnet 172.17.0.2 3306
gives me output:
root#9b078c1fed82:/# telnet 172.17.0.2 3306
Trying 172.17.0.2...
telnet: Unable to connect to remote host: Connection refused
Any idea, how can I telnet linked container?
Any help will be appreciated!
MySQL usually only listens on 127.0.0.1 by default. You need to modify the my.cnf during build and set bind-address to 0.0.0.0. You could accomplish this with something like:
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
After receiving the answer by #Andy Shinn, I Updated the container B Dockerfile as below:
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
RUN apt-get install -q -y mysql-server
RUN apt-get install -q -y mysql-client
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
EXPOSE 3306
And when I tried accessing mysql-server from container A as:
mysql -u root -ppassword -h 172.17.0.2
Gives me error:
ERROR 1130 (HY000): Host '172.17.0.3' is not allowed to connect to this MySQL server
Which I figured it out is because container A was trying to connect to db container through root user. But In mysql do not allow you to connect though root user remotely. So I work around creating another user in mysql-server with root privileges.

Resources