I am a complete beginner when it comes to networking and I am trying to set up a TCP tunnel on my machine using pagekite. I want to route all traffic from a TCP address to a port on my localhost, let's say 8080. I would then start a handler on localhost:8080 to deal with the incoming traffic. I can get this to work with ngrok simply by doing ngrok tcp 8080, but on a free ngrok plan I cannot reserve tcp addresses and ngrok is rather slow, so I opted to try and use pagekite.
Pagekite normally allows easy tunnelling to an HTTP address, but they have a guide here about how to use PuTTY along with Pagekite to create a TCP tunnel proxied by HTTP.
I followed their guide but could use some help figuring out if it does what I want it to do.
I am working on a Linux VM, so I first set up an SSH server with openssh like this: sudo service ssh start
I then exposed that SSH server using pagekite like this: python3 pagekite.py 22 ssh:user.pagekite.me
I then started PuTTY, and configured the Host Name to be user.pagekite.me on port 22, setup an HTTP proxy with the proxy hostname user.pagekite.me on port 443 and finally created a tunnel from the PuTTY machine with source port 8080 and destination localhost:8080.
Now I am not sure what this actually accomplished. I know that the PuTTY machine connected to the ssh server running on my VM and I am able to use the linux terminal from the PuTTY terminal but has this actually created a TCP tunnel from user.pagekite.me:8080 to localhost:8080? Additionally after doing this, if I try to setup the handler on localhost:8080 I get the following error:
Handler failed to bind to 0.0.0.0:8080
Rex::BindFailed The address is already in use or unavailable: (0.0.0.0:8080).
Again I am completely clueless when it comes to networking so if anyone could explain what it is I'm doing and if it is even possible to do what I want the way that I am doing it, that would be quite helpful.
I have a few web-based local applications for home-automation purposes, those are accessible through IP addresses with port numbers something like http://192.168.1.100:8080.
What I am trying to achieve is to link each individual IP and port number combination to an internal domain name, so that anyone can use domain and subdomain names rather than IP addresses.
For example a person can specify a URL of http://kitchen.home rather than an IP address with port number URL such as http://192.168.1.100:8080.
At the same time http://192.168.1.100:8081 could be mapped to a domain name such as hall.home so that a URL of http://hall.home could be used instead of that IP address and port number.
The access to a server should not require having to modify the hosts file of individual PCs but should be some kind of a domain name server that maps domain names to IP address and port number for any PC on the local network.
I have set up a DNS server on a PC running bind9 as a DNS server under Ubuntu 20.04 on my home network in order to have my own special domain names.
A few details about DNS and proxy server with HTTP
To map several different subdomain names within a domain to specific ports on the same PC, you will need a proxy server installed on the the PC as well as a DNS server for the local network. Domain names are mapped to a specific IP address with the DNS protocol and are not mapped to a specific port at the IP address.
In my case I have the same PC hardware with Ubuntu 20.04 that has (1) Bind9 for my DNS server and (2) Apache for my proxy server. I could use two different PCs, one as a DNS server to resolve domain names to IP addresses and the other offering various services accessed either through a single IP port using a proxy server listening on the port to dispatch connections to the services or directly to the service by specifying the correct port number of the desired service.
For example http://kitchen.home/ in a browser would instruct the browser to open a TCP connection to the IP address represented by the domain name kitchen.home using port 80, the default HTTP protocol port. A DNS server is used to resolve the domain name kitchen.home to an IP address. With a URL of http://kitchen.home:8081/, the browser will ask the DNS server to resolve the domain name kitchen.home to an IP address and then open a TCP connection to that IP address but using port 8081 rather than the standard HTTP port 80.
So for http://kitchen.home/ to map to port 8080 at that IP address and for http://hall.home/ to map to port 8081 at that IP address you need to combine a DNS server, which resolves the domain name, with a proxy server residing at port 80, the standard HTTP port. The proxy server will then redirect the request to a different port on the PC selected based on the subdomain name, hall or kitchen, of the entire domain specifier, kitchen.home or hall.home.
See https://stackoverflow.com/a/58122704/1466970 which describes setting up Nginx as a proxy server.
The Apache web server can also serve as a proxy server which is what I'm looking into. See this tutorial from DigitalOcean, How To Use Apache HTTP Server As Reverse-Proxy Using mod_proxy Extension as well as Setting up a basic web proxy in apache.
My environment
I have a Windows 10 PC downstairs with a Ubuntu 20.04 PC upstairs communicating through an Arris router to my cable internet provider. Both PCs are connected to my local home network with WiFi.
The Ubuntu 20.04 PC is my Subversion server using Apache web server. I spend most of my time with the Windows 10 PC downstairs, using PuTTY to connect to the Ubuntu PC with one or more terminal windows when needed. I plan to work with Visual Studio on the Windows 10 PC accessing Subversion through the Apache web server as well as using the Ubuntu PC as a database server (MySQL) and web server (Apache with Php) and microservices (golang and node.js).
I wanted to setup a DNS server on the Ubuntu PC and then point my Windows 10 PC to use the local DNS server for a special domain name while using standard DNS servers such as Google at 8.8.8.8 and 8.8.4.4.
What I did
The procedure I followed was (see How to Configure BIND9 DNS Server on Ubuntu 20.04 and see as well Domain Name Service (DNS) and Everything You Need To Know About Ubuntu DNS Servers):
install bind9 using sudo apt install bind9
create a firewall rule sudo ufw allow Bind9
modify the file /etc/bind/named.conf.options
modify the file /etc/bind/named.conf.local
create a copy of /etc/bind/db.local for my new domain name, home.x
modify the new file, /etc/bind/db.home.x, with the correct rules
The domain name I wanted to use locally was home.x with the idea that if I entered a web site URL of http://www.home.x/ the resulting page would be the Apache web server on my Ubuntu PC. Or if I entered http://home.x/svn the result would be the Subversion repository on my Ubuntu server.
Note: in order for Subversion access through Apache, I had to set that up. See enabling Subversion access via Apache web server and DAV on Ubuntu if you are interested in that.
Details on changes and modified files
The Ubuntu PC has an IP address of 192.168.0.4 on my local network. In the descriptions below, this IP address is used where ever the Ubuntu PC is referenced.
I added a forwarders section to the file /etc/bind/named.conf.options in order to forward any DNS requests that were unknown to my Ubuntu server to some other DNS server. The IP addresses I copied from the list of DNS servers returned by the Window 10 command ipconfig /all which I ran on my Windows 10 PC. The changed file is as follows:
rick#rick-MS-7B98:~/Documents$ cat /etc/bind/named.conf.options
options {
directory "/var/cache/bind";
// If there is a firewall between you and nameservers you want
// to talk to, you may need to fix the firewall to allow multiple
// ports to talk. See http://www.kb.cert.org/vuls/id/800113
// If your ISP provided one or more IP addresses for stable
// nameservers, you probably want to use them as forwarders.
// Uncomment the following block, and insert the addresses replacing
// the all-0's placeholder.
// forwarders {
// 0.0.0.0;
// };
// following forwards are to Google DNS servers at 8.8.8.8 and 8.8.4.4
forwarders {
8.8.8.8;
8.8.4.4;
209.55.27.13;
};
//========================================================================
// If BIND logs error messages about the root key being expired,
// you will need to update your keys. See https://www.isc.org/bind-keys
//========================================================================
dnssec-validation auto;
listen-on-v6 { any; };
};
After changing the file /etc/bind/named.conf.options, I then ran the check utility, which found no errors, and then restarted the bind service.
sudo named-checkconf
sudo systemctl restart bind9
Next I added a new zone directive to the file /etc/bind/named.conf.local to create a DNS entry for my new, local domain name home.x. The modified file looks like:
rick#rick-MS-7B98:~/Documents$ cat /etc/bind/named.conf.local
//
// Do any local configuration here
//
// Consider adding the 1918 zones here, if they are not used in your
// organization
//include "/etc/bind/zones.rfc1918";
zone "home.x" {
type master;
file "/etc/bind/db.home.x";
};
Finally I needed to create the file /etc/bind/db.home.x specified in the file directive of the zone directive. I did this by starting with a copy of the existing file /etc/bind/db.local by using the command
sudo cp /etc/bind/db.local /etc/bind/db.home.x
I then modified the file /etc/bind/db.home.x in order to specify the rules I needed to resolve the domain name of home.x as well as the subdomain of www.home.x to the IP address of my Ubuntu PC. The modified file looks like:
rick#rick-MS-7B98:~/Documents$ cat /etc/bind/db.home.x
;
; BIND data file for local loopback interface
;
$TTL 604800
# IN SOA home.x. root.home.x. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
# IN NS ns.home.x.
# IN A 192.168.0.4
# IN AAAA ::1
ns IN A 192.168.0.4
www IN A 192.168.0.4
At this point I could test that things worked from my Windows 10 PC by using the nslookup command from a command window. I tried it first without specifying the Ubuntu PC IP address and then with the Ubuntu PC IP address
C:\Users\rickc>nslookup home.x
Server: dns.google
Address: 8.8.8.8
*** dns.google can't find home.x: Non-existent domain
C:\Users\rickc>nslookup home.x 192.168.0.4
Server: UnKnown
Address: 192.168.0.4
Name: home.x
Addresses: ::1
192.168.0.4
I then used the Windows 10 Control Panel (Control Panel > Network and Internet > Network Connections) to see a list of my network adapters in order to modify the DNS server addresses my WiFi adapter was using. This setting is found by doing a right mouse click on the adapter to pop up a floating menu, select Properties to bring up the Properties dialog then select Internet Protocol Version 4 then click the Properties button on the dialog and modify the DNS server address. Below is a screen shot of the dialogs.
After modifying the DNS server address, I retried the nslookup without specifying a DNS server address and the command finds home.x:
C:\Users\rickc>nslookup home.x
Server: UnKnown
Address: 192.168.0.4
Name: home.x
Addresses: ::1
192.168.0.4
When I tested a URL of http://www.home.x/ the test page of my Apache web server is displayed. When I tested a URL of http://home.x/svn the web browser showed the directory tree of my Subversion repository. When I access my Subversion repository with http://home.x/svn/trunk/ within the Ankh Subversion plug-in with Visual Studio 2017 it works.
Other thoughts
One issue with this setup is that if my Ubuntu PC is not up and running then the Windows 10 PC will not have a functioning DNS server until either the Ubuntu PC is brought up or the DNS server address is set back to the original setting on the WiFi adapter. Previously, it was set to discover a DNS server. It may be that I can change this back and from a specified DNS server address, the IP address of my Ubuntu PC, and my Windows 10 PC will discover my Ubuntu PC as a DNS server anyway.
DNS names are independent of port numbers. A dns record correlates a name to an IP address. If you want to omit the port number from urls, you either need to host on the default ports (80/443), or use a proxy.
With a home network, the dns can be tricky, and usually depends greatly on the specific router you have. Your options are basically:
Maybe your router has an interface for defining host overrides. OpenWRT has such a thing, and some routers have a similar dns server on them that you can add host entries to.
Get a public domain name. You can have an A record for example.com that points at a local network address. The server itself may only be accessible inside your network, but the dns will be available anywhere.
Run your own dns server and tell your router to use it as the default resolver for your network.
Maybe mDNS could suit your needs for .local domains.
Hopefully straightforward. I know how to bind to the host only with
-p 127.0.0.1:$HOSTPORT:$CONTAINERPORT
The issue I'm encountering is that doing this preventing me from accessing the mapped host port over an ssh tunnel to the docker host.
Is there way to do this without having to block the port upstream from the docker host somewhere?
Just make the target of your ssh tunnel localhost or 127.0.0.1.
ssh -L local-port:127.0.0.1:container-port docker-host
Would forward your local-port to localhost:container-port on docker-host. No need to expose the container port to the external network.
We have a dockerized app that we imported on a Compute Engine instance with ubuntu 16.04.
It contains a nginx reverse proxy running on port 80 and in /etc/hosts we've added 127.0.0.1 mydockerizedapp
The GCE got an external IP address.
How can I set so that when I go on this external IP from a browser, I see the files served by the container nginx ?
You have to expose the ports of your container on the host machine by mapping it.
If you use the cli: --port-mappings=80:80:TCP
So I have a problem with Wordpress running in a Vagrant VM. The VM is an Ubuntu box, precise32, running under VirtualBox. It's configured for a private network, with no specific port forwarding rules:
config.vm.network "private_network", ip: "192.168.33.10"
The VM has a basic LAMP stack, with no added config -- no virtual hosts, for example.
From the host machine I can reach standard resources on the VM via HTTP just fine: the basic index.html, or anything I care to put into subdirectories.
I have WordPress installed into /wordpress. When I try to access
http://192.168.33.10/wordpress
from a browser on the host, I get redirected to
http://192.168.33.10:8080/wordpress
Which fails, because nothing is listening there.
When I create a specific hostname for the VM and add it to my hosts /etc/hosts file and access WP via the host name, I get the same result.
References I can find are mostly to do with running WP behind a reverse proxy. But I'm not doing that -- unless something in my Vagrant setup means I really am and just don't know it.
Any ideas?
I suggest to use port 8080 both in the guest and the server. Configure the webserver to listen on port 8080 and map that port to Vagrant host via forwarded_port.