I have a Bind DNS server on my local network to get *.home resolution, and a forwarders section for every other domains. I also have a local web server which serve some subdomains of *.mydomain.com from the outside, with my ISP box redirecting ports 80/443 to it.
But from any computer within my local network, if I try to access any of the *.mydomain.com address, the IP address returned is my public IP address and the ISP box doesn't redirect the request to my server. I used a workaround by adding the subdomains *.mydomain.com with the local IP address of the server in /etc/hosts on each local computer but it's not perfect since I have to change it if I move a computer to another network, plus I can't do this on my non-rooted Android devices.
The perfect solution would be to have my local DNS overrinding the DNS entries for *.mydomain.com with the local IP instead of the public one. I tried some Bind9 configurations for this, some using the "response-policy" directive, some without it, but in every case, I get the external IP ~50% of the time and I can't figure out why. This happens on every computer I use (some are on Ubuntu, some with Windows), also on my Android phones.
Using dig #mydns subdomain.mydomain.com return the local IP so I assume that the Bind configuration is OK, but dig subdomain.mydomain.com (or ping) doesn't always return the proper IP. I also tried to disable forwarders for the subdomain like this :
zone "mydomain.com" {
type master;
file "/etc/bind/db.mydomain";
forwarders {};
};
I flushed the DNS cache of my computers and there is no other DNS server on my network so I don't understand how this can happen. I also tried to log requests on the Bind server and I don't see requests with the external IP address returned (only those with the internal). So where did these requests go through ?
I finally found out what the problem was : my custom DHCP/DNS server was configured for IPv4 only, but my ISP box also provide IPv6 capabilities, so all the DNS requests from the devices on the network where sent on both protocols and half of the time, the request was answered by the ISP box instead of my custom DNS.
I disabled IPv6 for now and the problem is gone. I'll configure it for my custom server later.
Related
I am trying to get the big picture although my primary domain is not networking.
Some question's narrowed down for which I'm not getting enough/proper answers online
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
Do bigger companies like Amazon do port forwarding?
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
A gateway IP refers to a device on a network which sends local
network traffic to other networks. it sits between you and internet,or other network . its like a watchman.
Question 1 : google.com has multiple ip addresses lets say then , Yes, that is possible, and will need to be two A records. This is called Round-Robin DNS. Clients will semi-randomly use one of the two addresses.
question2: yes port forwarding happens more often than we think. ALL VPC's (virtual private clouds like AWS , GCP , Azure etc) use this as they dont want to expose servers/internal resources to the internet.
depending on the port number , particular service is exposed to requesting client. lets say we want to make a website public , then we explicitly expose port 80(http) 443(https) so that web crawlers and users can see them.
Port forwarding, sometimes called port mapping, allows computers or
services in private networks to connect over the internet with other
public or private computers or services.
google https://www.google.com:444/ wont work because they did not expose port 444 on their cloud router
but https://www.google.com:443/ will work because the server corresponding to google.com has explicitly left it open.
How IP is resolved:
Step 1 - Send a Request to Resolve a Domain Name
When you type www.google.com into a browser, in order to load the webpage, your computer asks for the IP address. Computers do not know in advance where they can find the necessary information, so they try searching through the DNS cache and for available external source. proceed from lower level caches to root/main servers.
Step 2+3 - Try to resolve an IP Locally
Before going externally, your computer loads the local DNS cache database to see if you already requested the IP for that domain name. Every computer has a temporary cache with the most recent DNS requests and attempts to connect to online sources. if required record is present locally its called "CACHE HIT" and query stops.
However A computer’s local DNS cache database does not always contain the necessary data to resolve a domain name this is called a "CACHE MISS" . In that case, the request goes further to your Internet Service Provider (ISP) and its DNS server.
Step 4 - ISPs Ask Outside DNS Servers to Provide an IP Address iff Cache miss
ISP DNS resolvers are configured to ask other DNS servers for correct IP address mapping until they can provide data back to the requester. These are iterative DNS queries.
When a DNS client sends such a request, the first responding server does not provide the needed IP address. Instead, it directs the request to another server that is lower in the DNS hierarchy, and that one to another until the IP address is fully resolved. There are a few stops in this process.
hierarchy looks like this (just for reference):
Root domain nameservers. Root servers themselves do not map IP addresses to domain names. Instead, they hold the information about all top-level domain (TLD) nameservers and point to their location. TLD is the rightmost section of a domain name... Root servers are critical since they are the first stop for all DNS lookup requests.
TLD nameservers. These servers contain the data for second-level domains, such as ‘phoenixnap’ in phoenixnap.com. Previously, the root server pointed to the location of the TLD server. Then, the TLD server needs to direct the request toward the server that contains the necessary data for the website we are trying to reach.
Authoritative nameserver. Authoritative servers are the final destination for DNS lookup requests. They provide the website’s IP address back to the recursive DNS servers. If the site has subdomains, the local DNS server will keep sending requests to the authoritative server until it finally resolves the IP address.
Step 5 - Receive the IP Address
Once the ISP’s recursive DNS server obtains the IP address by sending multiple iterative DNS queries, it finally returns it to your computer. The record for this request now stays cached on the hard drive. The browser can then fetch this IP from the cache and connect it to the website’s server.
ALL this happens in less than 1 second, most of the times. if you just registered a new domain it might take few hours to propagate this DNS cache globally hence newly registered websites do not show up sometimes.
About companies owning multiple IPs
Big companies have pool of IPs reserved for example 123.234.xxx.xxx which means a company has reserved 255*255 ips. they are mapped on a VPC(virtual private cloud)
and accessible vis a subnet masking and CIDR feature, like your EC2 instances on AWS
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
For sure it should, but it is mostly a Google management question that only they will be able to answer right. The thing is that we must understand how DNS query's work for this.
Let's take a look of it:
Device A requests the IP address through a DNS query of the device B.
To do this, it uses the network port 53 (Domain) on which it will ask, depending on which DNS server is being used at the time, which is usually the home router. Then the router will ask the ISP's DNS server, which will respond with a cached response, or the query with another server on top of it if it does not have one; All this process is followed until a reliable cache response is reached or until the authoritative response server is reached, that is, the name server that manages the domain in question.
Only the authoritative response server contains the reliable information of which IP of the domain which is going to be reached.
I suppose that within Google's servers and its network they use Google's own DNS servers, which are 8.8.8.8 and 8.8.4.4 where the DNS records are obtained and consulted by caching from many sites.
In general terms Google's IP will change depending on where you are, I made a DIG query to Google's authoritative servers, however, I received a result based on location to improve the route and loading time of the site which was 142.250.73.238.
Do bigger companies like Amazon do port forwarding?
Yes, they do. To handle queries with load balancers or similar and even for caching dns requests.
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
This has multiple answers. By the way, they actually can do a secure DNS query.
if they have N static IP address which resolves to a website, then they must be having N routers right?
They don't have to, but if they want to they can.
"Is this a fair assumption?"
No, the IP's doesn't depend on a router, the router only routes to a computer/server which can have multiple IP's. By the other hand, each thing (computer, server, etc... must have an IP which can be also a WAN IP).
i have problem with access to server with domains.
Test url: testpage.example.com.
Server is in local network with port fowarding (80, 443), configured as web server using caddy server as reverse proxy.
Case 1 (using Asus router):
Connected on local network behind router. Server is in the same network as my computer. Everything works like a charm in and out of my network.
Case 2 (using internet provider router):
Connected on local network behind router. Can't access server with domain. Works with direct IP. Outside network, works as in case 1.
I used same server.
Does anyone know why this problem occured? How can i solve it?
Thanks,
David
Two options I can think of:
You could add a record to whatever DNS server you're using in Case 2.
You could write a short script that runs whenever you change network connections to modify your hosts file accordingly.
This happens when the server you are trying to reach "testpage.example.com" resolves to your router's external IP address. Because your public IP address is the same as server's IP address (even though inside your home network you have different private IP addresses) your requests are lost in the ether.
As a workaround you can resolve the testpage.example.com manually on your local machine.
For Windows c:\windows\system32\etc\hosts
For Linux /etc/hosts
testpage.example.com 192.168.1.102 -> private IP of the machine serving the site.
I am attempting to spin up an application that listens on a port and responds to HTTP requests. I am on a Windows 8 machine connecting through a Netgear router that provides port forwarding. I have:
modified my DNS zone file of one of my domains to point to the IP address that is assigned to my cable modem
Added a port-forwarding rule to my router that sends requests to port 8080 to port 8081 on my computer
Opened port 8081 on my Windows Firewall
Executed netsh http add urlact http://+:8081/ user=Everyone listen=yes as administrator
Started up my app which uses the simple webserver solution found at http://codehosting.net/blog/BlogEngine/post/Simple-C-Web-Server.aspx which uses an HttpListener object with a prefix of http://+:8081/.
From any machine on my local network, I can browse to http://home.example.com:8080/blah/blah and everything works great. Whenever I attempt the same URL from a machine connected elsewhere on the Internet, the connection times out. I have tried using the IP address instead the domain name, and have tried disabling my Windows Firewall (temporarily), still with no luck.
I'm sure this is more of a network setup issue than a code issue, but I thought I would ask anyway to see if there is anything I can do. Sorry for the spaces in the urls above. This is my first post to SO, and I apparently don't have enough of a reputation to post more than a single link.
By "elsewhere on the Internet", I am assuming you are attempting to access it from a different ISP.
The thing about some ISPs is that unless you are paying for a "business class" connection, they will do all sorts of tricks to ensure that you remain a "consumer". What you need is an unNATed static IP address.
By this I mean that the IP address that you may have at your home may not be accessible to the outside world because the ISP is actually NATing (or other) that address to you. This is a fairly common practice because of limited IP4 addresses. If you really want a service accessible via the WWW, I would suggest moving your product to a VPN, or at least a commodity hosting provider.
Edit: Try a VPN service like Hamachi
I want to host my website from my home using a server and I also want to know if I can register my own domain name using my DNS server! If yes it would be really helpful on how to do so!
Thank you very much!
Setting up a home web server is simple. If you do it though, check and make sure that your ISP allows you to do so. Here is my recommendation to begin:
Register your domain name with godaddy.com, register.com or whomever you choose (ie.www.xyz.com)
Determine whether you will have a public dynamic IP address (will change constantly) or a static ip address (usually have to pay for this from your ISP)
DNS Setup
a. If you are using a public dynamic IP address (which I am sure you are in this case), then set up a dynamic dns free account (go here - http://www.dyndns.com/services/dns/dyndns/). You will need to set up a username and password, which you will later use on the DynDNS Updater you will need to install on your Windows web server. Create the zone for your domain (ie. xyz.com is a zone you create).
b. If you are using a static IP address, then make sure the name servers are 'Parked' with your domain provider, then set up a forward to the public static IP address of your home network (from here, you just set up port forwarding in your router/gateway to your web server and you should be done)
Web Server Configuration
I. set up a 'static' ip address on your network card which will manage the incoming connections. Make sure that IIS is listening for requests through this IP address via IIS Manager.
II. If you are going to use the dynamic dns configuration for DNS, then download the DynDNS Updater app to your windows server and it will manage the connection between your web server dns and dynamic dns servers (this is cool and easy)
III. Set up a test index.html in IIS Manager for your 'Default Site'. Open a web browser and open 'http://localhost' to make sure you can access the site.
IV. Make sure your Windows firewall allows traffic through port 80.
Router / Gateway Configuration - go to 'Port Forwarding', and forward all requests via port '80' to the static ip address of your webserver (ie. Allow Incoming Port '80' to 192.168.x.x')
From your web server, Go to 'http://www.whatismyip.com and see what your public ip address is.
Once your IIS is set up, your port forwarding has been configured, and dynamic dns is updated, ping the domain you registered 'ie. www.xyz.com' and see if it matches the IP address of your web server public ip.
From outside your local area network, try to access your domain 'http://www.xyz.com'. If success, then you are good to go. If no success, review all these steps again.
I have a website that I'm running locally at localhost:8000. I'd like to access my site from other devices on my network. I've tried to visit the site via my computer's internal IP address via: 127.0.0.1:8000but this doesn't work. Is this possible?
Three most probable things:
Check if there is a network route between the client and the server machine (commands like PING and TRACERT will help);
Check if the server machine has a firewall. If it does, there must be a rule allowing (opening) connections through that port (8000) in TCP;
Most likely, the problem is one of the two above. If not, there's one last thing:
Make sure the web server (the one that serves localhost:8000) is bound to listen to all IP addresses (not only 127.0.0.1).
To know that, search for servername bind all ip addresses on google. E.g., for apache HTTPD, it'd be apache bind all ip addresses.
here is what I do in similar cases:
search rejetto hfs on google, it is single executable less than 1Mb size and download it into your server machine.
after starting it, follow menu->IP addresses. these are all your possible addresess on the server side.
these are your IP addresses to enter, and your port is 8000 as you mentioned.
most possible IP addresses you will find are in the format of following:
192.168.?.?
10.0.0.?
169.254.?.?
besided you will probably need to add a firewall rule for your server app.