I was quite eager to understand, how does DNS server work.
Scenario
If I do a DNS query, it reaches to local DNS server, now local DNS server checks locally, if information is not present, it queries for other DNS servers (in a hierarchical manner, root DNS, Top Level DNS, etc, so on).
Thoughts & Question
I wondered, does the local DNS server change the source ip of my request packet it received and if does change my source ip to its source ip for querying to other Dns servers, like NAT does (please correct me if I am thinking wrong), how does it keep track of me (like NAT does), like when the response comes from Root/TL/Secondary Dns server, how does it know that this response has to be directed to me.
Yes, the recursive (caching) name server issues the query itself, so the root, TLD, and authoritative servers all see the query source as being the recursive server.
If the recursive server supports EDNS0, the edns-client-subnet will pass through the /24 netblock of the client, allowing the authoritative server to see the first three octets of the client that initiated the request.
DNS responses are correlated with requests through examining the QUERY field of the response packet.
Related
I am trying to get the big picture although my primary domain is not networking.
Some question's narrowed down for which I'm not getting enough/proper answers online
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
Do bigger companies like Amazon do port forwarding?
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
A gateway IP refers to a device on a network which sends local
network traffic to other networks. it sits between you and internet,or other network . its like a watchman.
Question 1 : google.com has multiple ip addresses lets say then , Yes, that is possible, and will need to be two A records. This is called Round-Robin DNS. Clients will semi-randomly use one of the two addresses.
question2: yes port forwarding happens more often than we think. ALL VPC's (virtual private clouds like AWS , GCP , Azure etc) use this as they dont want to expose servers/internal resources to the internet.
depending on the port number , particular service is exposed to requesting client. lets say we want to make a website public , then we explicitly expose port 80(http) 443(https) so that web crawlers and users can see them.
Port forwarding, sometimes called port mapping, allows computers or
services in private networks to connect over the internet with other
public or private computers or services.
google https://www.google.com:444/ wont work because they did not expose port 444 on their cloud router
but https://www.google.com:443/ will work because the server corresponding to google.com has explicitly left it open.
How IP is resolved:
Step 1 - Send a Request to Resolve a Domain Name
When you type www.google.com into a browser, in order to load the webpage, your computer asks for the IP address. Computers do not know in advance where they can find the necessary information, so they try searching through the DNS cache and for available external source. proceed from lower level caches to root/main servers.
Step 2+3 - Try to resolve an IP Locally
Before going externally, your computer loads the local DNS cache database to see if you already requested the IP for that domain name. Every computer has a temporary cache with the most recent DNS requests and attempts to connect to online sources. if required record is present locally its called "CACHE HIT" and query stops.
However A computer’s local DNS cache database does not always contain the necessary data to resolve a domain name this is called a "CACHE MISS" . In that case, the request goes further to your Internet Service Provider (ISP) and its DNS server.
Step 4 - ISPs Ask Outside DNS Servers to Provide an IP Address iff Cache miss
ISP DNS resolvers are configured to ask other DNS servers for correct IP address mapping until they can provide data back to the requester. These are iterative DNS queries.
When a DNS client sends such a request, the first responding server does not provide the needed IP address. Instead, it directs the request to another server that is lower in the DNS hierarchy, and that one to another until the IP address is fully resolved. There are a few stops in this process.
hierarchy looks like this (just for reference):
Root domain nameservers. Root servers themselves do not map IP addresses to domain names. Instead, they hold the information about all top-level domain (TLD) nameservers and point to their location. TLD is the rightmost section of a domain name... Root servers are critical since they are the first stop for all DNS lookup requests.
TLD nameservers. These servers contain the data for second-level domains, such as ‘phoenixnap’ in phoenixnap.com. Previously, the root server pointed to the location of the TLD server. Then, the TLD server needs to direct the request toward the server that contains the necessary data for the website we are trying to reach.
Authoritative nameserver. Authoritative servers are the final destination for DNS lookup requests. They provide the website’s IP address back to the recursive DNS servers. If the site has subdomains, the local DNS server will keep sending requests to the authoritative server until it finally resolves the IP address.
Step 5 - Receive the IP Address
Once the ISP’s recursive DNS server obtains the IP address by sending multiple iterative DNS queries, it finally returns it to your computer. The record for this request now stays cached on the hard drive. The browser can then fetch this IP from the cache and connect it to the website’s server.
ALL this happens in less than 1 second, most of the times. if you just registered a new domain it might take few hours to propagate this DNS cache globally hence newly registered websites do not show up sometimes.
About companies owning multiple IPs
Big companies have pool of IPs reserved for example 123.234.xxx.xxx which means a company has reserved 255*255 ips. they are mapped on a VPC(virtual private cloud)
and accessible vis a subnet masking and CIDR feature, like your EC2 instances on AWS
Is the IP that is resolved by the DNS server when I hit www.google.com is same as any of the Google router's Gateway IP?
For sure it should, but it is mostly a Google management question that only they will be able to answer right. The thing is that we must understand how DNS query's work for this.
Let's take a look of it:
Device A requests the IP address through a DNS query of the device B.
To do this, it uses the network port 53 (Domain) on which it will ask, depending on which DNS server is being used at the time, which is usually the home router. Then the router will ask the ISP's DNS server, which will respond with a cached response, or the query with another server on top of it if it does not have one; All this process is followed until a reliable cache response is reached or until the authoritative response server is reached, that is, the name server that manages the domain in question.
Only the authoritative response server contains the reliable information of which IP of the domain which is going to be reached.
I suppose that within Google's servers and its network they use Google's own DNS servers, which are 8.8.8.8 and 8.8.4.4 where the DNS records are obtained and consulted by caching from many sites.
In general terms Google's IP will change depending on where you are, I made a DIG query to Google's authoritative servers, however, I received a result based on location to improve the route and loading time of the site which was 142.250.73.238.
Do bigger companies like Amazon do port forwarding?
Yes, they do. To handle queries with load balancers or similar and even for caching dns requests.
If point 2 is true, I suppose they must be port forwarding with only 443 (https) port which means, to use multiple static IP across different data centers, they need to have that many routers. So, if they have N static IP address which resolves to a website, then they must be having N routers right? Is this a fair assumption?
This has multiple answers. By the way, they actually can do a secure DNS query.
if they have N static IP address which resolves to a website, then they must be having N routers right?
They don't have to, but if they want to they can.
"Is this a fair assumption?"
No, the IP's doesn't depend on a router, the router only routes to a computer/server which can have multiple IP's. By the other hand, each thing (computer, server, etc... must have an IP which can be also a WAN IP).
I'm a little confused about IP addresses.
I know that every web domain has an ip address.
Does the IP address represent the physical machine / host the website files are stored on?
Therefore when DNS lookup is performed, the domain's IP address is returned to the client. The client then uses this ip to contact the server that the web files reside on.
Is my understanding correct?
Many thanks
You are correct.
It is kind of like how some companies may say Dial PIZZAHUT instead of saying, Dial 74992488. PIZZAHUT is easier to remember, but you actually are dialling the number.
You're talking about HTTP protocol. Yes, FQDN (web domain) is resolved to IP address by DNS server. Client will connect to server IP address. Since you're probably using HTTP 1.1, HTTP request will contain also FQDN. This information is used by web server to perform several checks, like SSL certificate validation or Virtualhost management (several domains on a single IP address).
I have a basic question about DNS infrastructure.
I'm wondering how the IP addresses of upstream DNS servers are configured within DNS servers. For example, when my router needs to satisfy a DNS query on behalf of a machine on my LAN, it asks its upstream DNS server that it was given through DHCP. However, how does the upstream DNS server know how to reach the root DNS server or some authoritative DNS server if it doesn't have that information cached? Is the root DNS server's IP address hardcoded anywhere to achieve this? Are backbone DNS servers always configured with some DNS server upstream from it?
I recall setting up a Microsoft DNS server in which any requests that couldn't be satisfied by it would be forwarded. However, since an upstream DNS server wasn't configured, it forwarded those requests right to the root. This behavior makes sense, however, how did it know where to contact the root?
Your reasoning is correct.
Q: How does the upstream DNS server know how to reach the root DNS server or some authoritative DNS server if it doesn't have that information cached? Is the root DNS server's IP address hardcoded anywhere to achieve this?
A: Small scale DNS server (for example DNS server serving clients in one organization) will sometimes have (manually) configured forwarders (usually ISP nameservers) in order to benefit from big cache of ISPs nameservers and faster queries. From my experience, with faster internet links (and with less latency) in recent years, this setup is used less often. Instead, root hints are used.
Q: Is the root DNS server's IP address hardcoded anywhere to achieve this?
A: Yes. For Microsoft DNS server it is located in systemroot\System32\dns\cache.dns, for BIND it is usually in /etc/bind/db.root or /var/named/named.root. An updated copy (if needed) can be retrieved from https://www.internic.net/domain/db.cache
Q: Are backbone DNS servers always configured with some DNS server upstream from it?
A: As far as I know, never.
A recursive server has the (or at least a) list of root servers provided out-of-band. This is often called "root hints" or something similar. Once it knows how to talk to the root servers, everything else follows from that. In practice, a recursor will quite quickly come to cache the name server addresses for the more common TLDs (like .COM and .ORG), so it doesn't always have to start at the root. But the root server addresses are manually provided to start things off.
I am at a large University, with servers set up on the Univeristy network. The network has internal (10...) IP addresses, and external (129...) IP addresses. I have a ColdFusion-based business process which FTP's data from an external server.
The server on which the ColdFusion instance is based has several (about 10) IP addresses associated with it, both internal and external. It has one primary address, which has been, until recently, external.
Using Wireshark, we have been able see which IP is used, and it is never the primary, but other than that, does not seem to follow any logic.
We recently changed the primary IP for the server to an internal address, to comply with new University-wide security policies, and the FTP connection from Coldfusion stopped working. Using Wireshark, we've confirmed it is going out on one of the internal (10...*) IP addresses (not the primary).
Is there a way to control which IP ColdFusion uses for an FTP connection, either through system configuration, or programmatically?
Code for the FTP call:
var ftpService= new ftp(
username = partnerConfig.sftpLogin,
connection = "MyConnection",
password = partnerConfig.sftpPw,
fingerprint = partnerConfig.sftpFingerprint,
server = partnerConfig.sftpServer,
secure = "yes"
);
var result=ftpService.open();
var result2=ftpService.listdir(directory = partnerConfig.inFolder, name="dirlist");
var result3=result2.getResult();
EDIT: My server guy tells me that this University is fairly unique in how it configures the subnets. NO firewall exceptions are allowed from the 10.* subnet. The "Public" IP's are not public by default, simply the ones which are capable of having exceptions.
FURTHER EXPLANATION: It comes down to the Rules and IP addresses the main IT org at the university set up. the 10.x.x.x IPs are never allowed access outside the university ("internal"). 129.x.x.x IP, may have firewall exceptions ("external"). If I type ipconfig (it's Windows), I see about 10 static IP addresses, half of which are 10.x.x.x, and half of which are 129.x.x.x. When the "primary" IP for the machine was a 129.x.x.x address, Coldfusion chose one of the OTHER 129.x.x.x addresses for the outgoing IP for the connection. With the "primary" IP switched to a 10.x.x.x IP, ColdFusion is choosing one of the OTHER 10.x.x.x addresses for the outgoing connection. My choices, as far as I can tell are 1) Switch the primary back to a 129.x.x.x for the server; 2) Move the site to a different server with a primary of 129.x.x.x, and keep other sites on the server in compliance with the new policy; 3)See if I can figure out how to control which IP Coldfusion chooses from the 10 static IPs on the server.
Our admin figured it out. We needed to add a static route to the server, so that whenever ColdFusion (or any application) tries to connect to the specific destination we were FTPing to, it goes through a specific outgoing IP address.
This was not something I was familiar with, but a quick search gives the basics: http://technet.microsoft.com/en-us/library/dd469825.aspx
You have an internal IP address and you are not sure what the external is? This sound like a network bridge issue. What you are looking for is FTP proxy or some tool that associates an externally assigned IP to an internally assigned IP for port 22 (or whatever you set your port to). Look at: http://wiki.squid-cache.org/Features/FtpGateway
FTP reverse proxy
It sounds like your IPs (even internally may not be static)...or maybe it just feels that way. Make sure your internal IP is static.
Read about FTP port forwarding: Here are some port forwarding guides.
Other keywords FTP bridge, FTP one-to-one mapping, among others.
Good luck.
I had a similar question regarding the cfmail tag:
Force cfmail tag to send from a specfic server IP address
Unfortunately, I didn't get an answer that worked. ColdFusion seems to pick (seemingly at random) which source IP address is used.
Am I able to depend on a requestor's IP coming through on all web requests?
I have an asp.net application and I'd like to use the IP to identify unauthenticated visitors. I don't really care if the IP is unique as long as there is something there so that I don't get an empty value.
If not I guess I would have to handle the case where the value is empty.
Or is there a better identifier than IP?
You can get this from Request.ServerVariables["REMOTE_ADDR"].
It doesn't hurt to be defensive. If you're worried about some horrible error condition where this isn't set, check for that case and deal with it accordingly.
There could be many reasons for this value not to be useful. You may only get the address of the last hop, like a load balancer or SSL decoder on the local network. It might be an ISP proxy, or some company NAT firewall.
On that note, some proxies may provide the IP for which they're forwarding traffic in an additional HTTP header, accessible via
Request.ServerVariables["HTTP_X_FORWARDED_FOR"]. You might want to check this first, then fall back to Request.ServerVariables["REMOTE_ADDR"] or Request.UserHostAddress.
It's certainly not a bad idea to log these things for reference/auditing.
I believe that this value is set by your web sever and there is really no way to fake it as your response to there request wouldn't be able to get back to them if they set there IP to something else.
The only thing that you should worry about is proxies. Everyone from a proxy will get the same IP.
You'll always get an IP address, unless your web server is listening on some sort of network that is not an IP network. But the IP address won't necessarily be unique per user.
Well, web request is an http connection, which is a tcp connection and all tcp connections have two endpoints. So, it always exists. But that's about as much as you know about it. It's neither unique nor reliably accurate (with all the proxies and stuff).
Yes, every request must have an IP address, but as stated above, some ISP's use proxies, NAT or gateways which may not give you the individual's computer.
You can easily get this IP (in c#) with:
string IP = Context.Request.ServerVariables["REMOTE_ADDR"].ToString();
or in asp/vbscript with
IP = request.servervariables("REMOTE_ADDR")
IP address is not much use for identifying users. As mentioned already corporate proxies and other private networks can appear as a single IP address.
How are you authenticating users? Typically you would have them log in and then store that state in their session in your app.