When I type www.google.com at my browser address bar, what exactly happens technically and how entire stuff is loaded. Considering the same HTTP page is being loaded...
what is role of DNS server, IP address, MAC address, subnet mask, proxy setting, default gateway in this case.
Does it make any different if I am in different class of network?
You are asking about all things at once, it's a big concept.
Still in short.
When you type www.google.com (or any other site name) then the request goes to the DNS server which translates the URL into an IP address.
Read here more: http://en.wikipedia.org/wiki/Domain_Name_System
Then the request goes to server where the website is being hosted, the server which is providing hosting service for the website contains the website-stuff that has to be shown to the world.
Read about apache server: http://en.wikipedia.org/wiki/Apache_HTTP_Server
Subnet: http://en.wikipedia.org/wiki/Subnetwork
does it make any different if i am in different class of network?
No, it doesn't make any difference if you are in different class of network.
(Study about routers: http://en.wikipedia.org/wiki/Routers)
Points to help you out:
Every computer that belongs to a network -including yours has an ip address.
Every network has hosts under it. The network may be divided into subnets
The ip addressing is hierarchial.This helps in routing
IP addresses may be assigned manually or by the DHCP server
Manual-IP configuration
DHCP-Dynamic Host Configuration Protocol
All packets that are sent to your ip address come through your isp network - this includes switches and routers
Packets from other networks are forwarded by this ip address. Once they reach the nearest switch. Switches use your MAC address to send packets to your computer. The MAC address is obtained by ARP
The gateway address is the path through which packets are sent out of your network or your ISP's
Proxy servers are servers that allow connections through them
To understand more about how this works, download Wireshark:
Start the sniffer and then load google.com in your browser.
You will notice the following
The browser first sends a DNS request with the hostname to the DNS server of your ISP (or your network if any)- DNS finds out the ip address from the hostname.
The DNS server replies with the IP address of the server
The browser then sends the HTTP request
This is in the form e.g
GET /index.html HTTP/1.1
The server responds with the format
The data is sent to the user
Usually if a webpage is requested, it is in HTML format(with javascript,css,etc). This is then parsed and processed by the browser to get the webpage we see.
To test this :
ON LINUX,
type
telnet stackoverflow.com 80
in the terminal.
As soon as it gets connected,type the following (quickly before it gets disconnected):
GET /index.html HTTP/1.1 (enter)
Host: stackoverflow.com (enter)(enter)
to see the response
ON WINDOWS
Download the putty client and fill the host as stackoverflow.com, port as 80, and choose Connection type as Telnet.
As soon as it gets connected, repeat same steps as above to see the response.
The examples shown above illustrate how things work from Layer 7, but not Layer 3+ from your device's perspective. I would look at using tcpdump/wireshark to try and dump all of the network packets if you're interested in getting those kinds of low-level details. An example's provided below (run on FreeBSD).
Notes:
- Be sure to start wireshark/tcpdump before your web browser/client first so the packets get captured.
- Specify the right port when starting wireshark/tcpdump; filtering the connection via a DNS address might not work in all cases if the remote webserver has load balancing/failover setup.
Window with tcpdump:
# tcpdump -A tcp port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes
capability mode sandbox enabled
01:48:17.917640 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [P.], seq 1631738201:1631738227, ack 30720002, win 65535, length 26
E..BP.#.#...
...J}.t...PaBYY....P...F5..GET /index.html HTTP/1.1
01:48:17.918119 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 26, win 65535, length 0
E..(.U..#..{J}.t
....P......aBYsP...'+........
01:48:18.072501 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [P.], seq 26:28, ack 1, win 65535, length 2
E..*P.#.#...
...J}.t...PaBYs....P...F...
01:48:18.072662 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 28, win 65535, length 0
E..(.X..#..xJ}.t
....P......aBYuP...')........
01:48:18.074353 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [P.], seq 1:687, ack 28, win 65535, length 686
E....Y..#...J}.t
....P......aBYuP...Z...HTTP/1.0 400 Bad request: request header 'Host' missing
Content-type: text/html; charset="iso-8859-1"
<html>
<body>
<h3> Request denied by WatchGuard HTTP proxy. </h3>
<b> Reason: </b> request header 'Host' missing <br>
<hr size="1" noshade>
<b> Method: </b> GET <br>
<b> Host: </b> 74.125.239.116 <br>
<b> Path: </b> /index.html <br>
<hr size="1" noshade>
</body>
<!-- PAD --></html>
01:48:18.074512 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [F.], seq 687, ack 28, win 65535, length 0
E..(.Z..#..vJ}.t
....P......aBYuP...$z........
01:48:18.074683 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [.], ack 688, win 65014, length 0
E..(P.#.#...
...J}.t...PaBYu....P...F...
01:48:18.077023 IP 10.0.2.15.50636 > nuq05s01-in-f20.1e100.net.http: Flags [F.], seq 28, ack 688, win 65535, length 0
E..(P.#.#...
...J}.t...PaBYu....P...F...
01:48:18.077070 IP nuq05s01-in-f20.1e100.net.http > 10.0.2.15.50636: Flags [.], ack 29, win 65535, length 0
E..(.[..#..uJ}.t
....P......aBYvP...$y........
Window with telnet:
# telnet www.google.com 80
Trying 74.125.239.116...
Connected to www.google.com.
Escape character is '^]'.
GET /index.html HTTP/1.1
HTTP/1.0 400 Bad request: request header 'Host' missing
Content-type: text/html; charset="iso-8859-1"
<html>
<body>
<h3> Request denied by WatchGuard HTTP proxy. </h3>
<b> Reason: </b> request header 'Host' missing <br>
<hr size="1" noshade>
<b> Method: </b> GET <br>
<b> Host: </b> 74.125.239.116 <br>
<b> Path: </b> /index.html <br>
<hr size="1" noshade>
</body>
<!-- PAD --></html>
Connection closed by foreign host.
Related
I just recently learned the Scapy part and currently want to use Scapy for the TCP handshack test.
I currently use mininet to create two and execute the following code:
from scapy.all import *
ip=IP(src="10.0.0.1", dst="10.0.0.41")
TCP_SYN=TCP(sport=433, dport=443, flags="S", seq=100)
TCP_SYNACK=sr1(ip/TCP_SYN)
TCP_ACK=TCP(sport=433, dport=443, flags="A", seq=101, ack=TCP_SYNACK.seq + 1)
send(ip/TCP_ACK)
host 2 replied:
enter image description here
But host 2 did not reply to the SYN-ACK packet, even if host 2 was allowed to execute the following code
python3 -m http.server 80
Even if dport is changed to 80, host 1 will receive SYN-ACK from host 2, but host 1 will not reply with ACK
I have an HTTP server and a client (I use curl for testing). The problem I have is that even the simplest, "ping" request takes 3-5 seconds to accomplish. I have run tcpdump on the server and it seems that after establishing TCP connection, the client (curl) waits 3-4 seconds for sending the HTTP request:
09:06:55.079376 IP client > server: Flags [S], seq 1639466412, win 64240, options [mss 1358,nop,wscale 8,nop,nop,sackOK], length 0
09:06:55.079457 IP server > client: Flags [S.], seq 2519785093, ack 1639466413, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
09:06:55.087145 IP client > server: Flags [.], ack 1, win 1029, length 0
09:06:59.722060 IP client > server: Flags [P.], seq 1:83, ack 1, win 1029, length 82: HTTP: GET /ping HTTP/1.1
You can see the problem between 3 and 4 line of the tcpdump output.
Do you know what can cause this issue? Is this really on the client side or I should check also on the server side?
UPDATE
If I use telnet to connect to the server's port and send GET, the server replies immediately.
Flags [S] Syn
Flags [S.] - Syn Ack
Flags [.] - Placeholder, usually used for ACK.
Flags [P.] - Push Ack
Can you prepend time to your curl command? as shown below.
These hosts, is there latency between the two?
As a test on the same host (terminal):
terminal a $ python -m SimpleHTTPServer 8989
terminal b $ time curl -vvv localhost:8989
results look something like:
real 0m0.018s
user 0m0.006s
sys 0m0.005s
At this point its hard to say if this is client or server side, this depends on the data you are attempting to query. Can you share that query/curl command?
I am writing a pinging utility to check the connectivity. We have an IP range from X.X.X.0 - X.X.X.24
Ping X.X.X.08 - gives unknown host
Ping X.X.X.008- gives unknown host
Ping X.X.X.8 - gets successful response
How many digits should be there in the last octet?
But when I ping X.X.X.007 or X.X.X.07 or X.X.X.7 works, i get successful response.
Could some shade some light what i am missing?
This depends almost entirely on the implementation of your ping but the most likely cause is that the 08 is being treated as an octal number because it begins with 0.
And, since the valid octal digits are limited to 0..7, it assumes it's not a numeric IP address but instead a name to be looked up (in DNS, for example).
This octal behavior can be confirmed (under Windows) with the following transcript:
C:\Users\Pax> ping 192.168.1.061
Pinging 192.168.1.49 with 32 bytes of data:
Reply from 192.168.1.61: Destination host unreachable.
:
The 061 appears to have been treated as octal from the first line of the output, since 618 (6x8+1) = 4910. The subsequent lines, though they claim to be pinging the .61 address, are lying. That's because that's my actual machine and, if I do it without the leading zero, it works fine:
C:\Users\Pax>ping 192.168.1.61
Pinging 192.168.1.61 with 32 bytes of data:
Reply from 192.168.1.61: bytes=32 time<1ms TTL=128
:
If the octal bit contains a non-octal digit, that's when it starts complaining about the host itself rather than it just not being reachable (or, worse, pinging the wrong machine):
C:\Users\Pax>ping 192.168.1.61
Pinging 192.168.1.61 with 32 bytes of data:
Reply from 192.168.1.61: bytes=32 time<1ms TTL=128
:
C:\Users\Pax>ping 192.0168.1.61
Ping request could not find host 192.0168.1.61.
Please check the name and try again.
Normally, it doesn't matter if you have X.X.X.001 or .01 or .1 for the last octet.
See my ping results on 192.168.0.1 and .01 and .001 => they all result to .1
I am trying to get TCP. From TCP RFC 793 server and client select a random sequence numbers and after they increase it each time when it receives a new byte (it is wrong but just for example). To dump TCP packages I used tcpdump -n -i eth0 tcp:
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
04:32:20.732914 IP 10.10.0.2.43168 > 10.50.0.2.9: S 372254521:372254521(0)
win 5840 <mss 1460,sackOK,timestamp 3644068 0,nop,wscale 1>
04:32:20.766194 IP 10.50.0.2.9 > 10.10.0.2.43168: S 363863555:363863555(0)
ack 372254522 win 5792 <mss 536,sackOK,timestamp 3644074 3644068,nop,wscale 1>
04:32:20.766416 IP 10.10.0.2.43168 > 10.50.0.2.9: .
ack 1 win 2920 <nop,nop,timestamp 3644073 3644074>
04:32:25.502532 IP 10.10.0.2.43168 > 10.50.0.2.9: P 1:7(6)
ack 1 win 2920 <nop,nop,timestamp 3644548 3644074>
04:32:25.503272 IP 10.50.0.2.9 > 10.10.0.2.43168: .
ack 7 win 2896 <nop,nop,timestamp 3644548 3644548>
04:32:29.510131 IP 10.10.0.2.43168 > 10.50.0.2.9: F 7:7(0)
ack 1 win 2920 <nop,nop,timestamp 3644949 3644548>
04:32:29.513123 IP 10.50.0.2.9 > 10.10.0.2.43168: F 1:1(0)
ack 8 win 2896 <nop,nop,timestamp 3644949 3644949>
04:32:29.513356 IP 10.10.0.2.43168 > 10.50.0.2.9: .
ack 2 win 2920 <nop,nop,timestamp 3644949 3644949>
The first two packages look normally but from the third and so on it uses ack 1 instead of 363863556 and I can't get why?
It hasn't. You are running tcpdump without telling it that you want to see absolute sequence numbers (-S).
The default behavior for tcpdump is to translate sequence numbers to relative sequence numbers, which allow you to see how many bytes of data have transferred in either direction. In this specific case, you are seeing it go to 1 because, according to RFC-793, the SYN consumes one byte in the stream, so the proper response is SEQ+1. You will see the same thing happen going the other direction. (You will also find that the FIN consumes one byte). Following this, the ACK will increase by the number of bytes sent.
If you want to see the absolute sequence numbers, try again running tcpdump -n -i eth0 -S tcp
Using a Commodore 64 with a GLINK LT RS-232 adapter connected to a Digi-Connect SP configured to send RAW TCP to a static IP/port on my home network. Destination is a socat process passing traffic elsewhere with verbose logging.
Testing hitting socat from a bash shell on an OS X machine, issuing curl http://192.168.1.91:1234 I get a valid response and the socat log shows
> 2018/12/29 22:06:50.168550 length=81 from=0 to=80
GET / HTTP/1.1\r
Host: 192.168.1.91:1234\r
User-Agent: curl/7.54.0\r
Accept: */*\r
\r
< 2018/12/29 22:06:50.169509 length=144 from=0 to=143
(followed by http response)
But when I run the following on the C64 BASIC code SNIPPET:
100 OPEN 2,2,3,CHR$(6)+CHR$(0)
110 GET#2,A$: REM TOSS NULL TO OPEN RCVR CHANNEL
120 PRINT#2,"GET /"
...
the socat log shows:
> 2018/12/29 22:11:38.952005 length=1 from=167 to=167
G> 2018/12/29 22:11:38.983674 length=1 from=168 to=168
E> 2018/12/29 22:11:39.015464 length=1 from=169 to=169
T> 2018/12/29 22:11:39.051758 length=1 from=170 to=170
> 2018/12/29 22:11:39.084476 length=1 from=171 to=171
/> 2018/12/29 22:11:39.117131 length=1 from=172 to=172
I'm not sure if I'm doing something wrong on the C64 side to cause the individual char's to be sent or if it's an incorrect Digi-Connect setting.
Digi serial settings are:
TCP Client Settings:
Automatically establish TCP connections
Always connect and maintain connection
Establish connection to the following network service:
Server: 192.168.1.91
Service: Raw TCP
TCP Port: 1234
Enable TCP Keep-Alive: On
Basic Serial Settings
Baud: 300
Data bits: 8
Parity: None
Stop Bits: 0
Flow Control: Hardware
Advanced Serial Settings
(nothing configured here)
The individual characters was a red herring, the fix is simply to add a LF (ASCII 10) to the end of the message so the downstream web service understands that the message is finished.
Working example:
100 OPEN 2,2,3,CHR$(6)+CHR$(0)
110 PRINT#2,"GET /"
120 PRINT#2,CHR$(10)
(also of note - anything requiring correct upper/lower case will need PETSCII <-> ASCII conversions)