http header : Connection: Keep-Alive
After reading alot about this , I still can't understand how its working.
wiki :
A keepalive signal can
also be used to indicate to Internet infrastructure that the
connection should be preserved. Without a keepalive signal
intermediate NAT-enabled routers can drop the connection after
timeout.
I dont udnerstand :
a Server can have 1,000,000 cuncorrent connections .
John sends a request to the server.
Paul's compter is on the same lan near paul. paul also sends a request to the same server.
John's and paul organization is behind router.
How the hell the server knows how to keep connection alive for both paul and john ?
Also , when john sends request the second time , it "doesnt open a new conneciton" , so how does keep-alive is applied here ?
First of all, TCP/IP connection is not some thin wire that is temporarily connecting two computers. At the end of the day both TCP/IP and UDP are just a series of packets. It's the operating system that pretends you have a connection by putting the IP packets back together in correct order.
Now back to your question. Note that the problem is not really HTTP-specific, all of this works on TCP/IP layer. Say Paul has 192.168.0.100 and John has 192.168.0.101 internal IP addresses while NAT has public 1.2.3.4 address. When Paul connects to some server, his OS uses 192.168.0.100:54321 address (port is chosen randomly by OS). This request hits NAT which remembers that address and forwards request to external server. The external server sees 1.2.3.4:4321 (notice the different port) as the user is behind the NAT so internal IP is not visible.
When the external server (let it be web server) sends a reply, it sends it to 1.2.3.4:4321. NAT, on the other hand, remembers that 4321 port should be forwarded to 192.168.0.100:54321` - and so it is.
Now imagine John sends request to the same server. This TCP/IP connection is routed through NAT which remember that request from 192.168.0.101:32123 was made. This request is then forwarded using public 1.2.3.4:4322 (notice different port). When response arrives, NAT checks the port if it is 4322, it routes to 192.168.0.101:32123 (John). Otherwise (on port 4321) Paul will get his reply.
Note: do not confuse client ephemeral port with server port (80 in HTTP by default).
Related
The goal is to make an http request from the client browser to my server. Simple stuff; however I'm hitting a wall with the networking portion. In order to expose my server to WAN I have used one of my public IPs and NAT to translate to the private ip of my server on inbound traffic and to my public IP on outbound traffic.
The issue is that I can't make a connection. Specifically I can't get the last part of the TCP handshake. Using a test setup with Wireshark on the client and server I can see that the client send the SYN -> the server receives the SYN -> the server sends a SYN/ACK -> the client receives a SYN/ACK -> the the client send an ACK -> the server DOES NOT receive the ACK. It waits for a moment then does a retransmission. Eventually resetting.
I have tried adding various firewall rules even though I don't think it could be the firewall because the first packets make a successful round trip.
I've turned windows firewall off(the server)
I've tried disabling TCP checksum offloading
I've looked for network anti virus settings on the server and on the sonic wall(the router)
I would expect the the tcp connection to complete. I can't for the life of me think of a reason why the ACK would consistently go missing.
That is another thing. The behavior is consistent.
pings also work just fine.
NOTE: The server is actually a VM and the physical server that manages it is in my network.
Any guidance on what to try and where to look would be very much appreciated. Thanks.
UPDATE: I can make connection using port 5000(It's another port I have opened on the firewall). Port 80 still doesn't work though.
In my case this was caused by COX not allowing inbound traffic to port 80. I'm not sure why the first portions of the tcp handshake were getting through. If anyone can explain that part leave a comment.
If server A receives a UDP packet, can it forward it to server B and have server B directly respond to the end client without going back through server A since UDP is connectionless?
My gut feeling is no, unless both server A and server B have the same IP address (anycast maybe?)
But if the client is built to also be a UDP server, it could work in theory since the (logical) response from server B would just look like a new request. But then this really doesn't work in practice because of firewalls and NAT, for the same reason it's hard to host a game server from behind your firewall, router, and ISP.
If server A receives a UDP packet, can it forward it to server B and have server B directly respond to the end client without going back through server A since UDP is connectionless?
Yes, it can. However, there are a few caveats:
NAT devices, which most client are behind of. There are several types of NAT devices. A full-cone NAT will accept the redirected packet just fine, but an address-restricted NAT would require the packet to return from the same address it was sent to (hence, your anycast idea would work, as well as simply having server B spoof the IP of server A). A port restricted or symmetric NAT would also require the same source port (i.e. server B has to send the response from the same port server A received it from).
Other types of network middleboxes, such as firewalls might block the returning packet
Application - will the client application accept a response from a different IP address and port?
Browser can send a request to web server and get response, it is easy to understand because every domain does resolve to one or more public network IP, browser can find web servers via public network IP.
Some clients have public IP like PPPOE, so, when I establish a Websocket connection between browser and server, server can send data to browser on a device that owns a public network IP device, but not every client has public network IP.
My question is how does server find browser without public IP in a lan via Websocket?
Part of the magic is Network Address Translation and is performed by the routers between the server and the web browser's computer.
The simple answer is the server never has to find the client because once a browser sends a request to the server and a TCP connection is established, that connection can be maintained for as long as necessary.
The TCP protocol has the keepalive concept where every so often a peer sends a probe packet to the client with no data expecting and ACK packet in response. This way the connection remains alive despite network inactivity and can be terminated where peer does not reply.
The WebSocket Protocol, a TCP-based protocol also defines a similar concept Ping/Pong in which either of the peers can send a Ping probe on demand once a connection is established. The peer must respond to a Ping frame with a Pong frame as soon as is practical. This checks for dead peers, in which case the connection would be dead.
Given that you have multiple web browsers running, all which obviously listen on port 80, how would a browser figure if an incoming HTTP response was originated by itself? And whether or not catch the response and show it?
As part of the connection process a TCP/IP connection is assigned a client port. Browsers do not "listen on port 80"; rather a browser/clients initiate a request to port 80 on the server and waits for a reply on the client port from the server's IP.
After the client port is assigned (locally), each client [TCP/IP] connection is uniquely identified by (server IP, server port, client IP, client port) and the connection (and response sent over such) can be "connected back" to the correct browser. This same connection-identifying tuple is how a server doesn't confuse multiple requests coming from the same client/IP1
HTTP sits on top of the TCP/IP layer and doesn't have to concern itself with mixing up connection streams. (HTTP/2 introduces multiplexing, but that is a different beast and only affects connection from the same browser.)
See The Ephemeral Port Range for an overview:
A TCP/IPv4 connection consists of two endpoints, and each endpoint consists of an IP address and a port number. Therefore, when a client user connects to a server computer, an established connection can be thought of as the 4-tuple of (server IP, server port, client IP, client port). Usually three of the four are readily known -- client machine uses its own IP address and when connecting to a remote service, the server machine's IP address and service port number are required [leaving only the client port unknown and to be automatically assigned].
What is not immediately evident is that when a connection is established that the client side of the connection uses a port number. Unless a client program explicitly requests a specific port number, the port number used is an ephemeral port number. Ephemeral ports are temporary ports assigned by a machine's IP stack, and are assigned from a designated range of ports for this purpose. When the connection terminates, the ephemeral port is available for reuse, although most IP stacks won't reuse that port number until the entire pool of ephemeral ports have been used. So, if the client program reconnects, it will be assigned a different ephemeral port number for its side of the new connection.
See TCP/IP Client (Ephemeral) Ports and Client/Server Application Port Use for an additional gentle explanation:
To know where to send the reply, the server must know the port number the client is using. This [client port] is supplied by the client as the Source Port in the request, and then used by the server as the destination port to send the reply. Client processes don't use well-known or registered ports. Instead, each client process is assigned a temporary port number for its use. This is commonly called an ephemeral port number.
1 If there are multiple client computers (ie. different TCP/IP stacks each assigning possibly-duplicate ephemeral ports) using the same external IP then something like Network Address Translation must be used so the server still has a unique tuple per connection:
Network address translation (NAT) is a methodology of modifying network address information in Internet Protocol (IP) datagram packet headers while they are in transit across a traffic routing device for the purpose of remapping one IP address space into another.
thank you all for answers.
the hole listening thing over port 80 was my bad,I must have been dizzy last night :D
anyway,as I have read HTTP is connectionless.
browser initiates an HTTP request and after a request is made, the client disconnects from >the server and waits for a response. The server process the request and re-establish the >connection with the client to send response back.
therefor the browser does not maintain connection waiting for a response.so the answer is not that easy to just send the response back to the open socket.
here's the source
Pay attention browesers aren't listening on specific port to receive HTTP response. Web server listening on specific ports (usually 80 or 443). Browser open connection to web server, and send HTTP request to web server. Browser don't close connection before receive HTTP response. Web server writes HTTP response on opened connection.
Given that you have multiple web browsers running, all which obviously listen on port 80
Not obvious: just wrong. The HTTP server listens on port 80. The browsers connect to port 80.
how would a browser figure if an incoming HTTP response was originated by itself?
Because it comes back on the same connection and socket that was used to send the request.
And whether or not catch the response and show it?
Anything that comes back on the connected socket belongs to the guy who connected the socket.
And in any case all this is the function of TCP, not the browser.
I thought I understood the whole thing about NAT etc but now I came to a problem.
First what I assumed:
Because there are not enough IPv4 addresses available we need another system.
The devices of today at home for connecting to the internet are a combination of:
1) A modem at the physical-level to change the type of signals on the wire.
2) A switch at link-level so you can connect multiple computers to the device
3) A router to connect all the computers to the internet and go beyond your home-subnet etc.
4) A NAT to allow all the internal computers to connect to the outside
5) A portforwarder to let connections from the outside to the internal network
What I call a NAT:
When making a request to the outside: the NAT-part of the device changes the source-port and the source-ip of the request coming from an internal computer. The new source-ip will be your public-ip. The NAT-part will hold a record in a table with this mapping: "original-ip, original-port, new-port".
When a reponse comes back, the NAT will check the destination-port and compare this with the new-ports is in his table. If it finds a match the NAT will replace the destination-ip with original-ip and new-port with original-port. As a consequence the response will be forwarded to the internal computer that made the request.
So, the NAT-part is for when a connection is initialized from the inside. When this request traverses the NAT, 2 things are changed: source-ip and source-port.
Then the portforwarder:
This part of the device will accept connections initialized in the outside-world to your network. It will look at the destination-port of the incoming request and by making a rule for that port-number it may change the destination-port and the destination-ip of the request to an internal ip. With these rules a request from the outside can connect to a computer on your internal network and thus the portforwarder changes 2 things: the destination-ip and the destination-port.
A: Before I ask my question, how is this explanation?
Now my problem is with the response after a request came from the outside through the portforwarder. Assume the right rules are made and a request came through portforwarding on an internal computer. So in the portforwarder the destination-ip was changed to the internal-ip of the computer and the destination-port was changed to the port where the service is running on. If this internal-computer is a webserver it will generate a response. So the destination-ip will be the request's source-ip and the destination-port will be the request's source-port. The source-ip will be the internal-ip of the computer and the source-port will be the port of the service.
Now that response has to go to the outside. So I assume it goes through the NAT to the outside?
So after passing the NAT, the source-ip will be the public-ip and the source-port will be random. Now I tested this with wireshark. I contacted a webserver behind a NAT and I saw the reponse was coming from port 80 ?! How is this possible? This indicates that the response of the forwarded request did not pass the NAT?
I rethought the concept and my new hypothesis is that when a connection is initialized from the outside, it will pass the portforwarder and reach the right computer. This will create a response and when this response reaches our "all-in-one"device, this device can recognize it forwarded the request of the response and will not change the source-port.
B: Is this indeed the case or is it done in another way?
Wikipedia says about portforwarding: "The source address and port are, in this case, left unchanged. When used on machines that are not the default gateway of the network, the source address must be changed to be the address of the translating machine, or packets will bypass the translator and the connection will fail." (http://en.wikipedia.org/wiki/Port_forwarding)
This confirms that the response of a forwarded request MUST go through the portforwarder again and not through the NAT so the source-port wont be changed. The portforwarder will change the source-ip to the public-ip.
Can someone verify this or give me another explanation than mine?
Now I tested this with wireshark. I contacted a webserver behind a NAT
and I saw the reponse was coming from port 80 ?! How is this possible?
This indicates that the response of the forwarded request did not pass
the NAT?
The webserver inside the NAT does not have to be running on port 80. It certainly is set up at the NAT to port forward and respond as if it were at port 80, but that doesn't mean much about the port the web server is actually running on.
Here is some ASCII "art" that may help.
**Internal Network** **NAT Router** **External Computer**
Web Server running at IP 9.9.9.9 port 80 IP 20.20.20.20
IP 192.168.1.7 port 4567
Request web page at 9.9.9.9:80
Forwards port 80 traffic
to 192.168.1.7:4567
Replies with the web page
Puts 9.9.9.9:80 in the
source field and sends
the page on
Gets the page from "9.9.9.9:80"
even though it actually came
from 192.168.1.7:4567