I'm trying to understand UDP hole punching and I just don't quite get it.
In concept it seems simple but when I put it into practice I can't pull it off.
From what I understand there's a public server we call the hole-punch server. A client makes a request to hole-punch server (this is public). The hole-punch server spits out a public ip and port of the client that just made the request. So long as that port is open then essentially any random client can make a request to that client using that specific port and ip ?
The issue I guess I'm having is, the client is able to make a request to the server. The server is able to send data back to the client on that public port and ip however when another client tries to send a request to that client using that same port and ip it just doesn't go through and that's what's confusing me. If the server can make the request why can't another random client make that request?
The thing to know about UDP hole-punching is that many consumer-grade Internet routers/NAT-firewalls have a policy along the lines of "block any incoming UDP packets, except for UDP packets coming from an IP address that the user's local computer has recently sent a UDP packet to"; the idea being that if the local user is sending packets to a particular IP address, then the packets coming back from that same IP address are probably legitimate/desirable.
So in order to get UDP packets flowing between two firewalled/NAT'd computers, you have to get each of the two computers to first send a UDP packet to the other one; which is a bit of a chicken-and-egg problem since they can't know where to send the UDP packet without being able to communicate; the public server is what solves that problem. Since that server is public, both clients can communicate with the server (via UDP or TCP or HTTP or whatever), and that server can tell each client the IP address and port to send its UDP packets to. Once each client has sent some initial packets to the other, it should also (in most cases) then be able to receive UDP packets from the other client as well, at which point the server is no longer necessary as a go-between.
Related
Browser can send a request to web server and get response, it is easy to understand because every domain does resolve to one or more public network IP, browser can find web servers via public network IP.
Some clients have public IP like PPPOE, so, when I establish a Websocket connection between browser and server, server can send data to browser on a device that owns a public network IP device, but not every client has public network IP.
My question is how does server find browser without public IP in a lan via Websocket?
Part of the magic is Network Address Translation and is performed by the routers between the server and the web browser's computer.
The simple answer is the server never has to find the client because once a browser sends a request to the server and a TCP connection is established, that connection can be maintained for as long as necessary.
The TCP protocol has the keepalive concept where every so often a peer sends a probe packet to the client with no data expecting and ACK packet in response. This way the connection remains alive despite network inactivity and can be terminated where peer does not reply.
The WebSocket Protocol, a TCP-based protocol also defines a similar concept Ping/Pong in which either of the peers can send a Ping probe on demand once a connection is established. The peer must respond to a Ping frame with a Pong frame as soon as is practical. This checks for dead peers, in which case the connection would be dead.
When a firewall lets packets sent in response to a request pass, how does it know that certain packets are associated with a request sent from inside the network?
Does it keep track of TCP connections or does it use some kind of a session?
For example, my browser makes a request that goes through the firewall in the router and the firewall knows to let response packets through. On the other hand, if someone from outside makes a request to any computer inside my network, they will get blocked.
For TCP, it does keep track of connections.
For UDP, each outgoing packet creates (or refreshes) a temporary rule to forward inbound packets back to the local source of the outgoing UDP packet.
Given that you have multiple web browsers running, all which obviously listen on port 80, how would a browser figure if an incoming HTTP response was originated by itself? And whether or not catch the response and show it?
As part of the connection process a TCP/IP connection is assigned a client port. Browsers do not "listen on port 80"; rather a browser/clients initiate a request to port 80 on the server and waits for a reply on the client port from the server's IP.
After the client port is assigned (locally), each client [TCP/IP] connection is uniquely identified by (server IP, server port, client IP, client port) and the connection (and response sent over such) can be "connected back" to the correct browser. This same connection-identifying tuple is how a server doesn't confuse multiple requests coming from the same client/IP1
HTTP sits on top of the TCP/IP layer and doesn't have to concern itself with mixing up connection streams. (HTTP/2 introduces multiplexing, but that is a different beast and only affects connection from the same browser.)
See The Ephemeral Port Range for an overview:
A TCP/IPv4 connection consists of two endpoints, and each endpoint consists of an IP address and a port number. Therefore, when a client user connects to a server computer, an established connection can be thought of as the 4-tuple of (server IP, server port, client IP, client port). Usually three of the four are readily known -- client machine uses its own IP address and when connecting to a remote service, the server machine's IP address and service port number are required [leaving only the client port unknown and to be automatically assigned].
What is not immediately evident is that when a connection is established that the client side of the connection uses a port number. Unless a client program explicitly requests a specific port number, the port number used is an ephemeral port number. Ephemeral ports are temporary ports assigned by a machine's IP stack, and are assigned from a designated range of ports for this purpose. When the connection terminates, the ephemeral port is available for reuse, although most IP stacks won't reuse that port number until the entire pool of ephemeral ports have been used. So, if the client program reconnects, it will be assigned a different ephemeral port number for its side of the new connection.
See TCP/IP Client (Ephemeral) Ports and Client/Server Application Port Use for an additional gentle explanation:
To know where to send the reply, the server must know the port number the client is using. This [client port] is supplied by the client as the Source Port in the request, and then used by the server as the destination port to send the reply. Client processes don't use well-known or registered ports. Instead, each client process is assigned a temporary port number for its use. This is commonly called an ephemeral port number.
1 If there are multiple client computers (ie. different TCP/IP stacks each assigning possibly-duplicate ephemeral ports) using the same external IP then something like Network Address Translation must be used so the server still has a unique tuple per connection:
Network address translation (NAT) is a methodology of modifying network address information in Internet Protocol (IP) datagram packet headers while they are in transit across a traffic routing device for the purpose of remapping one IP address space into another.
thank you all for answers.
the hole listening thing over port 80 was my bad,I must have been dizzy last night :D
anyway,as I have read HTTP is connectionless.
browser initiates an HTTP request and after a request is made, the client disconnects from >the server and waits for a response. The server process the request and re-establish the >connection with the client to send response back.
therefor the browser does not maintain connection waiting for a response.so the answer is not that easy to just send the response back to the open socket.
here's the source
Pay attention browesers aren't listening on specific port to receive HTTP response. Web server listening on specific ports (usually 80 or 443). Browser open connection to web server, and send HTTP request to web server. Browser don't close connection before receive HTTP response. Web server writes HTTP response on opened connection.
Given that you have multiple web browsers running, all which obviously listen on port 80
Not obvious: just wrong. The HTTP server listens on port 80. The browsers connect to port 80.
how would a browser figure if an incoming HTTP response was originated by itself?
Because it comes back on the same connection and socket that was used to send the request.
And whether or not catch the response and show it?
Anything that comes back on the connected socket belongs to the guy who connected the socket.
And in any case all this is the function of TCP, not the browser.
As some one mentioned in other forum that interviewer has asked the question given below.
I dont know exact answer but I would say HTTP request ? Any suggestion and explainations
Imagine a user sitting at an Ethernet-connected PC. He has a browser open. He types "www.google.com" in the address bar and hits enter.
Now tell me what the first packet to appear on the Ethernet is .
Thanks
There's no guaranteed always-correct answer, but there are a few likely possibilities.
If the client is configured for DNS over UDP, then the first packet will be a UDP datagram containing a DNS query to resolve www.google.com to an IP address.
If the client is configured for DNS over TCP and the browser hasn't already got an established TCP connection to the DNS server, the first packet will be part of the connection handshake to DNS, and therefore the answer will be that a SYN packet is first out of the gate.
If the browser has been coded to maintain a long-lived TCP connection to the DNS server and assuming the DNS server has allowed the connection to stay alive, the first packet will be a DNS query, sent across the existing connection to that DNS server.
Finally, if the browser had recently visited www.google.com recently and is built to do some smart local caching of DNS query results then the first packet will be a SYN to establish a new connection to Google's web server.
If you want to be glib but absolutely precise about it, drop down a layer for your answer and say, "The first packet out will be an Ethernet frame containing a payload which supports whatever higher-level protocol is needed for the browser to serve up www.google.com". In fairness, the question is about the Ethernet layer...
Strictly speaking, with a completely blank slate, the first packet sent will be an ARP broadcast request ("Who has?") from the client PC attempting to discover the MAC address of its default gateway (or of its DNS server if that is on the same subnet as the client).
Interesting :) I just wiresharked it:
Client sends a SYN
Server replies with a SYN,ACK
Client sends an ACK
Client sends an HTTP GET
(like you mention in your comments the first is obviously the DNS lookup)
From what I understand, each HTTP request uses its own TCP connection (please correct me if i'm wrong). So, let's say that there are two current connections to the same server. For example, client side javascript code triggering a couple of AJAX POST requests using the XMLHttpRequest object, one right after the other, before getting the response to the first one. So we're talking about two connections to the same server, each waiting for a response in order to route it to each separate callback function.
Now here's the thing that I don't understand: The TCP packet includes source and destination ip and port, but won't both of these connections have the same src and dest ip addresses, and port 80? How can the packets be differentiated and routed to appropriately? Does it have anything to do with the packet sequence number which is different for each connection?
When your browser creates a new connection to the HTTP server, it uses a different source port.
For example, say your browser creates two connections to a server and that your IP address is 60.12.34.56. The first connection might originate from source port 60123 and the second from 60127. This is embedded in the TCP header of each packet sent to the server. When the server replies to each connection, it uses the appropriate port (e.g. 60123 or 60127) so that the packet makes it back to the right spot.
One of the best ways to learn about this is to download Wireshark and just observe traffic on your own network. It will show you this and much more.
Additionally, this gives insight into how Network Address Translation (NAT) works on a router. You can have many computers share the same IP address and the router will rewrite the request to use a different port so that two computers can simultaneously connect to places like AOL Instant Messenger.
They're differentiated by the source port.
The main reason for each HTTP request to not generate a separate TCP connection is called keepalives, incidentally.
A socket, in packet network communications, is considered to be the combination of 4 elements: server IP, server port, client IP, client port. The second one is usually fixed in a protocol, e.g. http usually listen in port 80, but the client port is a random number usually in the range 1024-65535. This is because the operating system could use those ports for known server protocols (e.g. 21 for FTP, 22 for SSH, etc.). The same network device can not use the same client port to open two different connections even to different servers and if two different clients use the same port, the server can tell them apart by their IP addresses. If a port is being used in a system either to listen for connection or to establish a connection, it can not be used for anything else. That's how the operating system can dispatch packets to the correct process once received by the network card.