The description of the questions goes like this:
Someone recorded all the IP packets of a TCP connection between a client and a server for 30 minutes. In the record, he didn't find any packet that was ACK-only. How is this possible?
What is claimed to be a possible solution: For all the record time, the server sent data to the client, which the client processed, but he didn't send any data back to the server.
I am having trouble understanding how can it be possible.
From what I see, since the client didn't send any data to the server, and there weren't any ACK-only packets in the record, then the server didn't get any ACK from the client. Logically, I would think that since no ACK is received by the server, it will always do re-transmit. But also, since the server doesn't get anything from the client for 30 minutes, which seems like a long time for me, it will conclude that the connection is broken and stop it. (maybe even send an ACK only, but I am not sure about it).
Moreover, from what I know, when using keepalive, the sender gets and ACK-only packet from his peer.
Can anyone help me understand this?
Help would be appreciated
Perhaps more details would be helpful here. What kind of server/client? What protocol is being used and for what purpose?
Is the connection running as expected and this is just viewed as strange traffic you are trying to understand or is the connection timing out?
Some devices or softwares can be set to a "No ACK" state which means that no ACKs are sent nor are they expected.
One reason for this is usually bandwidth. ACKs do consume bandwidth and there are cases where bandwidth is such a great premium that packets being lost is preferable to bandwidth being consumed by ACKs. That type of traffic would probably do better with a UDP protocol but that is a completely different topic.
Another reason is that you don't care if packets are lost. Again, this would be better off as UDP instead of TCP, but someone may be trying to work within strange parameters is bending the rules about what type of traffic to advertise as in order to get around some issue.
Hopefully this is helpful, but if it does not apply, then please put in more details about the connection so that we can better understand what may be happening.
I'm making a multiplayer game and often i want to test out if it perfectly works on global network, because sometime it's just works locally, so how could i do that without sending my client to friend to test it out.
If you want to test it for the "global network" - you have to test it that way. There are multiple things that can go wrong which are not an issue on a local network. Just off the top of my head
latency (important for a game)
NAT (common cause of problems depending on your game architecture - more so if P2P)
security
connection errors (Wifi/3G intermittent loss)
There are many aspects of networking that are often subtly different when you're talking over the internet rather than running on either localhost or a local network.
All manner of delays can occur, and this can throw out some poor assumptions in your code.
The TCP flow can be affected by TCP's flow control (when the TCP Window fills up the sender will stop sending and report the fact to you (or maybe not report it, if you're using async APIs)).
TCP reads that return 'complete messages' on localhost and your own network may start to return pieces of a message.
UDP datagrams may go missing and never arrive or may arrive multiple times or in any sequence.
So you're right to think that you need to test your code for these edge cases which rarely show up on your own network. You're also right to think that simply sending a client to a friend, or running it on a remote machine, is not enough.
One approach is to build a dedicated test client which sends known game play and checks that it gets expected responses (how hard this is depends on your protocol and your game). Once you have that working you then have the test client deliberately send data in such a way that the items above are tested. So, if you're using UDP, you might put some code in your test client so that it sometimes doesn't bother to send a UDP datagram at all. The client should think it sent it. The networking layer simply ditches it. This tests your UDP protocol for missing datagrams. Then send some datagrams multiple times, then send some out of sequence, etc. For TCP add delays, break logical "messages" into separate network sends with large delays between them; ideally send each distinct message type as a sequence of single bytes to check that the server's 'message accumulation code' works correctly.
Once you have done this you need to do the same for your client code, perhaps by adding a "fuzzing" option to your server's network code to do the same kind of thing...
Personally I tend to try and take a step back and do as much of this as possible in dedicated 'unit tests' (I know that some people will say that these aren't unit tests, call them what you like, just write them!). These tests exercise your networking layer using real networking (talking to a dummy server/client that the test creates) and validate the horrible edge cases.
I work in web development, but I don't have a great understanding of network protocols. I recall hearing an analogy that TCP, HTTP, and SSL can be thought of as a series of nested envelopes around the actual request content.
I also have a fuzzy idea TCP consists of packets, which are verified on the other end. But I'm sort of picturing the HTTP request being chopped into packets, too...
So basically, I don't understand this stuff well at all. Can anybody give a good overview of this? Also, is there a beginner-friendly book or other resource that you'd recommend?
Since I asked this question, I've learned more about this topic, so I'll take a crack at answering it myself.
The easiest way to picture the protocol stack is as a letter, wrapped in a series of envelopes. Each envelope has a different role in getting the letter to its recipient, and envelopes are added and removed as needed along the journey.
The Application Layer
The letter itself is an application-layer request. For example, you've typed "StackOverflow.com" in your browser and pressed enter. Your browser needs to ask the StackOverflow server for its home page. So it writes a letter saying, "Dear StackOverflow, would you please send me your home page?"
If the writer of the letter is your browser, the recipient of the letter is the web server program running on StackOverflow. The browser wants the web server to "write back" with a response in the form of a web page. Both the browser and server are applications - programs running on specific computers.
Because browsers speak HTTP, that's what it uses to make the request: the letter says something like "GET http://stackoverflow.com". The browser also writes down any cookie information it got from StackOverflow last time ("remember me? You told me my login ID was X") and adds some miscellaneous labeled information called "headers" (things like "I'm Firefox" and "I can accept HTML or text" and "it's OK with me if you compress the content with gzip"). All that information will help the server know how to personalize or customize its response.
At that point, the browser is basically done. It hands this letter to the operating system and says, "would you please send this for me?" The OS says, "Sure." It then does some work to connect to StackOverflow (more on that in a minute), then tells the browser, "I'm working on it. By the way, here's a little mail bin I made for you, called a socket. When I hear back from StackOverflow, I'll put its letter in there and you can read it just like a file." The browser then happily awaits the response.
The IP layer
To send the request from the browser to StackOverflow, the operating system has to do several things.
First, it has to look up the address for StackOverflow.com - specifically, the IP address. It does this using DNS (which I won't go into here). Once it knows the IP address, it will know how to wrap the request in one of the "envelopes" called the IP layer.
Why do we need the IP layer? Well, once upon a time, we didn't.
Why we need IP
Have you ever seen an old movie where someone makes a phone call by asking the operator to connect them? The operator would physically connect the wire from Person #1's house to the wire for Person #2's house. Before the protocol stack was invented, connecting computers was a lot like that phone call: you needed a dedicated wire from point to point.
So, for example, if the computer scientists at Stanford wanted to exchange data with the ones at Harvard, they'd pay a bunch of money to rent a dedicated wire between the two places (a "leased line"). Any data that went into one end came out reliably at the other end. However, this was very expensive: imagine paying for a separate line for every place you want to connect to!
People realized that this wouldn't scale up. We needed a way to have a network that was shared by all users, like a giant spiderweb of wires spread out all over the map. That way, each user would only need one connection to the network and could reach any other user through it.
But that presented a problem. If everyone's communications went on the same lines, how would the data get to the right place? Imagine a bunch of letters dumped on a conveyor belt. Obviously, every letter needs to be addressed to someone, or else they can't be delivered.
That's the basic idea of IP: every machine needs to have an IP address that uniquely identifies it. Messages are placed in IP packets, which are like envelopes with addresses and return addresses.
So, once the OS has looked up the IP address for Stackoverflow.com, it puts the HTTP request in an IP envelope. If it's a "long letter", too big for one envelope, the OS cuts it into pieces and puts it in several IP envelopes. Each envelope says something like "FROM: (your IP address); TO: (The Server's IP address." Like the HTTP request, the IP packet has some other miscellaneous header information, which we won't go into here, but the basic idea is just "to" and "from."
So, at this point, the letter is ready to go, right?
The messiness of IP
Not quite. This letter could easily get lost! See, with IP, we no longer have a dedicated line from place to place. If we did, we'd be sure that our letters were getting delivered: as long as the line wasn't broken, everything would go through.
But with IP, everyone's packets get dumped onto conveyor belts and carried along. The belts lead to little sorting stations, called "routers". If you imagine the routers like physical mail centers, you could picture one in, say, New York City.
"Here's a letter headed for Mexico City. I don't know exactly how to get there, but the station in Houston should be able to get it closer, so I'll send it there. Ah, here's a letter that's going to Atlanta. I'll send it to Charlotte; they should be able to forward it a step closer."
Generally this system works OK, but it's not as reliable as having your own dedicated line. Nearly anything could happen en route: a conveyor belt could break or catch fire, and everything on it could be lost. Or one could get bogged down for a while, so that its packets are delivered very late.
Besides that, because these conveyor belts and stations are used by everyone, nobody's letters get treated specially. So what happens if a router gets more letters than it can possibly handle? For a while, it can stack them in a corner (maybe in RAM), but eventually, it runs out of space.
What it does then may seem shocking: it starts throwing them away.
Yep. That's it. You might think that it would at least be kind enough to send back a note to you, saying, "sorry, we couldn't deliver your letter." But it doesn't. If you think about it, if the router is overwhelmed, it's probably because there's too much traffic on the lines already. Adding apology notes would only make the problem worse. So it throws away your packet and doesn't bother telling anyone.
Obviously, this is a problem for our HTTP request. We need it to get there, and we need the response to get back reliably, too.
To make sure it gets there, we want some kind of "delivery confirmation" service. For that, we'll wrap another envelope around our HTTP request before putting into IP packets. That layer is called TCP.
TCP
TCP stands for "transfer control protocol." It exists to control what would otherwise be a messy, error-prone delivery process.
As implied before, TCP lets us add some "delivery confirmation" to this messy delivery system. Before we wrap our HTTP request in IP packets, we first put it into TCP packets. Each one gets a number: packet 1 of 5, 2 of 5, etc. (The numbering scheme is actually more complicated and counts bytes rather than packets, but let's ignore that for now.)
The basic idea of TCP is this:
First, the client and server - in this case, your operating system and the StackOverflow server's operating system - do a "handshake" to establish a "connection". Both words needs quotes because the "handshake" is actually a few messages back and forth, proving that packets can get successfully there and back, and the "connection" is really nothing more than each side deciding that they'll keep track of the packets flowing between them.
Next, they send packets back and forth; the client maybe requesting a web page, and the server maybe sending it back (in as many packets as that takes).
As one side receives packets, it sends back confirmation messages, saying "so far I've received your packets up to packet 100" and so forth. If one party sends packets and doesn't hear a confirmation for a while, it will assume they were lost and re-send them.
(Getting confirmations when things arrive at the other end is better than getting error reports when a router drops things along the way for a couple of reasons. One is that confirmations go back over a working connection, whereas errors would further clog a non-working connection. Another is that we don't have to trust the intermediary routers to do the right thing; the client and server are the ones who care most about this particular conversation, so they're the ones who take charge of being sure that it works.)
Besides making sure that all the data gets to the other end, TCP also makes sure that the received data gets put back into the right order before handing it up the stack, in case earlier packets got resent and arrived later, or packets in the middle took a longer route, or whatever.
That's basically it - having this kind of delivery confirmation makes the unreliable IP network reliable.
Why wasn't it built straight into IP?
UDP
Well, confirmation has a drawback: it makes things slower. If something is missed, it must be repeated. In some cases, that would be a waste of time, because what you really want is a real-time connection. For example, if you're having a phone conversation over IP, or you're playing a real-time game over the internet, you want to know what's happening right now, even if it means you miss a bit of what happened a second ago. If you stop to repeat things, you'll fall out of sync with everyone else. In cases like that, you can use a cousin of TCP called UDP, which doesn't re-send lost packets. UDP stands for "user datagram protocol", but many people think of it as "unreliable data protocol". That's not an insult; sometimes reliability is less important than staying current.
Since both of these are valid use cases, it makes sense that the IP protocol stayed neutral on the issue of reliability; those who use it can choose whether to add reliability or not.
Both TCP and UDP add one other important piece of information to the request: a port number.
Port numbers
Remember, our original request is comes from a browser and is going to a web server program. But the IP protocol only has addresses that specify computers, not the applications running on them. The machine with StackOverflow's web server may also have other server programs that are listening for requests: a database server, an FTP server, etc. When that machine gets the request, how will it know which program should handle it?
It will know because the TCP request has a port number on it. This is just a number, nothing fancy, but by convention, certain numbers are interpreted to mean certain things. For example, using a port number of 80 is a conventional way of saying "this is a request for a web server." Then the server machine's operating system will know to hand that request to the web server program and not, say, the FTP server program.
When the TCP packets start streaming back to your computer, they will also have a port number, to let your machine know which program to give the response to. That number will vary based on the socket that your machine created initially.
Wait, what's a socket?
Sockets
Remember earlier when the browser asked the OS to send the request? The OS said it would set up a "mail bin" for any response it got back. That bin is called a socket.
You can think of a socket sort of like a file. A file is an interface that the OS provides. It says, "you can read and write data here, and I will take care of figuring out how to actually store it on the hard drive or USB key or whatever." The thing that uniquely identifies a file is the combination of path and filename. In other words, you can only have one file located in the same folder with the same name.
Similarly, a socket is an interface the OS provides. It says, "you can write requests here and read responses." The thing that uniquely identifies a socket is the combination of four things:
Destination IP
Destination Port
Source IP
Source Port
So, you can only have one socket on a system with the same combination of all of those. Notice that you could easily have several sockets open to the same destination IP and port - say, StackOverflow's web server - as long as they all have different source ports. The OS will guarantee that they do by choosing an arbitrary source port for each request, which is why you can have several tabs or several browsers all requesting the same web site simultaneously without anything getting confused; the packets coming back all say which port on your computer they're headed for, which lets the OS know "ah, this packet is for tab 3 in Firefox" or whatever.
Summing up so far
We've been thinking of the protocols as a series of envelops wrapped around the letter. In our example, the letter was an HTTP request, which got wrapped in TCP, then in IP. The IP packets get sent to the right destination computer. That computer removes the IP "envelope" and finds a TCP packet inside. The TCP packet has a port number, which lets the operating system know which port to collect its information in. It replies saying that it got that packet, and it puts its contents (the HTTP request) into the correct socket for the appropriate program to read from. When that program writes a reponse to the socket, the OS sends it back to the requester.
So our "stack" is:
An HTTP request (a "letter"). This is the application layer.
Wrapped in TCP packets ("envelopes"). This is the transport layer.
Wrapped in IP packets ("envelopes"). This is the IP layer.
It's important to understand that this stack is totally customizable. All of these "protocols" are just standard ways of doing things. You can put anything you want inside of an IP packet if you think the receiving computer will know what to do with it, and you can put anything you want inside a TCP or UDP packet if you think the receiving application will know what to do with it.
You could even put something else inside your HTTP request. You could say that some JSON data in there is the "phone number exchange protocol," and as long as both ends know what to do with it, that's fine, and you've just added a higher-level protocol.
Of course, there's a limit to how "high" you can go in the stack - that is, you can put a smaller envelope inside HTTP, and a smaller one inside that, etc, but eventually you won't have any room to go smaller; you won't have any bits for actual content.
But you can easily go "lower" in the stack; you can wrap more "envelopes" around the existing ones.
Other protocol layers
Once common "envelope" to wrap around IP is Ethernet. For example, when your computer decides to send IP packets to Google, it wraps them up as we've described so far, but to send them, it gives them to your network card. The network card may then wrap the IP packets in Ethernet packets (or token ring packets, if you've got an antique setup), addressing them to your router and sending them there. Your router removes those Ethernet "envelopes", checks the IP address, decides who the next closest router is, wraps another Ethernet envelope addressed to that router, and sends the packet along.
Other protocols could be wrapped as well. Maybe two devices are only connected wirelessly, so they wrap their Ethernet packets in a Wi-Fi or Bluetooth or 4G protocol. Maybe your packets need to cross a village with no electricity, so someone physically prints the packets on paper with numbered pages, rides them across town on a bicycle, and scans them into another computer in the order of the page numbers. Voila! A print-to-OCR protocol. Or maybe, I don't know, TCP over carrier pigeon would be better.
Conclusion
The protocol stack is a beautiful invention, and it works so well that we generally take it for granted.
It is a great example of abstracting functionality: each layer has its own work to do and can rely on others to deal with the rest.
The application layer is only concerned with applications talking to each other: "Firefox wants to talk to the web server at StackOverflow.com."
The transport layer is only concerned with getting a stream of packets delivered correctly from one app to another: "all the packets from port 123 on machine 1 need to get to port 80 on machine 2".
The IP layer is only concerned with routing individual packets: "this packet needs to get to the following IP address."
The link layer is only concerned with getting packets from one waypoint to the next: "this ethernet packet needs to get from the network card to the router."
The physical layer is only concerned with signal transmission: "these pulses need to be sent over this wire."
(Although these layer terms are borrowed from OSI, OSI was actually a competing standard to TCP/IP, and included things like the "session layer" and "presentation layer" that TCP/IP doesn't use. OSI was intended to be a more sane and standardized alternative to the scrappy hacked-together TCP/IP stack, but while it was still being discussed, TCP/IP was already working and was widely adopted.)
Because the layers can be mixed and matched as needed, the stack is flexible enough to accommodate nearly any use we can think of, so it's probably going to be around for a long time. And hopefully now you can appreciate it a bit more.
For the throughout description of TCP/IP networking (without physical layer, e.g., Ethernet), pick TCP/IP Illustrated by Stevens. If you going to do some low-level network programming, Unix network programming by the same author is the best.
There's a reason you'll often hear of TCP/IP implementations called a "stack". Part of the concept is that you have a low-level protocol (Ethernet, PPP, what-have-you), slightly higher-level protocols built on top of it (IP), and so on. It's quite similar to the OSI model, and can be described in terms of that model, though TCP/IP breaks up the layers just a bit differently. Anyway, programs generally send data using one of the upper-level protocols, and let the TCP/IP stack handle the details of getting the data from point A to point B.
TCP sits on top of IP and lets you think of the data flowing in and out as a pair of streams (one in, one out) rather than getting raw IP packets and having to figure out what to do with them. (Big BIG benefit: it simplifies multiplexing. Without TCP or UDP or the like, IP would be near useless -- only one program could normally communicate with the network at a given time.)
SSL sits on top of TCP, and lets you send data over the stream that TCP provides without having to get involved in the ugly details of encrypting and decrypting data, verifying certificates, etc.
HTTP sits on top of TCP (or SSL, in the case of HTTPS), and provides a way for a client and server to pass entire requests and responses, along with metadata describing them.
Network protocols are formal standards and policies comprised of rules, procedures and formats that define communication between two or more devices over a network. Network protocols govern the end-to-end processes of timely, secure and managed data or network communication.
There are several broad types of networking protocols, including:
• Network communication protocols: Basic data communication protocols, such as TCP/IP and HTTP.
• Network security protocols: Implement security over network communications and include HTTPS, SSL and SFTP.
• Network management protocols: Provide network governance and maintenance and include SNMP and ICMP.
The different layers of the Open Systems Interconnection (OSI) reference model are:
Application layer: This is the upper most layer in the OSI reference model. The application layer provides the means by which application processes can access network services, and is therefore associated with services that include direct support for applications.
Presentation layer: This layer in the OSI reference model deals with specifying the format which should be utilized to enable network data to be communicated between computers in the network. The presentation layer adds formatting, encryption, and data compression to the packet.
Session layer: This layer enables applications that reside on different computers to create and close network sessions. It also manages open network connections, or sessions that are open.
Transport layer: The transport layer is responsible for ensuring that data is delivered in sequence, error-free, and efficiently over the network. The transport layer also identifies duplicated packets, and drops them. Transport layer protocols include Transmission Control Protocol (TCP) and Sequenced Packet Exchange (SPX). These protocols open packets at the receiving computer, and reassemble the original messages as well.
Network layer: This layer of the OSI reference model provides addressing for messages for all networks. It translates your logical addresses and names to physical addresses, and then identifies the preferred route from the source computer to the destination computer.
Data Link layer: The Data Link layer prepares data for the physical connection by defining the means by which software drivers can access the physical medium. The Data Link layer transmits frames from the Network layer to the Physical layer.
Physical layer: This layer places the data on the physical medium which is carrying the data. It is responsible for the actual physical connection between two computers on the network that are exchanging data.
The function of protocols at the sending computer is summarized below:
• Segment data into smaller more manageable chunks or packets.
• Append addressing to the packets.
• Ensure that data is ready for sending via the network interface card (NIC) to the network cable
The function of protocols at the receiving computer is summarized below:
• Remove packets from the network cable, and move the packets through the NIC to the computer.
• Remove all information that relate to the sending of the packet. This is information added to the packet by the sending computer.
• Move the packets to the buffer for the reassembly process.
• Convey the data to the particular application.
Internet Protocol :
Internet protocol suite is the set of communication protocols that implement the protocol stack on which the internet runs. The Internet protocol suite is sometimes called the TCP/IP protocol suite, after TCP\IP, which refers to the important protocols in it, the Transmission Control Protocol(TCP) and the Internet Protocol(IP). The Internet protocol suite can be described by the analogy with the OSI model, but there are some differences. Also not all of the layers correspond well.
Protocol Stack:
A protocol stack is the complete set of protocol layers that work together to provide networking capabilities.
Transmission Control Protocol (TCP):
The Transmission Control Protocol is the core protocol of the internet protocol suite. It originated in the network implementation in which it complemented the Internet Protocol. Therefore the entire suite is commonly referred to as TCP/IP. TCP provides reliable delivery of a stream of octets over an IP network. Ordering and error-checking are main characteristics of the TCP. All major Internet applications such as World Wide Web, email and file transfer rely on TCP.
Internet Protocol(IP):
The Internet Protocol is the principal protocol in the Internet protocol suite for relaying data across networks. Its routing function essentially establishes the internet. Historically it was the connectionless datagram service in the original Transmission Control Program; the other being the connection oriented protocol(TCP). Therefore, the Internet protocol suite is referred as TCP/IP.
Is there a beginner-friendly book or other resource that you'd
recommend?
Data Communications and Networking by Behrouz Forouzan:
This contains introductory material and the explanation is beginner friendly. At the same time, it is not dumbed down and the material gets a bit more challenging as you go on. There are very good diagrams explaining concepts too. The typesetting is awesome and you'll have lots of interesting tips surrounding the content. The chapters are ordered according to the OSI stack as mentioned in other answers here. But a lot of the math and derivations for formulas for protocol efficiencies aren't explained.
Computer Networks by Andrew S. Tanenbaum
Everything found in Behrouz Forouzan + lots of equations.
My recommendation is to read the first book first and if you are particularly curious about the math, go to the second one.
We had computer networking on school and we had to buy this book it really helps. It explains every layer of the OSI model. (From the internetcabel and routers up to the tcp udp protecol layers up to the application layer). If you want to have more basic knowledge of how it all works this is a must read.