When connection sets up, there is:
Client ------SYN-----> Server
Client <---ACK/SYN---- Server ----①
Client ------ACK-----> Server
and when termination comes, there is:
Client ------FIN-----> Server
Client <-----ACK------ Server ----②
Client <-----FIN------ Server ----③
Client ------ACK-----> Server
my question is why ② and ③ can not set in the same package like ① which is ACK and SYN set in one package ???
After googling a lot, I recognized that the four-way is actually two pairs of two-way handshakes.
If termination is a REAL four-way actions, the 2 and 3 indeed can be set 1 at the same packet.
But this a two-phase work: the first phase (i.e. the first two-way handshake) is :
Client ------FIN-----> Server
Client <-----ACK------ Server
At this moment the client has been in FIN_WAIT_2 state waiting for a FIN from Server. As a bidirectional and full-duplex protocol, at present one direction has break down, no more data would be sent, but receiving still work, client has to wait for the other "half-duplex" to be terminated.
While the FIN from the Server was sent to Client, then Client response a ACK to terminate the connection.
Concluding note: the 2 and 3 can not merge into one package, because they belong to different states. But, if server has no more data or no data at all to be sent when received the FIN from client, it's ok to merge 2 and 3 in one package.
References:
http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm
http://www.tcpipguide.com/free/t_TCPConnectionEstablishmentProcessTheThreeWayHandsh-3.htm
http://www.tcpipguide.com/free/t_TCPOperationalOverviewandtheTCPFiniteStateMachineF-2.htm
I think of course the 2 and 3 can merge technically, but not flexible enough as not atomic.
The first FIN1 C to S means and only means: I would close on my way of communication.
The first ACK1 S to C means a response to FIN1. OK, I know your channel is disconnected but for my S(server) way connection, I'm not sure yet. Maybe my receiving buffer is not fully handled yet. The time I need is uncertain.
Thus 2 and 3 are not appropriate to merge. Only the server would have right to decide when his way of connection can be disconnected.
From Wikipedia - "It is also possible to terminate the connection by a 3-way handshake, when host A sends a FIN and host B replies with a FIN & ACK (merely combines 2 steps into one) and host A replies with an ACK.[14]"
Source:
Wikipedia - https://en.wikipedia.org/wiki/Transmission_Control_Protocol
[14] - Tanenbaum, Andrew S. (2003-03-17). Computer Networks (Fourth ed.). Prentice Hall. ISBN 0-13-066102-3.
Based on this document, we can see the detail process of the four way handshake as follows
The ACK (marked as ②) is send by TCP stack automatically. And the next FIN (marked as ③) is controlled in application level by calling close socket API. Application has the control to terminate the connection. So in common case, we didn't merge this two packets into one.
In contrast, the ACK/SYN (marked as ①) is send by TCP stack automatically. In the TCP connection establishment stage, the process is straightforward and easier, so TCP stack handles this by default.
If this is observed from the angle of coding, it is more reasonable to have 4 way than 3 way although both are possible ways for use.
When a connection is to be terminated by one side, there are multiple possibilities or states that the peer may have. At least one is normal, one is on transmitting or receiving, one is in disconnected state somehow all of a sudden before this initiation.
The way of termination should take at least above three into consideration for they all have high probabilities to happen in real life.
So it becomes more natural to find out why based on above. If the peer is in offline state, then things are quite easy for client to infer the peer state by delving into the packet content captured in the procedure since the first ack msg could not be received from peer. But if the two messages are combined together, then it becomes much difficult for the client to know why the peer does not respond because not only offline state could lead to the packet missing but also the various exceptions in the procedure of processing in server side could make this happen. Not to mention the client needs to wait large amount of time until timeout. With the additional 1 message, the two problems could become more easier
to handle from client side.
The reason for it looks like coding because the information contained in the message is just like the log of code.
In the Three-Way Handshake (connection setup) : The server must acknowledge (ACK) the client's SYN and the server must also send its own SYN containing the initial sequence number for the data that the server will send on the connection.
That why the server sends its SYN and the ACK of the client's SYN in a single segment.
In connection Termination : it takes four segments to terminate a connection since a FIN and an ACK are required in each direction.
(2) means that The received FIN (first segment) is acknowledged (ACK) by TCP
(3) means that sometime later the application that received the end-of-file will close its socket. This causes its TCP to send a FIN.
And then the last segment will mean that The TCP on the system that receives this final FIN acknowledges (ACK) the FIN.
Related
I'm an application developer looking to learn more about the transport layer of my requests that I've been making all these years. I've also been learning more of the backend and am building my own live data service with websockets, which has me curious about how data actually moves around.
As such I've learned about TCP, and I understand how it works, but there's still one term that confuses me-- a "TCP Connection". I have seen it everywhere, and actually there was a thread opened with the exact same question... but as the OP said in the comments, nobody actually answered the question:
TCP vs UDP - What is a TCP connection?
"when we say that there is a connection established between two hosts,
what does that mean? If I could get a magic microscope and inspect the
server or the client, and - a-ha! - find the connection, what would I
be looking at? Some variable allocated by the OS code? Some entry in
some kind of table? How and when does that gets there, and how and
when it is removed from there"
I've been reading to try to figure this out on my own,
Here is a nice resource that details HTTP flow, also mentions "TCP Connection"
https://blog.catchpoint.com/2010/09/17/anatomyhttp/
Here is another thread about HTTP Keep-alive, same "TCP Connection":
HTTP Keep Alive and TCP keep alive
My understanding:
When a client wants data from server, SYN/ACK handshake happens, this "connection" is established, and both parties agree on the starting sequence number, maximum packet size, etc.
as long as this "connection" is still open, client can request/receive data without doing another handshake. TCP Keep-alive sends a heartbeat to keep this "connection" open
1) Somehow a HTTP Header "Keep-alive" also keeps this TCP "connection" open, even though HTTP headers are part of the packet payload and it doesn't seem to make sense that the TCP layer would parse the HTTP headers?
To me it seems like a "connection" between two machines in the literal sense can never be closed, because a client is always free to hit a server with packets (like the first SYN packet, for example)
2) Is a TCP "connection" just the client and server saving the sequence number from the other's IP address? maybe it's just a flag that's saying "hey this client is cool, accept messages from them without a handshake"? So would closing a connection just be wiping that data out from memory?
... both parties agree on the starting sequence number
No, they don't "agree" one a number. Each direction has their own sequence numbering. So the client sends in the SYN to the server the initial sequence number (ISN) for the data from client to server, the server sends in its SYN the ISN for the data from server to client.
Somehow a HTTP Header "Keep-alive" also keeps this TCP "connection" open ...
Not really. With HTTP keep-alive the client just asks a server nicely to not close the connection after the HTTP response was sent so that another HTTP request can be sent using the same TCP connection. The server might decide to follow the clients wish or not.
To me it seems like a "connection" between two machines in the literal sense can never be closed,
Each side can send a packet with a FIN flag to signal that it will no longer send any data. If both sides has send the FIN the the connection is considered close since no one will send anything and thus nothing can be received. If one side decides that it does not want to receive any more data it can send a packet with a RST flag.
Is a TCP "connection" just the client and server saving the sequence number from the other's IP address?
Kind of. Each side saves the current state of the connection, i.e. IP's and ports involved, currently expected sequence number for receiving, current sequence number for sending, outstanding bytes which were not ACKed yet ... If no such state is there (for example one site crashed) then there is no connection.
... maybe it's just a flag that's saying "hey this client is cool, accept messages from them without a handshake"
If a packet got received which fits an existing state then it is considered part of the connection, i.e. it will be processed and the state will be updated.
So would closing a connection just be wiping that data out from memory?
Closing is telling the other that no more data will be send (using FIN) and if both side have done it both can basically remove the state and then there is no connection anymore.
Client Send SYN with client isn.
Server reply SYN ACK with server's isn.
Client resend SYN if timeout?
When client send data, it can accumulated confirm the server's isn.
I try to search, but can't find the answer.
I know how the tcp is designed now, I just don't know why it's designed like this. Why can't use a two way handshake.
It cannot use a two way handshake by definition. TCP/IP is formalized as a standard for communication across networks (Internetworking). Specifically, RFC 793 requires that:
The "three-way handshake" is the procedure used to establish a
connection. This procedure normally is initiated by one TCP and
responded to by another TCP. The procedure also works if two TCP
simultaneously initiate the procedure. When simultaneous attempt
occurs, each TCP receives a "SYN" segment which carries no
acknowledgment after it has sent a "SYN". Of course, the arrival of
an old duplicate "SYN" segment can potentially make it appear, to the
recipient, that a simultaneous connection initiation is in progress.
Proper use of "reset" segments can disambiguate these cases.
If you think about how the protocol works, you do not actually have a single full duplex connection. Instead, you have two simplex connections, each going in one direction. This is why the proper response to a SYN is a SYN-ACK. The server is acknowledging the synchronization request by sending the originators sequence number as the acknowledgement number plus one; it is simultaneously attempting to open its own connection to the client by sending a SYN request to synchronize.
The proper answer to a SYN will always be an ACK, even if the connection fails.
Regarding your question about sending a retry, yes; the client will send a retry (typically up to three, but it could be as many as eight in practice... There's no limit defined) attempting to elicit a response of some kind (preferably a SYN-ACK, but possibly a RST-ACK).
Who should first set the TCP FIN flag. Server, when finished sending data, or Client, when received complete data?
regards
FIN is sent when the application on that side requests that the connection be closed. This doesn't have to happen immediately after receiving a FIN from the other side, either - it's possible for one side to send a FIN, then the other side to send some more data before sending its own FIN.
Once the TCP connection is up, it's completely symmetrical - neither side is distinguishable as a "server" or "client". This implies that either side can send the first FIN. Since a host can send no more data on a connection after sending a FIN, it is usually the side that first knows that it has no more data to send that does so.
There is a good practical reason to design a network protocol so that the client is the one that first closes the connection - the "first closer" ends up in TIME_WAIT state for twice the Maximum Segment Lifetime, which is a few minutes. This occupies a small amount of resources for those minutes - but if this happens on the server side where the server is handling many hundreds of connections a second, those TIME_WAIT sockets will quickly add up. It's better to distribute this burden out among the clients.
This depends on the application layer protocol. - from http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm.
A TCP connection is normally
terminating using a special procedure
where each side independently closes
its end of the link. It normally
begins with one of the application
processes signalling to its TCP layer
that the session is no longer needed.
That device sends a FIN message to
tell the other device that it wants to
end the connection, which is
acknowledged. When the responding
device is ready, it too sends a FIN
that is acknowledged; after waiting a
period of time for the ACK to be
received, the session is closed.
In practice, it should usually be the client, but there is no reason why it must be so.
Why is data not transferred during the 3rd part of TCP 3-way handshake?
e.g.
(A to B)SYN
(B to A)ACK+SYN
(A to B) ACK.... why cant data be transferred along with this ACK?
I've always believed it was to keep the session establishment phase separate from the data transfer phase so that no real data is transferred until both ends of the session have agreed on the sequence numbers and session options, especially since packets arriving may be from a totally different, previous, session that just happens to have the same endpoints.
However, on further investigation, I'm not entirely certain that transmitting data with the handshake packets is disallowed. The section on TCP connection establishment in my Internetworking with TCP/IP1 book contains the following snippet:
Because of the protocol design, it is possible to send data along with the initial sequence numbers in the handshake segments. In such cases, the TCP software must hold the data until the handshake completes. Once a connection has been established, the TCP software can release data being held and deliver it to a waiting application program quickly.
Since it's certainly possible to construct a TCP packet with SYN (or ACK) and data, this may well be allowed. I've never seen it happen in the wild but, then again, I've never seen a hairy-eared dwarf lemur in the wild either, though I'm assured they exist.
It may be that it's the sockets software that prevents data going out before the session is fully established but TCP appears to consider it valid. It appears you can send data with a SYN-ACK packet (phase 2 of the connection establishment) since you have the other end's sequence number and options. Likewise, sending data with the phase 3 ACK packet appears to be possible as well.
The reason the TCP software holds on to the data until the handshake is fully complete is probably due to the reason mentioned above - only once both ends have agreed on the sequence numbers can you be sure that the data is not from a previous session.
1 Internetworking with TCP/IP Volume 1 Principles, Protocols and Architecture, 3rd edition, Douglas E. Comer, ISBN 0-13-216987-8.
There are 3 different accept versions in winsock. Aside from the basic accept which is there for standard compliance, there's also AcceptEx which seems the most advanced version (due to it's overlapped io capabilities), and WSAAccept. The latter supports a condition callback, which as far as I understand, allows the rejection of connection requests before they're accepted (when the SO_CONDITIONAL_ACCEPT option is enabled). None of the other versions supports this functionality.
Since I prefer to use AcceptEx with overlapped io, I wonder how come this functionality is only available in the simpler version?
I don't know enough about the inner workings of TCP to tell wehter there's actually any difference between rejecting a connection before it has been accepted, and disconnecting a socket right after a connection has been established? And if there is, is there any way to mimic the WSAAccept functionality with AcceptEx?
Can someone shed some light over this issue?
When a connection is established, the remote end sends a packet with the SYN flag set. The server answers with a SYN,ACK packet, and after that the remote end sends an ACK packet, which may already contain data.
There are two ways to break a TCP connection from forming. The first is resetting the connection - this is the same as the common "connection refused" message seen when connecting to a port nobody is listening to. In this case, the original SYN packet is answered with a RST packet, which terminates the connection immediately and is stateless. If the SYN is resent, RST will be generated from every received SYN packet.
The second is closing the connection as soon as it has been formed. On the TCP level, there is no way to close the connection both ways immediately - the only thing you can say is that "I am not going to send any more data". This happens so that when the initial SYN, SYN,ACK, ACK exchange has finished, the server sends a FIN packet to the remote end. In most cases, telling the other end with FIN that "I am not going to send any more data" makes the other end close the connection as well, and send it's own FIN packet. A connection terminated this way isn't any different from a normal connection where no data was sent for some reason. This means that the normal state tracking for TCP connections and the lingering close states will persist, just like for normal connections.
Now, on the C API side, this looks a bit different. When calling listen() on a port, the OS starts accepting connections on that port. This means that is starts replying SYN,ACK packets to connections, regardless if the C code has called accept() yet. So, on the TCP side, it makes no difference whether the connection is somehow closed before or after accept. The only additional concern is that a listening socket has a backlog, which means the number of non-accepted connections it can have waiting, before it starts saying RST to the remote end.
However, on windows, the SO_CONDITIONAL_ACCEPT call allows the application to take control of the backlog queue. This means that the server will not answer anything to a SYN packet until the application does something with the connection. This means, that rejecting connections at this level can actually send RST packets to the network without creating state.
So, if you cannot get the SO_CONDITIONAL_ACCEPT functionality enabled somehow on the socket you are using AcceptEx on, it will show up differently to the network. However, not many places actually use the immediate RST functionality, so I would think the requirement for that must mean a very specialized system indeed. For most common use cases, accepting a socket and then closing it is the normal way to behave.
I can't comment on the Windows side of things but as far as TCP is concerned, rejecting a connection is a bit different than disconnecting from it.
For one, disconnecting from a connection means there were more resources already "consumed" (e.g. ports state maintained in Firewalls & end-points, forwarding capacity used in switches/routers etc.) in both the network and the hosts. Rejecting a connection is less resource intensive.