TCP congestion window size in slow start phase - tcp

I have a question about the increasing rate of TCP sender's congestion window during the slow start phase.
Traditionally, the size of cwnd exponentially increases for every RTT. For instance, if the initial cwnd value is 1, it increases 2->4->8->16->.... .
In my case, since the sender uses linux kernel 3.5, the initial cwnd is 10.
I expected the cwnd increases as 10->20->40->... without delayed ACK (I turned it off at the receiver). However, when the receiver downloads a large size (over 1MB) of object from the sender over HTTP, cwnd increases as 10->12->19->29->.... I cannot understand this sequence.
I set RTT to 100ms and the link bandwidth is high enough. There is no loss during a session. I estimated the sender's cwnd by counting the number of packet the receiver received within one RTT.
Does anyone have idea for this behavior?
Thanks.

The dafault congestion control algorithm in kernel 3.5 is not TCP Reno, but CUBIC.
CUBIC has different behavior. It stems from BIC, and I know that CUBIC share slow start phase of BIC. Just see codes of BIC and CUBIC at /usr/src/yourkernelname/net/ipv4/tcp_cubic.c.
Moreover, delayed ack MAY result in such sequences. You know, 'traditional' TCP congestion control behavior is TCP reno without DACK nor SACK.
Know that even current TCP reno in Linux is not reno but new reno (reno with SACK).
Check the options of Delayed ack and Seletive ack using sysctl command.

Related

what is the diffrence between TCP TAHOE and TCP RENO

what is the diffrence between TCP TAHOE and TCP RENO.
what I want to know is about the behivor to 3-dup-ack and timeout?
what happend to cwind what happend to SST?
thanks!
TCP Tahoe and Reno are two forms of handling TCP congestion controls specifically when it comes to receiving 3 duplicate acks.
Tahoe: handles 3 duplicate acks similar (exactly?) to receiving a timeout. It first performs a fast retransmit. Then, it halves the ssthresh value to original congestion window size, and sets the new window size to 1 and staying in slow start.
Reno: The successor to Tahoe, goes into fast recovery mode upon receiving three duplicate acks thereby halving the ssthresh value. For each successive duplicate acks (fourth, fifth, sixth), cwind increases by 1. Once the receiver finally receives the missing packet, TCP will move to congestion avoidance or slowstate upon a timeout.
Source: TCP congestion control - TCP Tahoe and Reno

TCP Connection window size

A 10 Mbps congested non-buffered link is used to send a huge file between two hosts. The receiving host has a large buffer than the congestion window. Assume the turnaround time is 100 ms, and the TCP Reno connection is always in congestion avoidance phase
what is the max window size (in segment) that this TCP connection can achieve?
Im using the formula W*MSS/RTT=10Mbps
I have RTT which is 100ms but im not sure where to get the MSS(maximum segment size) to be able to solve for W

Proper way to calculate Link Throughput

I have read some articles online and I got a pretty good idea about the TCP and UDP in general. However, I still have some doubts which I am sure not completely clear to me.
What is the proper way to calculate throughput ?
(Can't we just divide Total number of bytes received by total time taken ?)
What is that key feature in TCP that makes it have much much higher
throughput than UDP ?
UPDATE:
I understood that TCP uses windows which is nothing but that much segments can be sent before actually waiting for Acknowledgements. But my doubt is that in UDP segments are continuously sent without even bothering about Acknowledgements. So there is no extra overheads in UDP. Then, why the throughput of TCP is much much higher than that of UDP ?
Lastly,
Is this true ?
TCP throughput = (TCP Window Size / RTT) = BDP / RTT = (Link Speed in Bytes/sec * RTT)/RTT = Link Speed in Bytes/sec
If so then TCP throughput is always equals to the Know Link speed. And since the RTTs cancels out each other, the TCP throughput does not even depends on RTT.
I have seen in some network analysis tools like iperf, passmark performance test etc. that the TCP/UDP Throughput changes with Block size.
How is throughput dependent on Block size ?
Is Block size equals TCP window or UDP datagram size ?
What is the proper way to calculate throughput?
There are multiple ways, depending on what exactly you want to measure. They all boil down to dividing some number of bits (or bytes) to some duration, as you mention; what varies is which bits you are counting or (more rarely) which moments of time you are considering for measuring the duration.
The factors you need to take into account are:
At which layer in the network stack are you measuring throughput?
If you measure at the application layer, all that matters is what useful data you transmit to the other endpoint. For example, if you are transferring a file of 6 kB, the amount of data you count when measuring throughput is 6 kB (that is 6,000 bytes, not bits, and note the multiplier of 1000, not 1024; these conventions are common in networking).
This is usually called goodput and it may be different from what is actually sent at the transport layer (as in TCP or UDP), for two reasons:
1. Overhead due to headers
Each layer in the network adds a header to the data that introduces some overhead due to its transmission time. Moreover, the transport layer breaks your data into segments; this is because the network layer (as in IPv4 or IPv6) has a maximum packet size called MTU, typically 1,500 B in Ethernet networks. This value includes the network layer header size (e.g. the IPv4 header, which is variable in length but usually 20 B long) and the transport layer header (for TCP, it is also variable in length but usually 40 B long). This leads to a maximum segment size MSS (number of data bytes, without headers, in one segment) of 1500 - 40 - 20 = 1440 bytes.
Thus if we want to send 6 kB of application-layer data, we must break it into 6 segments, 4 of 1440 bytes each and one of 240 bytes. However at the network layer we end up sending 6 packets, 4 of 1500 bytes each and one of 300 bytes, for a total of 6.3 kB.
Here I have not considered the fact that the link layer (as in Ethernet) adds its own header and possibly also a suffix, which increases the overhead further. For Ethernet this is 14 bytes for the Ethernet header, optionally 4 bytes for VLAN tag, then a CRC of 4 bytes and a gap of 12 bytes, for a total of 36 bytes per packet.
If you consider a fixed-rate link, say of 10 Mb/s, depending on what you measure you will get a different throughput. Normally you want one of these:
The goodput, i.e. application layer throughput, if what you want to measure is application performance. For this example, you divide 6 kB by the transfer duration.
The link-layer throughput, if what you want to measure is network performance. For this example, you divide 6 kB + TCP overhead + IP overhead + Ethernet overhead = 6.3 kB + 5 * 36 B = 6516 B by the transfer duration.
Retransmission overheads
The Internet is a best-effort network, meaning that the packets will be delivered if possible, but may also be dropped. Packet drops are corrected by the transport layer, in case of TCP; for UDP, there is no such mechanism, which means that either the application does not care if some parts of the data do not get delivered, or the application implements retransmission itself on top of UDP.
Retransmission reduce goodput for two reasons:
a. Some data needs to be sent again, which takes time. This introduces a delay which is inversely proportional to the rate of the slowest link in the network between the sender and the receiver (a.k.a the bottleneck link).
b. Detecting that some data was not delivered needs feedback from the receiver to the sender. Due to propagation delays (sometimes called latency; caused by the finite speed of light in the cable), feedback can only be received by the sender with some latency, which slows down the transmission even more. In most practical cases, this is the most significant contribution to the extra delay caused by the retransmission.
Clearly, if you use UDP instead of TCP and you do not care about packet loss, you will of course get better performance. But for many applications, data loss cannot be tolerated, so such a measurement is meaningless.
There are some applications that do use UDP for transferring data. One is BitTorrent, which may use either TCP or a protocol they designed called uTP, which emulates TCP on top of UDP, but aims at being more efficient with many parallel connections. Another transport protocol implemented over UDP is QUIC, which also emulates TCP and offers multiplexing multiple parallel transfers over a single connection, and forward error correction to reduce retransmissions.
I will discuss forward error correction a little since it is related to your question about throughput. A naive way of implementing it is by sending every packet twice; in case one gets lost, the other still has a chance of being received. This reduces the amount of retransmissions to half, but also halves your goodput since you send redundant data (note that the network or link layer throughput remains the same!). In some cases this is fine; especially if the latency is very large, such as on intercontinental or satellite links. Moreover, some mathematical methods exist where you don't have to send a full copy of the data; for instance for every n packets you send, you send another reduntant one which is the XOR (or some other arithmetic operation) of them; if the redundant one gets lost, it doesn't matter; if one of the n packets gets lost, you can reconstruct it based on the redundant one and the other n-1. You can thus configure the overhead introduced by forward error correction to whatever amount of bandwidth you can spare.
How you are measuring the transfer time
Is the transfer completed when the sender finished sending the last bit over the wire, or does it also include the time it takes for the last bit to travel to the receiver? Additionally, does it include the time it takes to get a confirmation from the receiver, stating that all data has been received successfully and no retransmission is neede?
It really depends on what you want to measure. Note that for large transfers, one extra round-trip-time is insignificant in most cases (unless you are communicating, for instance, with a probe on Mars).
What is that key feature in TCP that makes it have much much higher throughput than UDP?
This is not true, although a common misconception.
In addition to retransmitting data when needed, TCP will also adjust its sending rate so that it will not cause packet drops by congesting the network. The adjustment algorithm has been perfected over decades, and usually converges quickly to the maximum rate supported by the network (actually, the bottleneck link). For this reason it is usually difficult to beat TCP in throughput.
With UDP, there is no rate limiting at the sender. UDP lets the application send as much as it wants. But if you try to send more than the network can handle, some of the data will be dropped, lowering your throughput, and also making the admin of the network you are congesting very angry. This means that sending UDP traffic at high rates is impractical (unless the goal is to DoS a network).
Some media applications are using UDP but rate-limiting the transfer at the sender at a very small rate. This is typically used in VoIP applications or Internet Radio, where you require very little throughput but low latency. I suppose this is one of the reasons for the misconception that UDP is slower than TCP; that is not the case, UDP can be as fast as the network allows.
As I said before, there are protocols such as uTP or QUIC, implemented over UDP, which achieve performance similar to TCP.
Is this true ?
TCP throughput = (TCP Window Size / RTT)
Without packet loss (and retransmissions), this is correct.
TCP throughput = BDP / RTT = (Link Speed in Bytes/sec * RTT)/RTT = Link Speed in Bytes/sec
This is correct only if the window size is configured to the optimal value. BDP/RTT is the optimal (maximum possible) transfer rate in the network. Most modern operating systems should be able to auto-configure it optimally.
How is throughput dependent on Block size ? Is Block size equals TCP window or UDP datagram size?
I don't see any block size in the iperf documentation.
If you refer to the TCP window size, if it is smaller than BDP, then your throughput will be suboptimal (because you waste time waiting for ACKs instead of sending more data; if needed I can explain further). If it is equal or higher to the BDP, then you achieve optimal throughput.
It depends on how you define "Throughput". It usually can be one of the followings.
Number of bytes (or bits) sent in a fixed period of time;
Number of bytes (or bits) sent and received on the receiver end in a fixed period of time;
You can apply these definition to every layer when people talking about throughput. In application layer, 2nd definition means the bytes have really been received by the receiver end of the application. Some people refer to it as "goodput". In Transport layer, say TCP, 2nd definition means the corresponding TCP ACKs are received. To me, most of people should be only interested in the bytes are really received by the receiver end. So, 2nd definition is usually what people mean by "Throughput".
Now, once we have a clear definition of throughput (2nd definition). We can discuss how to measure the throughput correctly.
Usually, people either use TCP or UDP to measure the network throughput.
TCP: People usually measure TCP throughput only on the sender end. As for packets successfully received by the receiver end, ACKs will be sending back. So, sender itself will know how many bytes are sent and received on the receiver end. Divided this number by the measuring time, we will know the throughput.
But, there are two things need to be noticed during TCP throughput measurement:
Is sender side always full buffer during the measurement? i.e. During the measurement period, sender should always has packets to send. It is important for correct throughput measurement. e.g. if I set my measuring time to be 60 seconds, but my file has been finished transmission in 40 seconds. Then there are 20 seconds the network is actually idle. I will under-estimate the throughput.
TCP rate is regulated by its congestion window size, slow-start duration, sender window (and receiver window) size. Sub-optimal configuration of these parameters will result in under-estimated TCP throughput. Although most of the modern TCP implementation should have a quite good configuration of all of these, it is hard for a tester to 100% sure all these configurations are optimal.
Due to these limitations/risks of TCP in network throughput estimation, quite a number of researchers will use UDP for measuring network throughput.
UDP: As UDP has no ACK sending back once the packets are successfully received, people has to measure the throughput in the receiver end. Or, if the receiver end is not easily accessed, people can compare the logs on both sender and receiver sides to determine the throughput. But, this inconvenience is mitigated by some throughput measuring tools. For example, iperf has embedded sequence numbers in its customized payload, so that it can detect any loss. Also, a receiver's report will be sent to the sender to show the throughput.
As UDP by nature is just sending whatever it has to the network and not waiting for the feedback. Its throughput (remember the 2nd definition) once measured will be the actual capacity (or bandwidth) of the network.
So, usually, the throughput measured by UDP should be higher than that from TCP although the difference should be small (~5%-10%).
One biggest drawback of UDP throughput measuring is that, when using UDP one should also make sure that sender-side buffer must be full. (Otherwise, it results in under-estimated throughput as TCP). This step will be little tricky. In iperf, one can specify the sending rate by -b option. Increase -b value in different rounds of testing will converge the throughput measured. For example, in my gigabit ethernet, I first use -b 100k in the test. The throughput measured is 100Kbps. Then I perform the following iterations to converge the maximum throughput which is the capacity of my ethernet.
-b 1m --> throughput: 1Mbps
-b 10m --> throughput: 10Mbps
-b 100m --> throughput: 100Mbps
-b 200m --> throughput: 170Mbps
-b 180m --> throughput: 175Mbps (this should be quite close to the actual capacity)

How do delayed acknowledgements affect TCP's congestion avoidance phase?

From what I studied, congestion avoidance phase sets CWND = CWND + MSS * (MSS/CWND) every time a new acknowledgment is received. This is assuming we don't encounter duplicate ACKS or timeouts. But what happens if there are delayed acknowledgements ?
Here's what I think from research on delayed acks (no idea if this is correct):
Basically Delayed ACK is the destination retaining the ACK segment for a period of time expecting one of two things.
Either there will be more ACKS will be required to be sent before the timer is up because of new packets recieved by the receiver. OR the receiver will need to send some data back to the sender in which case it can piggy back the message on that packet.
How does this affect the congestion avoidance phase ?
This would be bad for congestion avoidance phase of TCP which depends on new Acks to increase CWND. This would cause delays in CWND window size change thus causing delay in the sending of packets. This means by the time that TCP could be sending packets to the receiver, it is actually not because acknowledgments are being delayed.
This affects the congestion avoidance phase the same way it affects the other phase (SS) : it will slow down the traffic. However, keep in mind there are two different network uses, the interactive one (such as telnet), and the bulk one. Delayed Acks are likely to be used with interactive protocols sending very small amounts of data, but this can bring new problems if Nagle's algorithm is used by the other side. When unsure, just disable delayed Acks.
That is a really good question. Since they keep the bottleneck buffer full, delayed Ack is not a big problem for traditional congestion control algorithms such as Reno and CUBIC.
For TCP Vegas which tries to keep the queue small, it is still not a problem because if it faces delayed Ack, it will reduce cwnd very slowly (one unit every RTT). Therefore a Vegas connection will not suffer from under-utilization unless delayed ack lasts for a very long time.

What effects can inconsistent latency have on TCP applications?

I am testing a GNU Radio program which can tunnel TCP traffic over a wireless link. We are having some strange results in testing, and in looking for a culprit I was curious about inconsistent latency.
How can inconsistent latency affect TCP applications? By inconsistent I mean widely different RTT for ACKs on a connection. For awhile ACks seem to be coming at a normal rate, then they disappear and we have retransmissions followed by the 'delayed' ACK.
For instance, say the first several ACK's received have a similar RTT. What would happen when the next ACK isn't receieved in twice the RTT of the previous ACKs? Whatever the issue is I see lots of retransmissions after a long wait for an ACK.
Now, more specifically, how can RTTs for ACKs which bounce between fast and slow affect a TCP connection?
Having said that, is there any way to tune the IP stack to handle this environment better?
TCP maintains a smoothed RTT (SRTT) to tell it how fast the intervening network is, i.e. how fast it can transmit. If the SRTT goes up TCP will slow down. If SRTT goes down TCP will speed up. If the actual RTT goes up and down violently, TCP may not react quickly enough, due to the smoothing, and transmit too fast, which would cause packet loss, which in turn causes retransmission, which wastes the bandwidth used by the lost packets. RTT smoothing is done via exponential decay with a gain of I think 0.2, so the old SRTT value has four times the weight of the current RTT when computing the new SRTT value.

Resources