I'm having an application on a micro controller with a small TCP/IP stack. This application waits for a connection on a specific TCP port. As soon as the TCP connection is established the micro controller need to send 8 KB of data. Because the TX buffer of the TCP socket is only 1 KB tall I will need to send 8 segments of data. The TX buffer size can't be changed!
But now I have the problem, that after every segment a delay of 200 ms occurs. I know that this is caused by the delayed ack (which is on Windows 200 ms). But in my use case this means the whole process includes 1400 ms of delay time, which is just wasted time.
Is there any possibility that I can force the PC to acknowledge the data instantly (maybe a bit in the TCP header)? I can't change anything on the PC side.
Or should I instead sent two 512 byte tall segments instead of the 1 KB segment? Would this fix / trick out the issue? I have read that the PC will acknowledge the data if a second segement arrives.
What is the right way to solve such a usecase?
Related
We're seeing this pattern happen a lot between two RHEL 6 boxes that are transferring data via a TCP connection. The client issues a TCP Window Full, 0.2s later the client sends TCP Keep-Alives, to which the server responds with what look like correctly shaped responses. The client is unsatisfied by this however and continues sending TCP Keep-Alives until it finally closes the connection with an RST nearly 9s later.
This is despite the RHEL boxes having the default TCP Keep-Alive configuration:
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_intvl = 75
...which declares that this should only occur until 2hrs of silence. Am I reading my PCAP wrong (relevant packets available on request)?
Below is Wireshark screenshot of the pattern, with my own packet notes in the middle.
Actually, these "keep-alive" packets are not used for TCP keep-alive! They are used for window size updates detection.
Wireshark treats them as keep-alive packets just because these packets look like keep-alive packet.
A TCP keep-alive packet is simply an ACK with the sequence number set to one less than the current sequence number for the connection.
(We assume that ip 10.120.67.113 refers to host A, 10.120.67.132 refers to host B.) In packet No.249511, A acks seq 24507484. In next packet(No.249512), B send seq 24507483(24507484-1).
Why there are so many "keep-alive" packets, what are they used for?
A sends data to B, and B replies zero-window size to tell A that he temporarily can't receive data anymore. In order to assure that A knows when B can receive data again, A send "keep-alive" packet to B again and again with persistence timer, B replies to A with his window size info (In our case, B's window size has always been zero).
And the normal TCP exponential backoff is used when calculating the persist timer. So we can see that A send its first "keep-alive" packet after 0.2s, send its second packet after 0.4s, the third is sent after 0.8, the fouth is sent after 1.6s...
This phenomenon is related to TCP flow control.
The source and destination IP addresses in the packets originating from client do not match the destination and source IP addresses in the response packets, which indicates that there is some device in between the boxes doing NAT. It is also important to understand where the packets have been captured. Probably a packet capture on the client itself will help understand the issue.
Please note that the client can generate TCP keepalive if it does not receive a data packet for two hours or more. As per RFC 1122, the client retries keepalive if it does not receive a keepalive response from the peer. It eventually disconnects after continuous retry failure.
The NAT devices typically implement connection caches to maintain the state of ongoing connections. If the size of the connection reaches limit, the NAT devices drops old connections in order to service the new connections. This could also lead to such a scenario.
The given packet capture indicates that there is a high probability that packets are not reaching the client, so it will be helpful to capture packets on client machine.
I read the trace slightly differently:
Sender sends more data than receiver can handle and gets zerowindow response
Sender sends window probes (not keepalives it is way to soon for that) and the application gives up after 10 seconds with no progress and closes the connection, the reset indicates there is data pending in the TCP sendbuffer.
If the application uses a large blocksize writing to the socket it may have seen no progress for more than the 10 seconds seen in the tcpdump.
If this is a straight connection (no proxies etc.) the most likely reason is that the receiving up stop receiving (or is slower than the sender & data transmission)
It looks to me like packet number 249522 provoked the application on 10.120.67.113 to abort the connection. All the window probes get a zero window response from .132 (with no payload) and then .132 sends (unsolicited) packet 249522 with 63 bytes (and still showing 0 window). The PSH flag suggests that this 63 bytes is the entire data written by the app on .132. Then .113 in the same millisecond responds with an RST. I can't think of any reason why the TCP stack would send a RST immediately after receiving data (sequence numbers are correct). In my view it is almost certain that the app on .113 decided to give up based on the 63 byte message sent by .132.
I'm trying to reverse engineer an application, and i need help understanding how TCP window size works. My MTU is 1460
My application transfers a file using TCP from point A to B. I know the following:
The file is split into segments of size 8K
Each segment is compressed
Then each segment is sent to point B over TCP. These segment for a text file can be of size 148 Bytes, and for a pdf 6000 Bytes.
For a text file, am i supposed to see the segments of 148 attached to one another to form one large TCP stream? and then it is split according to the Window Size?
Any help is appreciated.
The receiver application should see the data in teh same way, the sender application sent it. TCP uses byte-streaming and so it collects all the bytes in an in-order manner and delivers it to the application. MTU is largely an internal semantics to TCP and does not take into application-layer packet boundaries. If TCP has enough data to send in its send buffer (each TCP socket has its own send buffer, btw), then it will package its next segment worth MTU size and sends it; to be more precise, it deducts TCP and IP header from the MTU size.
I open a TCP socket and connect it to another socket somewhere else on the network. I can then successfully send and receive data. I have a timer that sends something to the socket every second.
I then rudely interrupt the connection by forcibly losing the connection (pulling out the Ethernet cable in this case). My socket is still reporting that it is successfully writing data out every second. This continues for approximately 1hour and 30 minutes, where a write error is eventually given.
What specifies this time-out where a socket finally accepts the other end has disappeared? Is it the OS (Ubuntu 11.04), is it from the TCP/IP specification, or is it a socket configuration option?
Pulling the network cable will not break a TCP connection(1) though it will disrupt communications. You can plug the cable back in and once IP connectivity is established, all back-data will move. This is what makes TCP reliable, even on cellular networks.
When TCP sends data, it expects an ACK in reply. If none comes within some amount of time, it re-transmits the data and waits again. The time it waits between transmissions generally increases exponentially.
After some number of retransmissions or some amount of total time with no ACK, TCP will consider the connection "broken". How many times or how long depends on your OS and its configuration but it typically times-out on the order of many minutes.
From Linux's tcp.7 man page:
tcp_retries2 (integer; default: 15; since Linux 2.2)
The maximum number of times a TCP packet is retransmitted in
established state before giving up. The default value is 15, which
corresponds to a duration of approximately between 13 to 30 minutes,
depending on the retransmission timeout. The RFC 1122 specified
minimum limit of 100 seconds is typically deemed too short.
This is likely the value you'll want to adjust to change how long it takes to detect if your connection has vanished.
(1) There are exceptions to this. The operating system, upon noticing a cable being removed, could notify upper layers that all connections should be considered "broken".
If want a quick socket error propagation to your application code, you may wanna try this socket option:
TCP_USER_TIMEOUT (since Linux 2.6.37)
This option takes an unsigned int as an argument. When the
value is greater than 0, it specifies the maximum amount of
time in milliseconds that transmitted data may remain
unacknowledged before TCP will forcibly close the
corresponding connection and return ETIMEDOUT to the
application. If the option value is specified as 0, TCP will
use the system default.
See full description on linux/man/tcp(7). This option is more flexible (you can set it on the fly, just right after a socket creation) than tcp_retries2 editing and exactly applies to a situation when you client's socket doesn't aware about server's one state and may get into so called half-closed state.
Two excellent answers are here and here.
TCP user timeout may work for your case: The TCP user timeout controls how long transmitted data may remain unacknowledged before a connection is forcefully closed.
there are 3 OS dependent TCP timeout parameters.
On Linux the defaults are:
tcp_keepalive_time default 7200 seconds
tcp_keepalive_probes default 9
tcp_keepalive_intvl default 75 sec
Total timeout time is tcp_keepalive_time + (tcp_keepalive_probes * tcp_keepalive_intvl), with these defaults 7200 + (9 * 75) = 7875 secs
To set these parameters on Linux:
sysctl -w net.ipv4.tcp_keepalive_time=1800 net.ipv4.tcp_keepalive_probes=3 net.ipv4.tcp_keepalive_intvl=20
PSH is a way to send data via TCP. Besides that, I can find very little info on how to implement it properly.
Here is what interests me:
Let's say, server window is 8000 bytes, and I send 2 requests with 150 and 600 bytes. Do I get some sort of confirmation that the data has been received? Can I somehow trigger a confirmacion?
I've seen some ACK packets, which does not contain PSH but do contain some sort of payload data (Wireshark marks it as "TCP segment data"). Is this data passed on to user, and if it is, why do we need PSH flag?
TCP PSH generally doesn't 'work' at all. Berkely-derived TCP implementations completely ignore it.
Source: W.R. Stevens, TCP/IP Illustrated, vol I: 20.5 PUSH Flag.
#Arsen: Answering to the second part of your question "why do we need PSH flag?"
The PSH flag in the TCP header informs the receiving host that the data should be pushed up to the receiving application immediately.
We are using PSH flag to exchange Time Stamp value between two servers.
i am assuming , if we set push flag , packet wont wait in receive buffer , it will directly send to receiver.
The data doesn't sit waiting in the receive buffer anyhow.
TCP apps must go out of their way to have the TCP layer bulk up a few packets and deliver full data buffers.
In fact its somewhat frustrating to see applications allocate 64KB buffers to receive data and see them getting a gazillion 1480/1472 byte messages.
When downloading a file from a Web Site, speeds of many megabytes per second can be acheived. If TCP needs to break up and individually send packets over 1500 bytes, then how are these speeds possible? Doesn't the client have to wait for every 1500 byte fragment which should take a while?
Thanks
Doesn't the client have to wait for every 1500 byte fragment which
should take a while
No. That's the magic of TCP, you don't have to ACK every segment, you can ACK once in a while. The server can push lots of segments before the client positively must acknowledge at least some.
TCP uses a concept called "windows". A sender can push data into a window, causing it to shrink. The receiver acknowledges data, causing the window to expand. If the receiver doesn't acknowledge data, the transfer grinds to a halt.
In modern TCP knowing when to acknowledge data is the gist of the protocol. Doing it too often or not often enough has enormous impacts on performance.