When the receiver of a TCP data stream closes its receive window (or, more exactly, the window is closed by the sender filling it up), there will be a flurry of packets which Wireshark would identify as TCP ZeroWindow and TCP Keep-Alive (repeated ACK with same sequence number). Some time later the receiver will send a new ACK to re-open the window (TCP Window Update) and data will start flowing again.
What TCP timer mechanism ensures that the window update packet is retransmitted if data does not start flowing?
Here is the sequence of packets that I see at the end of a connection (more data is expected):
No. Time Source Destination Protocol Length Info
122160 21:24:37.421824 192.168.15.121 72.21.81.253 TCP 60 41200 > http [ACK] Seq=462 Ack=2984241 Win=5152 Len=0
122162 21:24:37.445528 72.21.81.253 192.168.15.121 TCP 1514 [TCP segment of a reassembled PDU]
122163 21:24:37.445796 72.21.81.253 192.168.15.121 TCP 1514 [TCP segment of a reassembled PDU]
122164 21:24:37.446087 72.21.81.253 192.168.15.121 TCP 1514 [TCP segment of a reassembled PDU]
122171 21:24:37.481802 192.168.15.121 72.21.81.253 TCP 60 41200 > http [ACK] Seq=462 Ack=2988621 Win=784 Len=0
122184 21:24:37.744838 72.21.81.253 192.168.15.121 TCP 838 [TCP Window Full] [TCP segment of a reassembled PDU]
122185 21:24:37.745048 192.168.15.121 72.21.81.253 TCP 60 [TCP ZeroWindow] 41200 > http [ACK] Seq=462 Ack=2989405 Win=0 Len=0
122190 21:24:38.014841 72.21.81.253 192.168.15.121 TCP 60 [TCP Keep-Alive] http > 41200 [ACK] Seq=2989404 Ack=462 Win=15872 Len=0
122191 21:24:38.014993 192.168.15.121 72.21.81.253 TCP 60 [TCP ZeroWindow] 41200 > http [ACK] Seq=462 Ack=2989405 Win=0 Len=0
122232 21:24:38.534437 72.21.81.253 192.168.15.121 TCP 60 [TCP Keep-Alive] http > 41200 [ACK] Seq=2989404 Ack=462 Win=15872 Len=0
122233 21:24:38.534599 192.168.15.121 72.21.81.253 TCP 60 [TCP ZeroWindow] 41200 > http [ACK] Seq=462 Ack=2989405 Win=0 Len=0
122314 21:24:39.564525 72.21.81.253 192.168.15.121 TCP 60 [TCP Keep-Alive] http > 41200 [ACK] Seq=2989404 Ack=462 Win=15872 Len=0
122315 21:24:39.564680 192.168.15.121 72.21.81.253 TCP 60 [TCP ZeroWindow] 41200 > http [ACK] Seq=462 Ack=2989405 Win=0 Len=0
122361 21:24:43.403052 192.168.15.121 72.21.81.253 TCP 60 [TCP Window Update] 41200 > http [ACK] Seq=462 Ack=2989405 Win=119904 Len=0
122892 21:25:45.161896 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
122902 21:25:45.373289 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
122927 21:25:45.813267 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
122936 21:25:46.693275 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
122956 21:25:48.453337 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
123009 21:25:51.983392 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
123061 21:25:59.033566 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
123262 21:26:13.153852 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
123460 21:26:41.394469 192.168.15.121 72.21.81.253 TCP 60 41200 > http [FIN, ACK] Seq=462 Ack=2989405 Win=186720 Len=0
The receiver re-opens the window at 21:24:43 and hears nothing more from the sender. A minute later the receiver times out the connection (the close is initiated by the application) and sends a series of FIN-ACKs which remain unacknowledged.
It looks like communication with the sender is simply lost (the capture was taken on the receiver's network). If not, then should one always expect an acknowledgement to the FIN-ACKs, even after a period long enough for the peer to have forgotten the connection?
Despite what RFC1122 may say on the matter, is it (common) practice for large Internet servers to do something different in this regard, perhaps as counter-DoS measures?
As you noted, the receiver sends a new ACK with an updated window size once space is available in its receive window.
In order to handle the case where that ACK is lost, the sender keeps a "persist timer" that will occasionally re-transmit a packet (a "window probe") in order to test the waters and see if there is indeed unreported receive space.
The value of the persist timer is not specified explicitly but rather is a function of the calculated round-trip time along with some exponential backoff. Full details are in RFC1122 in section 4.2.2.14, here: https://www.rfc-editor.org/rfc/rfc1122#page-92
The trace provided looks like something in the middle is blocking return traffic from the server. My guess would be that your NAT gateway (192.168.. is not a valid address on the Internet so something in between is doing network address translation) has decided the connection is gone and is refusing to forward additional packets back from the server.
Related
I have an intermittent but reproducible issue with a TCP connection.
Sender (port 64613) is running on a Windows box while the receiver (port 14004) is on a RedHat 6 host.
Ports are not part of the picture: the same issue happens with different ports.
The connection works fine for over a minute or so, with data and acknowledgement packets flowing normally.
But then the following behavior happens: the Wireshark capture on the sender side shows a packet being sent (seq=3020828):
57788 2018-07-16 15:36:20.552618000 10.245.40.74 10.245.54.13 TCP 2974 64613 -> 14004 [ACK] Seq=3020828 Ack=73535403 Win=65536 Len=2920
for which no ACK is ever received back.
The sender also keeps re-transmitting it in a smaller packet, up to 5 times before eventually giving up the connection:
58376 2018-07-16 15:36:20.851770000 10.245.40.74 10.245.54.13 TCP 1514 [TCP Retransmission] 64613 -> 14004 [ACK] Seq=3020828 Ack=74313583 Win=1296 Len=1460
58378 2018-07-16 15:36:21.101721000 10.245.54.13 10.245.40.74 TCP 1350 14004 -> 64613 [PSH, ACK] Seq=74313583 Ack=3020828 Win=4096 Len=1296 [TCP segment of a reassembled PDU]
...
60992 2018-07-16 15:36:22.652682000 10.245.40.74 10.245.54.13 TCP 1514 [TCP Retransmission] 64613 -> 14004 [ACK] Seq=3020828 Ack=77762103 Win=1296 Len=1460
60994 2018-07-16 15:36:22.658427000 10.245.54.13 10.245.40.74 TCP 1514 14004 -> 64613 [ACK] Seq=77762103 Ack=3020828 Win=4096 Len=1460 [TCP segment of a reassembled PDU]
...
91947 2018-07-16 15:36:39.456903000 10.245.40.74 10.245.54.13 TCP 54 64613 -> 14004 [RST, ACK] Seq=3022288 Ack=118789563 Win=0 Len=0
The tcpcap on the receiver side shows the packet being received (just in smaller splits, due to intermediate network device):
13573 2018-07-16 15:36:20.526327000 10.245.40.74 10.245.54.13 TCP 1514 64613 -> 14004 [ACK] Seq=3020828 Ack=73535403 Win=65536 Len=1460
13575 2018-07-16 15:36:20.526360000 10.245.40.74 10.245.54.13 TCP 1514 64613 -> 14004 [ACK] Seq=3022288 Ack=73535403 Win=65536 Len=1460
Also, the retransmitted packets are received fine, up to the final RESET packet:
13878 2018-07-16 15:36:20.825430000 10.245.40.74 10.245.54.13 TCP 1514 [TCP Retransmission] 64613 -> 14004 [ACK] Seq=3020828 Ack=74313583 Win=1296 Len=1460
18810 2018-07-16 15:36:36.047313000 10.245.54.13 10.245.40.74 TCP 29254 14004 -> 64613 [ACK] Seq=114189103 Ack=3020828 Win=20480 Len=29200 [TCP segment of a reassembled PDU]
...
13991 2018-07-16 15:36:21.425465000 10.245.40.74 10.245.54.13 TCP 1514 [TCP Retransmission] 64613 -> 14004 [ACK] Seq=3020828 Ack=74834803 Win=1296 Len=1460
13993 2018-07-16 15:36:21.433178000 10.245.54.13 10.245.40.74 TCP 27794 14004 -> 64613 [ACK] Seq=74834803 Ack=3020828 Win=4096 Len=27740 [TCP segment of a reassembled PDU]
...
14388 2018-07-16 15:36:22.626436000 10.245.40.74 10.245.54.13 TCP 1514 [TCP Retransmission] 64613 -> 14004 [ACK] Seq=3020828 Ack=77762103 Win=1296 Len=1460
14390 2018-07-16 15:36:22.632029000 10.245.54.13 10.245.40.74 TCP 32174 14004 -> 64613 [PSH, ACK] Seq=77762103 Ack=3020828 Win=4096 Len=32120 [TCP segment of a reassembled PDU]
...
19416 2018-07-16 15:36:39.431628000 10.245.40.74 10.245.54.13 TCP 60 64613 -> 14004 [RST] Seq=3020828 Win=0 Len=0
As you can see above, every time the receiver replies with the ACK message for the previous sequential number:
ACK for 3020828 instead of the new one (3020828 + 1460).
Eventually, the connection drops.
What could be a reason for the receiver to pick up new packets but producing ACKs for previous seq number?
What could be a reason for the receiver to pick up new packets but producing ACKs for previous seq number?
I would suggest that the recipient (i.e. the application at the RedHat system) has stopped reading the data. This will fill up the socket buffer and once the socket buffer is full no more data will be accepted by the OS kernel. The application is still writing though data to the original sender. But since the applications read socket buffer is full only the ACK for the last (and still unread) data in the socket buffer will be send together with the data written to the original sender.
I wrote a proxy server which support http and https connection, When I use with http all work fine, but when I work with https , wireshark report this error
'Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)'
Although the browsing work fine but performance is impacted as after this I see TCP RST and because of that SSL negotiation happen again.
Any clue on what could be wrong ?
169.254.119.252 169.254.1.66 TCP 66 54589 > ff-fms [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
169.254.1.66 169.254.119.252 TCP 60 ff-fms > 54589 [SYN, ACK] Seq=0 Ack=1 Win=16384 Len=0 MSS=1456
169.254.119.252 169.254.1.66 TCP 54 54589 > ff-fms [ACK] Seq=1 Ack=1 Win=65520 Len=0
169.254.119.252 169.254.1.66 TCP 245 [TCP segment of a reassembled PDU]
169.254.1.66 169.254.119.252 TCP 912 [TCP segment of a reassembled PDU]
169.254.119.252 169.254.1.66 TCP 252 54588 > ff-fms [PSH, ACK] Seq=192 Ack=859 Win=64662 Len=198[Reassembly error, protocol TCP: New fragment overlaps old data (retransmission?)]
169.254.1.66 169.254.119.252 TCP 91 [TCP segment of a reassembled PDU]
169.254.119.252 169.254.1.66 TCP 54 54587 > ff-fms [RST, ACK] Seq=391 Ack=955 Win=0 Len=0
Sorry for the bad formatting. But frame 7, I do not understand why a TCP retransmission was intiated. My understanding is that the retransmission occurs when an ACK is not returned in time and the send assumes that packet loss occurred and retransmits everything. What ACK not being returned resulted in this?
TCP: 57190 > 6007 [PSH, ACK] Seq=1 Ack=1 Win=64162 Len=200
TCP: 57190 > 6007 [PSH, ACK] Seq=201 Ack=1 Win=64162 Len=200
TCP: 57190 > 6007 [PSH, ACK] Seq=401 Ack=1 Win=64162 Len=200
TCP: 57190 > 6007 [PSH, ACK] Seq=601 Ack=1 Win=64162 Len=62
TCP: 6007 > 57190 [ACK] Seq=1 Ack=4294966897 Win=64240 Len=0 SLE=1 SRE=601
TCP: [TCP Dup ACK 5#1] 6007 > 57190 [ACK] Seq=1 Ack=4294966897 Win=64240 Len=0 SLE=1 SRE=663
TCP: [TCP Retransmission] 57190 > 6007 [PSH, ACK] Seq=4294966897 Ack=1 Win=64162 Len=1062
TCP: 6007 > 57190 [ACK] Seq=1 Ack=663 Win=63178 Len=0 SLE=1 SRE=663
TCP: 6007 > 57190 [PSH, ACK] Seq=1 Ack=663 Win=63178 Len=78
TCP: 57190 > 6007 [ACK] Seq=663 Ack=79 Win=64084 Len=0
Thanks,
Kyle
In TCP the receiver replies with ACK# of the next packet that it expects. If that packet is sent by the sender but is lost in the network and the receiver receives the packet next to the packet it was expecting, it will result in receiver sending another ACK with ACK# of the packet it is expecting next (in this case, the lost packet). And the sender after receiving the second duplicate ACK understands that the receiver hasn't received that particular payload (must have been lost) and it re-transmits it.
I am writing a program that fakes TCP requests and collects the data to store in a local buffer. For this, in the system connected to the client i have configured the iptables to keep all the incoming packets to a queue before routing. Then i use the netfilter library to read the packets from the queue. After this using RAW sockets I send the fake TCP packets to the client. With this I am able to fake the SYN/ACK packet in response to the SYN request from the client.
But issue happens when I try to fake an ACK to the client in response to the incoming data. In this case the real ip of the source comes in the packet and not the faked one. Please see 7th trace below marked with ">>>". In this the source ip is shown as 192.168.10.10 where as it has to be 212.58.246.81. In the 4th trace(i.e. SYN/ACK packet) its showing as fine.
3 0.073852000 192.168.10.100 212.58.246.81 TCP 38307 > http [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=502233100 TSER=0 WS=6
4 0.103102000 212.58.246.81 192.168.10.100 TCP http > 38307 [SYN, ACK] Seq=0 Ack=1 Win=31744 Len=0
5 0.103147000 192.168.10.100 212.58.246.81 TCP 38307 > http [ACK] Seq=1 Ack=1 Win=5840 Len=0
6 0.103349000 192.168.10.100 212.58.246.81 HTTP GET /go/rss/int/news/-/sport2/hi/tennis/9519762.stm HTTP/1.1
>>> 7 1.118729000 192.168.10.10 192.168.10.100 TCP http > 38307 [ACK] Seq=1 Ack=1 Win=31744 Len=0
8 1.118788000 192.168.10.100 192.168.10.10 TCP 38307 > http [RST] Seq=1 Win=0 Len=0
9 3.102627000 192.168.10.100 212.58.246.81 HTTP [TCP Retransmission] GET /go/rss/int/news/-/sport2/hi/tennis/9519762.stm HTTP/1.1
10 3.148590000 192.168.10.10 192.168.10.100 TCP [TCP Dup ACK 7#1] http > 38307 [ACK] Seq=1 Ack=1 Win=31744 Len=0
11 3.148606000 192.168.10.100 192.168.10.10 TCP 38307 > http [RST] Seq=1 Win=0 Len=0
Also I have tried out "sendip" command like below to send a fake TCP ACK
sendip -p ipv4 -p tcp -is 212.58.246.81 -id 192.168.10.100 -ts 80 -td 4567 -tfa 1 -tfs 0 -d "Data" 192.168.10.100
here tfa and tfs stands for ack and syn flags respectively.
This also didnt work as expected and its shown as orginating from 192.168.10.10 instead of 212.58.246.81. But if I set both flags(syn and ack) as 1 then its working fine.
The OS is Ubuntu. Can anyone please let me know where I am going wrong. Thanks a lot for your help.
Is it okay if a machine sets the TCP windowsize to zero after receiving a FIN?
I've got the following packet dump from wireshark of the end of the connection and I'm just wondering if this is a valid way to end a connection or if something is wrong.
192.168.1.1 192.168.1.6 TCP 3450 > 102 [FIN, ACK] Seq=48 Ack=50 Win=65486 Len=0
192.168.1.6 192.168.1.1 TCP [TCP ZeroWindow] 102 > 3450 [ACK] Seq=50 Ack=49 Win=0 Len=0
192.168.1.6 192.168.1.1 TCP 102 > 3450 [FIN, PSH, ACK] Seq=50 Ack=49 Win=0 Len=0
192.168.1.1 192.168.1.6 TCP 3450 > 102 [ACK] Seq=49 Ack=51 Win=65486 Len=0
BTW: .1 is a regular windows PC while .6 is a Siemens PLC. (S7-400)
After some investigation it looks like a weird but valid way to end a TCP conversation.
I see nothing wrong with sending a zero window after a FIN ACK... presumably 192.168.1.6 sent a FIN to 192.168.1.1, so they are now closing the connection.
192.168.1.6 192.168.1.1 TCP [TCP ZeroWindow] 102 > 3450 [ACK] Seq=50 Ack=49 Win=0 Len=0
But immediately setting a PSH flag and sending no data (Len=0) right after that ACK, looks rather odd (but not technically wrong) to me...
192.168.1.6 192.168.1.1 TCP 102 > 3450 [FIN, PSH, ACK] Seq=50 Ack=49 Win=0 Len=0