how tcp stack distinguish close and shutdown? - tcp

As we known that:
/////////////////////////////////////////////////////////
close() will terminate both directions on a tcp connection
shutdown() can block communication in one or both directions
/////////////////////////////////////////////////////////
here,what puzzled me is how tcp stack can tell them apart?
I have write an example program:
firstly I use :
....
connect(192.168.1.100) //there is a server process running in 192.168.1.100
....
close(socketfd);
sleep(1000);
then I use wireshark to dump the packets:
01 -->syn
02 <--syn,ack
03 -->ack
04 -->fin,ack
05 <--ack
netstat -an |grep 192.168.1.100
I have run it for about 5 minutes, it print:
"tcp 0 0 ... FIN_WAIT2 " at first,then after about 2 minutes there is no output,seems that the connection has been destroyed.
then, I use :
....
connect(192.168.1.100)
....
shutdown(socketfd,SHUT_WR);
sleep(1000);
use wireshark to dump the packets:
01 -->syn
02 <--syn,ack
03 -->ack
04 -->fin,ack
05 <--ack
...
netstat -an |grep 192.168.1.100
run it for about 10 minutes, it always print:
"tcp 0 0 ... FIN_WAIT2"
from the output of wireshark ,it seems there is no different for close and shutdown,
but from the output of netstat ,its behaviour is different.
so why the behaviour is different?

There is no difference on the wire between just closing the socket and doing shutdown(SHUT_RDWR). Either will cause the TCP stack to emit a FIN and stop accepting packets from the other end. The only difference is in your program: after shutdown the file descriptor remains valid, but after close it doesn't. To be sure, you cannot do much with the still-valid file descriptor after shutdown besides closing it, but it is valid nonetheless.
In your case you use SHUT_WR, not SHUT_RDWR, so your socket is still prepared to receive data from the other end. But your server doesn't send any data. You would see a difference between close and shutdown(SHUT_WR) if your server sent some data after the client had closed its end. The close() client would respond to the server's data with RST whereas the shutdown(SHUT_WR) client would accept the server's data and ACK it. A shutdown(SHUT_RDWR) client would behave the same as the close() client.
In most client-server TCP protocols (like HTTP for instance), the server will terminate its end of the connection anyway as soon as it receives a FIN from the client. So with those protocols it doesn't make much of a difference whether the client half-closes or full-closes the socket because the server will promptly close its end of the connection anyway. I can see that your server does not behave like this though, because I don't see it sending its own FIN (only your client sends a FIN).

Related

I would like to resolve when recv function doesn't return 0 when tcpip is disconnected

Is there a case that recv function doesn't return 0 when tcpip is disconnected physicially? There is no calling return 0. So, Tcpip is no closed. What should I do to resolve this issue?
Recv returns 0 only if TCP connection is closed explicitly by remote side.
Another cases of disconnect:
If TCP connection is explicitly broken (TCP RST or ICMP unreachable packet received) then you can get error when read from the socket.
If transit network goes down or remote host disappears then:
if timeout explicitly specified then read waits and return timeout error
if timeout is not specified then read waits maximum timeout (depends on system for Linux it is about 2 hours)
In last case you may:
Try to enable keepalive requests on socket (SO_KEEPALIVE socket options)
Do keepalive by yourself on application layer:
Send something to the peer and expect answer from it
If nothing received within timeout - close connection

Nodemcu: UDP communication gets influenced if there is/isn't udp listener?

I have a weird problem with nodemcu firmware (2.1.0) on an ESP8266, where I'm running out of ideas what else I could try to solve the issue.
I have a simple lua script running, which is listening on UDP for commands to switch a relay on and off, and sending alive messages via UDP every 60 seconds to a defined IP address.
If there is nothing listening on the server side which is supposed to get the UDP "alive" messages, ESP reacts fine, all good.
As soon as I start netcat to listen to the UDP packages coming from ESP, the ESP will start to hang every couple of minutes for at least 30-60 seconds.
It is specifically confusing as I'm expecting UDP to be a connectionless protocol. So how can a listener on UDP influence the behavior of the sender?
These are the relevant parts of the lua script:
[...]
alive=60000
[...]
function srvupd(s)
if (connected==1) then
s = s .." "..ip
srv:send(serverport, serveradr, s.."\n")
end;
end;
if (alive>0) then
tmr.alarm(2, alive, 1, function()
srvupd("alive")
end)
end
srv=net.createUDPSocket()
srv:listen(80)
srv:on("sent", function()
srv:close();
srv:listen(80);
end)
srv:on("receive",function(client,request, port, ip)
if (alive>0) then tmr.stop(2) end
print(string.format("received '%s' from %s:%d", request, ip, port))
buf="unknown"
if (request == "ch1on") then gpio.write(relay1, relayon);buf="ok" end
[...]
client:send(port, ip, buf)
if (alive>0) then tmr.start(2) end
end)
And this is how I use netcat to listen to the UDP messages from ESP in a bash script:
#!/bin/bash
while true
do
msg=$(netcat -4 -u -n -l -D 192.168.0.5 2701 -w0 -q0)
echo -e "$msg"
done
In the situation where ESP is not reacting to UDP commands anymore, the alive messages are still sending every minute. The UDP commands are even received by the ESP, because as soon as processing continues, a "channel-on" command sent some time ago gets executed.
These temporary blockings of ESP only happens when I listen to its UDP messages.
I've checked all kind of combinations, like separate UDP-sockets for the listener and the alive sending on the ESP, closing and opening the server, after message was sent (like in the current version above) etc.
I've even tried to receive commands via TCP and only send the alive messages via UDP.
Behaviour remains the same. All works, as long as there is nothing receiving the UDP messages from ESP. As soon as I start netcat, ESP startes to hang within a couple of minutes.
Any ideas? As it is UDP it is already difficult to understand how it can happen at all.
kind regards
Tjareson
The issue is solved meanwhile. A friend of mine was pointing me to the only common basis for the UDP issue, which is ARP.
The behaviour only occured, when ESP was in a different network than the udp-listener. (like 192.168.1.x and 192.168.5.y)
Even if it remains a bit unclear, the guessing is that probably netcat is making an ARP request when receiving a message and that this somehow isn't properly processed by the router, if taking place between two different networks.
After putting the listener bashscript in the same network (basically by giving the raspberry on which it runs a second IP in the network the ESP is in), the blocked ESP communication didn't happen again.

TCP keep-alive gets involved after TCP zero-window and closes the connection erroneously

We're seeing this pattern happen a lot between two RHEL 6 boxes that are transferring data via a TCP connection. The client issues a TCP Window Full, 0.2s later the client sends TCP Keep-Alives, to which the server responds with what look like correctly shaped responses. The client is unsatisfied by this however and continues sending TCP Keep-Alives until it finally closes the connection with an RST nearly 9s later.
This is despite the RHEL boxes having the default TCP Keep-Alive configuration:
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_intvl = 75
...which declares that this should only occur until 2hrs of silence. Am I reading my PCAP wrong (relevant packets available on request)?
Below is Wireshark screenshot of the pattern, with my own packet notes in the middle.
Actually, these "keep-alive" packets are not used for TCP keep-alive! They are used for window size updates detection.
Wireshark treats them as keep-alive packets just because these packets look like keep-alive packet.
A TCP keep-alive packet is simply an ACK with the sequence number set to one less than the current sequence number for the connection.
(We assume that ip 10.120.67.113 refers to host A, 10.120.67.132 refers to host B.) In packet No.249511, A acks seq 24507484. In next packet(No.249512), B send seq 24507483(24507484-1).
Why there are so many "keep-alive" packets, what are they used for?
A sends data to B, and B replies zero-window size to tell A that he temporarily can't receive data anymore. In order to assure that A knows when B can receive data again, A send "keep-alive" packet to B again and again with persistence timer, B replies to A with his window size info (In our case, B's window size has always been zero).
And the normal TCP exponential backoff is used when calculating the persist timer. So we can see that A send its first "keep-alive" packet after 0.2s, send its second packet after 0.4s, the third is sent after 0.8, the fouth is sent after 1.6s...
This phenomenon is related to TCP flow control.
The source and destination IP addresses in the packets originating from client do not match the destination and source IP addresses in the response packets, which indicates that there is some device in between the boxes doing NAT. It is also important to understand where the packets have been captured. Probably a packet capture on the client itself will help understand the issue.
Please note that the client can generate TCP keepalive if it does not receive a data packet for two hours or more. As per RFC 1122, the client retries keepalive if it does not receive a keepalive response from the peer. It eventually disconnects after continuous retry failure.
The NAT devices typically implement connection caches to maintain the state of ongoing connections. If the size of the connection reaches limit, the NAT devices drops old connections in order to service the new connections. This could also lead to such a scenario.
The given packet capture indicates that there is a high probability that packets are not reaching the client, so it will be helpful to capture packets on client machine.
I read the trace slightly differently:
Sender sends more data than receiver can handle and gets zerowindow response
Sender sends window probes (not keepalives it is way to soon for that) and the application gives up after 10 seconds with no progress and closes the connection, the reset indicates there is data pending in the TCP sendbuffer.
If the application uses a large blocksize writing to the socket it may have seen no progress for more than the 10 seconds seen in the tcpdump.
If this is a straight connection (no proxies etc.) the most likely reason is that the receiving up stop receiving (or is slower than the sender & data transmission)
It looks to me like packet number 249522 provoked the application on 10.120.67.113 to abort the connection. All the window probes get a zero window response from .132 (with no payload) and then .132 sends (unsolicited) packet 249522 with 63 bytes (and still showing 0 window). The PSH flag suggests that this 63 bytes is the entire data written by the app on .132. Then .113 in the same millisecond responds with an RST. I can't think of any reason why the TCP stack would send a RST immediately after receiving data (sequence numbers are correct). In my view it is almost certain that the app on .113 decided to give up based on the 63 byte message sent by .132.

Linux Doesn't Respond to SYN on ESTABLISHED connection

So I have a remote device using a Lantronics XPort module connecting to a VPS. They establish a TCP connection and everything is great. The server ACKs everything.
At some point the remote device stops transmitting data. 30 seconds goes by.
The device then starts sending SYN packets as if trying to establish a new connection. The device is configured to maintain a connection to the server, and it always uses the same source port. (I realize this is bad, but it is hard for me to change)
The server sees a SYN packet from the same (source ip, source port), so the server thinks the connection is ESTABLISHED. The server does not respond to the SYN packet.
Why does the server not respond with ACK as described in Figure 10 in RFC 793? ( https://www.ietf.org/rfc/rfc793.txt )
How can I get the server to kill the connection or respond with an ACK?
It could be the case that during that 30 second silence, the device is waiting for an ACK from the server, and that ACK was dropped somewhere along the line. In this case, I think it should retransmit.
Server is running Ubuntu with kernel 3.12.9-x86_64-linode37
Thank you for any help!
My first suggestion is change the client to use the same connection or to gracefully close the connection before re-opening.
As you DO NOT have control over client and all that can do is on server, you can try this:
Configure keep-alive to be sent after 10 secs of silence and probe only once. If client does not respond, server closes the connection. By doing this, the server should be in listening mode again within 10 seconds of silence without client responding. You can play with the following sysctl's and arrive at optimal values.
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_keepalive_probes = 1
======
Also, regarding missing-ack that you have mentioned in your question, TCP takes care of those things. 30 seconds is too long a time for the first re-transmission from sender. If client/device does not get an ack after 30 seconds, it will/should not try to open a new connection. If you are seeing that, it is an insane-TCP stack at the client. What is that device and which OS/TCP-stack is it using?
it is kernel version has different behavior, ignore any syn packet in kernel 3.12.9-x86_64. but server ack a ack packet, client receive the ack resent rst, and sent new syn in kernel 4.9.0.
incoming-tcp-syns-possibilities
TCP packets ignored on ESTABLISHED connection

Can an application affect TCP retransmits

I'm troubleshooting some communications issues and in the network traces I am occasionally coming across TCP sequence errors. One example I've got is:
Server to Client: Seq=3174, Len=50
Client to Server: Ack=3224
Server to Client: Seq=3224, Len=50
Client to Server: Ack=3224
Server to Client: Seq=3274, Len=10
Client to Server: Ack=3224, SLE=3274, SRE=3284
Packets 4 & 5 are recorded in the trace (which is from a router in between the client and server) at almost exactly the same time so they most likely crossed in transit.
The TCP session has got out of sync with the client missing the last two transmissions from the server. Those two packets should have been retransmitted but they weren't, the next log in the trace is a RST packet from the Client 24 seconds after packet 6.
My question is related to what could be responsible for the failure to retransmit the server data from packets 3 & 5? I would assume that the retransmit would be at the operating system level but is there anyway the application could influence it and stop it being sent? A thread blocking or put to sleep or something like that?
Only one packet has been lost from server to client - packet 3. Packet 6 contains a selective acknowledgement (SACK) for packet 5, so that got through.
In answer to your specific question, no, application-level issues shouldn't prevent TCP retransmissions.

Resources