I am working in a legacy VB6 project that involves TCP Communication using sockets.
Application requires long lived socket connections but i doubt to how much extent it is possible.
Application runs in windows OS.
If after a socket connection in established and some data is exchanged at regular intervals say 30 sec,
will the connection lives forever or is there any constraints from OS or TCP stack or network provider?
There is no technical limitation against having a TCP socket connected for months on end.
Skype will have a 24/7 TCP session established to a server as long as it's running.
The idea of TCP is that you:
syn→
← synack
ack→
And now your connection is established. Barring some technical issue (rebooted machines), that TCP connection will stay up forever. Yes there may be various edge firewalls that have to maintain stateful information about that TCP/IPv6 (or TCP/IPv4) connection - but that is the firewall's problem.
Their job is to route traffic. If they can't handle n open connections: then they need to be redesigned.
Related
I was discussing Websocket overhead with a fellow engineer and we were both uncertain of how a Websocket actually detects the status of a client connection.
Is there a "status" packet sent to the client/server periodically?
Does it have anything to do with ping or pong in the low level API?
What about frames?
How does the Websocket detect that it's disconnected on the client? the server?
I'm surprised I couldn't find this answered on SO, but that may be my mistake. I found this answer addresses scalability, but that's not what I'm asking here. This answer touches on implementation, but not at the depth I'm pursuing here.
A webSocket connection is a TCP connection that uses the webSocket protocol. By default, a server or client knows when the connection has disappeared only when the underlying TCP realizes that the connection has been closed and the webSocket layer is listening for a close event on the connection so it is informed that way.
The webSocket protocol does not, by itself, require heartbeat packets that can regularly test if the connection is still working. The TCP socket may still appear to be alive, but the connection may not actually still work. The other end could have disappeared or been interrupted in between and one or both endpoints might not know that at any given time.
Socket.io which is built on top of webSocket, uses the ping and pong packets to implement a heartbeat which regularly tests the connection and will, in fact, detect a non-functioning connection at the client, close the socket and then reconnect automatically.
Is there a "status" packet sent to the client/server periodically?
Not by default for a regular webSocket connection.
Does it have anything to do with ping or pong in the low level API?
It is up to a client or server if they want to send ping or pong packets themselves to implement some sort of connection validation detection.
What about frames?
webSocket frames are the data format for sending data over a webSocket. They don't have anything to do with this issue.
How does the Websocket detect that it's disconnected on the client? the server?
Described above. Unless the client/server implement their own ping/pong system to detect when a connection has gone awry, they just rely on TCP signaling to know when a connection has been closed by the other end. A webSocket connection can be non-functional without the client or server knowing it until they try to send.
When a browser window/tab opens a webSocket connection and then the window/tab is retargeted to a new URL, the browser will close all resources associated with that window/tab including any webSocket connections. If the link between client and server is functional at that point, then the server will be told the underlying TCP connection (and thus the webSocket) has been closed. If the network link goes down and then the user moves that window/tab to a new URL, the server will not necessarily know that the connection is non-functional without relying on ping/pong type signalling to regularly test the connection.
If the browser crashes, the OS should close any sockets opened by that process.
If the computer/OS crashes, the sockets will likely not get closed normally (though this may be somewhat OS dependent and may be crash-dependent too).
Consider such a scenario, there are client-a and server-b. server-b has TCP keepalive disabled. And server-b does not have any application logic to check whether a TCP connection is open. client-a establishes a TCP connection to server-b. There is no data transfer between client-a and server-b afterward. In such a case, will the server ever close the idle connection? Or the server will have the TCP connection stay open forever.
After reading Longest Open TCP Connection?, I guess that such a TCP connection will stay open forever. Is my guess correct?
There is no limit in the TCP connection itself. Client and server could in theory stay connected for years without exchanging any data and without any packet flow. Problems are usually caused by middleboxes like NAT router or firewalls which keep a state and expire the state after some inactivity. Any new packets sent within the connection cannot be delivered then because no associated state exists anymore in the middlebox.
I'm running service A and service B on the same host. They connect to each other using TCP loopback. The sender always creates a new TCP connection each time it sends a message and closes the connection immediately. Because most of the closed connections stay in TIME_WAIT state, it runs out of ephemeral ports soon given the QPS is high.
I wonder if using unix domain socket solves this problem. Does it have the same TIME_WAIT state as TCP?
TIME_WAIT is a TCP construct designed to protect against problems due to "half-open" connections, or misaligned connection state between remote and local parties. See [3] state diagram on p23, half-open connections around p33.
With Unix sockets, the OS completely knows the connection state of both ends of the socket - it's local-only. There's no FIN/FIN-ACK handshake in Unix sockets, and TIME_WAIT is not necessary.
I'm not sure if TIME_WAIT is used with local TCP connections. It should not be necessary for the same reason that Unix sockets don't need it, but is likely still present to a) avoid optimizing for the unusual local-only case and b) avoid deviations in behavior between "localhost" TCP connections and remote connections.
And a quick thing to look at from your command line: netstat -anop. You'll never see unix sockets in a TIME_WAIT state. :)
So yes: Unix domain sockets should address the "time-wait loading" issue. If your interface is local-only, Unix sockets have less overhead to deal with as well - some slight performance gains to be had there. And once connected or listening, their behavior from an API standpoint is indistinguishable from TCP sockets.
References:
The TIME-WAIT state in TCP and Its Effect on Busy Servers
AF_UNIX domain - why use local file names only? which discusses why the Unix-socket-on-a-remote-share trick doesn't work
IETF RFC 793 - Transmission Control Protocol
I have client and server component. Server may be installed behind the firewall or load balancer. Many sites/forums suggested to use TCP keep-alive feature to avoid connection termination due to inactivity.
The question is whether the keep-alive message from client will actually reach to server?
I tried to simulate the deployment using tcptrace utility and found that the keep-alive messages does not reach to server still the client was getting ACK for keep alive message.
I am not sure whether LB/FW work in same manner.
Is the keep-alive good option to avoid connection termination due to inactivity over socket in case of firewall and load balancer?
The answer is, of course: "it depends".
Many firewalls and load balancers maintain separate frontend and backend TCP connections, e.g.:
client <-- TCP --> firewall/balancer <-- TCP --> server
For situations like this, using TCP keepalive will not work as you'd expect. Why not? The TCP keepalive works for that TCP session only, and the keepalive probe packets are more like "administrative overhead" packets that data-bearing packets. This means that a) using TCP keepalive on the client end only means keeping the TCP connection to the firewall/balancer alive, and b) the firewall/balancer does not "forward" those keepalive probe packets across to the backend connection.
So is using TCP keepalive useful? Yes. There are other types of proxies which work at lower layers in the OSI stack, and which do forward those packets; using TCP keepalive is good for keeping your idle connection alive through those types of network intermediaries.
If your client/server application uses a long-lived, possibly idle TCP connection through firewalls/balancers, the best way to ensure that that connection is not torn down (sometimes politely, e.g. with a RST packet sent by the firewall/balancer, sometimes silently) is to use a "ping" or "heartbeat" message at the application layer. (Think of this as an "application keepalive".) This is just some kind of message that is sent e.g. from the client to the server. A simple and effective technique is to have the client periodically send some bytes to the server, which the server echoes back to the client. The client knows which bytes it sent, and when it receives those same bytes back from the server, it knows that everything in the network path is still working as expected.
Hope this helps!
Oddly I didn't find this info by googling. What is the cost of establishing connection using Unix Domain sockets versus TCP sockets?
Right now I have to do connection pooling with TCP sockets because reconnecting is quite expensive. I wonder if I can simplify my client by simply switching to Unix Domain sockets and getting rid of connection pooling.
If you look into the code, you'll see that Unix Domain sockets execute far less code than TCP sockets.
Messages sent through TCP sockets have to go all the way through the networking stack to the loopback interface (which is a virtual network interface device typically called "lo" on Unix-style systems), and then back up to the receiving socket. The networking stack code tacks on TCP and IP headers, makes routing decisions, forwards a packet to itself through "lo", then does more routing and strips the headers back off. Furthermore, because TCP is a networking protocol, the connection establishment part of it has all kinds of added complexity to deal with dropped packets. Most significantly for you, TCP has to send three messages just to establish the connection (SYN, SYN-ACK, and ACK).
Unix Domain sockets simply look at the virtual file system (or the "abstract namespace") to find the destination socket object (in RAM) and queue the message directly. Furthermore, even if you are using the file system to name your destination socket, if that socket has been accessed recently, its file system structures will be cached in RAM, so you won't have to go to to disk. Establishing a connection, for a Unix Domain socket involves creating a new socket object instance in RAM (i.e., the socket that gets returned by accept(), which is something that has to be done for TCP too) and storing a pointer in each of the two connected socket objects (so they each have a pointer to the other socket later when they need to send). That's pretty much it. No extra packets are needed.
By the way, this paper suggests that Unix Domain sockets are actually faster than even Pipes for data transfers:
http://osnet.cs.binghamton.edu/publications/TR-20070820.pdf
Unfortunately, they didn't do specific measurements of connection establishment costs, but as I have said, I've looked at the Linux source code and it's really quite a lot simpler than the TCP connection establishment code.
Connecting to a server using TCP sockets may involve network traffic, as well as the TCP three-way handshake.
Local sockets (formerly known as Unix domain sockets) are all local, but need to access a physical file on disk.
If you only do local communication then local sockets might be faster as there is less overhead from the protocol. If your application needs to connect remotely then you can't use local sockets.
By the way, if you're only communicating locally, and not over a network, a pair named pipes (or anonymous if you're forking) might be even better.