I had to implement an application which in very short, sent packets every few seconds to a server, when the server received them it send a response to the client which only then proceeded to send another packet. This sounds all good but we were using TCP and the responses came as soon as the server got the packet and not post-processing or anything like that. So that makes me wonder, why would you do something like this? The client had a queue where I kept all the packets and did something like this:
try {
send packet // exception is thrown if connection is lost
remove packet from queue
} catch exception {
try to reconnect
}
so in this case the packet gets removed from the queue only if the send was successful.
Any idea about this? Is this best practice? I would appreciate if someone could clear this for me.
Thanks
One option would be to put the packets into a queue and send them. After sending move them into a "pending" queue. Once the other end has processed them, you mark them as completed. Then you are up against other problems. What if the other end processes them but the ack never gets to your end? This is a relatively researched problem and I suggest you research distributed transactions and two phase commit, if you need to be sure.
Sending isn't enough in some cases. If it's absolutely critical that the data you're shoving out the door MUST be received, then you should wait for an acknowledgement that the packet was received/processed by the remote end.
Even if the network-level stuff works perfectly and the packets arrive at the destination, that destination machine might still crash or otherwise lose the data. If you remove-on-send, then that data's gone. Waiting for acknowledgements from the remote end would at least give you the ability to resend packets that were corrupted/lost.
Related
The description of the questions goes like this:
Someone recorded all the IP packets of a TCP connection between a client and a server for 30 minutes. In the record, he didn't find any packet that was ACK-only. How is this possible?
What is claimed to be a possible solution: For all the record time, the server sent data to the client, which the client processed, but he didn't send any data back to the server.
I am having trouble understanding how can it be possible.
From what I see, since the client didn't send any data to the server, and there weren't any ACK-only packets in the record, then the server didn't get any ACK from the client. Logically, I would think that since no ACK is received by the server, it will always do re-transmit. But also, since the server doesn't get anything from the client for 30 minutes, which seems like a long time for me, it will conclude that the connection is broken and stop it. (maybe even send an ACK only, but I am not sure about it).
Moreover, from what I know, when using keepalive, the sender gets and ACK-only packet from his peer.
Can anyone help me understand this?
Help would be appreciated
Perhaps more details would be helpful here. What kind of server/client? What protocol is being used and for what purpose?
Is the connection running as expected and this is just viewed as strange traffic you are trying to understand or is the connection timing out?
Some devices or softwares can be set to a "No ACK" state which means that no ACKs are sent nor are they expected.
One reason for this is usually bandwidth. ACKs do consume bandwidth and there are cases where bandwidth is such a great premium that packets being lost is preferable to bandwidth being consumed by ACKs. That type of traffic would probably do better with a UDP protocol but that is a completely different topic.
Another reason is that you don't care if packets are lost. Again, this would be better off as UDP instead of TCP, but someone may be trying to work within strange parameters is bending the rules about what type of traffic to advertise as in order to get around some issue.
Hopefully this is helpful, but if it does not apply, then please put in more details about the connection so that we can better understand what may be happening.
I am talking about only one case here.
client sent a request to server -> server received it and returned a response -> unfortunately the response dropped.
I have only one question about this.
Is this case even possible? If it's possible then what should the response code be, or will client simply see it as read timeout?
As I want to sync status between client/server and want 100% accuracy no matter how poor the network is, the answer to this question can greatly affect the client's 'retry on failure' strategy.
Any comment is appreciated.
Yes, the situation you have described is possible and occurs regularly. It is called "packet loss". Since the packet is lost, the response never reaches the client, so no response code could possibly be received. Web browsers will display this as "Error connecting to server" or similar.
HTTP requests and responses are generally carried inside TCP packets. If a TCP packet carrying the HTTP response does not arrive in the expected time window, the request is retransmitted. The request will only be retransmitted a certain number of times before a timeout error will occur and the connection is considered broken or dead. (The number of attempts before TCP timeout can be configured on both the client and server sides.)
Is this case even possible?
Yes. It's easy to see why if you picture a physical cable between the client and the server. If I send a request down the cable to the server, and then, before the server has a chance to respond, unplug the cable, the server will receive the request, but the client will never "hear" the response.
If it's possible then what should the response code be, or will client simply see it as read timeout?
It will be a timeout. If we go back to our physical cable example, the client is sitting waiting for a response that will never come. Hopefully, it will eventually give up.
It depends on exactly what tool or library you're using how this is wrapped up, however - it might give you a specific error code for "timeout" or "network error"; it might wrap it up as some internal 5xx status code; it might raise an exception inside your code; etc.
I have been testing a program which has simple communication between two machines over a 1Gbps line. While running TCP communications over the line I occasionally receive write errors on the client side (due to a timeout) when the network is totally flooded (running at or close to 100% usage). This generally happens when I am running multiple instances of the same program going to different ports.
My question is, is it possible to get a write error but still receive the message on the server side. It appears that is what is happening, and I am not quite sure why. Could it be that the ACK coming back to the client is what is timing out?
Yes, that is possible. TCP does not guarantee you that data you sent successfully is received and that data that is sent unsuccessfully is not received. This problem is unsolvable. It is called the Generals Problem. There is always a way to loose messages/packets such that the sender comes to the wrong conclusion. TCP guarantees that the receiver receives the same stream of bytes that the sender sent, but possibly cut off at an arbitrary point.
This unreliability has performance reasons, too. TCP data is buffered on both hosts as well as on the network. Acknowledgement is delayed.
You have to live with this. If you make your scenario more concrete I can suggest some strategies of dealing with this.
send puts data into the TCP send buffer.
If the send buffer has no enough space, send will block util the data is completely or partly copied into the send buffer, or the designed timeout arrives.
Read timeout and write timeout is OK. You should check and process them. The way is restarting read/write operation after timeout. You also pay attention to other read/write error except timeout.
Say I am capturing data from TCP using RECV function in c++.
I might sound stupid but I would like to know will I get any speed up if I capture the packet through a simple sniffer (maybe using PCAP) and process it?
Thanks
No, it probably won't speed up anything. I rather expect it to be even slower and more memory-consuming. (overhead, overhead, overhead...).
Additionally, it won't work at all.
No payload will be exchanged if there isn´t a real client
which creates a proper connection with the peer.
If there is a connection and you´re relying only on the sniffer without proper receiving the payload in the client, the whole transfer will stop after some amount of data. (Because the buffer is full, and the sender won't send anymore until there is space again).
That means you must call recv, which makes sniffing useless in the first place.
UDP package are like once gone you will never know that it is received or not. So it is not gurenteed weather package will be received. I have learned that mostly DNS use UDP package to send the request. How does the loss of DNS requested using UDP are handled?
If no response packet is received within a certain amount of time, the request is re-sent. Dan Bernstein suggests that most clients will retry up to four times.
There are two ways to decide that a UDP datagram has been lost. Neither is completely reliable.
The most common is a timeout. You send a message and wait for a response. If you don't get a response after some amount of time, you assume that either the message or the response were lost. At that point you can either try again, or give up. It is also possible that the message or response is just taking a really long time to get through the network, so you must account for duplicates. Note that all packet-switched communication, including TCP, works this way. TCP just hides the details for you.
Another method is to look for ICMP messages telling you that a packet has been dropped. For example, ICMP_UNREACH_PORT, ICMP_UNREACH_HOST, or ICMP_UNREACH_HOST_PROHIB. But not only are these messages rarely sent and subject to loss themselves, but you can sometimes receive them even when a message did get through successfully. At best, if you get an ICMP message you can think of it as a suggestion of what might have happened.
Most DNS implementations use a short timeout because duplication is no big deal. After a few repeats to one DNS server, it will try another one (assuming multiple servers are available). Most implementations will also cache information about which servers are responding and which are not.