I've heard it said that providing exactly-once delivery is almost impossible. At the same time, TCP is said to provide guaranteed delivery. If TCP does not provide exactly-once guaranteed delivery then does it provide at-most-once or at-least-once
We could say that TCP provides at-least-once delivery and exactly-one processing, with regards to the following definitions:
At-least-once delivery: a TCP message will be delivered at least once to the destination. More specifically, it will keep re-transmitting with specific timeouts if no ACK(knowledgement) is received, so that it will eventually be delivered. However, if some of these re-transmissions were not lost (but just delayed), then more than one copies of the message will be delivered.
Exactly-once processing: each TCP message will be processed by the destination node exactly once. More specifically, the destination will watch out for duplicate messages (checking the IDs of each received message). So, even if a message is delivered twice, the destination node will only process it (pass it to the application level) once and ignore the duplicates received later.
Exactly once is clearly impossible. What if the network connection is severed and never recovers?
Related
When a server has only 1 UDP socket, and many clients are sending UDP packets to it, what would be the best approach to handle all of the incoming packets?
I think this can also be a problem with TCP packets, since there's a limited thread count, which cannot cover all client TCP socket receive events.
But things are better in this situation because there's 1 TCP socket per client, and even if the network buffer is full, packet receiving is blocked until the queue has space (let me know if I'm wrong).
UDP packets, however, are discarded when the buffer is full, and there's only 1 socket, so the chances of that happening are higher.
How can I solve this problem? I've searched for a while, but I couldn't get a clear answer. Should I implement my own queueing system? Or just maximize the network buffer size?
There is no way to guarantee you won't drop UDP messages. No matter what you do, if the rate of packets being sent is too large, you will drop some, either on the receiving host or somewhere in the network.
Some things that can help include:
Implementing an internal queue for messages in your Java app, and handing them over to a thread pool to process.
Increasing the kernel's message buffering.
But neither of these can deal with the case where the average message arrival rate is higher that the receiver's ability to process them or the network capacity. This will inevitably lead to lost messages (requests).
I've searched for a while, but I couldn't get a clear answer.
That is because there isn't one! Some problems are fundamentally unsolvable. For others, the best answer depends on factors that are too hard to measure or predict.
(If you want certainty ... don't use networking!)
In the TCP case, what you should do is use a (long-term) socket for each client. Depending on the number of sockets you need to support, you could either:
Dedicate a server-side thread to each socket (and client).
Use java.nio.channels.Selector and a thread pool.
You will still get problems if the rate of requests exceeds your server's ability to process them. However, the TCP connections will ensure that requests are not lost, and that the clients get some "back pressure".
Like this figure,enter image description here
I can find many openflow1.3 which are multipart requests in this packet, but I don't know why happened here?
Actually, isn't it only one openflow1.3 here ?
It is related to openflowjava do serialize, wireshark, NIC, tcp nagle's algorithim?
Thanks!
TCP is a byte stream, i.e. packet has no semantic from the perspective of the application layer (i.e. OpenFlow). At the transport level there can be multiple application layer "messages" in a single TCP packet, messages cross packet boundaries etc - it does not matter for the application. While often TCP packet boundaries are also message boundaries because of timing in the application and maybe a disabling of NAGLE algorithm, the assumption that TCP packet boundaries are always message boundaries is wrong and any reliance on this will often cause sporadic and hard to reproduce problems.
And based on this what you see is also not a "multipart request". These are just multiple OpenFlow messages (application level) send at the same time or shortly after each other and they are put together into the same transport level entity (packet) since this way it is less overhead to transport each message.
Client Send SYN with client isn.
Server reply SYN ACK with server's isn.
Client resend SYN if timeout?
When client send data, it can accumulated confirm the server's isn.
I try to search, but can't find the answer.
I know how the tcp is designed now, I just don't know why it's designed like this. Why can't use a two way handshake.
It cannot use a two way handshake by definition. TCP/IP is formalized as a standard for communication across networks (Internetworking). Specifically, RFC 793 requires that:
The "three-way handshake" is the procedure used to establish a
connection. This procedure normally is initiated by one TCP and
responded to by another TCP. The procedure also works if two TCP
simultaneously initiate the procedure. When simultaneous attempt
occurs, each TCP receives a "SYN" segment which carries no
acknowledgment after it has sent a "SYN". Of course, the arrival of
an old duplicate "SYN" segment can potentially make it appear, to the
recipient, that a simultaneous connection initiation is in progress.
Proper use of "reset" segments can disambiguate these cases.
If you think about how the protocol works, you do not actually have a single full duplex connection. Instead, you have two simplex connections, each going in one direction. This is why the proper response to a SYN is a SYN-ACK. The server is acknowledging the synchronization request by sending the originators sequence number as the acknowledgement number plus one; it is simultaneously attempting to open its own connection to the client by sending a SYN request to synchronize.
The proper answer to a SYN will always be an ACK, even if the connection fails.
Regarding your question about sending a retry, yes; the client will send a retry (typically up to three, but it could be as many as eight in practice... There's no limit defined) attempting to elicit a response of some kind (preferably a SYN-ACK, but possibly a RST-ACK).
I would like to ask how does Rebus HTTP Gateway acknowledge message delivery so when OutboundService sends the message how it knows it can commit or rollback the transaction.
Intuitive answer would be that HTTP response acknowledges it however looking at the code
https://github.com/rebus-org/Rebus/blob/5fef6b400feaf569e0d6517ad9ee3f6da2f31820/src/Rebus.HttpGateway/Outbound/OutboundService.cs#L139
it seems no action is taken after reading the response.
Thanks in advance :)
It does a very simple "acknowledge" in the way that if no error occurs, then the message is assumed to have been delivered safely to the destination queue.
This means that the ubiquitous at least once-delivery guarantee holds across gateways as well, although the risk of receiving the same message twice will of course be greatly increased.
If it's important for you to process each message only once, you need to make your receiver idempotent - but that's generally the rule when you're doing messaging without distributed transactions, so it's no different from scenarios where there's no HTTP gateway involved.
UDP package are like once gone you will never know that it is received or not. So it is not gurenteed weather package will be received. I have learned that mostly DNS use UDP package to send the request. How does the loss of DNS requested using UDP are handled?
If no response packet is received within a certain amount of time, the request is re-sent. Dan Bernstein suggests that most clients will retry up to four times.
There are two ways to decide that a UDP datagram has been lost. Neither is completely reliable.
The most common is a timeout. You send a message and wait for a response. If you don't get a response after some amount of time, you assume that either the message or the response were lost. At that point you can either try again, or give up. It is also possible that the message or response is just taking a really long time to get through the network, so you must account for duplicates. Note that all packet-switched communication, including TCP, works this way. TCP just hides the details for you.
Another method is to look for ICMP messages telling you that a packet has been dropped. For example, ICMP_UNREACH_PORT, ICMP_UNREACH_HOST, or ICMP_UNREACH_HOST_PROHIB. But not only are these messages rarely sent and subject to loss themselves, but you can sometimes receive them even when a message did get through successfully. At best, if you get an ICMP message you can think of it as a suggestion of what might have happened.
Most DNS implementations use a short timeout because duplication is no big deal. After a few repeats to one DNS server, it will try another one (assuming multiple servers are available). Most implementations will also cache information about which servers are responding and which are not.