I would like to ask how does Rebus HTTP Gateway acknowledge message delivery so when OutboundService sends the message how it knows it can commit or rollback the transaction.
Intuitive answer would be that HTTP response acknowledges it however looking at the code
https://github.com/rebus-org/Rebus/blob/5fef6b400feaf569e0d6517ad9ee3f6da2f31820/src/Rebus.HttpGateway/Outbound/OutboundService.cs#L139
it seems no action is taken after reading the response.
Thanks in advance :)
It does a very simple "acknowledge" in the way that if no error occurs, then the message is assumed to have been delivered safely to the destination queue.
This means that the ubiquitous at least once-delivery guarantee holds across gateways as well, although the risk of receiving the same message twice will of course be greatly increased.
If it's important for you to process each message only once, you need to make your receiver idempotent - but that's generally the rule when you're doing messaging without distributed transactions, so it's no different from scenarios where there's no HTTP gateway involved.
Related
Since i had bad introduction that confused people, I am editing the question and removing the introduction I previously made.
Now, here is the business case for which now I have concerns. C# pseudocode:
Array.ForEach(files, filename =>
{
try
{
WcfServiceClient wcfClient = new WcfServiceClient();
wcfClient.SomeMethodWhichPostsFile(filename);
}
catch (Exception ex)
{
LogException(ex)
}
}
);
I am confused because of existence of WSBinding which is reliable, and basicHTTPBinding is not. I know that WSBinding with reliable sessions guarantees delivery, order, content is encrypted etc. But in the case I described with pseudocode, according to my opinion I have all of these supported even with basicHttpBinding and HTTPS over TCP. TCP provides me reliability, order guarantees and HTTPS encryption.
(1. is removed) Am I right related to previous? Or to rephrase: is there an example to show that basicHttpBindind under specified conditions can not provide the same features as WS binding with reliable sessions?
My business case requires to accept WCF calls by the order they are issued. If I send them synchronously from the client in a foreach loop (as shown in pseudo code), I assume the order at the server is guaranteed regardless if those are sent within one TCP connection or not, since I am waiting for the response and then I send another request. Even loadbalancer can not disorder messages here since there is no parallelization, messages are sent one by one synchronously.
I assume disordering could happen only if I send messages without waiting for response in fire and forget manner and I use different TCP connections.
So, am I right here? :)
There are different meanings of the term reliability and the interpretation also depends on the context. Your interpretation of reliability in your question is message is delivered. The interpretation of reliability in the source you cite is instead message is delivered exactly once. Your confusion comes from taking the statement "HTTP is not reliable" which was meant for one interpretation of reliability and using it in the your different interpretation of reliability.
HTTP cannot guarantee that the message gets only delivered once, it can at most guarantee that the message is delivered at least once. It can happen that the underlying TCP connection breaks while sending the request or receiving the response. In this case the client might ignore the problem or retry, which might result in no message delivered (ignoring errors while sending request) but also the same message delivered multiple times (retrying if connections breaks during response). By retrying until the response is received successfully one can guarantee that the message is received at least once, which is your interpretation of reliability but not the one from the statement you cite.
I am talking about only one case here.
client sent a request to server -> server received it and returned a response -> unfortunately the response dropped.
I have only one question about this.
Is this case even possible? If it's possible then what should the response code be, or will client simply see it as read timeout?
As I want to sync status between client/server and want 100% accuracy no matter how poor the network is, the answer to this question can greatly affect the client's 'retry on failure' strategy.
Any comment is appreciated.
Yes, the situation you have described is possible and occurs regularly. It is called "packet loss". Since the packet is lost, the response never reaches the client, so no response code could possibly be received. Web browsers will display this as "Error connecting to server" or similar.
HTTP requests and responses are generally carried inside TCP packets. If a TCP packet carrying the HTTP response does not arrive in the expected time window, the request is retransmitted. The request will only be retransmitted a certain number of times before a timeout error will occur and the connection is considered broken or dead. (The number of attempts before TCP timeout can be configured on both the client and server sides.)
Is this case even possible?
Yes. It's easy to see why if you picture a physical cable between the client and the server. If I send a request down the cable to the server, and then, before the server has a chance to respond, unplug the cable, the server will receive the request, but the client will never "hear" the response.
If it's possible then what should the response code be, or will client simply see it as read timeout?
It will be a timeout. If we go back to our physical cable example, the client is sitting waiting for a response that will never come. Hopefully, it will eventually give up.
It depends on exactly what tool or library you're using how this is wrapped up, however - it might give you a specific error code for "timeout" or "network error"; it might wrap it up as some internal 5xx status code; it might raise an exception inside your code; etc.
I've heard it said that providing exactly-once delivery is almost impossible. At the same time, TCP is said to provide guaranteed delivery. If TCP does not provide exactly-once guaranteed delivery then does it provide at-most-once or at-least-once
We could say that TCP provides at-least-once delivery and exactly-one processing, with regards to the following definitions:
At-least-once delivery: a TCP message will be delivered at least once to the destination. More specifically, it will keep re-transmitting with specific timeouts if no ACK(knowledgement) is received, so that it will eventually be delivered. However, if some of these re-transmissions were not lost (but just delayed), then more than one copies of the message will be delivered.
Exactly-once processing: each TCP message will be processed by the destination node exactly once. More specifically, the destination will watch out for duplicate messages (checking the IDs of each received message). So, even if a message is delivered twice, the destination node will only process it (pass it to the application level) once and ignore the duplicates received later.
Exactly once is clearly impossible. What if the network connection is severed and never recovers?
UDP package are like once gone you will never know that it is received or not. So it is not gurenteed weather package will be received. I have learned that mostly DNS use UDP package to send the request. How does the loss of DNS requested using UDP are handled?
If no response packet is received within a certain amount of time, the request is re-sent. Dan Bernstein suggests that most clients will retry up to four times.
There are two ways to decide that a UDP datagram has been lost. Neither is completely reliable.
The most common is a timeout. You send a message and wait for a response. If you don't get a response after some amount of time, you assume that either the message or the response were lost. At that point you can either try again, or give up. It is also possible that the message or response is just taking a really long time to get through the network, so you must account for duplicates. Note that all packet-switched communication, including TCP, works this way. TCP just hides the details for you.
Another method is to look for ICMP messages telling you that a packet has been dropped. For example, ICMP_UNREACH_PORT, ICMP_UNREACH_HOST, or ICMP_UNREACH_HOST_PROHIB. But not only are these messages rarely sent and subject to loss themselves, but you can sometimes receive them even when a message did get through successfully. At best, if you get an ICMP message you can think of it as a suggestion of what might have happened.
Most DNS implementations use a short timeout because duplication is no big deal. After a few repeats to one DNS server, it will try another one (assuming multiple servers are available). Most implementations will also cache information about which servers are responding and which are not.
Say my network connection drops for a few seconds and I miss some SignalR server-pushed messages.
When I regain network connectivity are the messages I missed lost? or does signalR handle them and push them out when I reconnect?
If it can't handle missed messages, then what is the recommended approach for ensuring consistency?
Periodically (2-3 mins) poll to check server-data?
Somehow detect loss of network on the client side and do an ajax call to get the data on network restoration?
something else?
Here are a couple of thoughts:
If you aren't sending a lot of messages per second, consider sending no data in the messages themselves. Instead, the message is just a "ping" to the clients telling them to go get the server data when they can. Combine that with a periodic poll, as you said, and you can be assured that you won't miss messages. They just might be delayed.
If you are sending a lot of messages quickly, how about adding a sequential ID to each one? Think of a SQL Identity column. Your clients would need to keep track of the most recent ID received. After a network reconnect, the client could ask for all messages since [Last ID]. If a message is received whose ID is not contiguous with the most recently received, you know that there was a disconnect and can ask the server for the missing information.