I need asynchronous message sending. I read about non-blocking sending on MPI (e.g. MPI_Isend or MPI_Ibcast) and functions to wait/test that sending request (MPI_Wait*/MPI_Test*). However I realize (through testing and web info) that non-blocking send might never get progress on receiver unless that sender invokes functions MPI_Wait*/MPI_Test* after it sends. How can I get a real (and optimal) asynchronous sending?
Regards
Related
While discussing asynchronous messaging on page 67 of the Microservices Patterns book by Chris Richardson (2019), the author writes:
Synchronous—The client expects a timely response from the service and might even block while it waits.
Asynchronous - The client doesn’t block, and the response, if any, isn’t necessarily sent immediately
Given that, it seems that moving from "synchronous" to "asynchronous" communication actually just swaps one synchronous service (e.g., Service A) with a different synchronous service (e.g., a listening port on the message broker like Active MQ, Kafka, IBM MQ, AWS Kinesis, etc.).
That's because the client, presumably, must still block (or at least use 1 thread or connection from a pool) while communicating with the broker, instead of communicating directly with Service A--especially since the client probably expects a broker response (e.g., SUCCESS) for reliability purposes.
Is that analysis correct?
Yes, your analysis is correct.
Working on your case, the broker's client library provides the asynchronous functionality to the caller code (ServiceA for example), which means that it doesn't block the ServiceA's thread until the operation is finished, but it lets you provide a callback that will be invoked (with the results of the async operation) when it is finished.
Now the question is: who will invoke that callback? Well, some code from the broker's client library, which runs on a thread that presumably does some periodic checks to see if the operation is finished (or any other logic that will eventually emit this result).
So yes, there has to be some background thread that does some synchronous work to grab those results.
According to the documentation, MPI_Ssend and MPI_Issend are a blocking and a non-blocking send operations, both synchronous. The MPI specification says that a synchronous send completes when the receiver has started to receive the message and after that it is safe to update the send buffer:
The functions MPI_WAIT and MPI_TEST are used to complete a nonblocking
communication. The completion of a send operation indicates that the
sender is now free to update the locations in the send buffer (the
send operation itself leaves the content of the send buffer
unchanged). It does not indicate that the message has been received,
rather, it may have been buffered by the communication subsystem.
However, if a synchronous mode send was used, the completion of the
send operation indicates that a matching receive was initiated, and
that the message will eventually be received by this matching receive.
Bearing in mind that a synchronous send is considered to be completed when it's just started to be received, I am not sure of the following:
It is possible that only a part of the data has been read from the send buffer at the moment when MPI_Ssend or MPI_Issend signal about send completion? For example, the first N bytes have been sent and received while the next M bytes are still being sent.
How can the caller be safe to modify the data until the whole message is received? Does it mean that the data is necessarily copied to the system buffer? As far as I understand, the MPI standard permits the use of a system buffer but does not require it. Moreover, from here I read that MPI_Issend() doesn't ever buffer data locally.
MPI_Ssend() (or the MPI_Wait() related to MPI_Issend()) returns when :
the receiver has started to receive the message
and the send buffer can be reused
the second condition is met if the message was fully received, or the MPI library buffers the data locally.
I did not read that the MPI standard prohibits data buffering.
From the standard, MPI 3.1 chpt 3.4 page 37
A send that uses the synchronous mode can be started whether or not a matching
receive was posted. However, the send will complete successfully only if a matching receive is
posted, and the receive operation has started to receive the message sent by the synchronous
send. Thus, the completion of a synchronous send not only indicates that the send buffer
can be reused, but it also indicates that the receiver has reached a certain point in its
execution, namely that it has started executing the matching receive. If both sends and
receives are blocking operations then the use of the synchronous mode provides synchronous
communication semantics: a communication does not complete at either end before both
processes rendezvous at the communication. A send executed in this mode is non-local.
In the MPI Standard Section 3.4 (page 37):http://mpi-forum.org/docs/mpi-3.0/mpi30-report.pdf
the synchronous send completion means
1. the send-buffer can be reused
2. the receiver has started to receive data.
The standard says "has started" instead of "has completed", so I have a question about this: Imagine a case:
The sender calls MPI_Ssend, then a receiver is matched and has started to receive data. At this time, the send is complete and returned. As the MPI standard said, the send-buffer can be reused, so the sender modifies some data of the send-buffer. At the same time, the receiver is receiving data very slowly (e.g. network is very bad), so how can we guarantee the data finally received by the receiver is same as the original data stored in sender's send-buffer?
Ssend is synchronous. It means that Ssend cannot return before the corresponding Recv is called.
Ssend is Blocking. It means that the function return only when it is safe to touch the "send-buffer".
Synchronous and blocking are 2 different thing, I know it can be confusing.
Most implementation of Send works as follow (MPICH,OpenMPI,CRAY-MPI):
For small message the send-buffer is copied to the memory which is reserved for MPI. As soon as the copy is done the send return.
For large message, no copy are done, therefore the Send return once the entire send-buffer has been send to the network (which cannot be done before the Revc has been called, to avoid to overload the network memory)
So a MPI_Send is: Blocking, asynchronous for small message,synchronous for large one.
A Ssend works as follow:
As soon as the Recv is started AND the send-buffer is either copied or fully in the network, the Ssend return.
Ssend should be avoided as much as one can. As it slow down the communication (due to the fact that the network need to tell the sender that the recv has started)
Does the MPI standard implement the request-reply communication pattern?
Reading about MPI I found that there are point-to-point routines like:
Synchronous send
Blocking send / blocking receive
Non-blocking send /non-blocking receive
Buffered send
Combined send/receive
"Ready" send
Maybe a developer can implement the request-reply communication pattern using these routines, but it seems that MPI does not directly implement it.
Edit: For clarifying purposes the request-reply (request-response) is a message exchange pattern in which a requestor sends a request message to a replier system which receives and processes the request, ultimately returning a message in response. This is a simple, but powerful messaging pattern which allows two applications to have a two-way conversation with one another over a channel. This pattern is especially common in client–server architectures. It may be synchronous or asynchronous.
This is not available as-is.
That being said, this is trivial to implement.
The requestor can MPI_Sendrecv() and the replier can MPI_Recv() the request and then MPI_Send() the answer.
I understand the concept of Synchronous and Asynchronous in the context of threading in a program, but I'm not sure what that means in communication.
More specifically, I'm confused about what it means to have an asynchronous communication between a server and a client...
In synchronous communication, and please correct me if I'm wrong, one side sends a message, then waits to receive a response, and when the response has arrived, it again sends a message and so on...
What happens in asynchronous mode?
I'm always imagining a two way pipe where there are no rules or protocols about whose turn is it to transmit information, and both sides just shoots bytes into the pipe whenever the feel like, and in both sides, the reading and writing to the pipe happens in two different threads. Is that the case?
That is, again, just a wild guess, if anyone have an explanation I'd love to read.
You are right about the synchronous communication. For Asynchronous communication it works like this:
The client sends a message to the server and optionally specifies what to do upon receiving a response from the server. In the mean time the client can go on doing other stuff, however when the server sends the response, the client knows what to do with that response and handles the response. This is typically done through a "callback" function.
Try to imagine this as sending and receiving email, you can send an email, but because you do not know how long it will take before the addressee sends you an email back you go on with your daily life. The addressee receives your email and sends you a response back. Upon receiving the email you decide the next step.
I hope this explanation helps you conceptualize synchronous communication between client and server.