Need clarification to my understanding of isend and issend as given in Send Types
My understanding is that isend will return once the send buffer is free, i.e. when all the data has been released. Issend on the other hand returns only when it receives an ack from the receive of getting/not getting the entire data. Is this all there is to it?
Both MPI_Isend() and MPI_Issend() return immediately, but in both cases you can't use the send buffer immediately.
Think of the difference that there is between MPI_Send() and MPI_Ssend():
MPI_Send() can be buffered or it can be synchronous if the buffer is too
large to be buffered locally, and in this case it waits to complete sending the
data to the corresponding receive operation.
MPI_Ssend() is always synchronous: it always waits to complete sending the data
to the corresponding receive operation.
The inner working of the corresponding "I"-operations is very similar, except for the fact that they both don't block (return immediately): the difference is only when the MPI library signals to the user program that you can use the send-buffer (that is: MPI_Wait() returns or MPI_Test() returns true - the so called send-complete operation of the non-blocking send):
with MPI_Isend() this can happen either when the data has been copied locally in a buffer owned by the MPI library, if below the "synchronous threshold", or when the data has been actually moved to the sibling task: the send-complete operation can be local, in case the underlying send operation is buffered.
With MPI_Issend() MPI doesn't ever buffer data locally and the "buffer-free condition" is returned only after the data has been actually transferred (and probably ack'ed, at low level): the send-complete operation is non-local.
The MPI standard document is quite pedantic on these aspects. See section 3.7 Nonblocking Communication.
Correct. Obviously both of those will only be true when the request that you get back from the call to MPI_ISEND or MPI_ISSEND is completed via a MPI_WAIT* or MPI_TEST* function.
Related
Written as part of a TCP/IP client-server:
Server:
write(nfds,data1,sizeof(data1));
usleep(1000);
write(nfds,data2,sizeof(data2));
Client:
read(fds,s,sizeof(s));
printf("%s",s);
read(fds,s,sizeof(s));
printf("%s",s);
Without usleep(1000) between the two calls to write(), the client prints data1 twice. Why is this?
Background:
I am doing a Client-Server program where the server has to send two consecutive pieces of information after their acquisition, via the network (socket); nfds is the file descriptor we get from accept().
In the client side, we receive these information via read; here fds is the file descriptor obtained via socket().
My issue is that when I am NOT using the usleep(1000) between the write() functions, the client just prints the info represented by data1 twice, instead of printing data1 and then data2. When I put in the usleep() it's fine. Exactly WHY is this happening? Is write() blocking the operation till the buffer is read or is read() blocking the operation till info is written into the buffer? Or am I completely off the page?
You are making several false assumptions. There is nothing in TCP that guarantees that one send equals one receive. There is a lot of buffering, at both ends, and there are deliberate delays in sending to as to coalesce packets (the Nagle algorithm). When you call read(), or recv() and friends, you need to store the result into a variable and examine it for each of the following cases:
-1: an error: examine/log/print errno, or strerror(), or call perror(), and in most cases close the socket and exit the reading loop.
0: end of stream; the owner has closed the connection; close the socket and exit the reading loop.
a positive value but less than you expected: keep reading and accumulate the data until you have everything you need.
a positive value that is more than you expected: process the data you expected, and save the rest for next time.
exactly what you expected: process the data, discard it all, and repeat. This isn the easy case, and it is rare, but it is the only case you are currently programming for.
Don't add sleeps into networking code. It doesn't solve problems, it only delays them.
As we know, there is a thing called an MPI send buffer used during a send action.
And for following code:
MPI_Isend(data, ..., req);
...
MPI_Wait(req, &status)
Is it safe to use data between MPI_Isend and MPI_Wait ?
That means, will MPI_Isend use data as the internal send buffer?
And more, if I don't use data anymore, could I indicate MPI to use data as the send buffer rather than waste time to copy data?
BTW, I've heard of MPI_Bsend, but I don't think it could save memory and time in this case.
MPI provides two kinds of operations: blocking and non-blocking. The difference between the two is when it is safe to reuse the data buffer passed to the MPI function.
When a blocking call like MPI_Send returns, the buffer is no longer needed by the MPI library and can be safely reused. On the other hand, non-blocking calls only initiate the corresponding operation and let it continue asynchronously. Only after a successful call to a routine like MPI_Wait or after a positive test result from MPI_Test one can safely reuse the buffer.
As for how the library utilises the user buffer, that is very implementation-specific. Shorter messages are usually copied to internal (for the MPI library) buffers for performance reasons. Longer messages are usually directly read from the user buffer and sent to the network, therefore the buffer will be in use by MPI until the whole message has been sent.
It is absolutely not save to use data between MPI_Isend and MPI_Wait.
Between MPI_Isend and MPI_Wait you actually don't know when data can be reused. Only after MPI_Wait you can be sure that data is sent and you can reuse it.
If you don't use data anymore you should call MPI_Wait at the end of your program.
Is gen_tcp:send() asynchronous? Assume I'll send some byte array using gen_tcp:send/2. Will process continue to work:
a) Immediately
b) At the time data will arrive in target's inner buffer
c) When the target gets the data from buffer
Thank You in advance.
gen_tcp:send/2 is synchronous. It means that the call returns only after the given packet is really sent. Usually it happens immediately, however if TCP window is full gen_tcp:send/2 blocks until the data is sent. So it means that the call can theoretically block infinitely (for example when receiver does not read data from socket on its side).
Fortunately there are some options to avoid such situation. There are two options {send_timeout, Integer} and {send_timeout_close, Boolean} for sockets which can be specified by the call inet:setopts/2. The first one allows to specify a longest time to wait for a send operation.
When the limit is exceeded, the send operation will return {error, timeout}. Default value of that option is infinity (and it is the reason of infinite block). Also unfortunately it is unknown how much of data was sent if {error, timeout} was returned. In that case it is better to close the socket. If the second option {send_timeout_close, Boolean} is set to true then the socket will be close automatically if {error, timeout} occurs.
In a spatially decomposed 2D domain, I need to send particles to the 8 neighbors. I know how many I'm sending but not how many I'll receive from these neighbors.
I had implemented a code with MPI_Send(), MPI_Probe() and MPI_Recv() but I realized that it caused deadlocks whenever the message was too big.
I decided to go for non-blocking communications but then I can't figure out in what order MPI_Isend, MPI_Irecv and MPI_Iprobe should be called? I definitely need to know the size my receiving buffer should be allocated to before actually calling MPI_Irecv so I'm tempted by the order MPI_Isend() then MPI_Iprobe() then MPI_Irecv(), but the problem is that MPI_Iprove() always returns a flag equal to false and I get stuck in the while loop. As far as I understand there no obligation for MPI to actually complete the send before the call to MPI_Wait(), therefore I understand that MPI_Iprobe might never return true. But if so, how does one receives an unknown size message in non-blocking MPI point-to-point communications?
You don't have to make all 3 operations non-blocking. You can use an MPI_ISEND with a regular MPI_PROBE and/or MPI_RECV. It sounds like that might be a better option for you.
I don't know exactly,
In my case.
I was tested UPnP via Linux, I just use recvfrom.
I got a HTTP response not expected counts. (In this time, I expected 3)
So, I do put sleep(1) in while(), It works!
I have a question is 'why'?
recvfrom returns to buffer per one packets. <-- this is what I know, and is there a relationship with this?
You can use recvfrom() function for both connection and connection-less sockets.if you are using this function in connection-less socket, If a message is too long to fit in the supplied buffer, the excess bytes are discarded. To avoid this kind of situations "you can set the flag MSG_WAITALL that Requests the function block until the full amount of data requested can be returned. The function may return a smaller amount of data if a signal is caught, if the connection is terminated, if MSG_PEEK was specified, or if an error is pending for the socket."
if you are using recvfrom() function in a stream-based sockets such as SOCK_STREAM, message boundaries are ignored. In this case, data is returned to the user as soon as it becomes available, and no data is discarded.
In your case , instead of using sleep() you can set MSG_WAITALL flag that will block your socket untill full amount of data requested can be returned. and there is no relationship between recvfrom() and sleep() functions.