The MPI_Irecv and MPI_Isend operations return an MPI_Request that can be later marked as cancelled with MPI_Cancel. Is there a similar mechanism for blocking MPI_Probe and MPI_Mprobe ?
The context of the question is the latest implementation of Boost.MPI request handlers using Probe.
EDIT - Here is an example of how an hypothetical MPI_Probecancel could be used:
#include <mpi.h>
#include <chrono>
#include <future>
using namespace std::literals::chrono_literals;
// Executed in a thread
void async_cancel(MPI_Probe *probe)
{
std::this_thread::sleep_for(1s);
int res = MPI_Probecancel(probe);
}
int main(int argc, char* argv[])
{
int provided;
MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &provided);
if (rank == 0)
{
// A handle to the probe (similar to a request)
MPI_Probe probe;
// Start a thread
// `probe` will be filled with the next call, pretty ugly
// Ideally, this should be done in two steps like MPI_Irecv, MPI_Wait
auto res = std::async(std::launch::async, &async_cancel, &probe);
MPI_Message message;
MPI_Status status;
MPI_MProbe(1, 123, MPI_COMM_WORLD, &message, &status, &probe);
if (!probe.cancelled)
{
int buffer;
MPI_Mrecv(&buffer, 1, MPI_INT, &message, &status);
}
}
else
std::this_thread::sleep_for(2s);
MPI_Finalize();
return 0;
}
First, the premise / nomenclature of your question is wrong. It is the nonblocking calls. MPI_Irecv and MPI_Isend which return a request object that you may cancel. For these calls, you cancel the local operation.
MPI_Probe and MPI_Mprobe are in fact blocking. You cannot possibly cancel these operations in the sense that control flow will only leave when a message is available.
On the other hand, MPI_Iprobe and MPI_Improbe are nonblocking, meaning they always complete immediately, telling you whether a message is available.
For neither kind of probe call, there is any kind of local state left after the completion. So there is nothing that could be cancelled locally after the functions return.
That said, if a probe tells you that a message is available, you should definitely receive it. Otherwise a send operation may bock and you would leak resources on all side. But that's just a receive operation.
Edit: Regarding your idea to cancel a ongoing local MPI_Probe in a concurrent thread: This is not directly supported.
Theoretically, you could emulate this on a conforming implementation with MPI_THREAD_MULTIPE by running the probe on MPI_ANY_SOURCE and send a message to the same rank from the other thread. That, of course, has the consequence that you change must probe on message from any incoming rank.
Realistically, if you have to do this, you would probably just use a loop like while(!cancelled) MPI_Iprobe();.
That said, I would again question the scenario: How would another thread on your rank suddenly know to cancel a local MPI_Probe operation? It would probably have to be based on information received from a remote rank - in which case that would be covered by actually being able to receive information from it, i.e. the actual Probe would complete.
Maybe for some high-level abstraction it makes some sense to offer a local cancel, but in an actual practical situation I would believe you could design a idiomatic pattern without needing this.
Related
I want an simple and clear example of how to do the signal and slot mechanism in queued connection.
Take the below line as example. Think that obj1 is backend functionality object emitting signal asynchronously from another thread and is connected to a slot in GUI ( main thread):
connect(obj1, SIGNAL(Mysignal(vector<mystruct> )), this, slot(myslot(vector <mystruct>)))
I have read that you have to register the types(meta types).
Please give a clear, simple and ready to use code lines for the above example that I would need, so that errors during run time like vector, my struct or string not defined, etc.. I don't face.
Also, is there a better way to handle this like sending pointers like:
connect(obj1, SIGNAL(Mysignal(obj2 *)), this, slot(myslot(obj2 *)))
Obj2 contains the vector of mystruct. Will i still need to register the obj2 with those metatypes?
If somebody has experience in this, please share all your good practices and simple code snippets, I am new to the queued connections with arguments. Please help.
If you want a queued connection, you need to call connect with a 5. parameter Qt::QueuedConnection. Otherwise, you get a direct connection inside the thread where you sent the signal from. Edit: See Tobys comment below.
You must wrap a QVector<> into a typedef, otherwise registering will not work (bug? in Qt from the stoneage). Also do not use references to your typedef, will not work either.
Header
typedef struct {
int a;
int b;
} mystruct;
typedef QVector<mystruct> myvector;
Q_DECLARE_METATYPE(myvector);
Source
void MainWindow::test()
{
qRegisterMetaType<myvector>();
connect(this, SIGNAL(sigRec(myvector)), SLOT(slotRec(myvector)), Qt::QueuedConnection);
mystruct x = {1,2};
myvector v;
v.append(x);
emit sigRec(v);
}
void MainWindow::slotRec(myvector s)
{
}
In my program, I have wrapped up some MPI communicators in to a data structure. Unfortunately, sometimes the destructor of an object of this type might get called before it has been initialized. In my destructor, I of course call MPI_Comm_Free. But if this is called on an invalid communicator the code crashes.
I've been looking through the MPI standard, but I can't find a function to test if a communicator is valid. I also assume I can't use MPI_Comm_set_errhandler to try and catch the free exception because there isn't a valid communicator to set the handler of. I could maintain a flag value of my own saying if the communicator is valid, but I prefer to avoid duplicating state information like that. Is there any built in way I can safely check if a communicator is valid?
Here is a basic program demonstrating my problem:
#include <mpi.h>
typedef struct {
MPI_Comm comm;
} mystruct;
void cleanup(mystruct* a) {
MPI_Comm_free(&(a->comm));
}
int main(int argc, char* argv[]) {
MPI_Init(&argc, &argv);
mystruct a;
/* Some early exit condition triggers cleanup without
initialization */
cleanup(&a);
MPI_Finalize();
return 0;
}
MPI_COMM_NULL is a constant used for invalid communicators. However, you cannot determine if an MPI communicator has been initialized. In C, it is impossible to determine if a variable has been initialized. Non-static variables start with an indeterminate value, reading it causes undefined behavior.
You must initialized the communicator with MPI_COMM_NULL yourself. This only make sense if cannot possibly create actual communicator during initialization.
Note: MPI_Comm_free also sets comm to MPI_COMM_NULL.
I'm trying to implement MPI_Bcast, and I'm planning to do that by MPI_Send and MPI_Recv but seems I can not send message to myself?
The code is as follow
void My_MPI_Bcast(void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm) {
int comm_rank, comm_size, i;
MPI_Comm_rank(comm, &comm_rank);
MPI_Comm_size(comm, &comm_size);
if(comm_rank==root){
for(i = 0; i < comm_size; i++){
MPI_Send(buffer, count, datatype, i, 0, comm);
}
}
MPI_Recv(buffer, count, datatype, root, 0, comm, MPI_STATUS_IGNORE);
}
Any suggestion on that? Or I should never send message to oneself and just do a memory copy?
Your program is erroneous on multiple levels. First of all, there is an error in the conditional:
if(comm_rank=root){
This does not compare comm_rank to root but rather assigns root to comm_rank and the loop would then only execute if root is non-zero and besides it would be executed by all ranks.
Second, the root process does not need to send data to itself since the data is already there. Even if you'd like to send and receive anyway, you should notice that both MPI_Send and MPI_Recv peruse the same buffer space, which is not correct. Some MPI implementations use direct memory copy for self-interaction, i.e. the library might use memcpy() to transfer the message. Using memcpy() with overlapping buffers (including using the same buffer) leads to an undefined behaviour.
The proper way to implement linear broadcast is:
void My_MPI_Bcast(void *buffer, int count, MPI_Datatype datatype, int root, MPI_Comm comm)
{
int comm_rank, comm_size, i;
MPI_Comm_rank(comm, &comm_rank);
MPI_Comm_size(comm, &comm_size);
if (comm_rank == root)
{
for (i = 0; i < comm_size; i++)
{
if (i != comm_rank)
MPI_Send(buffer, count, datatype, i, 0, comm);
}
}
else
MPI_Recv(buffer, count, datatype, root, 0, comm, MPI_STATUS_IGNORE);
}
The usual ways for a process to talk to itself without deadlocking is:
using a combination of MPI_Isend and MPI_Recv or a combination of MPI_Send and MPI_Irecv;
using buffered send MPI_Bsend;
using MPI_Sendrecv or MPI_Sendrecv_replace.
The combination of MPI_Irecv and MPI_Send works well in cases where multiple sends are done in a loop like yours. For example:
MPI_Request req;
// Start a non-blocking receive
MPI_Irecv(buff2, count, datatype, root, 0, comm, &req);
// Send to everyone
for (i = 0; i < comm_size; i++)
MPI_Send(buff1, count, datatype, i, 0, comm);
// Complete the non-blocking receive
MPI_Wait(&req, MPI_STATUS_IGNORE);
Note the use of separate buffers for send and receive. Probably the only point-to-point MPI communication call that allows the same buffer to be used both for sending and receiving is MPI_Sendrecv_replace as well as the in-place modes of the collective MPI calls. But these are implemented internally in such way that at no time the same memory area is used both for sending and receiving.
This is an incorrect program. You cannot rely on doing a blocking MPI_Send to yourself...because it may block. MPI does not guarantee that your MPI_Send returns until the buffer is available again. In some cases this could mean it will block until the message has been received by the destination. In your program, the destination may never call MPI_Recv, because it is still trying to send.
Now in your My_MPI_Bcast example, the root process already has the data. Why need to send or copy it at all?
The MPI_Send / MPI_Recv block on the root node can be a deadlock.
Converting to MPI_Isend could be used to resolve the issue. However, there may be issues because the send buffer is being reused and root is VERY likely to reach the MPI_Recv "early" and then may alter that buffer before it is transmitted to other ranks. This is especially likely on large jobs. Also, if this routine is ever called from fortran there could be issues with the buffer being corrupted on each MPI_Send call.
The use of MPI_Sendrecv could be used only for the root process. That would allow the MPI_Send's to all non-root ranks to "complete" (e.g. the send buffer can be safely altered) before the root process enters a dedicated MPI_Sendrecv. The for loop would simply begin with "1" instead of "0", and the MPI_Sendrecv call added to the bottom of that loop. (Why is a better questions, since the data is in "buffer" and is going to "buffer".)
However, all this begs the question, why are you doing this at all? If this is a simple "academic exercise" in writing a collective with point to point calls, so be it. BUT, your approach is naive at best. This overall strategy would be beaten by any of the MPI_Bcast algorithms in any reasonably implemented mpi.
I think you should put MPI_Recv(buffer, count, datatype, root, 0, comm, MPI_STATUS_IGNORE); only for rank=root otherwise it will probably hang
#include <Arduino.h>
#include "include/MainComponent.h"
/*
Turns on an LED on for one second, then off for one second, repeatedly.
*/
MainComponent* mainComponent;
void setup()
{
mainComponent = new MainComponent();
mainComponent->beginComponent();
}
void loop()
{
mainComponent->runComponent();
}
is there any callback to release memory in Arduino ?(e.g to call delete mainComponent)
or this will happen automatically as the loop ends?
what is the strategy to ensure freeing the memory allocated in that code snippet?
SCENARIO :"I wanted to access the object in both methods , so the object is declared in the global scope then instantiated at setup."
What happen when loop() terminated ? will mainComponent still remain in the memory?
If it was in OS NO , process will terminated then resources will be deallocated.
So in Arduino how can I achieve above SCENARIO , by ensuring memory will be deallocated when the controller is switched off ?
What is confusing you is that the main() function is hidden by the basic Arduino IDE. Your programs have a main() function just like on any other platform, and have a lifecycle same as when run on a computer with OS. If you look under arduino___\hardware\cores\aduino, you will find a file main.cpp, which is included into your binaries:
int main(void)
{
init();
//...
setup();
for (;;) {
loop();
if (serialEventRun) serialEventRun();
}
return 0;
}
Considering this file you will now see, that while you exit the loop(), it is continuously called. Your program never exits. In general, your best pattern is to new objects once and never delete, like you have done here. If you are new'ing and delete'ing objects repeatedly on a microcontroller, you are not thinking about lifecycles and resources wisely.
So
"is the new'd object deleted at return from loop()?" No, the program is still running and it stays on the heap.
"What happens at power off? Is there a way to clean up?" The moment the supply voltage drops too low, the microcontroller will stop executing instructions. Power supervisor circuitry prevents the controller from doing anything erratic as the voltage drops (should prevent) When the voltage is conpletely drained, all the RAM is lost. Without adding circuitry, you have no way to execute any clean up at power off.
"Do I need to clean up?" No, at power up, everything is reset to a known state. Operation cannot be affected by anything left behind in RAM (presumes you initialize all your variables).
When using a serial port via POSIX, it's recommended to save the original attributes using tcgetattr() before changing them with tcsetattr(), and then restore them before closing the port. What about when a program is terminated by pressing control-C or when the program receives SIGINT? I haven't seen this covered in any of the serial tutorials.
Apparently an atexit() function wouldn't be sufficient, because it's not called by the default SIGINT handler. So it seems installation of a signal handler would be necessary that restores the attributes to any serial ports still open. Is it even safe to call tcsetattr() from a signal handler?
One might simply dismiss this issue as insignificant, but it's common to terminate a program with control-C, especially one that can take tens of seconds to complete operations. If it's OK not to preserve serial port settings in this case, then there seems little reason to preserve them at all. If anything, it might be better not to bother, rather than do it inconsistently.
I found some examples of source code doing the above, but nothing well-documented. I guess I'm interested in some discussion of whether this is a good idea. Thanks.
After further research I think I've answered this to my satisfaction.
First, in the man page for signal I noticed that a signal handler is specifically allowed to call tcsetattr(), along with a few others:
The signal handler routine must be very careful, since processing elsewhere was interrupted at some arbitrary point. POSIX has the concept of "safe function". If a signal interrupts an unsafe function, and handler calls an unsafe function, then the behavior is undefined. Safe functions are listed explicitly in the various standards. The POSIX.1-2003 list is ... `raise()` ... `signal()` ... `tcsetattr()` [trimmed to relevant ones]
This strongly suggests that the POSIX committee had this exact kind of thing in mind, and leads to a straight forward approach where you change the SIGINT handler once you've opened serial and saved its attributes, then in your handler, restore them and the old SIGINT handler, then re-raise the signal:
static void (*prev_sigint)( int );
static termios saved_attr;
static int fd;
static void cleanup( int ignored )
{
tcsetattr( fd, TCSANOW, &saved_attr );
signal( SIGINT, prev_sigint );
raise( SIGINT );
}
int main( void )
{
open_serial_and_save_attrs();
prev_sigint = signal( SIGINT, cleanup );
...
}