I want to know what exactly are the two, and how they are different. What are the advantage or disadvantage of the two types of calls? Really appreciate using some small example code.
These are detailed, e.g., in here: http://www.netlib.org/utk/papers/mpi-book/node14.html
It's not really proper to talk about advantages of either. They are for different purposes. An example of a local call MPI_Comm_rank, since it doesn't need input from other processes and an example of a non-local call is MPI_Send, since it will have to communicate with some other process that will recieve the message.
Related
I am confused by a number of aspects of QTcpSocket::write.
The documentation suggests that it can write fewer bytes than the length of the buffer being sent. This implies that multiple calls are potentially needed. What is the recommended way to deal with this (bearing in mind following points)?
My initial attempt at calling write did not appear actually to send any data. I found that calling waitForBytesWritten solved this. If I need multiple write calls as per previous point, how do I use waitForBytesWritten in conjunction with these? Do I associate a waitForBytesWritten with each write or do I loop over write and then use waitForBytesWritten.
The documentation suggests that waitForBytesWritten can fail randomly on Windows so ideally I do not want to rely on it at all. It suggests using the bytesWritten signal instead but I have found very little information on how one is supposed to use this properly. In particular, if I have to deal with my concern in the first point, do I not get into the recursive call situation warned about in the documentation of bytesWritten?
I'm trying to understand how to use MPI_Intercomm_create to create a communication handle from one group to another. These two groups are also written in their own C files so there is no way of one group directly accessing the other's communication handle unless I use a global variable or the like. How do I get the "peer_comm" (3rd argument of the call) for the other group? Or am I just not understanding something?
MPI_Intercomm_create() operates on communicator (e.g. MPI_Comm) and not on groups (e.g. MPI_Group) so let's use the right semantic here.
If you launch several binaries with the same mpirun command line, then they are all in MPI_COMM_WORLD, and this is likely what you want to use for peer_comm.
If you use MPI_Comm_spawn() in order to launch "the other binaries", then it returns your inter communicator, so you likely do not even need MPI_Intercomm_create().
I strongly encourage you to write a Minimal, Complete, and Verifiable example. Not only it will help you clear some confusion, you will more likely get a precise answer once the issue is clearly stated.
According to the documentation for zumero_sync:
If a large amount of information needs to be pulled from the server,
this function may need to be called more than once.
In my Android app that uses Zumero that's no problem; I just keep calling zumero_sync until the return value doesn't start with "0;".
However, now I'm trying to write an admin script that also syncs with my server dbfiles. I'd like to use the sqlite3 shell, and have the script pass the SQL to execute via command line arguments. I need to call zumero_sync in a loop (which SQLite doesn't support) to make sure the db is fully synced. If I had to, I could invoke sqlite3 in a loop (reading its output, looking for "0;"), or even write a C++ app to call the SQLite/Zumero functions natively. But it certainly would be easier if a single zumero_sync was enough.
I guess my real question is: could zumero_sync be changed so it completes the sync before returning? If there are cases where the existing behavior is more useful, maybe there could be a parameter for specifying which mode to use?
I see two basic questions here:
(1) Why does zumero_sync() work the way it does?
(2) Can it work differently?
I'll answer (2) first, since it's easier: Yes, it could work differently. Rather, we could (and probably will, soon, you brought this up) implement an additional function, named something like zumero_sync_complete(), which performs [the guts of] zumero_sync() in a loop and returns after the sync is complete.
We didn't implement zumero_sync_complete() because it doesn't add much value. It's a simple loop, so you can darn well write it yourself. :-)
Er, except in scripting environments which don't support loops. Like the sqlite3 shell.
Answer to (1):
The Zumero sync protocol is designed to give the server the flexibility to return partial results if it wants to do so. And for the sake of reducing load on the server (and increasing its scalability) it often does want to do exactly that.
Given that, one reason to expose this to the client is to increase the client's flexibility as well. As long we're making multiple roundtrips, we might as well give the client an opportunity to do something (like, maybe, update a progress bar) in between them.
Another thing a client might want to do in between loop iterations is handle an error.
Or, in the case of a multithreaded client, it might want to deal with changes that happened on the client while the sync is going on.
Which raises the question of how locking should be managed? Do we hold the sqlite write lock during the entire loop? Or only when absolutely necessary?
Bottom line: A robust app would probably want to implement the loop itself so that it can make its own decisions and retain full control over things.
But, as you observe, the sqlite3 shell doesn't have loops. And it's not an app. And it doesn't have threads. Or progress bars. So it's a use case where a simpler-and-less-powerful form of zumero_sync() would make sense.
I'm very new to unix programming, so please bear with me. :)
I'd like to pass data between two processes. I was going to use named pipes, but read about these "half-duplex" pipes, and it was very intriguing, so I figured that I would give them a try first.
I have two issues with these pipes thus far:
I haven't figured out how to get execlp to run another application from my child process
Even if I could, debugging is tough because I've only been able to set breakpoints in the parent process
I'm sure there are reasons for these issues. I am starting to wonder if it makes sense to just forget about them and just use named pipes so I can debug each application in a separate instance of eclipse.
If there is any relevant information, please let me know. The code I am using is essentially what it found on tldp.org.
EDIT -- I renamed my question to be about unix pipes in general. I had assumed that for named pipes, I wouldn't have to use fork(), but all of the examples I have seen so far require it. So regardless of half-duplex or named pipes, I'm going to need to be able to debug the child process somehow!
EDIT #2 -- this example clearly shows that what I had seen before (on an IBM link) regarding named pipes wasn't necessarily true.
I recommend two tools:
strace -ff should give you a trace of all significant events, allowing you to examine in detail what's going on, namely, all reads and writes;
lsof allows you to dump file descriptors of involved processes, clearly showing what is connected to what else and, in particular, if you forgot to close() some descriptors and the whole thing deadlocks.
Looking into asynchronous address resolution in winsock it seems that the only two options are either to use the blocking gethostbyname on a seperate thread, or use WSAAsyncGetHostByName. The latter is designed for some reason to work with window messages, instead of overlapped operations and completion ports/routines.
Is there any version of gethostbyname that works asynchronously with overlapped operations in a similiar manner to the rest of the winsock API?
Unfortunately there isn't at present, although GetAddrInfoEx() has placeholders for all the right things for async operation via all of the 'usual' routes (including IOCP) so I expect there will be eventually... Unfortunately, at this time, the docs say that all of these must be set to NULL and are marked as 'reserved'. :(
I'm just about to write one (have been for a while)... It's unfortunate that WSAAsyncGetHostByName doesn't even allow concurrent name resolution, so it's pretty useless as a base for what I want; but, then again, since it doesn't handle IPv6 that also makes it pretty useless to me. I expect I'll start from scratch; possibly using something like this (beerware) as a base.
Sorry, there is no overlapped version of gethostbyname().