Sending an Array of Strings using Message Passing Interface - mpi

Question

I would like to send an array of strings from the master to a slave thread using Messgae Passing Interface (MPI).
i.e. String [] str = new String [10]
str[0]= "XXX" ... etc
How can I do that while avoiding to send each of the elements in this array as a chain of characters?
I succeeded to send an array of integers in one send operation ... but I don't know how to do that when it is about an array of strings

Answer 1

I don't know Java, but I'll give you the C answer. The concepts -- particularly the two approaches one might take to solve this - are the same in any language, though.
Imagine if this were a simple c-string (some characters terminated with '\0'). There are two approaches:
over-provision memory and receive up to some limit,
or send a message indicating how much data to expect.
Do you have a maximum length? (e.g. PATH_MAX or something like that). If you do not need every byte of memory, you could do
MPI_Send(str, strlen(str), MPI_CHAR, slave_rank, slave_tag, MPI_COMM_WORLD);
and you'd pair that with
MPI_Recv(str, MAX_LENGTH, MPI_CHAR, master_rank, slave_tag, MPI_COMM_WORLD);
If you don't like having slop at the end, you'll have to do it in two messages:
len=strlen(str) + 1; /* +1 for the NULL byte */
MPI_Send(&len, 1, MPI_INT, slave_rank, slave_tag, MPI_COMM_WORLD);
MPI_Send(str, strlen(str), MPI_CHAR, slave_rank, slave_tag, MPI_COMM_WORLD);
and you'd match that with
MPI_Recv(&len, 1, MPI_INT, master_rank, slave_tag, MPI_COMM_WORLD);
payload= malloc(len);
MPI_Recv(&payload, len, MPI_CHAR, master_rank, slave_tag, MPI_COMM_WORLD);

Answer 2

Sending arrays of strings, especially if of varying sizes, is quite an involving process. There are several options but the most MPI-friendly one is to use the packing and unpacking facilities of MPI, exposed in mpiJava as Comm.Pack, Comm.Unpack, and Comm.Pack_size.
You could do something of the sort:
Sender
byte[][] bytes = new byte[nStr][];
int[] lengths = new int[nStr];
int bufLen = MPI.COMM_WORLD.Pack_size(1, MPI.INT);
bufLen += MPI.COMM_WORLD.Pack_size(nStr, MPI.INT);
for (int i = 0; i < nStr; i++) {
bytes[i] = str[i].getBytes(Charset.forName("UTF-8"));
lengths[i] = bytes[i].length;
bufLen += MPI.COMM_WORLD.Pack_size(lengths[i], MPI.BYTE);
}
byte[] buf = new byte[bufLen];
int position = 0;
int nStrArray[] = new int[1];
nStrArray[0] = nStr;
position = MPI.COMM_WORLD.Pack(nStrArray, 0, 1, MPI.INT,
buf, position);
position = MPI.COMM_WORLD.Pack(lengths, 0, nStr, MPI.INT,
buf, position);
for (int i = 0; i < nStr; i++) {
position = MPI.COMM_WORLD.Pack(bytes[i], 0, lengths[i], MPI.BYTE,
buf, position);
}
MPI.COMM_WORLD.Send(buf, 0, bufLen, MPI.PACKED, rank, 0);
Having string lengths in an auxiliary array and packing it at the beginning of the message simplifies the receiver logic.
Receiver
Assumes that the sender is rank 0.
Status status = MPI.COMM_WORLD.Probe(0, 0);
int bufLen = status.Get_count(MPI.PACKED);
byte[] buf = new byte[bufLen];
MPI.COMM_WORLD.Recv(buf, 0, bufLen, MPI.PACKED, status.source, status.tag);
int position = 0;
int nStrArray[] = new int[1];
position = MPI.COMM_WORLD.Unpack(buf, position,
nStrArray, 0, 1, MPI.INT);
int nStr = nStrArray[0];
int lengths[] = new int[nStr];
position = MPI.COMM_WORLD.Unpack(buf, position,
lengths, 0, nStr, MPI.INT);
String[] str = new String[nStr];
for (int i = 0; i < nStr; i++) {
byte[] bytes = new byte[lengths[i]];
position = MPI.COMM_WORLD.Unpack(buf, position,
bytes, 0, lengths[i], MPI.BYTE);
str[i] = new String(bytes, "UTF-8");
}
Disclaimer: I don't have MPJ Express installed and my Java knowledge is very limited. The code is based on the mpiJava specification, the MPJ Express JavaDocs, and some examples found on the Internet.