I was wondering if there was a smart way to do this. Let's say I have three nodes, 0, 1, 2. And let's say each node has an array, a0, a1, a2. If the contents of each node is something like
a0 = {0, 1, 2, 1}
a1 = {1, 2, 2, 0}
a2 = {0, 0, 1, 2}
Is there a clever communication pattern so to move each number to it's corresponding node, i.e.
a0 = {0, 0, 0, 0}
a1 = {1, 1, 1, 1}
a2 = {2, 2, 2, 2}
The approach I have in mind, would involve sorting and temporary buffers, but I was wondering if there was a smarter way?
You can use MPI_Alltoallv for this in the following way:
Sort the local_data (a) by corresponding node of each element in increasing order.
Create a send_displacements array such that send_displacements[r] indicates the index of the first element in the local_data that refers to node r.
Create a send_counts array such that send_counts[r] equals the number of elements in local_data that correspond to node r. This can be computed send_counts[r] = send_displacements[r+1] - send_displacements[r] except for the last rank.
MPI_Alltoall(send_counts, 1, MPI_INT, recv_counts, 1, MPI_INT, comm)
Compute recv_displacements such that recv_displacements[r] = sum(recv_counts[r'] for all r' < r).
Prepare a recv_data with sum(recv_counts) elements.
MPI_Alltoallv(local_data, send_counts, send_displacements, MPI_INT, recv_data, recv_counts, recv_displacements, MPI_INT, comm)
Related
I understand the general usage of MPI_alltoall, which can be described by the following figure
But in practice, it is almost not always that the number of processes will equal to the number grids. The above case assume process = grid = 4. If numbers are not equal, I will have rectangular grids. Below I show an example showing a similar alltoall operation, but nonequal number of grids and processes (grid = 8, process = 2).
My question is then very straightforward, how should I achieve that?
I have looked over alltoallv, but I don't think it will work.
Any suggestions are welcome.
Thank you
a "natural" alltoall would be
MPI_Alltoall(sbuf, 4, MPI_INT, rbuf, 4, MPI_INT, MPI_COMM_WORLD);
and you would end up with
P0 = { A0, A1, A2, A3, C0, C1, C2, C3}
P1 = { B0, B1, B2, B3, D0, D1, D2, D3}
your case is a bit convoluted and you have to use (complex) derived datatypes. (note I did not free the intermediate datatypes in order to keep the code readable)
MPI_Datatype tmp, stype, rtype;
/* derived datatype for send */
MPI_Type_vector(2, 1, 4, MPI_INT, &tmp); /* {0, 4} */
MPI_Type_create_resized(tmp, 0, 4, &tmp); /* next type starts at 1 */
MPI_Type_contiguous(2, tmp, &tmp); /* {0, 4, 1, 5} */
MPI_Type_create_resized(tmp, 0, 8, &stype); /* next type starts at 2, likely unnecessary */
MPI_Type_commit(&stype);
/* derived datatype for recv */
MPI_Type_vector(2, 2, 4, MPI_INT, &tmp); /* {0, 1, 4, 5 } */
MPI_Type_create_resized(tmp, 0, 8, &rtype); /* next type starts at 2 */
MPI_Type_commit(&rtype);
/* all2all */
/* thanks to the derived datatypes :
P0 sends {A0, B0, A1, B1} to P0 and {A2, B2, A3, B3} to P1
P0 receives {A0, B0, .., .., A1, B1, .., ..} from itself, and
{ .., .., C0, D0, .., .., C1, D1} from P1 } */
MPI_Alltoall(sbuf, 1, stype, rbuf, 1, rtype, MPI_COMM_WORLD);
Example:
I have a solution list a:
a = [1, 1, 0, 0, 0]
and input lists bs:
b1 = [1, 1, 0, 0, 0]
b2 = [0, 1, 1, 0, 0]
b3 = [0, 0, 1, 1, 0]
...
bn = [1, 0, 0, 0, 1]
If I compare a to either b1, b2, ..., bn, I expected to get True value from the comparisons. For sure, this simple expression will not work:
if a == b:
...
because in Python only identical lists can be equal.
Is there any beautiful math that I can easily implement it in programming languages? Now I am thinking about building some hash function but I'm still not sure how?
Note 1) it can be easily implemented by just using for loop but I need some thing more robust. 2) this is maybe also related to problem of this post Cyclic group
A simple solution could be to adjust the a and b values:
a_original = [5, 2, 3, 1, 4]
a_formatted = sorted(a_original)
Then, you can just use the formatted variables. A simple "for" loop can be used to format all of your variables.
Hope this helps!
The group elements are generated by (g, 1, 1, ...), (1, g, 1, ... ), (1, 1, g, ...) ... that is have a form (g^i1, g^i2, ... ) where g^p = 1 for some p.
There is an edge between elements in the group where elements at some index have powers of g different by 1 modulo p, that is g^1 and g^2 have a bidirectional edge for p = 4, but g^1 and g^3 do not.
Does there always exist a Hamiltonian cycle in such a group? What kind of structure does it have?
Example
For g = 1 under addition modulo 2 (0, 0, ... 1, 0, 0, 0) generate vertices of a hypercube. So the question is if there is a Hamiltonian cycle in the hypercube.
I'm trying to implement an algorithm to build a decision tree from a dataset.
I wrote a function to calculate the information gain between a subset and a particular partition, then I try all the possible partition and want to choose the "best" partition, in the sense that it's got the lowest entropy.
This procedure must be recursive, hence, after the first iteration, it needs to work for every subset of the partition you got in the previous step.
These are the data:
X = {{1, 0, 1, 1}, {1, 1, 1, 1}, {0, 1, 1, 1}, {1, 1, 1, 0}, {1, 1, 0, 0}}
Xfin[0]=X
This is the function: for every subset of the partition, it tries all the possible partitions and calculate the IG. Then it selects the partition with IGMAX:
Partizioneottimale[X_, n_] :=
For[l = 1, l <= Length[Flatten[X[n], n - 1]], l++,
For[v = 1, v <= m, v++,
If[IG[X[n][[l]], Partizione[X[n][[l]], v]] == IGMAX[X[n][[l]]],
X[n + 1][[l]] := Partizione[X[n][[l]], v]]]]
then I call it:
Partizioneottimale[Xfin, 0]
and it works fine for the first one:
Xfin[1]
{{{1, 0, 1, 1}, {1, 1, 1, 1}, {0, 1, 1, 1}, {1, 1, 1, 0}}, {{1, 0, 0, 0}}}
That is the partition with lowest entropy.
But it doesn't work for the next ones:
Partizioneottimale[Xfin, 1]
Set delayed::steps : Xfin[1+1] in the part assignment is not a symbol
Has anybody any idea about how to solve this?
Thanks
without unraveling all your logic a simple fix is this:
Partizioneottimale[X_, n_] := (
xnp1 = Table[Null, {Length[Flatten[X[n], n - 1]]}] ;
For[l = 1, l <= Length[Flatten[X[n], n - 1]], l++,
For[v = 1, v <= m, v++,
If[IG[X[n][[l]], Partizione[X[n][[l]], v]] == IGMAX[X[n][[l]]],
xnp1[[l]] = Partizione[X[n][[l]], v]]]] ;
X[n+1] = xnp1 ; )
Considering the following list :
dalist = {{1, a, 1}, {2, s, 0}, {1, d, 0}, {2, f, 0}, {1, g, 1}}
I would like to count the number of times a certain value in the first column takes a certain value in column 3.
So in this example my desired output would be:
{{1,1,2},
{1,0,1},
{2,1,0},
{2,0,2}}
or :
Where the latest sublist {2,0,2} being read as: When the value is 2 in the first column, a corresponding value (same row in matrices world) in column 3 of 0 is present twice.
I hope this is not to confusing. I added the second Column to convey the fact that the columns are distant to each other.
If possible, no reordering should happen.
EDIT :
{1,2,3,4,5}
{1,0}
are the exact values taken by the columns I am actually dealing with in my data.
I know I am missing the correct description. Please edit if you can and know it. Thank you
From what I understood, this should do it:
In[11]:= dalist = {{1, a, 1}, {2, s, 0}, {1, d, 0}, {2, f, 0}, {1, g, 1}}
Out[11]= {{1, a, 1}, {2, s, 0}, {1, d, 0}, {2, f, 0}, {1, g, 1}}
In[12]:= Map[Flatten, Tally[dalist[[All, {1, 3}]]]]
Out[12]= {{1, 1, 2}, {2, 0, 2}, {1, 0, 1}}
In your sample, you don't actually have the combination {2,1}, but you have the combination {2,0} twice, not once - thus the output is different from what you anticipated. That is, if I understood the question correctly.
I tried to come up with something brand new using Sasha's assumptions about the required output, but it got more similar to his code than I thought it would be. Still the differences are interesting enough to post.
{#1, #2, Count[dalist[[All, {1, 3}]], {##}]} & ###
Tuples[
{DeleteDuplicates#dalist[[All, 1]],
DeleteDuplicates#dalist[[All, 3]]}
]
Edit
With your clarification about the input the code can be simplified and actually improved to:
{#1, #2, Count[dalist[[All, {1, 3}]], {##}]}& ###Tuples[{Range[5],{0,1}}]
The first version is correct only if at least one example of each possible outcome is actually present in each column.
You can use a combination of Outer and Count:
In[39]:= Flatten[Outer[
{#1, #2, Count[dalist, {#1, _, #2}]} &,
DeleteDuplicates#dalist[[All, 1]],
DeleteDuplicates#dalist[[All, -1]] ], 1]
Out[39]= {{1, 1, 2}, {1, 0, 1}, {2, 1, 0}, {2, 0, 2}}
Here is a variation of Sjoerd's second method, that may or may not be easier to read and adapt.
Join ## Table[{i, j, dalist[[All, {1,3}]] ~Count~ {i, j}}, {i,5}, {j,0,1}]
One may use Array in the same manner:
Join ## Array[{##, dalist[[All, {1,3}]] ~Count~ {##}} &, {5,2}, {1,0}]
If your table is large, it will be worthwhile to do the extraction only once:
With[{x = dalist[[All, {1,3}]]},
Join ## Array[{##, x~Count~{##}} &, {5,2}, {1,0}]
]