I have this array [1 2 3 4 5 6 7 8 9] and i am performing scan operation on that.
I have 3 mpi tasks and each task gets 3 elements then each task calculates its scan and returns result to master task
task 0 - [1 2 3] => [1 3 6]
task 1 - [4 5 6 ] => [4 9 15]
task 2 - [7 8 9] => [7 15 24]
Now task 0 gets all the results [1 3 6] [4 9 15] [7 15 24]
How can I combine these results to produce final scan output?
final scan output of array would be [1 3 6 10 15 21 28 36 45]
can anyone help me please?
Are you trying to implement your own scan operation? Since this is not what MPI_SCAN does. It applies the scan operation elementwise over each i-th element of the input array stored on each node and the result will be more like:
rank 0 - [1 2 3] => [ 1 2 3]
rank 1 - [4 5 6] => [ 5 7 9]
rank 2 - [7 8 9] => [12 15 18]
Nevertheless, in order to obtain the result that you want, you should add 6 (the last element from the first scan in task 0) to all elements in the next scans:
[ 1 3 6][ 4 9 15][ 7 15 24]
+6 -------------->
=
[ 1 3 6][10 15 21][13 21 30]
Then you should add 15 (the last element from the scan in task 1 before 6 was added) to all elements in the next scans and so forth.
[ 1 3 6][10 15 21][13 21 30]
+15 ---->
=
[ 1 3 6][10 15 21][28 36 45]
Alternatively you could add 6 only to the results from the second scan, then add 21 to the results from the third scan and so forth.
Maybe you can find some clever way to do that using MPI operations.
Related
I'd like to know how can I operate with CartesianIndex. For example I have array
julia> A = rand(1:5, 10, 2)
10×2 Array{Int64,2}:
2 5
1 1
4 5
4 1
2 1
4 1
2 4
1 5
2 5
4 4
and I want to save all numbers which stay near (in pair) with number 1. I can use c=findall(x->x==1, A), but I will have a cartensian indexes of "1".
There is function x=getindex.(c, [1 2]) it makes an array which I can change, but I don't know how to convert it back to CartesianIndex. And I think that must be a better way to do this.
A[view(A.==1,:,[2,1])]
This literally returns "all numbers which stay in pair with number 1".
The order of returned numbers is columnar. If you want to return it by rows:
A'[view(A.==1,:,[2,1])']
Example:
julia> A = rand(1:5, 10, 2)
10×2 Array{Int64,2}:
1 4
3 3
1 3
3 3
5 1
1 5
2 1
3 3
1 3
2 3
julia> A'[view(A.==1,:,[2,1])']
6-element Array{Int64,1}:
4
3
5
5
2
3
If you rather want full rows than use filter!:
julia> filter!((x)->(1 in x), collect(eachrow(A)))
6-element Array{SubArray{Int64,1,Array{Int64,2},Tuple{Int64,Base.Slice{Base.OneTo{Int64}}},true},1}:
[1, 4]
[1, 3]
[5, 1]
[1, 5]
[2, 1]
[1, 3]
When I'm comparing, for example, the following line (These numbers are sample data and can vary).
(map < '[1 2 3 4 5 6] [4 2 3 4 5 9])
I want be able to check if the output contains a false boolean. If it does I want to do some logic, otherwise I want to do some different logic.
This is what I have so far:
(if(map < '[1 2 3 4 5 6] [4 2 3 4 5 9])
// True logic
// False logic)
Thus far it always goes to the true line and never the false line with what ever data I insert. Is there a way I can acheive this?
You could use some with the predicate false?:
(some false? (map < '[1 2 3 4 5 6] [4 2 3 4 5 9])) ;;=> true
(if (some false? (map < '[1 2 3 4 5 6] [4 2 3 4 5 9]))
'True
'False) ;;=> True
I am coding an MPI fortran program and have, let say, three vectors of different length in three ranks. I would like to combine them together in a "concatenate" way such as:
Rank 0: a0 = [1 2 3 4 5]
Rank 1: a1 = [3 5 7 9]
Rank 2: a2 = [2 4 6 8 10 12]
Combine them to:
Rank 0: a = [1 2 3 4 5 3 5 7 9 2 4 6 8 10 12]
Could you please tell me how I can do that ?
Since vectors have different sizes based on the ranks, you can use MPI_Gatherv() in order to achieve the expected result
I am working on random broadcasting between MPI processes. Basically, I want to have a sort of randomized decentralized communication. I like broadcasting as opposed to send and receive style communication for the supposed speed increase, but I get a slight "bug", or feature depending on your definition.
When I run the below,
from mpi4py import MPI
import random
import numpy
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
def stoch_action(n):
assert n >= 1
sample = numpy.array(range(n))
if rank == 0:
sample = numpy.array(random.sample(range(size), n))
print "0 Master Rank", rank, "took sample", sample
comm.Bcast([sample, MPI.INT], root=0)
print "1 Rank", rank, "got sample", sample
if rank in sample:
print "2 Rank", rank, "recognizes duty"
stoch_out = numpy.array([rank, rank, rank])
stoch_root = rank
else:
stoch_out = numpy.array([0, 0, 0])
stoch_root = random.sample(sample, 1)[0]
print "2 Rank", rank, "without duty", stoch_root
comm.Bcast([stoch_out, MPI.INT],
root=stoch_root)
print "3 Rank", rank, "recieved", stoch_out, "from", stoch_root
stoch_action(3)
I get the undesired output,
$ mpiexec -n 8 python stoch.py | sort -n
0 Master Rank 0 took sample [0 7 4]
1 Rank 0 got sample [0 7 4]
1 Rank 1 got sample [0 7 4]
1 Rank 2 got sample [0 7 4]
1 Rank 3 got sample [0 7 4]
1 Rank 4 got sample [0 7 4]
1 Rank 5 got sample [0 7 4]
1 Rank 6 got sample [0 7 4]
1 Rank 7 got sample [0 7 4]
2 Rank 0 recognizes duty
2 Rank 1 without duty 7
2 Rank 2 without duty 7
2 Rank 3 without duty 4
2 Rank 4 recognizes duty
2 Rank 5 without duty 0
2 Rank 6 without duty 7
2 Rank 7 recognizes duty
3 Rank 0 recieved [0 0 0] from 0
3 Rank 1 recieved [7 7 7] from 7
3 Rank 2 recieved [0 0 0] from 7
3 Rank 3 recieved [7 7 7] from 4
3 Rank 4 recieved [4 4 4] from 4
3 Rank 5 recieved [7 7 7] from 0
3 Rank 6 recieved [0 0 0] from 7
3 Rank 7 recieved [7 7 7] from 7
Notice that rank 3 process received [7 7 7] from 4. This shouldn't happen as rank 4 should be broadcasting [4 4 4]. You can see this in other places as well.
I am using OpenMPI (OpenRTE) 1.6.5 and Mpi4Py 1.3.1. On Ubuntu 15.04.
Is there any way to fix it? Ideally, in this example, if a vector is received from a process, the array should be filled with the rank of the process.
Take a function which inputs a set, set of sets, collection, nested vector with collections buried within, etc. So, the inputs sometimes are not of the same type. How would one go about converting such input into a nested vector structure? In the first case of just a set, I would use
(into [] #{1 2 3 4})
converting the into into a vector.
But the case of sets within sets, I am unable to output nested vectors, i.e.
#{1 2 3 #{1 2 3}}
Similarly, I might have an input such as
[1 2 3 (4 5 6)]
and I want to output
[1 2 3 [4 5 6]]
The idea is that sometimes I need to go within the depth and pick out a collection or set to turn into a vector. Is it possible to have a function, which in general can handle the many different structural inputs and output a nested vector structure. Namely can a function generalize the aforementioned examples? I simplified the samples somewhat, for instance I might have inputs such as
[[[1 2 3 4] [#{1 2 3 4} 2 3 4] [(1 2 3 4) 2 3 4]]]
To give more insight into the function I am trying to work on consider the function C from the language R. The importance of this function lies in the importance of vector structures within statistics/data analysis.
user=> (use 'clojure.walk)
user=> (clojure.walk/postwalk (fn [e] (if (coll? e) (vec e) e)) '(1 2 3 #{4 5 6 (7 8 9)}))
[1 2 3 [4 5 6 [7 8 9]]]
a naive reduce implementation:
(defn to-vectors [c]
(reduce #(conj %1 (if (coll? %2) (to-vectors %2) %2))
[] c))
user=> (to-vectors '[[[1 2 3 4] [#{1 2 3 4} 2 3 4] [(1 2 3 4) 2 3 4]]])
[[[1 2 3 4] [[1 2 3 4] 2 3 4] [[1 2 3 4] 2 3 4]]]