dynamic scheduling by Julia - julia

I have a function for dynamic scheduling and I can use it for simple array and functions, for example I can use it for this code:
scheduling:
#everywhere function pmap(f,lst)
np=nprocs()
n=length(lst)
results=Vector{Any}(n)
i=1
nextidx()=(idx=i;i+=1;idx)
#sync begin
for p=1:np
if p != myid() || np==1
#sync begin
while true
idx=nextidx()
if idx > n
break
end
results[idx]= remotecall_fetch(f,p,lst[idx])
end
end
end
end
end
results
end
function:
#everywhere f(x)=x+1
f (generic function with 1 method)
array:
julia> arrays=SharedArray{Float64}(10)
10-element SharedArray{Float64,1}:
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
julia> arrays=[1 2 3 4 5 6 7 8 9 10]
1×10 Array{Int64,2}:
1 2 3 4 5 6 7 8 9 10
result:
#everywhere function fsum(x)
x+1
end
pmap(fsum,arrays)
10-element Array{Any,1}:
2
3
4
5
6
7
8
9
10
11
my question is if I had this function and arrays, how should I use scheduling function?
function:
#everywhere f(x,y)=x.+y
julia> x=SharedArray{Float64}(2,2)
2×2 SharedArray{Float64,2}:
0.0 0.0
0.0 0.0
julia> y=SharedArray{Float64}(2,2)
2×2 SharedArray{Float64,2}:
0.0 0.0
0.0 0.0
julia> x=[1 2;3 4]
2×2 Array{Int64,2}:
1 2
3 4
julia> y=[6 7;8 9]
2×2 Array{Int64,2}:
6 7
8 9
I wanted to call it by pmap(f,x,y) but I got this error:
ERROR: MethodError: no method matching pmap(::#f, ::Array{Int64,2}, ::Array{Int64,2})
You may have intended to import Base.pmap
Closest candidates are:
pmap(::Any, ::Any) at REPL[1]:2
and I have another question too, How we can be sure our problem is running in different process? How we can monitor it?

pmap splats the arguments, and so this works:
f(x,y) = x+y; pmap(f,1:5,6:10)
You probably re-defined pmap using what you have in the OP which doesn't splat the arguments and thus fails. You do not need to write your own here: if you just use the built-in version it will work.

Related

How to delete a selected element in a range construct in Julia?

From here I found that in a range construct one cannot find and replace its elements via array functions... How can be do it anyway?
Suppose I want to delete the elements 2,6,7,8,13,19 in range(1, step=1, stop=21). Or more generally, suppose a is a random array that contains numbers in the range [1,21] and one wants to delete these elementes in the given range.
You cannot delete from a range object, since that is immutable, but you can filter it:
julia> filter(x -> x ∉ [2,6,7,8,13,19], a)
15-element Array{Int64,1}:
1
3
4
5
9
10
11
12
14
15
16
17
18
20
21
However, if a is a "real" array, you can use filter! to operate in-place.
Another solution that if often convenient is to use InvertedIndices.jl package which exports Not and you can just use indexing:
julia> r = 1:21
1:21
julia> x = [2,6,7,8,13,19]
6-element Array{Int64,1}:
2
6
7
8
13
19
julia> r[Not(x)]
15-element Array{Int64,1}:
1
3
4
5
9
10
11
12
14
15
16
17
18
20
21

append an array as a column to an pre initialized matrix

I wish to use indexing to populate a pre initialized matrix with the results of my array for loop output:
A = Float64.(reshape(1.0:81.0,9,9))
# initialize output
B = zeros(Int64, 2, 9)
j = 1
for j in 1:size(A,2) # loop cols
out = [sum(A[:,j]),j]
out = reshape(out,2,1) # make column to append
# append out to the B
global B = hcat(B,out) # this grows...
end
I initialized B = zeros(Int64, 2, 9)
same dims as the expected output of the sum operation.
in my real world example - I am iterating through j, columns and, i rows - the output is then an array... rather than use hcat() to append the array to my output can I do it with indexing?
In the above it uses hcat() which will then append to the existing B so it grows. I have since tried initializg with rows 2 and cols 0 so hcat() builds to correct output dim:
B = zeros(Int64, 2, 0)
I am doubting if hcat() will be memory efficient (excuse using global for example sakes) - if I couldn't do it with indexing I can populate it for another inner loop at my [i,j]. But perhaps someone has a way I can append an array as a column to existing pre initialized output?
The recommendation is to pre-allocate B and fill it afterwards. I wrap the code in a function as it simplifies benchmarking:
function f2()
A = reshape(1:81,9,9)
B = zeros(Int64, 2, 9 + size(A,2))
for j in 1:size(A,2) # loop cols
B[:, j + 9] .= (sum(view(A, :, j)), j)
end
B
end
Your old code is:
function f1()
A = Float64.(reshape(1.0:81.0,9,9))
B = zeros(Int64, 2, 9)
j = 1
for j in 1:size(A,2) # loop cols
out = [sum(A[:,j]),j]
out = reshape(out,2,1) # make column to append
# append out to the B
B = hcat(B,out)
end
B
end
And here is a comparison:
julia> #btime f1()
8.567 μs (83 allocations: 7.72 KiB)
2×18 Array{Float64,2}:
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 45.0 126.0 207.0 288.0 369.0 450.0 531.0 612.0 693.0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
julia> #btime f2()
73.662 ns (1 allocation: 368 bytes)
2×18 Array{Int64,2}:
0 0 0 0 0 0 0 0 0 45 126 207 288 369 450 531 612 693
0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9
And you can see that the difference is very significant.
Some more minor comments to your original code:
there is no need to call Float64. on reshape(1.0:81.0,9,9), the object already holds elements that have Float64 values
in your code there was an inconsistency that initilally B held Int64 and A held Float64 - i have made this consistent (I chose Int64, but equally well you could use Float64)
sum(A[:,j]) unnecessarily allocated a new object; it is faster to use a view
You did not have to call reshape(out,2,1) on out before hcat as vectors are already treated as columnar objects

Function to re-orient faces in Julia?

Tried to find Julia implementation of re-orienting faces of a mesh. Does an implementation exist?
Geometries from external sources not necessary have their faces oriented consistently (even if possible). If visualized in GLVisualize.jl, coloring vertices of faces not oriented consistently does not come out nicely.
Example Julia code using GLVisualize with a screenshot linked with non-consistent mesh.
using GLVisualize, GeometryTypes, GLWindow, GLAbstraction, Colors
vertices = [
0.0 0.0 0.0
10.0 0.0 0.0
10.0 20.0 0.0
0.0 20.0 0.0
0.0 0.0 5.0
10.0 0.0 5.0
10.0 20.0 5.0
0.0 20.0 5.0
]
faces = [
7 6 5 # 5 6 7 for consistent orientation
5 7 8
1 4 3
1 3 2
1 2 6
1 6 5
2 3 7
2 7 6
3 4 8
3 8 7
4 1 5
4 5 8
]
v = [ Point{3,Float32}( vertices[i,1:3] ) for i=1:size(vertices,1) ]
f = [ Face{3,UInt32}( faces[i,3:-1:1] ) for i=1:size(faces,1) ]
mesh = GLNormalAttributeMesh(
vertices=v, faces=f,
attributes=RGBA{Float32}[RGBA{Float32}( 0.0, 1.0, 0.0, 1.0 )], attribute_id=zeros( length( v ) )
)
window = glscreen()
_view( visualize( mesh ), window )
renderloop( window )
Brick from the top with non-consistent mesh
I could write a slow brute-force algorithm, which would be slow. Better to ask first.
A recent article suggests the algorithm can be O(N), see https://arxiv.org/pdf/1512.02137.pdf
Note: one should be careful not to look at source implementation of mesh re-orientation in software with non-compatible licenses.

Parse 2D-Array in Julia

In Julia I can create 2D-arrays with
[1 2 3 4 ; 5 6 7 8]
2×4 Array{Int64,2}:
1 2 3 4
5 6 7 8
The problem is, that I need to parse a 2D-array supplied as an argument to a script - that is as a String.
For example
$ julia script.jl "[1 2 3 4 ; 5 6 7 8]"
and in the script something like:
c = parse.(ARGS[1])
and c should be a 2×4 array.
I am flexible regarding the format of the input String.
The usecase is, that I want to call an optimization problem implemented in Julia + JuMP from within Java.
Check out the readdlm function, which will allow you to parse the text received from ARGS as an array:
using DelimitedFiles
a = readdlm(IOBuffer(ARGS[1]),',',';')
display(a)
Running:
$ julia argscript.jl "1,2,3,4;5,6,7,8"
2×4 Array{Float64,2}:
1.0 2.0 3.0 4.0
5.0 6.0 7.0 8.0
You can force the array element type in the script:
a = readdlm(IOBuffer(ARGS[1]),',',Int,';')
You could even enforce the matrix dimensions by passing two more arguments:
using DelimitedFiles
n = parse(Int,ARGS[1])
m = parse(Int,ARGS[2])
a = readdlm(IOBuffer(ARGS[3]),',',Int,';',dims=(n,m))
Running:
$ julia argscript.jl 2 3 "3,2,1;2,6,8"
2×3 Array{Int64,2}:
3 2 1
2 6 8
$ julia argscript.jl 2 4 "3,2,1;2,6,8"
ERROR: LoadError: at row 2, column 1 : ErrorException("missing value at row 1 column 4"))

How to create lagged variables in Julia?

Is there a function to create lagged variables in Julia without resorting any packages?
Specifically, I want to emulate the R's embed function in Julia.
> embed(1:8, 3)
[,1] [,2] [,3]
[1,] 3 2 1
[2,] 4 3 2
[3,] 5 4 3
[4,] 6 5 4
[5,] 7 6 5
[6,] 8 7 6
After a couple of hours of browsing Julia manual, I gave up looking for suitable function in Julia. This ugly function (by R standard) is what I have so far. Is there any built-in function or any room for improvement?
julia> function embed(x, k)
n = length(x)
m = zeros(n - k + 1, k)
for i in 1:k
m[:, i] = x[(k-i+1):(n-i+1)]
end
return m
end
embed (generic function with 1 method)
julia> embed(1:8,3)
6x3 Array{Float64,2}:
3.0 2.0 1.0
4.0 3.0 2.0
5.0 4.0 3.0
6.0 5.0 4.0
7.0 6.0 5.0
8.0 7.0 6.0
You can dismiss zeros for cell to skip initialization. You can also do
embed(x,k) = hcat([x[i+k-1:-1:i] for i in 1:length(x)-k+1]...)'
Explanation
Create reverse stride indexes using [i+k-1:-1:i] and for
Take that list of items, and make it the arguments of hcat by using ...
Concatenate the strides (passed as arguments)
Transpose the result using '
EDIT: Assuming length(x) ⋙ k, you can also use:
embed(x,k) = hcat([x[k-i+1:length(x)-i+1] for i in 1:k]...)
Which gives the same results, but iterates less, and thus does less allocations.

Resources