Julia: Building multiple types in parallel - julia

I am trying to use parallelism in Julia to construct several large types (specifically, gaussian mixture models via sklearn through PyCall).
If I were doing this in series, I would do:
models = Array(GMM, N)
for i = 1 : N
params = ...
models[i] = train_gmm(params)
end
However, I should be able to do this is parallel. I am having trouble figuring out where to start, since SharedArrays and #parallel don't seem to be the right thing for me.
I was trying to use #spawn, but found the following:
function f1()
rand(10000000)
rand(10000000)
rand(10000000)
rand(10000000)
rand(10000000)
end
function f2()
a = #spawn rand(10000000)
b = #spawn rand(10000000)
c = #spawn rand(10000000)
d = #spawn rand(10000000)
e = #spawn rand(10000000)
a_r = fetch(a)
b_r = fetch(b)
c_r = fetch(c)
d_r = fetch(d)
e_r = fetch(e)
end
f1()
f2()
println(#elapsed(f1()))
println(#elapsed(f2()))
f1 takes 0.21 seconds, f2 takes 0.32 seconds! Is there something about #spawn that I am missing?
EDIT
It looks like doing:
function f1()
[sum(rand(100000000)),
sum(rand(100000000)),
sum(rand(100000000)),
sum(rand(100000000)),
sum(rand(100000000))]
end
function f2()
a = #spawn sum(rand(100000000))
b = #spawn sum(rand(100000000))
c = #spawn sum(rand(100000000))
d = #spawn sum(rand(100000000))
e = #spawn sum(rand(100000000))
[fetch(a), fetch(b), fetch(c), fetch(d), fetch(e)]
end
Causes f2() to run faster than f1() and it is more in-line with what I want. I'll go with this unless someone has a better, official way.
Thank you.

I think your edit has got it correct.
IAINMAC:~ idunning$ julia -p 3
julia> #everywhere function foo()
sleep(2)
end
julia> #time [foo(), foo(), foo()]
elapsed time: 6.017959282 seconds (294088 bytes allocated)
3-element Array{Nothing,1}:
nothing
nothing
nothing
julia> function bar()
a = #spawn foo()
b = #spawn foo()
c = #spawn foo()
[fetch(a), fetch(b), fetch(c)]
end
bar (generic function with 1 method)
julia> #time bar()
elapsed time: 2.030760103 seconds (199720 bytes allocated)
3-element Array{Nothing,1}:
nothing
nothing
nothing
or even more elegantly with pmap:
julia> #everywhere function foo(a::Int)
sleep(a)
end
julia> #time pmap(foo,1:3)
elapsed time: 3.004821524 seconds (448540 bytes allocated)
3-element Array{Any,1}:
nothing
nothing
nothing
julia> #time map(foo,1:3)
elapsed time: 6.006557822 seconds (1368 bytes allocated)
3-element Array{Nothing,1}:
nothing
nothing
nothing

Related

Access struct fields within a function by string argument

Say I have a struct,
struct MyStruct
a
b
end
Is there some way to write a function like the following
function doSomething(x::MyStruct,fieldName::String)
y = x.fieldName
return f(y)
end
I could not find anything about this in the documentation / forums.
You can access fields with Symbols, so you can convert the string to a symbol and then use getproperty:
julia> struct MyStruct
a
b
end
julia> function doSomething(x::MyStruct, name::String)
s = Symbol(name)
return getproperty(x, s)
end
doSomething (generic function with 1 method)
julia> doSomething(MyStruct(1, 2), "a")
1
Note, however, that this will probably be very inefficient, since the compiler most likely can't see through this and thus your code might be type-unstable, see https://docs.julialang.org/en/v1/manual/performance-tips/.
If you plan to get the value only few times redrikekre's solution is OK. However, using metaprogramming you can write code without efficiency penalty, see the getDoSomething2() function below.
Consider those three functions:
function doSomethingNative(x)
return x.a
end
function doSomething(x, name::String)
return getproperty(x, Symbol(name))
end
function getDoSomething2(name::String)
field = Symbol(name)
code = quote
(obj) -> obj.$field
end
return eval(code)
end
Now the setup:
using BenchmarkTools
struct MyStruct
a
b
end
x = MyStruct(5,6)
Now the benchmarks:
julia> #btime doSomethingNative($x)
0.001 ns (0 allocations: 0 bytes)
5
julia> #btime doSomething($x,"a")
36.266 ns (0 allocations: 0 bytes)
5
julia> const doSomething2 = getDoSomething2("a");
julia> #btime doSomething2($x)
0.001 ns (0 allocations: 0 bytes)
5
If you run #code_native doSomethingNative(x) and #code_native doSomething2(x) you will see that the assembly output is identical.

async Elixir vs async Julia

In Elixir I can run code asynchronously like this
defmodule Async do
def example do
process_id = Task.async(fn ->
#some code you want its result later
end)
#do some stuff
async_result = Task.await(process_id)
end
end
and, if i don't need any results I can do like this
defmodule Async do
def example do
process_id = Task.start_link(fn ->
#some code you want its result later
end)
#do some stuff
:ok
end
end
What's the equivalent of above in Julia lang?
if you do not care about the result, you can use #async:
function foo()
sleep(100)
sum(1:100)
end
julia> #async foo()
Task (runnable) #0x00007f983b9d5f90
julia>
in the above example you get back the control of the terminal, without having to wait until the end of the execution of foo()
if you want to know the result, and keep an asynchronous behavior, you can use Task, schedule and fetch together:
julia> a() = sum(1:10000)
a (generic function with 1 method)
julia> b = Task(a)
Task (runnable) #0x00007f983b9d5cf0
julia> schedule(b)
Task (done) #0x00007f983b9d5cf0
julia> fetch(b)
50005000
Here is a small piece of code I use to illustrate the different behaviors:
module async
function example1()
a() = #async sleep(2)
b = Task(a)
schedule(b)
println(fetch(b))
sleep(4)
fetch(b)
end
function example2()
a() = sleep(2)
b = Task(a)
schedule(b)
fetch(b)
end
function example3()
a() = sum(1:10000)
b = Task(a)
schedule(b)
fetch(b)
end
end;
when I run this code, I get:
julia> async.example1()
Task (runnable) #0x00007f983b9d5510
Task (done) #0x00007f983b9d5510
julia> async.example2()
julia> async.example3()
50005000
async.example2() does not return any result, but keep the terminal busy around 2 seconds since fetch waits for the task to complete before giving back the hand.

Recursive call signature keeps changing

I am going to implement a program that uses recursion quite a bit. So, before I started to get stack overflows exceptions, I figured it would be nice to have a trampoline implemented and use thunks in case it was needed.
A first try I did was with factorial. Here the code:
callable(f) = !isempty(methods(f))
function trampoline(f, arg1, arg2)
v = f(arg1, arg2)
while callable(v)
v = v()
end
return v
end
function factorial(n, continuation)
if n == 1
continuation(1)
else
(() -> factorial(n-1, (z -> (() -> continuation(n*z)))))
end
end
function cont(x)
x
end
Also, I implemented a naive factorial to check if, as a matter of fact, I would be preventing stack overflows:
function factorial_overflow(n)
if n == 1
1
else
n*factorial_overflow(n-1)
end
end
The results are:
julia> factorial_overflow(140000)
ERROR: StackOverflowError:
#JITing with a small input
julia> trampoline(factorial, 10, cont)
3628800
#Testing
julia> trampoline(factorial, 140000, cont)
0
So, yes, I am avoiding StacksOverflows. And yes, I know the result is nonsense as I am getting integers overflows, but here I just cared about the stack. A production version of course would have that fixed.
(Also, I know for the factorial case there is a built-in, I wouldn't use either of these, I made them for testing my trampoline).
The trampoline version takes a lot of time when running for the first time, and then it gets quick... when computing the same or lower values.
If I did trampoline(factorial, 150000, cont) I will have some compiling time again.
It seems to me (educated guess) that I am JITing many different signatures for factorial: one for every thunk generated.
My question is: can I avoid this?
I think the problem is that every closure is its own type, which is specialized on the captured variables. To avoid this specialization, one can instead use functors, that are not fully specialized:
struct L1
f
n::Int
z::Int
end
(o::L1)() = o.f(o.n*o.z)
struct L2
f
n::Int
end
(o::L2)(z) = L1(o.f, o.n, z)
struct Factorial
f
c
n::Int
end
(o::Factorial)() = o.f(o.n-1, L2(o.c, o.n))
callable(f) = false
callable(f::Union{Factorial, L1, L2}) = true
function myfactorial(n, continuation)
if n == 1
continuation(1)
else
Factorial(myfactorial, continuation, n)
end
end
function cont(x)
x
end
function trampoline(f, arg1, arg2)
v = f(arg1, arg2)
while callable(v)
v = v()
end
return v
end
Note that the function fields are untyped. Now the function run much faster on the first run:
julia> #time trampoline(myfactorial, 10, cont)
0.020673 seconds (4.24 k allocations: 264.427 KiB)
3628800
julia> #time trampoline(myfactorial, 10, cont)
0.000009 seconds (37 allocations: 1.094 KiB)
3628800
julia> #time trampoline(myfactorial, 14000, cont)
0.001277 seconds (55.55 k allocations: 1.489 MiB)
0
julia> #time trampoline(myfactorial, 14000, cont)
0.001197 seconds (55.55 k allocations: 1.489 MiB)
0
I just translated every closure in your code into a corresponding functor. This might not be needed and probably there are be better solutions, but it works and hopefully demonstrates the approach.
Edit:
To make the reason for the slowdown more clear, one can use:
function factorial(n, continuation)
if n == 1
continuation(1)
else
tmp = (z -> (() -> continuation(n*z)))
#show typeof(tmp)
(() -> factorial(n-1, tmp))
end
end
This outputs:
julia> trampoline(factorial, 10, cont)
typeof(tmp) = ##31#34{Int64,#cont}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,#cont}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}}}}}
typeof(tmp) = ##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,##31#34{Int64,#cont}}}}}}}}}
3628800
tmp is a closure. Its automatically created type ##31#34 looks similar to
struct Tmp{T,F}
n::T
continuation::F
end
The specialization on the type F of the continuation field is the reason for the long compilation times.
By using L2 instead, which is not specialized on the corresponding field f, the continuation argument to factorial has always the type L2 and the problem is avoided.

Julia pi approximation slow

I have pi approximation code very similar to that on official page:
function piaprox()
sum = 1.0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
m = parse(Int,ARGS[1])
opak = parse(Int,ARGS[2])
#time for i = 0:opak
piaprox()
end
When I try to compare time of C and Julia, then Julia is significantly slower, almost 38 sec for m = 100000000 (time of C is 0.1608328933 sec). Why this is happening?
julia> m=100000000
julia> function piaprox()
sum = 1.0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
piaprox (generic function with 1 method)
julia> #time piaprox()
28.482094 seconds (600.00 M allocations: 10.431 GB, 3.28% gc time)
I would like to mention two very important paragraphs from Performance Tips section of julia documentation:
Avoid global variables A global variable might have its value, and
therefore its type, change at any point. This makes it difficult for
the compiler to optimize code using global variables. Variables should
be local, or passed as arguments to functions, whenever possible.....
The macro #code_warntype (or its function variant code_warntype()) can
sometimes be helpful in diagnosing type-related problems.
julia> #code_warntype piaprox();
Variables:
sum::Any
#s1::Any
i::Any
It's clear from #code_warntype output that compiler could not recognize types of local variables in piaprox(). So we try to declare types and remove global variables:
function piaprox(m::Int)
sum::Float64 = 1.0
i::Int = 0
for i = 2:m-1
sum = sum + (1.0/(i*i))
end
end
julia> #time piaprox(100000000 )
0.009023 seconds (11.10 k allocations: 399.769 KB)
julia> #code_warntype piaprox(100000000);
Variables:
m::Int64
sum::Float64
i::Int64
#s1::Int64
EDIT
as #user3662120 commented, the super fast behavior of the answer is result of a mistake, without a return value LLVM might ignore the for loop, by adding a return line the #time result would be:
julia> #time piaprox(100000000)
0.746795 seconds (11.11 k allocations: 400.294 KB, 0.45% gc time)
1.644934057834575

Julia-Lang Metaprogramming: turn expression into function with expression-dependent arguments

Given a dictionary of values,
values = {:A => 3, :B => 1}
turn an (arbitrary) expression like
expr = :(2*A)
into a function foo(values) that evaluates the expression, so in this case foo(values) = 6. The resulting function will be called millions of times, so speed is an important consideration. I am happy to adopt a slightly different approach if necessary, as long as it can be automatised.
Things I tried:
The conversion using convert(Function, expr), as suggested here. Fails for me (Julia 0.3.8-pre):
convert has no method matching convert(::Type{Function}, ::Expr)
Using #eval one can do
#eval foo(A) = $(expr)
and then call foo(values[:A]), but that would require knowing that expr depends on A (and only on A).
I wrote a function find_vars(exp) to return the symbols in expr (in this case [:A]), but couldn't find how to use them in the #eval approach.
Base.Cartesian has an unexported function lreplace which may be what you're after. Then you can do something like:
julia> values = Dict(:A=>3, :B=>1)
Dict{Symbol,Int64} with 2 entries:
:B => 1
:A => 3
julia> import Base.Cartesian.lreplace
julia> expr = :(2*A)
:(2A)
julia> function lreplace_all(expr, d)
for (k, v) in d
expr = lreplace(expr, k, v)
end
expr
end
lreplace_all (generic function with 1 method)
julia> lreplace_all(expr, values)
:(2 * 3)
julia> #eval foo(A) = $(lreplace_all(:(2A), values))
foo (generic function with 1 method)
julia> foo(1)
6
Although, since A is defined by the values dict, it makes more sense to define foo as a zero-argument function (unless I've missed something).
EDIT:
After rereading your question it seems like you want to pass in the actual dictionary to the function rather than have the values available at compile time as I've done above. In that case, we have get a little creative:
First we need an lreplace like function that will work with expressions which is easy enough
julia> dictreplace!(ex, s, v) = ex
dictreplace! (generic function with 1 method)
julia> dictreplace!(ex::Symbol, s, v) = s == ex ? v : ex
dictreplace! (generic function with 2 methods)
julia> function dictreplace!(ex::Expr, s, v)
for i=1:length(ex.args)
ex.args[i] = dictreplace!(ex.args[i], s, v)
end
ex
end
dictreplace! (generic function with 3 methods)
julia> dictreplace(ex, s, v) = dictreplace!(copy(ex), s, v)
dictreplace (generic function with 1 method)
Now we want to replace every occurence of a symbol in our dict keys with a dictionary lookup
julia> function dictreplace_all(expr, kys, dsym)
for k in kys
expr = dictreplace(expr, k, :($(dsym)[$(QuoteNode(k))]))
end
expr
end
dictreplace_all (generic function with 1 method)
julia> dictreplace_all(:(2A), keys(values), :d)
:(2 * d[:A])
julia> #eval foo(args) = $(dictreplace_all(:(2A), keys(values), :args))
foo (generic function with 1 method)
julia> values[:A] = -99
-99
julia> foo(values)
-198
Thanks to the solution by #ptb and another metaprogramming question I found a simpler yet slower solution:
function foo(values, expr)
expr = quote
A = values[:A]
B = values[:B]
return $(expr)
end
eval(expr)
end
Reading in the values from the dictionary can also be done programmatically by replacing the inner evaluation by
$([:($k = $v) for (k, v) in values]...)
return $(expr)

Resources