I have a function that optionally uses threads for its main loop, doing so when an argument usingthreads is true. At the moment, the code looks like this:
function dosomething(usingthreads::Bool)
n = 1000
if usingthreads
Threads.#threads for i = 1:n
#20 lines of code here
end
else
for i = 1:n
#same 20 lines of code repeated here
end
end
end
Less nasty than the above would be to put the "20 lines" in a separate function. Is there another way?
You could use a macro that changes its behavior depending on the result of Threads.nthreads():
macro maybe_threaded(ex)
if Threads.nthreads() == 1
return esc(ex)
else
return esc(:(Threads.#threads $ex))
end
end
Without threading, this macro will be a no-op:
julia> #macroexpand #maybe_threaded for i in 1:5
print(i)
end
:(for i = 1:5
#= REPL[2]:2 =#
print(i)
end)
But when threading is enabled and e.g. JULIA_NUM_THREADS=4 it will expand to the threaded version:
julia> #maybe_threaded for i in 1:5
print(i)
end
41325
Edit: Upon rereading the question, I realize this doesn't really answer it but it might be useful anyway.
You can use ThreadsX as suggested in this discourse link.
The answer from the thread (all credit to oxinabox):
using ThreadsX
function foo(multi_thread=true)
_foreach = multi_thread ? ThreadsX.foreach : Base.foreach
_foreach(1:10) do ii
#show ii
end
end
Related
I have a function within which I would like to evaluate a list of expressions using interpolated literals passed as arguments to the function and assign the result to a new array. Previously I've been able to do this fairly simply:
array_of_expr = Any[:(A * 5), :(B * 6 * T)]
arglist = [:A; :B; :T]
test_input = [1e5, 1e1, 100]
#eval begin
function interpolate_all($(arglist...))
result_array = $(Expr(:vcat, array_of_expr...))
end
end
interpolate_all(test_input...)
which returns the expected result:
2-element Array{Float64,1}:
500000.0
6000.0
(I know the code seems needlessly complicated with the #eval--it's because in the full version of the code, arglist is ~500 items long. This has been causing some compiler errors in the full version of the code, so my attempts here to loop over array_of_expr are part of my testing to find out the exact error, and also help me better understand metaprogramming/variable scope while I'm at it.)
I can index array_of_expr and evaluate an individual elements manually in this MWE:
#eval begin
function interpolate_one($(arglist...))
result_array = similar(array_of_expr)
println("evaluating $(array_of_expr[2])")
result_array = $(array_of_expr[2])
println(result_array)
end
end
interpolate_one(test_input...)
which returns:
evaluating B * 6 * T
6000.0
Which is the expected behavior. But if I try to loop over array_of_expr, I encounter various errors. The following ignores the iterator i and just prints the symbolic expression:
#eval begin
function interpolate_1by1($(arglist...))
result_array = similar(array_of_expr)
# this doesn't work, it just gives the symbolic expression, basically ignoring the $:
for i in range(1, length=length(array_of_expr))
result_array[i] = ($array_of_expr[i])
end
println(result_array)
end
end
interpolate_1by1(test_input...)
The following reports that i is not defined, which I understand is because expressions are evaluated in the global scope, not local:
#eval begin
function interpolate_1by1($(arglist...))
result_array = similar(array_of_expr)
# this doesn't work, i is undefined:
for i in range(1, length=length(array_of_expr))
result_array[i] = $(array_of_expr[i])
end
end
end
interpolate_1by1(test_input...)
Is there any way to make this work? I have tried the tactics in the cited SE answer, but did not have success.
you may unroll the loop at compile time:
#eval begin
function interpolate_1by1($(arglist...))
result_array = similar($array_of_expr)
$((quote
result_array[$i] = $(array_of_expr[i])
end for i in 1:length(array_of_expr))...)
return result_array
end
end
which after expansion is similar to
function interpolate_1by1($(arglist...))
result_array = similar($array_of_expr)
result_array[1] = $(array_of_expr[1])
result_array[2] = $(array_of_expr[2])
...
return result_array
end
I'm very new to Julia but I've got a some background in Scheme/Rust/F#.
Today I wanted to make yesterday's AoC nicer without an explicit number of nested loops.
I arrived at this working solution, but I don't like the last if. In the languages mentioned above I would call a function (or use a computation expression) that gives me the first result that is not None. For Julia, I expected something to do that. It does, but unexpectedly in an eager fashion.
So When I tried return something(find(r, n, start + 1, which), find(r, n - 1, start + 1, extended)), that also evaluated the second argument when the first already had a result—and thus crashed.
Is there a macro/lazy version or something that I didn't find? How are you supposed to handle a case like that?
I also thought about (short-circuited) or'ing them together, but I guess Julia's strictness in that matter spoils that.
using DataStructures
function find(r::Array{Int}, n, start = 1, which = nil())::Union{Int,Nothing}
if start <= length(r)
extended = cons(start, which)
with_current = sum(i -> r[i], extended)
if with_current == 2020 && n == 1
return prod(i -> r[i], extended)
else
# Unfortunately no :(
#return something(find(r, n, start + 1, which), find(r, n - 1, start + 1, extended))
re = find(r, n, start + 1, which)
if isnothing(re)
return find(r, n - 1, start + 1, extended)
else
re
end
end
end
end
Let me comment more on it why it is not possible given the discussion in the comments.
In Julia function arguments are evaluated eagerly, so Julia evaluates both find(r, n, start + 1, which) and find(r, n - 1, start + 1, extended) before passing them to something function.
Now, with macros you have (I am not writing in a fully general case for simplicity and I hope I got the hygiene right :)):
julia> macro something(x, y)
quote
local vx = $(esc(x))
isnothing(vx) ? $(esc(y)) : vx
end
end
#something (macro with 1 method)
julia> #something 1 2
1
julia> #something nothing 2
2
julia> #something 1 sqrt(-1)
1
julia> #something nothing sqrt(-1)
ERROR: DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
(in a full-blown version of the macro varargs and Some should be handled to replicate something exactly)
Piqued by Bogumił's answer I wanted to write my first Julia macro. It took some time and numerous attempts to figure out syntax, hygiene and escaping but I'm quite happy now.
I thought it might be worth sharing and provide opportunity for suggestions/improvements.
A lazy #something analog to Base.something
function _something_impl(thing)
:(something($(esc(thing))))
end
function _something_impl(thing, rest...)
quote
local evalued = $(esc(thing))
if isnothing(evalued)
$(_something_impl(rest...))
else
something(evalued)
end
end
end
macro something(things...)
_something_impl(things...)
end
Version without exceptions
As I found exceptions raised from a macro like this not quite suitable, I also made a version that falls back to nothing.
function _something_nothing_impl(thing)
quote
local evaluated = $(esc(thing))
if isa(evaluated, Some)
evaluated.value
else
evaluated
end
end
end
function _something_nothing_impl(thing, rest...)
quote
local evalued = $(esc(thing))
if isnothing(evalued)
$(_something_nothing_impl(rest...))
else
something(evalued)
end
end
end
macro something_nothing(things...)
_something_nothing_impl(things...)
end
Now I guess the recursive middle function could also generated by a macro. :)
Like in R:
a <- 2
or even better
a ← 2
which should translate to
a = 2
and if possible respect method overloading.
= is overloaded (not in the multiple dispatch sense) a lot in Julia.
It binds a new variable. As in a = 3. You won't be able to use ← instead of = in this context, because you can't overload binding in Julia.
It gets lowered to setindex!. As in, a[i] = b gets lowered to setindex!(a, b, i). Unfortunately, setindex! takes 3 variables while ← can only take 2 variables. So you can't overload = with 3 variables.
But, you can use only 2 variables and overload a[:] = b, for example. So, you can define ←(a,b) = (a[:] = b) or ←(a,b) = setindex!(a,b,:).
a .= b gets lowered to (Base.broadcast!)(Base.identity, a, b). You can overload this by defining ←(a,b) = (a .= b) or ←(a,b) = (Base.broadcast!)(Base.identity, a, b).
So, there are two potentially nice ways of using ←. Good luck ;)
Btw, if you really want to use ← to do binding (like in 1.), the only way to do it is using macros. But then, you will have to write a macro in front of every single assignment, which doesn't look very good.
Also, if you want to explore how operators get lowered in Julia, do f(a,b) = (a .= b), for example, and then #code_lowered f(x,y).
No. = is not an operator in Julia, and cannot be assigned to another symbol.
Disclaimer: You are fully responsible if you will try my (still beginner's) experiments bellow! :P
MacroHelper is module ( big thanks to #Alexander_Morley and #DanGetz for help ) I plan to play with in future and we could probably try it here :
julia> module MacroHelper
# modified from the julia source ./test/parse.jl
function parseall(str)
pos = start(str)
exs = []
while !done(str, pos)
ex, pos = parse(str, pos) # returns next starting point as well as expr
ex.head == :toplevel ? append!(exs, ex.args) : push!(exs, ex)
end
if length(exs) == 0
throw(ParseError("end of input"))
elseif length(exs) == 1
return exs[1]
else
return Expr(:block, exs...) # convert the array of expressions
# back to a single expression
end
end
end;
With module above you could define simple test "language":
julia> module TstLang
export #tst_cmd
import MacroHelper
macro tst_cmd(a)
b = replace("$a", "←", "=") # just simply replacing ←
# in real life you would probably like
# to escape comments, strings etc
return MacroHelper.parseall(b)
end
end;
And by using it you could probably get what you want:
julia> using TstLang
julia> tst```
a ← 3
println(a)
a +← a + 3 # maybe not wanted? :P
```
3
9
What about performance?
julia> function test1()
a = 3
a += a + 3
end;
julia> function test2()
tst```
a ← 3
a +← a + 3
```
end;
julia> test1(); #time test1();
0.000002 seconds (4 allocations: 160 bytes)
julia> test2(); #time test2();
0.000002 seconds (4 allocations: 160 bytes)
If you like to see syntax highlight (for example in atom editor) then you need to use it differently:
function test3()
#tst_cmd begin
a ← 3
a ← a + a + 3 # parser don't allow you to use "+←" here!
end
end;
We could hope that future Julia IDEs could syntax highlight cmd macros too. :)
What could be problem with "solution" above? I am not so experienced julian so many things. :P (in this moment something about "macro hygiene" and "global scope" comes to mind...)
But what you want is IMHO good for some domain specific languages and not to redefine basic of language! It is because readability very counts and if everybody will redefine everything then it will end in Tower of Babel...
I was practicing with some exercise like P3.jl. When I run following code, it hasn't any error and the answer is correct.
N = 10
A = rand(N,N)
SA = convert(SharedArray, A)
T = 100
Jacobi_parallel(SA,N,T)
But when I run with N=16386, it has an error:
This case is not supported yet
How can i resolve it?
Thank you.
In this part of the code:
#everywhere function Jacobi_blockwise_parallel_one_time_step(A,x,y,k,threshold)
if k <= threshold
Jacobi_serial_one_block_one_time_step(A,x,y,k)
else
k1 = floor(Int64, k/2)
k2 = k - k1
if (k1 != k2) error("This case is not supported yet") end
lrefs = [#spawn Jacobi_blockwise_parallel_one_time_step(A,x,y,k1,threshold),
#spawn Jacobi_blockwise_parallel_one_time_step(A,x+k1,y,k1,threshold),
#spawn Jacobi_blockwise_parallel_one_time_step(A,x,y+k1,k1,threshold),
#spawn Jacobi_blockwise_parallel_one_time_step(A,x+k1,y+k1,k1,threshold)];
pmap(fetch, lrefs)
end
end
When you call it the first time with k = N-2 (i.e. for N=2048 this triggers the error for me), k = 1023, and when you get to the 'spawn' statements, in the second call 'k' becomes 511 and k2 becomes 512, so this error is triggered.
Interestingly, the error does not appear if you run without adding extra worker processes first. Presumably the #spawn macro behaves differently / unexpectedly when there's no defined workers, I'm not sure. Either way though, I don't think the lrefs bit is doing what the author thinks it's doing (maybe it did in an older version of julia?)
Im looking for a function like Pythons
"foobar, bar, foo".count("foo")
Could not find any functions that seemed able to do this, in a obvious way. Looking for a single function or something that is not completely overkill.
Julia-1.0 update:
For single-character count within a string (in general, any single-item count within an iterable), one can use Julia's count function:
julia> count(i->(i=='f'), "foobar, bar, foo")
2
(The first argument is a predicate that returns a ::Bool).
For the given example, the following one-liner should do:
julia> length(collect(eachmatch(r"foo", "bar foo baz foo")))
2
Julia-1.7 update:
Starting with Julia-1.7 Base.Fix2 can be used, through ==('f') below, as to shorten and sweeten the syntax:
julia> count(==('f'), "foobar, bar, foo")
2
What about regexp ?
julia> length(matchall(r"ba", "foobar, bar, foo"))
2
I think that right now the closest built-in thing to what you're after is the length of a split (minus 1). But it's not difficult to specifically create what you're after.
I could see a searchall being generally useful in Julia's Base, similar to matchall. If you don't care about the actual indices, you could just use a counter instead of growing the idxs array.
function searchall(s, t; overlap::Bool=false)
idxfcn = overlap ? first : last
r = findnext(s, t, firstindex(t))
idxs = typeof(r)[] # Or to only count: n = 0
while r !== nothing
push!(idxs, r) # n += 1
r = findnext(s, t, idxfcn(r) + 1)
end
idxs # return n
end
Adding an answer to this which allows for interpolation:
julia> a = ", , ,";
julia> b = ",";
julia> length(collect(eachmatch(Regex(b), a)))
3
Actually, this solution breaks for some simple cases due to use of Regex. Instead one might find this useful:
"""
count_flags(s::String, flag::String)
counts the number of flags `flag` in string `s`.
"""
function count_flags(s::String, flag::String)
counter = 0
for i in 1:length(s)
if occursin(flag, s)
s = replace(s, flag=> "", count=1)
counter+=1
else
break
end
end
return counter
end
Sorry to post another answer instead of commenting previous one, but i've not managed how to deal with code blocks in comments :)
If you don't like regexps, maybe a tail recursive function like this one (using the search() base function as Matt suggests) :
function mycount(what::String, where::String)
function mycountacc(what::String, where::String, acc::Int)
res = search(where, what)
res == 0:-1 ? acc : mycountacc(what, where[last(res) + 1:end], acc + 1)
end
what == "" ? 0 : mycountacc(what, where, 0)
end
This is simple and fast (and does not overflow the stack):
function mycount2(where::String, what::String)
numfinds = 0
starting = 1
while true
location = search(where, what, starting)
isempty(location) && return numfinds
numfinds += 1
starting = location.stop + 1
end
end
one liner: (Julia 1.3.1):
julia> sum([1 for i = eachmatch(r"foo", "foobar, bar, foo")])
2
Since Julia 1.3, there has been a count method that does exactly this.
count(
pattern::Union{AbstractChar,AbstractString,AbstractPattern},
string::AbstractString;
overlap::Bool = false,
)
Return the number of matches for pattern in string.
This is equivalent to calling length(findall(pattern, string)) but more
efficient.
If overlap=true, the matching sequences are allowed to overlap indices in the
original string, otherwise they must be from disjoint character ranges.
│ Julia 1.3
│
│ This method requires at least Julia 1.3.
julia> count("foo", "foobar, bar, foo")
2
julia> count("ana", "bananarama")
1
julia> count("ana", "bananarama", overlap=true)
2