I'm wondering if I found a bug in Julia's BenchmarkTools or if there's something deeper happening here that I don't understand. Running the following script
function test()
function func1(n)
sum(1:n)
end
function func2(n)
ans = 0
for i = 1:n
ans += i
end
return ans
end
#time func1(100000)
#time func2(100000)
end
works exactly as expected and times both functions. However, using #btime instead of #time gives me an undefined error:
ERROR: UndefVarError: func1 not defined
If I move the internal functions outside test(), both timing versions work fine, but in my actual tests this is not something I can easily do. I prefer using #btime to #time, as it's more accurate and robust, but here I clearly can't. Can someone explain if this is a bug or what's going on here?
Try adding $ to your #btime calls:
function test()
function func1(n)
sum(1:n)
end
function func2(n)
ans = 0
for i = 1:n
ans += i
end
return ans
end
#btime $func1(100000)
#btime $func2(100000)
end
This interpolates the function definition and now the inner function will be visible to the benchmark.
Related
Recently started to try and learn Julia through examples. I am basically trying to figure out how to access a struct property from within a function inside the struct itself. E.g:
struct Test
a::Int
foo::Function
function Test()
return new(777, xfoo)
end
function xfoo()
println(a)
end
end
t = Test()
t.foo()
I get:
ERROR: LoadError: UndefVarError: a not defined
Stacktrace:
[1] (::var"#xfoo#1")()
# Main /tmp/j.jl:10
[2] top-level scope
# /tmp/j.jl:15
in expression starting at /tmp/j.jl:15
Am I using Julia wrong or am I missing something?
Julia is not object oriented language so object oriented patterns are usually not a good idea.
Hence xfoo should be outside of Test:
function xfoo(t::Test)
println(t.a)
end
There are packages that try to emulate OOP with Julia (however this is not a Julian pattern): https://github.com/Suzhou-Tongyuan/ObjectOriented.jl
You can also easily find quite a lot of discussion behind the design decision no to make Julia OOP. Start with: https://discourse.julialang.org/t/why-there-is-no-oop-object-oriented-programming-in-julia/86723
Workaround
Just out of curiosity one can find some workaround to attach a function to a struct (not a recommended design pattern!). For an example:
mutable struct MyTest
a::Int
foo::Function
function MyTest()
s = Ref{MyTest}()
s[] = new(777, () -> println(s[].a))
s[]
end
end
And some sample usage:
julia> t = MyTest();
julia> t.foo()
777
julia> t.a = 900;
julia> t.foo()
900
I am trying to call a general method from a specific one, but cannot figure out how.
function fn(x)
# generic
...
end
function fn(x :: String)
# I want to call the generic version here
val = fn(x)
# do something with val and then return it
...
end
Is this possible?
A workaround is using a helper function that can be called from both generic and specific methods. e.g.
function helper(x)
# main work is here
...
end
function fn(x)
# generic
helper(x)
end
function fn(x :: String)
val = helper(x)
# now use the val to do something
...
end
Without using such helpers, is there a way to control the dispatch to select a particular method to use? Is there something like :before and :after keywords and call-next-method from lisp CLOS in Julia?
You can use the invoke function:
julia> function fn(x)
#info "generic $x"
end
fn (generic function with 1 method)
julia> function fn(x :: String)
#info "before"
invoke(fn, Tuple{Any}, x)
#info "after"
end
fn (generic function with 2 methods)
julia> fn(10)
[ Info: generic 10
julia> fn("10")
[ Info: before
[ Info: generic 10
[ Info: after
(just to be clear - the printing of "before" and "after" is only to highlight what gets executed in what sequence - the only thing that is related to method dispatch here is the invoke function)
I am trying to write a simple Julia code for parallel computing.
I wrote a simple code based on this doc: https://docs.julialang.org/en/latest/manual/parallel-computing
#everywhere function test(x)
return x * 2.0
end
nprocess = 5
addprocs(nprocess)
responses = Vector{Any}(nworkers())
for i in 1:nworkers()
responses[i] = remotecall(test, i+1, i)
end
for res in responses
wait(res)
end
However, I got this error message.
ERROR: LoadError: On worker 2:
UndefVarError: #test not defined
I think the #everywhere macro doesn't work correctly.
I'm using Julia 0.6.0.
Does anyone know how to fix it?
The #everywhere and the addprocs are in the reverse order (causing the added workers not to know about the function test). The other way around it works and doesn't do the UndefVarError:
nprocess = 5
addprocs(nprocess)
responses = Vector{Any}(nworkers())
#everywhere function test(x)
return x * 2.0
end
for i in 1:nworkers()
responses[i] = remotecall(test, i+1, i)
end
for res in responses
wait(res)
end
Best be explained by an example:
I define a type
type myType
α::Float64
β::Float64
end
z = myType( 1., 2. )
Then suppose that I want to pass this type as an argument to a function:
myFunc( x::Vector{Float64}, m::myType ) =
x[1].^m.α+x[2].^m.β
Is there a way to pass myType so that I can actually use it in the body of the function in a "cleaner" fashion as follows:
x[1].^α+x[2].^β
Thanks for any answer.
One way is to use dispatch to a more general function:
myFunc( x::Vector{Float64}, α::Float64, β::Float64) = x[1].^α+x[2].^β
myFunc( x::Vector{Float64}, m::myType = myFunc(x,m.α,m.β)
Or if your functions are longer, you may want to use Parameters.jl's #unpack:
function myFunc( x::Vector{Float64}, m::myType )
#unpack m: α,β #now those are defined
x[1].^α+x[2].^β
end
The overhead of unpacking is small because you're not copying, it's just doing a bunch of α=m.α which is just making an α which points to m.α. For longer equations, this can be a much nicer form if you have many fields and use them in long calculations (for reference, I use this a lot in DifferentialEquations.jl).
Edit
There's another way as noted in the comments. Let me show this. You can define your type (with optional kwargs) using the #with_kw macro from Parameters.jl. For example:
using Parameters
#with_kw type myType
α::Float64 = 1.0 # Give a default value
β::Float64 = 2.0
end
z = myType() # Generate with the default values
Then you can use the #unpack_myType macro which is automatically made by the #with_kw macro:
function myFunc( x::Vector{Float64}, m::myType )
#unpack_myType m
x[1].^α+x[2].^β
end
Again, this only has the overhead of making the references α and β without copying, so it's pretty lightweight.
You could add this to the body of your function:
(α::Float64, β::Float64) = (m.α, m.β)
UPDATE: My original answer was wrong for a subtle reason, but I thought it was a very interesting bit of information so rather than delete it altogether, I'm leaving it with an explanation on why it's wrong. Many thanks to Fengyang for pointing out the global scope of eval! (as well as the use of $ in an Expr context!)
The original answer suggested that:
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
would return a list comprehension which had assignment side-effects, since it conceptually would result in something like [α=1., β=2., etc]. The assumption was that this assignment would be within local scope. However, as pointed out, eval is always assessed at global scope, therefore the above one-liner does not do what it's meant to. Example:
julia> type MyType
α::Float64
β::Float64
end
julia> function myFunc!(x::Vector{Float64}, m::MyType)
α=5.; β=6.;
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
x[1] = α; x[2] = β; return x
end;
julia> myFunc!([0.,0.],MyType(1., 2.))
2-element Array{Float64,1}:
5.0
6.0
julia> whos()
MyType 124 bytes DataType
myFunc 0 bytes #myFunc
α 8 bytes Float64
β 8 bytes Float64
I.e. as you can see, the intention was for the local variables α and β to be overwritten, but they didn't; eval placed α and β variables at global scope instead. As a matlab programmer I naively assumed that eval() was conceptually equivalent to Matlab, without actually checking. Turns out it's more similar to the evalin('base',...) command.
Thanks again to Fengyand for giving another example of why the phrase "parse and eval" seems to have about the same effect on Julia programmers as the word "it" on the knights who until recently said "NI". :)
I was wondering if there was a way to programmatically or dynamically import a set of modules into Julia? For example, if I have a list of files that conform to some naming convention that are present at startup that I can grab using something like:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
which might return a list of files ["mod00.jl", "mod02.jl", "mod05.jl"], is there a way to then import each of the modules in those files. This would be equivalent to having:
import mod00
import mod02
import mod05
in the code if I knew that those modules were available when I wrote the code. Or maybe there is some other approach to doing this that is better. Any suggestions would be much appreciated.
Update
I attempted to do this via a macro, but with no luck. For example:
macro import_mod(modn)
quote
import $modn
end
end
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
#import_mod modn
end
end
When I run this, I get ERROR: syntax: invalid "import" statement. I tried various escaping strategies but all failed similarly.
NOTE: Please refer to user1712368's (Jameson Nash, Julia dev) answer, this discussion at the julia-users mailing list and this entry of the Julia manual, to know about why this is not the correct answer.
This is how a multiple import expression looks like, using Julia version 0.3.4:
julia> parse("import Foo, Bar")
:($(Expr(:toplevel, :($(Expr(:import, :Foo))), :($(Expr(:import, :Bar))))))
julia> dump(ans)
Expr
head: Symbol toplevel
args: Array(Any,(2,))
1: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Foo
typ: Any
2: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Bar
typ: Any
typ: Any
Here is a macro which does this programmatically, it takes a modules argument that could be a :call or :vcat Expr or a Symbol, which has to evaluate to a Vector{Symbol}:
julia> macro dynamic_import(modules)
(modules = eval(modules))::Vector{Symbol}
ex = Expr(:toplevel)
for m in modules
push!(ex.args, Expr(:import, m))
end
return ex
end
You could also generalize this line:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
By abstracting it into a function that takes a Regex and a String directory path as arguments and returns a Vector{Symbol}:
julia> function needed_modules(rx::Regex, dir::String=".")
module_files = filter(rx, readdir(dir))
module_syms = map(m -> symbol(split(m, '.')[1]), module_files)
end
needed_modules (generic function with 2 methods)
So you may use it like this:
julia> #dynamic_import [:Mod01, :Mod02] # :vcat expression
julia> rx = r"^Mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx) # :call expression
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:Mod01
:Mod02
julia> #dynamic_import modules # Symbol
Finally you could wrap it all into a module so you may use using DynamicImport:
Note: currently I get this when trying to run the same examples from a module:
julia> using DynamicImport
julia> #dynamic_import [:mod01, :mod02]
julia> rx = r"^mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx)
ERROR: rx not defined
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:mod01
:mod02
julia> #dynamic_import modules
ERROR: modules not defined
But it works fine If I define the objects inside the REPL, I guess this is an issue involving hygiene, which is something I'm not experienced with, so I'll ask at the julia-users mailing list.
The function version of import X is require("X"), so you should be able to do:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
require(modn)
end
end
Then, assuming each of these defined a module of the same name, you can collect them into an array:
modules = Module[]
...
if isdefined(Main, symbol(modn))
push!(modules, getfield(Main, symbol(modn))
else
warn("importing $modn did not defined a module")
end
...
return modules
I'm not thrilled with this solution, but it seems to work. Basically I dynamically create a temporary file contains the import statements that I then include it. So:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
path, f = mktemp()
for modname in module_files
modn = splitext(modname)[1]
write(f, "import $modn\n")
end
close(f)
include(path)
rm(path)
end
I would be curious to know if there is a more idiomatic strategy since I'm new to Julia and just wadding through its capabilities.
Edit
The strategy of setting up a temporary file is likely unnecessary since Julia also provides an include_string method.
On the Julia-user list, #Isaiah also suggests using eval(Expr(:import, symbol(modn)))