In some languages (like Python) there are function decorators which appear like macros and sit above a function definition. Decorators provide some additional functionality to the function itself.
Does Julia support this idea of function decorators in any way? Would it be possible to use macros to accomplish the same goal?
Here is one way to achieve this with a macro using ExprTools.jl:
import ExprTools
macro dec(decorator, inner_def)
inner_def = ExprTools.splitdef(inner_def)
outer_def = copy(inner_def)
fname = get(inner_def, :name, nothing)
if fname !== nothing
#assert fname isa Symbol
inner_def[:name] = Symbol(fname, :_inner)
end
outer_def[:body] = Expr(:call,
:($decorator($(ExprTools.combinedef(inner_def)))),
get(outer_def, :args, [])...,
get(outer_def, :kwargs, [])...,
)
return esc(ExprTools.combinedef(outer_def))
end
julia> _time(f) = (x...) -> (t0=time(); val=f(x...); println("took ", time()-t0, " s"); val)
_time (generic function with 1 method)
julia> #dec _time function foo(x, y)
return x + y
end
foo (generic function with 1 method)
julia> foo(1, 2)
took 9.5367431640625e-7 s
3
However, at least in my experience, this comes up surprisingly rarely in idiomatic Julia programs. I would first think about whether your specific problem might be better solved just by multiple dispatch with plain higher-order functions and traits, which will usually lend itself to better composability.
To expand the comment of #SilvioMayolo, Julia has pretty elegant macro capability, that is more general than the Python decorator.
Indeed Julia's macro take as input any expression, and the macro can hook over the Abstract Syntax Tree to return a modified expression.
The keyword you want to look for is "metaprogramming".
Here you can find the official documentation and here a convenient tutorial.
Related
Some of the parameters to a simulation I am writing are functions. When the output is generated, I want to put the definition of these functional parameters in the output. I have in mind a macro that somehow saves the definition as a string, and then defines it. For example, here's what I do now:
borda_score_fn(p) = exp(1/p)
global g_borda_score_fn_string = "exp(1/p)"
And then I write g_borda_score_fn_string to my output. But this is really ugly!
What I would like to do is something like this:
#paramfn borda_score_fn(p) = exp(1/p)
And later be able to both call borda_score_fn(p), and have the form (i.e., "exp(1/p)") available for writing to my output log. (The string form might get stashed in a global dict, actually, they both could.)
I have tried many version of this, but can't get the right set of parses and calls to get it to work. Any help would be appreciated.
This may be a bit different than what you have in mind, but one perhaps "Julian" approach might be to have the function itself return the form string via multiple dispatch, rather than defining a whole new global variable just for that. For example, say we have a type
struct Form end
that we can use for dispatch, then we can write
borda_score_fn(p) = exp(1/p)
borda_score_fn(::Form) = "exp(1/p)"
which can then be retrieved just by calling the function with our type
julia> borda_score_fn(2)
1.6487212707001282
julia> borda_score_fn(Form())
"exp(1/p)"
That might actually be not bad on its own. But, if you want a macro to do both parts at once, then something along the lines of
macro paramfn(e)
name = esc(e.args[1].args[1])
str = string(e.args[2].args[2])
f = esc(e)
quote
$name(::Form) = $str
$f
end
end
would let you write
julia> #paramfn borda_score_fn(p) = exp(1/p)
borda_score_fn (generic function with 2 methods)
julia> borda_score_fn(1)
2.718281828459045
julia> borda_score_fn(Form())
"exp(1 / p)"
For completeness, here's how you can do it in a way more similar to your original approach, but more idiomatically than with a global variable:
julia> module FormOf
export formof, #paramfn
function formof end
macro paramfn(expr)
name = esc(expr.args[1].args[1])
form_str = string(expr.args[2].args[2])
quote
$(esc(expr))
$FormOf.formof(::typeof($name)) = $form_str
$name
end
end
end
Main.FormOf
julia> FormOf.#paramfn borda_score_fn(p) = exp(1/p)
borda_score_fn (generic function with 1 method)
julia> FormOf.formof(borda_score_fn)
"exp(1 / p)"
However, since it defines a new method of FormOf.formof, this only works in global scope:
julia> function bla()
FormOf.#paramfn fn(p) = exp(1/p)
fn(10) + 1
end
ERROR: syntax: Global method definition around REPL[45]:10 needs to be placed at the top level, or use "eval".
Stacktrace:
[1] top-level scope
# REPL[50]:1
#cbk's solution does not have this limitation.
I would like to extend an abstract type's method in a concrete type.
I can do it with a new method but this introduces more complexity:
abstract type AbstractTask end
function complete(task::AbstractTask)
complete_concrete_task(task)
println("Done!")
end
struct Task <: AbstractTask
name::String
end
complete_concrete_task(task::Task) = println(task.name)
coding = Task("coding")
complete(coding)
In Python, I would use the super operator. Is there an equivalent in Julia?
Thanks in advance!
I would say that using dispatch the way you did introduces much less cognitive complexity than a call to super. But you can use invoke select methods of supertypes:
julia> abstract type AbstractTask end
julia> function complete(task::AbstractTask)
println("Done!")
end
complete (generic function with 1 method)
julia> struct Task <: AbstractTask
name::String
end
julia> complete(task::Task) = (println(task.name); invoke(complete, Tuple{AbstractTask}, task))
complete (generic function with 2 methods)
julia> coding = Task("coding")
Task("coding")
julia> complete(coding)
coding
Done!
Of course this requires you not to forget to add it to every subtype method, which is the additional complexity I mean.
The good thing is that invoke with a constant type parameter will be compiled away (or so I have read), so there's not even overhead from dispatch.
The docs for generated functions at some point say the following:
Some operations that should not be attempted include:
...
Interacting with the contents or methods of Core.Compiler in any way.
What exactly is meant by "interacting" in this context? Is just using things from Core.Compiler from within a generated function undefined behaviour?
My use case is to detect builtin functions from within an IRTools dynamo (which constructs generated functions), but you can refer to the following dummy code (inspired by Cassette.canrecurse), which contains the actual "interactions" I want to perform:
julia> #generated function foo(f, args...)
mod = Core.Compiler.typename(f).module
is_builtin = ((f <: Core.Builtin) && !(mod === Core.Compiler))
if is_builtin
quote
println("$f is builtin")
f(args...)
end
else
:(f(args...))
end
end
foo (generic function with 1 method)
julia> foo(+, 1, 2)
3
julia> foo(Core.tuple, 1, 2)
tuple is builtin
(1, 2)
Which seems to work without problems.
Best be explained by an example:
I define a type
type myType
α::Float64
β::Float64
end
z = myType( 1., 2. )
Then suppose that I want to pass this type as an argument to a function:
myFunc( x::Vector{Float64}, m::myType ) =
x[1].^m.α+x[2].^m.β
Is there a way to pass myType so that I can actually use it in the body of the function in a "cleaner" fashion as follows:
x[1].^α+x[2].^β
Thanks for any answer.
One way is to use dispatch to a more general function:
myFunc( x::Vector{Float64}, α::Float64, β::Float64) = x[1].^α+x[2].^β
myFunc( x::Vector{Float64}, m::myType = myFunc(x,m.α,m.β)
Or if your functions are longer, you may want to use Parameters.jl's #unpack:
function myFunc( x::Vector{Float64}, m::myType )
#unpack m: α,β #now those are defined
x[1].^α+x[2].^β
end
The overhead of unpacking is small because you're not copying, it's just doing a bunch of α=m.α which is just making an α which points to m.α. For longer equations, this can be a much nicer form if you have many fields and use them in long calculations (for reference, I use this a lot in DifferentialEquations.jl).
Edit
There's another way as noted in the comments. Let me show this. You can define your type (with optional kwargs) using the #with_kw macro from Parameters.jl. For example:
using Parameters
#with_kw type myType
α::Float64 = 1.0 # Give a default value
β::Float64 = 2.0
end
z = myType() # Generate with the default values
Then you can use the #unpack_myType macro which is automatically made by the #with_kw macro:
function myFunc( x::Vector{Float64}, m::myType )
#unpack_myType m
x[1].^α+x[2].^β
end
Again, this only has the overhead of making the references α and β without copying, so it's pretty lightweight.
You could add this to the body of your function:
(α::Float64, β::Float64) = (m.α, m.β)
UPDATE: My original answer was wrong for a subtle reason, but I thought it was a very interesting bit of information so rather than delete it altogether, I'm leaving it with an explanation on why it's wrong. Many thanks to Fengyang for pointing out the global scope of eval! (as well as the use of $ in an Expr context!)
The original answer suggested that:
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
would return a list comprehension which had assignment side-effects, since it conceptually would result in something like [α=1., β=2., etc]. The assumption was that this assignment would be within local scope. However, as pointed out, eval is always assessed at global scope, therefore the above one-liner does not do what it's meant to. Example:
julia> type MyType
α::Float64
β::Float64
end
julia> function myFunc!(x::Vector{Float64}, m::MyType)
α=5.; β=6.;
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
x[1] = α; x[2] = β; return x
end;
julia> myFunc!([0.,0.],MyType(1., 2.))
2-element Array{Float64,1}:
5.0
6.0
julia> whos()
MyType 124 bytes DataType
myFunc 0 bytes #myFunc
α 8 bytes Float64
β 8 bytes Float64
I.e. as you can see, the intention was for the local variables α and β to be overwritten, but they didn't; eval placed α and β variables at global scope instead. As a matlab programmer I naively assumed that eval() was conceptually equivalent to Matlab, without actually checking. Turns out it's more similar to the evalin('base',...) command.
Thanks again to Fengyand for giving another example of why the phrase "parse and eval" seems to have about the same effect on Julia programmers as the word "it" on the knights who until recently said "NI". :)
I was wondering if there was a way to programmatically or dynamically import a set of modules into Julia? For example, if I have a list of files that conform to some naming convention that are present at startup that I can grab using something like:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
which might return a list of files ["mod00.jl", "mod02.jl", "mod05.jl"], is there a way to then import each of the modules in those files. This would be equivalent to having:
import mod00
import mod02
import mod05
in the code if I knew that those modules were available when I wrote the code. Or maybe there is some other approach to doing this that is better. Any suggestions would be much appreciated.
Update
I attempted to do this via a macro, but with no luck. For example:
macro import_mod(modn)
quote
import $modn
end
end
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
#import_mod modn
end
end
When I run this, I get ERROR: syntax: invalid "import" statement. I tried various escaping strategies but all failed similarly.
NOTE: Please refer to user1712368's (Jameson Nash, Julia dev) answer, this discussion at the julia-users mailing list and this entry of the Julia manual, to know about why this is not the correct answer.
This is how a multiple import expression looks like, using Julia version 0.3.4:
julia> parse("import Foo, Bar")
:($(Expr(:toplevel, :($(Expr(:import, :Foo))), :($(Expr(:import, :Bar))))))
julia> dump(ans)
Expr
head: Symbol toplevel
args: Array(Any,(2,))
1: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Foo
typ: Any
2: Expr
head: Symbol import
args: Array(Any,(1,))
1: Symbol Bar
typ: Any
typ: Any
Here is a macro which does this programmatically, it takes a modules argument that could be a :call or :vcat Expr or a Symbol, which has to evaluate to a Vector{Symbol}:
julia> macro dynamic_import(modules)
(modules = eval(modules))::Vector{Symbol}
ex = Expr(:toplevel)
for m in modules
push!(ex.args, Expr(:import, m))
end
return ex
end
You could also generalize this line:
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
By abstracting it into a function that takes a Regex and a String directory path as arguments and returns a Vector{Symbol}:
julia> function needed_modules(rx::Regex, dir::String=".")
module_files = filter(rx, readdir(dir))
module_syms = map(m -> symbol(split(m, '.')[1]), module_files)
end
needed_modules (generic function with 2 methods)
So you may use it like this:
julia> #dynamic_import [:Mod01, :Mod02] # :vcat expression
julia> rx = r"^Mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx) # :call expression
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:Mod01
:Mod02
julia> #dynamic_import modules # Symbol
Finally you could wrap it all into a module so you may use using DynamicImport:
Note: currently I get this when trying to run the same examples from a module:
julia> using DynamicImport
julia> #dynamic_import [:mod01, :mod02]
julia> rx = r"^mod[0-9][0-9].jl$";
julia> #dynamic_import needed_modules(rx)
ERROR: rx not defined
julia> modules = needed_modules(rx)
2-element Array{Symbol,1}:
:mod01
:mod02
julia> #dynamic_import modules
ERROR: modules not defined
But it works fine If I define the objects inside the REPL, I guess this is an issue involving hygiene, which is something I'm not experienced with, so I'll ask at the julia-users mailing list.
The function version of import X is require("X"), so you should be able to do:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
println(module_files)
for modname in factor_files
modn = modname[1:end-3]
println(modn)
require(modn)
end
end
Then, assuming each of these defined a module of the same name, you can collect them into an array:
modules = Module[]
...
if isdefined(Main, symbol(modn))
push!(modules, getfield(Main, symbol(modn))
else
warn("importing $modn did not defined a module")
end
...
return modules
I'm not thrilled with this solution, but it seems to work. Basically I dynamically create a temporary file contains the import statements that I then include it. So:
function test_mimport()
module_files = filter(r"^mod[0-9][0-9].jl$", readdir())
path, f = mktemp()
for modname in module_files
modn = splitext(modname)[1]
write(f, "import $modn\n")
end
close(f)
include(path)
rm(path)
end
I would be curious to know if there is a more idiomatic strategy since I'm new to Julia and just wadding through its capabilities.
Edit
The strategy of setting up a temporary file is likely unnecessary since Julia also provides an include_string method.
On the Julia-user list, #Isaiah also suggests using eval(Expr(:import, symbol(modn)))