The code at the end of this post constructs a function which is bound to the variables of a given dictionary. Furthermore, the function is not bound to the actual name of the dictionary (as I use the Ref() statement).
An example:
julia> D = Dict(:x => 4, :y => 5)
julia> f= #mymacro4(x+2y, D)
julia> f()
14
julia> DD = D
julia> D = nothing
julia> f()
14
julia> DD[:x] = 12
julia> f()
22
Now I want to be able to construct exactly the same function when I only have access to the expression expr = :(x+2y).
How do I do this? I tried several things, but was not able to find a solution.
julia> f = #mymacro4(:(x+2y), D)
julia> f() ### the function evaluation should also yield 14. But it yields:
:(DR.x[:x] + 2 * DR.x[:y])
(I actually want to use it within another macro in which the dictionary is automatically created. I want to store this dictionary and the function within a struct, such that I'm able to call this function at a later point in time and manipulate the objects in the dictionary. If necessary, I may post the complete example and explain the complete problem.)
_freevars2(literal) = literal
function _freevars2(s::Symbol)
try
if typeof(eval(s)) <: Function
return s
else
return Meta.parse("DR.x[:$s]")
end
catch
return Meta.parse("DR.x[:$s]")
end
end
function _freevars2(expr::Expr)
for (it, s) in enumerate(expr.args)
expr.args[it] = _freevars2(s)
end
return expr
end
macro mymacro4(expr, D)
expr2 = _freevars2(expr)
quote
let DR = Ref($(esc(D)))
function mysym()
$expr2
end
end
end
end
suppose that I have a macro that is defined as :
macro foomacro(ex::Expr)
dump(ex)
ex
end
Currently I would like to pass my expression as a parsed string so that I may pass a rather complicated and case dependent expression that has been obtained via string concatenation.
However, trying :
#foomacro 1+2+3
gives the expected result 6 whereas
#foomacro parse("1+2+3")
returns the parsed expression :(1+2+3) instead of actually parsing it...
As far as I understand this both macros should be receiving the same expression but this is clearly not the case.
How do I get this MWE to work ?
ps: I figured out this fix but I feel like it is very dirty and "incorrect"
macro foomacro(ex::Expr)
if ex.head == :call
#in this case the user is calling the macro via a parsed string
dump(ex)
return ex
end
dump(ex)
ex
end
ps: if this is of any relevance, currently the code is running on 0.6.4 and if possible I'd rather not update to 1.0 yet since this would postpone my actual project to much...
You're mixing up levels. Let's introduce an intermediate function for clarity:
function foomacro_impl(expr)
dump(expr)
expr
end
macro foomacro(expr)
foomacro_impl(expr)
end
If run, the expression #foomacro <someexpr> will be parsed, the <someexpr> part passed to foomacro_impl, and the result treated as an expression and inserted instead of the original expression. That means that writing #foomacro 1+2+3 is equivalent to having written
let expr = :(1+2+3)
dump(expr)
expr
end
which returns
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Int64 1
3: Int64 2
4: Int64 3
:(1 + 2 + 3)
an Expr that evaluates to 6.
On the other hand, in #foomacro Meta.parse("1+2+3"), the whole argument, parse("1+2+3"), is used as expr:
julia> let expr = :(Meta.parse("1+2+3"))
dump(expr)
expr
end
Expr
head: Symbol call
args: Array{Any}((2,))
1: Expr
head: Symbol .
args: Array{Any}((2,))
1: Symbol Meta
2: QuoteNode
value: Symbol parse
2: String "1+2+3"
:(Meta.parse("1+2+3"))
So the result of the macro call is the expression Meta.parse("1+2+3"), which evaluates to another expression :(1 + 2 + 3), since it is a call to parse. The two forms are thus not receiving the same expression!
But there are ways to manually parse an expression and pass it to a macro:
You can do as I did, and use a separate "macro implementing function". Then, the expression returned by #foomacro bla is equivalent to foomacro_impl(Meta.parse(bla)). (This approach, BTW, is very useful for testing, and I recommend it most of the times.)
You can use the macro #eval to construct an expression, splice into it, and evaluate it immediately:
julia> #eval #foomacro $(Meta.parse("1+2+3"))
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Int64 1
3: Int64 2
4: Int64 3
6
(Or similarly, use eval and manually constructed Expr values.)
Suppose I have an expression like :(Main.i / (0.5 * Main.i * sin(Main.i)).
I would like to replace each occurence of Main.i into some other symbol. Is there an idiomatic way to do this in Julia?
Main.i is not a symbol but an expression, which you can check by doing dump(:(Main.i)).
Here is a quick writeup what I think might match your needs:
function expr_replace(expr, old, new)
expr == old && return new
if expr isa Expr
expr = deepcopy(expr) # to avoid mutation of source
for i in eachindex(expr.args)
expr.args[i] = expr_replace(expr.args[i], old, new)
end
end
expr
end
Does it work for you?
EDIT: Here is a version that does a minimal safe use of deepcopy:
function expr_replace(expr, old, new)
function f(expr)
expr == old && return deepcopy(new)
if expr isa Expr
for i in eachindex(expr.args)
expr.args[i] = f(expr.args[i])
end
end
expr
end
f(deepcopy(expr))
end
In general I guess this will not matter that much as you probably will not want to pass 100 lines of code through this function.
I'm writing a genetic program in order to test the fitness of randomly generated expressions. Shown here is the function to generate the expression as well a the main function. DIV and GT are defined elsewhere in the code:
function create_single_full_tree(depth, fs, ts)
"""
Creates a single AST with full depth
Inputs
depth Current depth of tree. Initially called from main() with max depth
fs Function Set - Array of allowed functions
ts Terminal Set - Array of allowed terminal values
Output
Full AST of typeof()==Expr
"""
# If we are at the bottom
if depth == 1
# End of tree, return function with two terminal nodes
return Expr(:call, fs[rand(1:length(fs))], ts[rand(1:length(ts))], ts[rand(1:length(ts))])
else
# Not end of expression, recurively go back through and create functions for each new node
return Expr(:call, fs[rand(1:length(fs))], create_single_full_tree(depth-1, fs, ts), create_single_full_tree(depth-1, fs, ts))
end
end
function main()
"""
Main function
"""
# Define functional and terminal sets
fs = [:+, :-, :DIV, :GT]
ts = [:x, :v, -1]
# Create the tree
ast = create_single_full_tree(4, fs, ts)
#println(typeof(ast))
#println(ast)
#println(dump(ast))
x = 1
v = 1
eval(ast) # Error out unless x and v are globals
end
main()
I am generating a random expression based on certain allowed functions and variables. As seen in the code, the expression can only have symbols x and v, as well as the value -1. I will need to test the expression with a variety of x and v values; here I am just using x=1 and v=1 to test the code.
The expression is being returned correctly, however, eval() can only be used with global variables, so it will error out when run unless I declare x and v to be global (ERROR: LoadError: UndefVarError: x not defined). I would like to avoid globals if possible. Is there a better way to generate and evaluate these generated expressions with locally defined variables?
Here is an example for generating an (anonymous) function. The result of eval can be called as a function and your variable can be passed as parameters:
myfun = eval(Expr(:->,:x, Expr(:block, Expr(:call,:*,3,:x) )))
myfun(14)
# returns 42
The dump function is very useful to inspect the expression that the parsers has created. For two input arguments you would use a tuple for example as args[1]:
julia> dump(parse("(x,y) -> 3x + y"))
Expr
head: Symbol ->
args: Array{Any}((2,))
1: Expr
head: Symbol tuple
args: Array{Any}((2,))
1: Symbol x
2: Symbol y
typ: Any
2: Expr
[...]
Does this help?
In the Metaprogramming part of the Julia documentation, there is a sentence under the eval() and effects section which says
Every module has its own eval() function that evaluates expressions in its global scope.
Similarly, the REPL help ?eval will give you, on Julia 0.6.2, the following help:
Evaluate an expression in the given module and return the result. Every Module (except those defined with baremodule) has its own 1-argument definition of eval, which evaluates expressions in that module.
I assume, you are working in the Main module in your example. That's why you need to have the globals defined there. For your problem, you can use macros and interpolate the values of x and y directly inside the macro.
A minimal working example would be:
macro eval_line(a, b, x)
isa(a, Real) || (warn("$a is not a real number."); return :(throw(DomainError())))
isa(b, Real) || (warn("$b is not a real number."); return :(throw(DomainError())))
return :($a * $x + $b) # interpolate the variables
end
Here, #eval_line macro does the following:
Main> #macroexpand #eval_line(5, 6, 2)
:(5 * 2 + 6)
As you can see, the values of macro's arguments are interpolated inside the macro and the expression is given to the user accordingly. When the user does not behave,
Main> #macroexpand #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
:((Main.throw)((Main.DomainError)()))
a user-friendly warning message is provided to the user at parse-time, and a DomainError is thrown at run-time.
Of course, you can do these things within your functions, again by interpolating the variables --- you do not need to use macros. However, what you would like to achieve in the end is to combine eval with the output of a function that returns Expr. This is what the macro functionality is for. Finally, you would simply call your macros with an # sign preceding the macro name:
Main> #eval_line(5, 6, 2)
16
Main> #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
ERROR: DomainError:
Stacktrace:
[1] eval(::Module, ::Any) at ./boot.jl:235
EDIT 1. You can take this one step further, and create functions accordingly:
macro define_lines(linedefs)
for (name, a, b) in eval(linedefs)
ex = quote
function $(Symbol(name))(x) # interpolate name
return $a * x + $b # interpolate a and b here
end
end
eval(ex) # evaluate the function definition expression in the module
end
end
Then, you can call this macro to create different line definitions in the form of functions to be called later on:
#define_lines([
("identity_line", 1, 0);
("null_line", 0, 0);
("unit_shift", 0, 1)
])
identity_line(5) # returns 5
null_line(5) # returns 0
unit_shift(5) # returns 1
EDIT 2. You can, I guess, achieve what you would like to achieve by using a macro similar to that below:
macro random_oper(depth, fs, ts)
operations = eval(fs)
oper = operations[rand(1:length(operations))]
terminals = eval(ts)
ts = terminals[rand(1:length(terminals), 2)]
ex = :($oper($ts...))
for d in 2:depth
oper = operations[rand(1:length(operations))]
t = terminals[rand(1:length(terminals))]
ex = :($oper($ex, $t))
end
return ex
end
which will give the following, for instance:
Main> #macroexpand #random_oper(1, [+, -, /], [1,2,3])
:((-)([3, 3]...))
Main> #macroexpand #random_oper(2, [+, -, /], [1,2,3])
:((+)((-)([2, 3]...), 3))
Thanks Arda for the thorough response! This helped, but part of me thinks there may be a better way to do this as it seems too roundabout. Since I am writing a genetic program, I will need to create 500 of these ASTs, all with random functions and terminals from a set of allowed functions and terminals (fs and ts in the code). I will also need to test each function with 20 different values of x and v.
In order to accomplish this with the information you have given, I have come up with the following macro:
macro create_function(defs)
for name in eval(defs)
ex = quote
function $(Symbol(name))(x,v)
fs = [:+, :-, :DIV, :GT]
ts = [x,v,-1]
return create_single_full_tree(4, fs, ts)
end
end
eval(ex)
end
end
I can then supply a list of 500 random function names in my main() function, such as ["func1, func2, func3,.....". Which I can eval with any x and v values in my main function. This has solved my issue, however, this seems to be a very roundabout way of doing this, and may make it difficult to evolve each AST with each iteration.
How could I check the element type of a nested array assumming I don't know the level of nesting?:
julia> a = [[[[1]]]]
1-element Array{Array{Array{Array{Int64,1},1},1},1}:
Array{Array{Int64,1},1}[Array{Int64,1}[[1]]]
julia> etype(a)
Int64?
As it is often the case for type computations, a recursive approach works very well:
nested_eltype(x) = nested_eltype(typeof(x))
nested_eltype{T<:AbstractArray}(::Type{T}) = nested_eltype(eltype(T))
nested_eltype{T}(::Type{T}) = T
This has no runtime overhead:
julia> #code_llvm nested_eltype([[[[[[[[[[1]]]]]]]]]])
define %jl_value_t* #julia_nested_eltype_71712(%jl_value_t*) #0 {
top:
ret %jl_value_t* inttoptr (i64 140658266768816 to %jl_value_t*)
}
There is probably a clever way to do this with one line, but in the meantime, you can use recursive calls to eltype to do this inside a while loop:
function nested_eltype(x::AbstractArray)
y = eltype(x)
while y <: AbstractArray
y = eltype(y)
end
return(y)
end
Note that this works for nested arrays of any dimension, ie it doesn't just have to be Vector like the example in your question...
This is a generated version using .depth attribute:
#generated function etype{T}(x::T)
ex = :(x)
for i = 1:T.depth
ex = :($ex |> eltype)
end
ex
end
julia> a = [[[[1]]]]
1-element Array{Array{Array{Array{Int64,1},1},1},1}:
Array{Array{Int64,1},1}[Array{Int64,1}[[1]]]
julia> etype(a)
Int64