I'm trying to understand the ASTs of Julia methods. An example is the following function:
function inner(o, p)
s = A1(o, p)
s = s + A2(o, p)
end
Calling show(code_lowered(inner, (Int64, Int64))[1]) will display something like
:($(Expr(:lambda, {:o,:p}, {{:#s908,:s},{{:o,:Any,0},{:p,:Any,0},{:#s908,:Any,18},{:s,:Any,2}},{}}, quote # /home/pool/projekt/julia/grouping.jl, line 7:
s = A1(o,p) # line 8:
#s908 = +(s,A2(o,p))
s = #s908
return #s908
end)))
My question is how to interpret the part {{:#s908,:s},{{:o,:Any,0},{:p,:Any,0},{:#s908,:Any,18},{:s,:Any,2}},{}} it seems to be two local variables {:#s908,:s}. After this there are a number of symbols listed, both function arguments and local variables again, each have type information and an Int, what is this for? I'm guessing it is listing variables in the local scope and the Int is some sort of attribute? What are the possible values and meanings of these? Are they documented somewhere?
If you want the true AST, you don't need to use code_lowered, just quote:
julia> ex = quote function inner(o, p)
s = A1(o, p)
s = s + A2(o, p)
end end
quote # none, line 1:
function inner(o,p) # none, line 2:
s = A1(o,p) # line 3:
s = +(s,A2(o,p))
end
end
julia>
julia> dump(ex)
Expr
head: Symbol block
args: Array(Any,(2,))
1: Expr
head: Symbol line
args: Array(Any,(2,))
1: Int64 1
2: Symbol none
typ: Any
2: Expr
head: Symbol function
args: Array(Any,(2,))
1: Expr
head: Symbol call
args: Array(Any,(3,))
typ: Any
2: Expr
head: Symbol block
args: Array(Any,(4,))
typ: Any
typ: Any
typ: Any
You're looking at the annotated AST after removing syntactic sugar.
Related
I have a vector
a = collect(1:4)
What is the meaning of a <- a in Julia [result - false]
As per the documentation <- is not an assignment operator in Julia.
Here is a way to check it (so that in the future you will be able to solve similar problems more easily):
julia> dump(:(a <- a))
Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol <
2: Symbol a
3: Expr
head: Symbol call
args: Array{Any}((2,))
1: Symbol -
2: Symbol a
So as you can see a <- a is the same as a < (-a) thus you are comparing, using the < operator the a vector with the -a vector.
macro test1(name,arg)
println(arg.args[2])
typeof(arg.args[2])
end
#test1 test1 (
(arg1, (:max=>10))
)
I have the macro above and I'm trying to pass a Dict parameter as the second parameter. But the problem is that I cannot access it. The macro will always interpret the second parameter as an expression, and when I try to access arg.args[2].args it shows:
Vector{Any} (alias for Array{Any, 1})
so I don't know how to pass the Dict as it is.
I just want to get the second argument like:
Dict{Symbol, Int64} with 1 entry:
:max => 10
This is because macros work on code before the code is compiled. Source code is first parsed to Symbols, literals (integers, floats, strings, etc), or Expr (expressions). At this point, all expressions contain only these three things.** After the macro is done and returns an expression, that expression is compiled into runtime code where more complicated objects like Dicts can exist.
The code below illustrates the difference before and after compiling. Note how 1+5 and Dict() were expressions in the macro body, but is afterward evaluated to an Int64 and a Dict.
# splat arbitrary number of Expr
macro peekExpr(expr1, expr2, expr3tuple...)
println(typeof(expr1), " ", expr1)
println(typeof(expr2), " ", expr2)
println(typeof(expr3tuple), " ", expr3tuple)
:($expr1, $expr2, $expr3tuple)
end
evaluated = #peekExpr 1+5 Dict() Int64 10 max::Int64
#= printout
Expr 1 + 5
Expr Dict()
Tuple{Symbol,Int64,Expr} (:Int64, 10, :(max::Int64))
=#
for item in evaluated println(typeof(item), " ", item) end
#= printout
Int64 6
Dict{Any, Any} Dict{Any, Any}()
Tuple{Symbol,Int64,Expr} (:Int64, 10, :(max::Int64))
=#
**PS: Bear in mind that Expr can contain other objects if you interpolate runtime objects into them (x = Dict(); :(a in $x).args[end] vs :(a in Dict()).args[end]). It's just that macros do not work at a phase where it can access runtime objects. #peekExpr $x will only see a Expr(:$, :x).
Using eval against the following expression is straightforward:
exp = :(x + 1)
x = 1
eval(exp)
I cannot figure out how to evaluate the following expression that involves an escaped variable:
x = 1
exp = :($(esc(x)) + 1)
A dump of the exp gives:
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Expr
head: Symbol escape
args: Array{Any}((1,))
1: Int64 1 <-- current value of x
3: Int64 1
I figured out how to replace the escaped variable value like this:
exp.args[2].args[1] = 2
dump(exp) is now:
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Expr
head: Symbol escape
args: Array{Any}((1,))
1: Int64 2 <-- Replaced value
3: Int64 1
I am missing one last step that I could not find in spite of searching the docs and stackoverflow.
I am not exactly sure what you want to accomplish, however esc normally makes sense only inside macros, so if have object like this you need to wrap it around a macro.
julia> exp = :($(esc(x)) + 1)
:($(Expr(:escape, 33)) + 1)
julia> macro call_exp(); exp; end
#call_exp (macro with 1 method)
julia> x=33;
julia> #call_exp
34
suppose that I have a macro that is defined as :
macro foomacro(ex::Expr)
dump(ex)
ex
end
Currently I would like to pass my expression as a parsed string so that I may pass a rather complicated and case dependent expression that has been obtained via string concatenation.
However, trying :
#foomacro 1+2+3
gives the expected result 6 whereas
#foomacro parse("1+2+3")
returns the parsed expression :(1+2+3) instead of actually parsing it...
As far as I understand this both macros should be receiving the same expression but this is clearly not the case.
How do I get this MWE to work ?
ps: I figured out this fix but I feel like it is very dirty and "incorrect"
macro foomacro(ex::Expr)
if ex.head == :call
#in this case the user is calling the macro via a parsed string
dump(ex)
return ex
end
dump(ex)
ex
end
ps: if this is of any relevance, currently the code is running on 0.6.4 and if possible I'd rather not update to 1.0 yet since this would postpone my actual project to much...
You're mixing up levels. Let's introduce an intermediate function for clarity:
function foomacro_impl(expr)
dump(expr)
expr
end
macro foomacro(expr)
foomacro_impl(expr)
end
If run, the expression #foomacro <someexpr> will be parsed, the <someexpr> part passed to foomacro_impl, and the result treated as an expression and inserted instead of the original expression. That means that writing #foomacro 1+2+3 is equivalent to having written
let expr = :(1+2+3)
dump(expr)
expr
end
which returns
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Int64 1
3: Int64 2
4: Int64 3
:(1 + 2 + 3)
an Expr that evaluates to 6.
On the other hand, in #foomacro Meta.parse("1+2+3"), the whole argument, parse("1+2+3"), is used as expr:
julia> let expr = :(Meta.parse("1+2+3"))
dump(expr)
expr
end
Expr
head: Symbol call
args: Array{Any}((2,))
1: Expr
head: Symbol .
args: Array{Any}((2,))
1: Symbol Meta
2: QuoteNode
value: Symbol parse
2: String "1+2+3"
:(Meta.parse("1+2+3"))
So the result of the macro call is the expression Meta.parse("1+2+3"), which evaluates to another expression :(1 + 2 + 3), since it is a call to parse. The two forms are thus not receiving the same expression!
But there are ways to manually parse an expression and pass it to a macro:
You can do as I did, and use a separate "macro implementing function". Then, the expression returned by #foomacro bla is equivalent to foomacro_impl(Meta.parse(bla)). (This approach, BTW, is very useful for testing, and I recommend it most of the times.)
You can use the macro #eval to construct an expression, splice into it, and evaluate it immediately:
julia> #eval #foomacro $(Meta.parse("1+2+3"))
Expr
head: Symbol call
args: Array{Any}((4,))
1: Symbol +
2: Int64 1
3: Int64 2
4: Int64 3
6
(Or similarly, use eval and manually constructed Expr values.)
I followed the documentation of julia:
julia> :(a in (1,2,3))
:($(Expr(:in, :a, :((1,2,3)))))
Now that :(a in (1,2,3))==:($(Expr(:in, :a, :((1,2,3))))), why does julia express this expression in this way？ And what does $ exactly means? It seems to me that $ just evaluates the next expression in a global scope. I found the documentation unclear about this.
The reason :(a in (1,2,3)) is displayed awkwardly as :($(Expr(...))) is because the show function for Expr typed objects (show_unquoted in show.jl) does not understand the in infix operator and fallbacks into a generic printing format.
Essentially it is the same as :(1 + 1) except that show_unquoted recognizes + as an infix operator and formats it nicely.
In any case, :(...) and $(...) are inverse operators in some sense, so :($(..thing..)) is exactly like ..thing.., which in this case is Expr(:in,:a,:((1,2,3))).
One can see this weirdness in :(1+1) for example. The output is of Expr type, as typeof(:(1+1))==Expr confirms. It is actually Expr(:+,1,1), but typing Expr(:+,1,1) on the REPL will show :($(Expr(:+,1,1))) - the generic formatting style of Expr typed objects.
Fixing show.jl to handle in could be a nice change. But the issue is harmless and concerns display formatting.
$ is the interpolation command, Julia use this notation to interpolate Strings as well as Expression:
julia> a=1;
julia> "test $a" # => "test 1"
julia> :(b+$a) # => :(b + 1)
When you type a command in Julia REPL, it tries to evaluates the command and if the code do not have ; char at the end it prints the result, so it's more related to printing functions, that what will be seen on REPL, when a command executes.
so if you want to see the real contents of a variable one possibility is to use dump function:
julia> dump(:(a+b))
Expr
head: Symbol call
args: Array(Any,(3,))
1: Symbol +
2: Symbol a
3: Symbol b
typ: Any
julia> dump(:(a in b))
Expr
head: Symbol in
args: Array(Any,(2,))
1: Symbol a
2: Symbol b
typ: Any
It's clear from above tests, that both expressions use a common data structure of Expr with head, args and typ without any $ inside.
Now try to evaluate and print result:
julia> :(a in b)
:($(Expr(:in, :a, :b)))
julia> :(a+b)
:(a + b)
We already know that both command create a same structure but REPL can't show the result of :(a in b) better that an Expr of result of another Expr and it's why there in a $ inside. But when dealing with :(a+b), REPL do more intelligently and understands that this:
Expr
head: Symbol call
args: Array(Any,(3,))
1: Symbol +
2: Symbol a
3: Symbol b
typ: Any
is equal to :(a+b).