I have a macro works if defined in main.
macro check_set(d,v)
nm = string(v)
quote
global $v
if haskey($d,$nm)
$v = $d[$nm]
end
end
end
However, when I put it inside a Module (macro defined inside the module) to use in a function (also defined inside the module), I get scope issues.
export setVars
function setVars(val::Dict)
global max_iter
#check_set val max_iter
end
In main I call setVars(config) where config is a dictionary as expected. I get:
ERROR: UndefVarError: val not defined
Adding #macroexpand I see:
begin
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:156 =#
global max_iter
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:157 =#
if Stuff.haskey(Stuff.val, "max_iter")
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:158 =#
Stuff.max_iter = Stuff.val["max_iter"]
end
end
So the ERROR makes sense, it is looking for Stuff.val when val is locally scoped to the function.
How do I get it to look for the locally scoped val?
Your issue is related to macro hygiene. In your case, since you want d and v to refer to variable names in the scope of the macro call environment, you must "escape" them with esc
Here would be a minimal working example along the lines of what you wrote:
module Stuff
macro check_set(d,v)
nm = string(v)
d = esc(d)
v = esc(v)
quote
global $v
if haskey($d,$nm)
$v = $d[$nm]
end
end
end
end #module Stuff
We can check that this expands to what is expected:
julia> #macroexpand Stuff.#check_set val max_iter
quote
#= REPL[1]:8 =#
global max_iter
#= REPL[1]:9 =#
if Main.Stuff.haskey(val, "max_iter")
#= REPL[1]:10 =#
max_iter = val["max_iter"]
end
end
And that it also behaves as expected at run time:
julia> function setVars(val::Dict)
Stuff.#check_set val max_iter
end
setVars (generic function with 1 method)
julia> config = Dict("max_iter" => 42)
Dict{String,Int64} with 1 entry:
"max_iter" => 42
julia> setVars(config)
42
julia> max_iter
42
Related
I am using the following function to strip the line numbers from Julia ASTs:
function filter_lineno(ex::Expr)
filter!(ex.args) do e
isa(e, LineNumberNode) && return false
if isa(e, Expr)
(e::Expr).head === :line && return false
filter_lineno(e::Expr)
end
return true
end
return ex
end
But this doesn't seem to work properly when there are macros in the code. Here is a failing example:
expr = Meta.parse("begin run(``) end")
filter_lineno(expr)
I get the following error:
BoundsError: attempt to access 2-element Array{Any,1} at index [3]
Another example when processing docstrings:
expr = Meta.parse("begin \"Here is the doc\"\nmodule X end end")
filter_lineno(expr)
Yields the following:
quote
Core.#doc module X
end
end
What is wrong with this function, and how can I fix it?
Just use the MacroTools package:
julia> using MacroTools
julia> cc = Meta.parse("begin \"Here is the doc\"\nmodule X end end")
quote
#= none:1 =#
#= none:1 =# Core.#doc "Here is the doc" module X
#= none:2 =#
#= none:2 =#
end
end
julia> MacroTools.striplines(cc)
quote
Core.#doc "Here is the doc" module X
end
end
This is a duplicate question, which I've answered here on SO before:
Just use Base.remove_linenums!(ex) and it should work (TM).
What is wrong with my code? Do I have to declare x before using it?
function f(n::Int64, t::Int64)
A = ones(n,n)
for i=0:t
if i > 0
A[x,a] = rand()*A[x,a] + rand()
end
y = rand(1:n)
b = rand(1:n)
if i > 0
A[x,a] = rand()*A[x,a] + rand()*A[y,b]
end
x = y
a = min(b, rand(1:n))
end
return A
end
Here is the error thrown when trying to call f:
UndefVarError: x not defined
I think that the reason is more complex, as similar code in Python would work.
For example compare (Python):
>>> def f():
... for i in range(3):
... if i > 0:
... print(a)
... a = i
...
>>> f()
0
1
to (Julia):
julia> function f()
for i in 0:2
if i > 0
println(a)
end
a = i
end
end
f (generic function with 1 method)
julia> f()
ERROR: UndefVarError: a not defined
So what is the difference? As the Julia manual explains here you have:
for loops, while loops, and comprehensions have the following behavior: any new variables introduced in their body scopes are freshly allocated for each loop iteration, as if the loop body were surrounded by a let block
This means that in your code variables a and x as they are local to the for loop are freshly allocated in each iteration of the loop. Because of this the variable has to be assigned to before it is accessed inside the loop.
Therefore it is not needed to assign a value to x and a before the loop. It is enough to define them in scope outer to the loop (even without assigning of the value). For example like this:
julia> function f(n::Int64, t::Int64)
A = ones(n,n)
local x, a
for i=0:t
if i > 0
A[x,a] = rand()*A[x,a] + rand()
end
y = rand(1:n)
b = rand(1:n)
if i > 0
A[x,a] = rand()*A[x,a] + rand()*A[y,b]
end
x = y
a = min(b, rand(1:n))
end
return A
end
f (generic function with 1 method)
julia> f(1,1)
1×1 Array{Float64,2}:
0.94526289614139
Now it works because x and a are not freshly allocated in each iteration of the loop.
In my original toy example it would look like:
julia> function f()
local a
for i in 0:2
if i > 0
println(a)
end
a = i
end
end
f (generic function with 2 methods)
julia> f()
0
1
and you see that you get exactly what you had in Python.
I have issue after calling my macro:
#introspectable square(x) = x * x
Then when calling
square(3)
i should be able to get 9, cause the function call has been specialized to execute an attribute of the structure which is Julia code, however when I enter the macro, the code seems to be directly evaluated.
What i have tried:
struct IntrospectableFunction
name
parameters
native_function
end
(f::IntrospectableFunction)(x) = f.native_function(x)
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = tuple(expr.args[1].args[2:end]...)
body = expr.args[2].args[2]
:( global $name = IntrospectableFunction( :( name ), $parameters, :( body ) ))
end
#introspectable square(x) = x * x
square(3)
The answer should be 9 , however i get "Object of type symbol are not callable ". However if i replace :( body ) with x -> x * x i get the desired result, my objective is generalizing the macro-call.
I usually find it easier to work with expressions in macros (it is not the shortest way to write things, but, from my experience, it is much easier to control what gets generated).
Therefore I would rewrite your code as:
macro introspectable(expr)
name = expr.args[1].args[1]
parameters = expr.args[1].args[2:end]
anon = Expr(Symbol("->"), Expr(:tuple, parameters...), expr.args[2].args[2])
constr = Expr(:call, :IntrospectableFunction, QuoteNode(name), Tuple(parameters), anon)
esc(Expr(:global, Expr(Symbol("="), name, constr)))
end
Now, as you said you wanted generality I would define your functor like this:
(f::IntrospectableFunction)(x...) = f.native_function(x...)
(in this way you allow multiple positional arguments to be passed).
Now let us test our definitions:
julia> #introspectable square(x) = x * x
IntrospectableFunction(:square, (:x,), getfield(Main, Symbol("##3#4"))())
julia> square(3)
9
julia> #macroexpand #introspectable square(x) = x * x
:(global square = IntrospectableFunction(:square, (:x,), ((x,)->x * x)))
julia> #introspectable toarray(x,y) = [x,y]
IntrospectableFunction(:toarray, (:x, :y), getfield(Main, Symbol("##5#6"))())
julia> toarray("a", 10)
2-element Array{Any,1}:
"a"
10
julia> #macroexpand #introspectable toarray(x,y) = [x,y]
:(global toarray = IntrospectableFunction(:toarray, (:x, :y), ((x, y)->[x, y])))
julia> function localscopetest()
#introspectable globalfun(x...) = x
end
localscopetest (generic function with 1 method)
julia> localscopetest()
IntrospectableFunction(:globalfun, (:(x...),), getfield(Main, Symbol("##9#10"))())
julia> globalfun(1,2,3,4,5)
(1, 2, 3, 4, 5)
julia> function f()
v = 100
#introspectable localbinding(x) = (v, x)
end
f (generic function with 1 method)
julia> f()
IntrospectableFunction(:localbinding, (:x,), getfield(Main, Symbol("##11#12")){Int64}(100))
julia> localbinding("x")
(100, "x")
(note that it is useful to use #macroexpand to make sure our macro works as expected)
EDIT - how to handle a minimal multiple dispatch
I am writing a non-macro example because it is related to the data structure:
Use e.g. such a definition:
struct IntrospectableFunction
name::Symbol
method_array::Vector{Pair{Type{<:Tuple}, Function}}
end
function (f::IntrospectableFunction)(x...)
for m in f.method_array
if typeof(x) <: first(m)
return last(m)(x...)
end
end
error("signature not found")
end
and now you can write:
julia> square = IntrospectableFunction(:square, [Tuple{Any}=>x->x*x,Tuple{Any,Any}=>(x,y)->x*y])
IntrospectableFunction(:square, Pair{DataType,Function}[Tuple{Any}=>##9#11(), Tuple{Any,Any}=>##10#12()])
julia> square(3)
9
julia> square(2,3)
6
Keep in mind that the approach I present is not perfect and universal - it just serves to give a very simple example how you could do it.
Is there a build-in Julia function for stripping LineNumberNode in Expr? especially for macrocalls:
julia> ex = :(#foo 1)
:(#= REPL[5]:1 =# #foo 1)
julia> dump(ex)
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #foo
2: LineNumberNode
line: Int64 1
file: Symbol REPL[5]
3: Int64 1
Tried MacroTools.striplines, but
julia> ex = :(#foo 1+1)
:(#= REPL[7]:1 =# #foo 1 + 1)
julia> MacroTools.striplines(ex) |> dump
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #foo
2: LineNumberNode
line: Int64 1
file: Symbol REPL[7]
3: Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 1
My use-case is to compare two different exprs constructed in different files(so different line number info). My current workaround is to explicitly write Expr(:macrocall, Symbol("#foo"), nothing, :(1+1)) which is a little bit verbose.
The built-in function is Base.remove_linenums!:
julia> ex = quote begin
x = 3
y = 2
z = 4
foo(x) = 3
end
end
quote
#= REPL[2]:1 =#
begin
#= REPL[2]:2 =#
x = 3
#= REPL[2]:3 =#
y = 2
#= REPL[2]:4 =#
z = 4
#= REPL[2]:5 =#
foo(x) = begin
#= REPL[2]:5 =#
3
end
end
end
julia> Base.remove_linenums!(ex)
quote
begin
x = 3
y = 2
z = 4
foo(x) = begin
3
end
end
end
Credit to Alex Arslan for reminding me of it.
Not built in, but MacroTools.jl has MacroTools.striplines(ex) which removes the LineNumberNodes from an expression.
Since your goal is to be able to compare Exprs maybe replace LineNumberNodes with nothing. This allows to make comparisons and the Exprs still work. See the example below:
julia> macro hello(world)
println("hello ",world)
end
#hello (macro with 1 method)
julia> m1 = :(#hello "world")
:(#= REPL[99]:1 =# #hello "world")
julia> m2 = :(#hello "world")
:(#= REPL[100]:1 =# #hello "world")
julia> m1 == m2
false
julia> replace!(arg -> typeof(arg) <: LineNumberNode ? nothing : arg, m1.args);
julia> replace!(arg -> typeof(arg) <: LineNumberNode ? nothing : arg, m2.args);
julia> dump(m1)
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #hello
2: Nothing nothing
3: String "world"
julia> eval(m1)
hello world
julia> m1 == m2
true
Of course if your code is nested you will have to make the replace its elements recursively over the entire Expr's AST.
You can consider defining the following function to achieve what you want by comparing two expressions for equality ignoring line number nodes:
function cmpexpr(ex1::Expr, ex2::Expr)
ex1.head === ex2.head || return false
length(ex1.args) === length(ex2.args) || return false
for (a1, a2) in zip(ex1.args, ex2.args)
typeof(a1) === typeof(a2) || return false
if a1 isa Expr
cmpexpr(a1, a2) || return false
elseif !(a1 isa LineNumberNode)
isequal(a1, a2) || return false
end
end
return true
end
julia 0.5.1
I want to create a function inside a quote that can be used after the specified macro has been used. Here is an example of what I mean
macro wat()
quote
type Foo end
global bar() = begin end
global function bar2()
end
type Baaz end
end
end
#wat
Foo()
Baaz()
bar()
bar2()
Now when I run this the last line crashes, because bar2 is undefined. I do not understand why because in my understanding bar() and bar2() should be equal and bar is just syntactic sugar for bar2. But they are apparently not equal and I do not understand why the one works and other does not.
Secondly is there a way to define bar and bar2 inside that quote without the global-keyword and still being available after the macro has been executed?
My motivation for wanting the bar2 notation is that I can specify a return-type with this syntax.
global bar3()::Void = begin end
Is not allowed syntax.
In the returned expressions of Julia macros, names of local variables are replaced with unique symbols:
julia> macro foo()
quote
x = 1
global y = 2
end
end
#foo (macro with 1 method)
julia> macroexpand(:(#foo))
quote # REPL[1], line 4:
#1#x = 1
global y = 2
end
This feature is called macro hygiene and avoids accidental clashes with variables at the call site.
To escape from this behavior, one has to use esc:
julia> macro bar()
quote
x = 1
end |> esc
end
#bar (macro with 1 method)
julia> macroexpand(:(#bar))
quote # REPL[1], line 3:
x = 1
end
Often, one doesn't want to escape the whole returned expression but only specific parts of it:
julia> macro myshow(expr)
quote
x = $(esc(expr))
println($(string(expr)), " = ", x)
x
end
end
#myshow (macro with 1 method)
julia> x = pi/2
1.5707963267948966
julia> macroexpand(:(#myshow sin(x)))
quote # REPL[1], line 3:
#1#x = sin(x) # REPL[1], line 4:
(Main.println)("sin(x)", " = ", #1#x) # REPL[1], line 5:
#1#x
end
julia> #myshow sin(x)
sin(x) = 1.0
1.0
julia> x
1.5707963267948966
For details, I recommend to read the corresponding section in the manual.