I am using the following function to strip the line numbers from Julia ASTs:
function filter_lineno(ex::Expr)
filter!(ex.args) do e
isa(e, LineNumberNode) && return false
if isa(e, Expr)
(e::Expr).head === :line && return false
filter_lineno(e::Expr)
end
return true
end
return ex
end
But this doesn't seem to work properly when there are macros in the code. Here is a failing example:
expr = Meta.parse("begin run(``) end")
filter_lineno(expr)
I get the following error:
BoundsError: attempt to access 2-element Array{Any,1} at index [3]
Another example when processing docstrings:
expr = Meta.parse("begin \"Here is the doc\"\nmodule X end end")
filter_lineno(expr)
Yields the following:
quote
Core.#doc module X
end
end
What is wrong with this function, and how can I fix it?
Just use the MacroTools package:
julia> using MacroTools
julia> cc = Meta.parse("begin \"Here is the doc\"\nmodule X end end")
quote
#= none:1 =#
#= none:1 =# Core.#doc "Here is the doc" module X
#= none:2 =#
#= none:2 =#
end
end
julia> MacroTools.striplines(cc)
quote
Core.#doc "Here is the doc" module X
end
end
This is a duplicate question, which I've answered here on SO before:
Just use Base.remove_linenums!(ex) and it should work (TM).
Related
I would like to run the function f() on all my 4 processors (intel i7) and fetch the rands sum, as follows:
using Distributed;
#everywhere function f()
return sum(rand(10000))
end
#sync for w in workers()
#async begin
res = #spawnat w f()
values[w-1] = fetch(res)
end
end
But, getting the following error:
ERROR: TaskFailedException
nested task error: MethodError: no method matching setindex!(::typeof(values), ::Float64, ::Int64)
Stacktrace:
[1] macro expansion
# ./REPL[58]:4 [inlined]
[2] (::var"#68#70"{Channel{Any}, Int64})()
# Main ./task.jl:411
Stacktrace:
[1] sync_end(c::Channel{Any})
# Base ./task.jl:369
[2] top-level scope
# task.jl:388
Please guide me in resolving the issue!
For your code the easiest way would be (assuming Julia has been run with -p 4 command line parameter or you have run addprocs(4):
julia> values = #distributed (append!) for i in 1:4
[f()]
end
4-element Vector{Float64}:
5001.232864826896
4999.244031827526
4966.883114472259
5014.022690758762
If you want to do #spawns yourself this code works:
julia> values = Vector{Float64}(undef, 4);
julia> #sync for w in workers()
#async values[w-1] = fetch(#spawnat w f())
end
julia> values
4-element Vector{Float64}:
5029.967318172736
4993.1064528029
5016.491407076979
5062.0706219606345
However your code mostly didn't work because the type of your values was not Vector{Float64}. Here is how to replicate your error:
julia> vv()=0
vv (generic function with 1 method)
julia> vv[1]=11
ERROR: MethodError: no method matching setindex!(::typeof(vv), ::Int64, ::Int64)
I have a macro works if defined in main.
macro check_set(d,v)
nm = string(v)
quote
global $v
if haskey($d,$nm)
$v = $d[$nm]
end
end
end
However, when I put it inside a Module (macro defined inside the module) to use in a function (also defined inside the module), I get scope issues.
export setVars
function setVars(val::Dict)
global max_iter
#check_set val max_iter
end
In main I call setVars(config) where config is a dictionary as expected. I get:
ERROR: UndefVarError: val not defined
Adding #macroexpand I see:
begin
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:156 =#
global max_iter
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:157 =#
if Stuff.haskey(Stuff.val, "max_iter")
#= /home/dpazzula/Documents/Stuff/src/Stuff.jl:158 =#
Stuff.max_iter = Stuff.val["max_iter"]
end
end
So the ERROR makes sense, it is looking for Stuff.val when val is locally scoped to the function.
How do I get it to look for the locally scoped val?
Your issue is related to macro hygiene. In your case, since you want d and v to refer to variable names in the scope of the macro call environment, you must "escape" them with esc
Here would be a minimal working example along the lines of what you wrote:
module Stuff
macro check_set(d,v)
nm = string(v)
d = esc(d)
v = esc(v)
quote
global $v
if haskey($d,$nm)
$v = $d[$nm]
end
end
end
end #module Stuff
We can check that this expands to what is expected:
julia> #macroexpand Stuff.#check_set val max_iter
quote
#= REPL[1]:8 =#
global max_iter
#= REPL[1]:9 =#
if Main.Stuff.haskey(val, "max_iter")
#= REPL[1]:10 =#
max_iter = val["max_iter"]
end
end
And that it also behaves as expected at run time:
julia> function setVars(val::Dict)
Stuff.#check_set val max_iter
end
setVars (generic function with 1 method)
julia> config = Dict("max_iter" => 42)
Dict{String,Int64} with 1 entry:
"max_iter" => 42
julia> setVars(config)
42
julia> max_iter
42
Is there a build-in Julia function for stripping LineNumberNode in Expr? especially for macrocalls:
julia> ex = :(#foo 1)
:(#= REPL[5]:1 =# #foo 1)
julia> dump(ex)
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #foo
2: LineNumberNode
line: Int64 1
file: Symbol REPL[5]
3: Int64 1
Tried MacroTools.striplines, but
julia> ex = :(#foo 1+1)
:(#= REPL[7]:1 =# #foo 1 + 1)
julia> MacroTools.striplines(ex) |> dump
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #foo
2: LineNumberNode
line: Int64 1
file: Symbol REPL[7]
3: Expr
head: Symbol call
args: Array{Any}((3,))
1: Symbol +
2: Int64 1
3: Int64 1
My use-case is to compare two different exprs constructed in different files(so different line number info). My current workaround is to explicitly write Expr(:macrocall, Symbol("#foo"), nothing, :(1+1)) which is a little bit verbose.
The built-in function is Base.remove_linenums!:
julia> ex = quote begin
x = 3
y = 2
z = 4
foo(x) = 3
end
end
quote
#= REPL[2]:1 =#
begin
#= REPL[2]:2 =#
x = 3
#= REPL[2]:3 =#
y = 2
#= REPL[2]:4 =#
z = 4
#= REPL[2]:5 =#
foo(x) = begin
#= REPL[2]:5 =#
3
end
end
end
julia> Base.remove_linenums!(ex)
quote
begin
x = 3
y = 2
z = 4
foo(x) = begin
3
end
end
end
Credit to Alex Arslan for reminding me of it.
Not built in, but MacroTools.jl has MacroTools.striplines(ex) which removes the LineNumberNodes from an expression.
Since your goal is to be able to compare Exprs maybe replace LineNumberNodes with nothing. This allows to make comparisons and the Exprs still work. See the example below:
julia> macro hello(world)
println("hello ",world)
end
#hello (macro with 1 method)
julia> m1 = :(#hello "world")
:(#= REPL[99]:1 =# #hello "world")
julia> m2 = :(#hello "world")
:(#= REPL[100]:1 =# #hello "world")
julia> m1 == m2
false
julia> replace!(arg -> typeof(arg) <: LineNumberNode ? nothing : arg, m1.args);
julia> replace!(arg -> typeof(arg) <: LineNumberNode ? nothing : arg, m2.args);
julia> dump(m1)
Expr
head: Symbol macrocall
args: Array{Any}((3,))
1: Symbol #hello
2: Nothing nothing
3: String "world"
julia> eval(m1)
hello world
julia> m1 == m2
true
Of course if your code is nested you will have to make the replace its elements recursively over the entire Expr's AST.
You can consider defining the following function to achieve what you want by comparing two expressions for equality ignoring line number nodes:
function cmpexpr(ex1::Expr, ex2::Expr)
ex1.head === ex2.head || return false
length(ex1.args) === length(ex2.args) || return false
for (a1, a2) in zip(ex1.args, ex2.args)
typeof(a1) === typeof(a2) || return false
if a1 isa Expr
cmpexpr(a1, a2) || return false
elseif !(a1 isa LineNumberNode)
isequal(a1, a2) || return false
end
end
return true
end
julia 0.5.1
I want to create a function inside a quote that can be used after the specified macro has been used. Here is an example of what I mean
macro wat()
quote
type Foo end
global bar() = begin end
global function bar2()
end
type Baaz end
end
end
#wat
Foo()
Baaz()
bar()
bar2()
Now when I run this the last line crashes, because bar2 is undefined. I do not understand why because in my understanding bar() and bar2() should be equal and bar is just syntactic sugar for bar2. But they are apparently not equal and I do not understand why the one works and other does not.
Secondly is there a way to define bar and bar2 inside that quote without the global-keyword and still being available after the macro has been executed?
My motivation for wanting the bar2 notation is that I can specify a return-type with this syntax.
global bar3()::Void = begin end
Is not allowed syntax.
In the returned expressions of Julia macros, names of local variables are replaced with unique symbols:
julia> macro foo()
quote
x = 1
global y = 2
end
end
#foo (macro with 1 method)
julia> macroexpand(:(#foo))
quote # REPL[1], line 4:
#1#x = 1
global y = 2
end
This feature is called macro hygiene and avoids accidental clashes with variables at the call site.
To escape from this behavior, one has to use esc:
julia> macro bar()
quote
x = 1
end |> esc
end
#bar (macro with 1 method)
julia> macroexpand(:(#bar))
quote # REPL[1], line 3:
x = 1
end
Often, one doesn't want to escape the whole returned expression but only specific parts of it:
julia> macro myshow(expr)
quote
x = $(esc(expr))
println($(string(expr)), " = ", x)
x
end
end
#myshow (macro with 1 method)
julia> x = pi/2
1.5707963267948966
julia> macroexpand(:(#myshow sin(x)))
quote # REPL[1], line 3:
#1#x = sin(x) # REPL[1], line 4:
(Main.println)("sin(x)", " = ", #1#x) # REPL[1], line 5:
#1#x
end
julia> #myshow sin(x)
sin(x) = 1.0
1.0
julia> x
1.5707963267948966
For details, I recommend to read the corresponding section in the manual.
In a different question on Stack Overflow the answer included the following function:
julia> function nzcols(b::SubArray{T,2,P,Tuple{UnitRange{Int64},UnitRange{Int64}}}) where {T,P<:SparseMatrixCSC}
return collect(i+1-start(b.indexes[2])
for i in b.indexes[2]
if b.parent.colptr[i]<b.parent.colptr[i+1] &&
inrange(b.parent.rowval[nzrange(b.parent,i)],b.indexes[1]))
end
nzcols (generic function with 3 methods)
And it was parsed without error. When adding a new-line before where clause for readability, an error suddenly appeared:
julia> function nzcols(b::SubArray{T,2,P,Tuple{UnitRange{Int64},UnitRange{Int64}}})
where {T,P<:SparseMatrixCSC}
return collect(i+1-start(b.indexes[2])
for i in b.indexes[2]
if b.parent.colptr[i]<b.parent.colptr[i+1] &&
inrange(b.parent.rowval[nzrange(b.parent,i)],b.indexes[1]))
end
ERROR: syntax: space before "{" not allowed in "where {"
Finally, when the parameter list parenthesis is moved to the where line, the error disappears again:
julia> function nzcols(b::SubArray{T,2,P,Tuple{UnitRange{Int64},UnitRange{Int64}}}
) where {T,P<:SparseMatrixCSC}
return collect(i+1-start(b.indexes[2])
for i in b.indexes[2]
if b.parent.colptr[i]<b.parent.colptr[i+1] &&
inrange(b.parent.rowval[nzrange(b.parent,i)],b.indexes[1]))
end
nzcols (generic function with 3 methods)
What is the logic behind this syntax and should it be fixed?
This is similar to many other syntaxes in the language; if the parser has a "complete" syntax at the end of a line, it'll use that and move on.
julia> parse("begin; 1 \n+ 2; end")
quote # none, line 1:
1 # none, line 2:
+2
end
julia> parse("begin; 1 +\n 2; end")
quote # none, line 1:
1 + 2
end
Note that this means you can still break the where clause onto a separate line, but the where itself needs to be on the same line as the end of the function.