I am generating some code that is later being evaluated. Even though the generated code is correct and evaluating it line by line does not cause issues, it fails to be correctly evaluated as a whole.
eval(quote
macro m() "return" end
#m()
end)
Returns:
ERROR: LoadError: UndefVarError: #m not defined
eval(quote macro m() "return" end end)
eval(#m())
Returns: "return"
Macro expansion is done before evaluation, so when macro expansion occurs in this code the definition of the macro in the first expression in the block happens too late to affect the expansion of the second expression in the block. There is one special case which does what you want: the :toplevel expression type. This is automatically used for top-level global expressions in modules but you can manually construct expressions of this type like this:
ex = Expr(:toplevel,
:(macro m() "return" end),
:(#m())
)
And sure enough, this does what you want:
julia> eval(ex)
"return"
Since Julia doesn't have locally scoped macros, this macro definition has to take place in global scope already so presumably this should work anywhere that the original macro would have worked—i.e. a macro definition should be valid in all the same places that a top-level compound expression is valid.
Related
I would like to build a Julia application where a user can specify a function using a configuration file (and therefore as a string). The configuration file then needs to be parsed before the function is evaluated in the program.
The problem is that while the function name is known locally, it is not known in the module containing the parser. One solution I have come up with is to pass the local eval function to the parsing function but that does not seem very elegant.
I have tried to come up with a minimal working example here, where instead of parsing a configuration file, the function name is already contained in a string:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
# in a real application, parseconfig would parse the configuration file to extract funstr
function parseconfig(funstr)
return eval(Meta.parse(funstr))
end
function parseconfig(funstr, myeval)
return myeval(Meta.parse(funstr))
end
end
# test 1 -- succeeds
f1 = MyFuns.myfun
println("test1: $(f1(1))")
# test 2 -- succeeds
f2 = MyUtil.parseconfig("MyFuns.myfun", eval)
println("test2: $(f2(1))")
# test 3 -- fails
f3 = MyUtil.parseconfig("MyFuns.myfun")
println("test3: $(f3(1))")
The output is:
test1: 3
test2: 3
ERROR: LoadError: UndefVarError: MyFuns not defined
So, the second approach works but is there a better way to achieve the goal?
Meta.parse() will transform your string to an AST. What MyFuns.myfun refers to depends on the scope provided by the eval() you use.
The issue with your example is that the eval() inside MyUtil will evaluate in the context of that module. If that is the desired behavior, you simply miss using MyFuns inside MyUtil.
But what you really want to do is write a macro. This allows the code to be included when parsing your program, before running it. The macro will have access to a special argument __module__, which is the context where the macro is used. So __module__.eval() will execute an expression in that very scope.
foo = "outside"
module MyMod
foo = "inside"
macro eval(string)
expr = Meta.parse(string)
__module__.eval(expr)
end
end
MyMod.#eval "foo"
# Output is "outside"
See also this explanation on macros:
https://docs.julialang.org/en/v1/manual/metaprogramming/index.html#man-macros-1
And for the sake of transforming the answer of #MauricevanLeeuwen into the framework of my question, this code will work:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
macro parseconfig(funstr)
__module__.eval(Meta.parse(funstr))
end
end
f4 = MyUtil.#parseconfig "MyFuns.myfun"
println("test4: $(f4(1))")
I have a script that defines a function, and later intended to call the function but forgot to add the parentheses, like this:
function myfunc()
println("inside myfunc")
end
myfunc # This line is silently ignored. The function isn't invoked and there's no error message.
After a while I did figure out that I was missing the parentheses, but since Julia didn't give me an error, I'm wondering what that line is actually doing? I'm assuming that it must be doing something with the myfunc statement, but I don't know Julia well enough to understand what is happening.
I tried --depwarn=yes but don't see a julia command line switch to increase the warning level or verbosity. Please let me know if one exists.
For background context, the reason this came up is that I'm trying to translate a Bash script to Julia, and there are numerous locations where an argument-less function is defined and then invoked, and in Bash you don't need parentheses after the function name to invoke it.
The script is being run from command line (julia stub.jl) and I'm using Julia 1.0.3 on macOS.
It doesn't silently ignore the function. Calling myfunc in an interactive session will show you what happens: the call returns the function object to the console, and thus call's the show method for Function, showing how many methods are currently defined for that function in your workspace.
julia> function myfunc()
println("inside myfunc")
end
myfunc (generic function with 1 method)
julia> myfunc
myfunc (generic function with 1 method)
Since you're calling this in a script, show is never called, and thus you don't see any result. But it doesn't error, because the syntax is valid.
Thanks to DNF for the helpful comment on it being in a script.
It does nothing.
As in c, an expression has a value: in c the expression _ a=1+1; _ has the value _ 2 _ In c, this just fell out of the parser: they wanted to be able to evaluate expressions like _ a==b _
In Julia, it's the result of designing a language where the code you write is handled as a data object of the language. In Julia, the expression "a=1+1" has the value "a=1+1".
In c, the fact that
a=1+1;
is an acceptable line of code means that, accidentally,
a;
is also an acceptable line of code. The same is true in Julia: the compiler expects to see a data value there: any data value you put may be acceptable: even for example the data value that represents the calculated value returned by a function:
myfunc()
or the value that represents the function object itself:
myfunc
As in c, the fact that data values are everywhere in your code just indicates that the syntax allows data values everywhere in your code and that the compiler does nothing with data values that are everywhere in your code.
I would like to write a macro #unpack t which takes an object t and copies all its fields into local scope. For example, given
immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
the expression #unpack foo should expand into
i = foo.i
x = foo.x
Unfortunately, such a macro cannot exist since it would have to know the type of the passed object. To circumvent this limitation, I introduce a type-specific macro #unpackFoo foo with the same effect, but since I'm lazy I want the compiler to write #unpackFoo for me. So I change the type definition to
#unpackable immutable Foo
i::Int
x::Float64
end
which should expand into
immutable Foo
i::Int
x::Float64
end
macro unpackFoo(t)
return esc(quote
i = $t.i
x = $t.x
end)
end
Writing #unpackable is not too hard:
macro unpackable(expr)
if expr.head != :type
error("#unpackable must be applied on a type definition")
end
name = isa(expr.args[2], Expr) ? expr.args[2].args[1] : expr.args[2]
fields = Symbol[]
for bodyexpr in expr.args[3].args
if isa(bodyexpr,Expr) && bodyexpr.head == :(::)
push!(fields,bodyexpr.args[1])
elseif isa(bodyexpr,Symbol)
push!(fields,bodyexpr)
end
end
return esc(quote
$expr
macro $(symbol("unpack"*string(name)))(t)
return esc(Expr(:block, [:($f = $t.$f) for f in $fields]...))
end
end)
end
In the REPL, this definition works just fine:
julia> #unpackable immutable Foo
i::Int
x::Float64
end
julia> macroexpand(:(#unpackFoo foo))
quote
i = foo.i
x = foo.x
end
Problems arise if I put the #unpackFoo in the same compilation unit as the #unpackable:
julia> #eval begin
#unpackable immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
#unpackFoo foo
end
ERROR: UndefVarError: #unpackFoo not defined
I assume the problem is that the compiler tries to proceed as follows
Expand #unpackable but do not parse it.
Try to expand #unpackFoo which fails because the expansion of #unpackable has not been parsed yet.
If we wouldn't fail already at step 2, the compiler would now parse the expansion of #unpackable.
This circumstance prevents #unpackable from being used in a source file. Is there any way of telling the compiler to swap steps 2. and 3. in the above list?
The background to this question is that I'm working on an iterator-based implementation of iterative solvers in the spirit of https://gist.github.com/jiahao/9240888. Algorithms like MinRes require quite a number of variables in the corresponding state object (8 currently), and I neither want to write state.variable every time I use a variable in e.g. the next() function, nor do I want to copy all of them manually as this bloats up the code and is hard to maintain. In the end, this is mainly an exercise in meta-programming though.
Firstly, I would suggest writing this as:
immutable Foo
...
end
unpackable(Foo)
where unpackable is a function which takes the type, constructs the appropriate expression and evals it. There are a couple of advantages to this, e.g. that you can apply it to any type without it being fixed at definition time, and the fact that you don't have to do a bunch of parsing of the type declaration (you can just call fieldnames(Foo) == [:f, :i] and work with that).
Secondly, while I don't know your use case in detail (and dislike blanket rules) I will warn that this kind of thing is frowned upon. It makes code harder to read because it introduces a non-local dependency; suddenly, in order to know whether x is a local or global variable, you have to look up the definition of a type in a whole different file. A better, and more general, approach is to explicitly unpack variables, and this is available in MacroTools.jl via the #destruct macro:
#destruct _.(x, i) = myfoo
# now we can use x and i
(You can destruct nested data structures and indexable objects too, which is nice.)
To answer your question: you're essentially right about how Julia runs code (s/parse/evaluate). The whole block is parsed, expanded and evaluated together, which means in your example you're trying to expand #unpackFoo before it's been defined.
However, when loading a .jl file, Julia evaluates blocks in the file one at a time, rather than all at once.
This means that you can happily write a file like this:
macro foo()
:(println("hi"))
end
#foo()
and run julia foo.jl or include("foo.jl") and it will run fine. You just can't have a macro definition and its use in the same block, as in your begin block above.
Try having a look at Parameters package by Mauro (https://github.com/mauro3/Parameters.jl). It has an #unpack macro and the accompanying machinery similar to what you suggest you need.
I often see something like the following in Julia:
convert(::Type{Point{Float64}}, ::Float64)
how does the (:: work? And what is the terminology for this?
Your answer can be found in the Julia documentation for defining conversions. To quote (with types switched to make it even more straightforward to read):
The type of the first argument of this method is a singleton
type, Type{Point{Float64}}, the only instance of which is
Point{Float64}. Thus, this method is only invoked when the first
argument is the type value Point{Float64}. Notice the syntax used
for the first argument: the argument name is omitted prior to the ::
symbol, and only the type is given. This is the syntax in Julia for a
function argument whose type is specified but whose value is never
used in the function body. In this example, since the type is a
singleton, there would never be any reason to use its value within the
body.
(Emphasis mine)
You will also encounter the foo(::SomeType) syntax in error messages, when trying to invoke a function with arguments of the wrong type (after all you can't show the argument names of a variant that does not exist). E.g:
julia> foo(x::Bool) = 3
foo (generic function with 1 method)
julia> foo(5)
ERROR: `foo` has no method matching foo(::Int64)
In Julia, it is possible to view the AST of a user defined function:
julia> myFunc(x) = 5*x+3
myFunc (generic function with 1 method)
julia> tmp = dump(quote myFunc end)
Expr
head: Symbol block
args: Array(Any,(2,))
1: Expr
head: Symbol line
args: Array(Any,(2,))
1: Int64 1
2: Symbol none
typ: Any
2: Symbol myFunc
typ: Any
Which is the AST I am interested in. However, the variable tmp doesn't contain the Expr representing the syntactic tree I am expecting:
julia> tmp
julia> typeof(tmp)
Nothing (constructor with 1 method)
Is there another way to get this Expr? (the one that is displayed when running dump(quote myFunc end) )
dump does not give you the result; it is just a way of printing the value. (As you saw, it prints as a side-effect and returns a nothing.)
What you gave dump was an AST containing the name of your function, not the function itself. dump is not printing out a representation of your function: it is saying it has a block of one line containing the symbol myFunc.
If you want the AST, you should run code_typed(myFunc,(Any,)) or code_lowered(myFunc,(Any,)). For other functions, you will need different and/or more specific type signatures as the second argument.
If you are only planning to call myFunc with Ints or Float64s or whatever, use that instead of Any -- it will make a difference to code_typed's output, since the type inference will change.
I wrote a blog post documenting the code_typed/code_lowered set of functions: http://blog.leahhanson.us/julia-introspects.html
(I also spend time in that post looking at their output, the Expr type and explaining it's structure.)
The Metaprogramming section of the official manual will probably be useful to you in working with ASTs, if you haven't already read it.
You can't access the AST of a function, because a function is a a collection of methods (that might be implemented differently) in Julia. If it suits your needs you should use the documented code_typed function, where you specify the types of the arguments to select the right method. There are also some hints in Access the AST for generic functions in Julia, but that is not documented functionality, so it might change without warning.