I have been trying to write a Julia macro that takes Cmd objects and runs them inside a loop. The catch is that I want to use the local loop variables for interpolation in the command. I would write a function and use eval(), but eval() uses the global scope and so can't see the local loop variables.
Below is a simple example that demonstrates how string interpolation works, but the command interpolation fails:
macro runcmd(cmdExpr)
quote
for i in 1:2
println("i: $i")
run($cmdExpr)
end
end
end
#runcmd(`echo $i`)
outputs
i: 1
ERROR: i not defined
in anonymous at none:5
If I expand the macro I get
quote # none, line 3:
for #261#i = 1:2 # line 4:
println("i: $#261#i") # line 5:
run(Base.cmd_gen((("echo",),(i,))))
end
end
I am guessing the #261# part that is missing from the cmd_gen argument's reference to i is related to the problem, but I don't know for sure.
You're right that the problem is because of the mismatch between the index #261#i and the reference to it in echo $i. An easy way to resolve the problem is to escape the i used in the index:
macro runcmd(cmdExpr)
quote
for $(esc(:i)) in 1:2
println("i: $i")
run($cmdExpr)
end
end
end
Then we get:
julia> #runcmd(`echo $i`)
i: 1
1
i: 2
2
There's more discussion in Julia's metaprogramming documentation.
Related
In fortran, we can simply use & to do line continuation.
But I wonder, in Julia, is there a safe way to do line continuation?
I heard that sometimes Julia cannot identify line continuation and it can cause bugs? Because after, Julia does not seem to have any symbol to do line continuation. Can Julia correctly recognize the line continuation?
Like I define the below function with long arguments,
function mean_covar_init(kmix::Int64,dim_p::Int64,weight::Array{Float64,1},sigma::Array{Float64,2},mu::Array{Float64,2})
return nothing
end
If I do things like
function mean_covar_init(kmix::Int64
,dim_p::Int64
,weight::Array{Float64,1}
,sigma::Array{Float64,2}
,mu::Array{Float64,2})
return nothing
end
Is it safe? Thank you very much!
If you do the thing like you have presented it is safe because Julia sees ( in the first line of code so it will look for a closing ).
However a problematic code would be:
f() = 1
+ 2
The reason is that the f() = 1 part is a valid and complete function definition. Therefore you need to make sure to signal Julia that the line is incomplete. The three most typical ways to do it are:
Move the + to the end of first line:
f() = 1 +
2
Use ( and ) as a wrapper:
f() = (1
+ 2)
Use begin and end:
f() = begin 1
+ 2 end
Let me give another example with macros, which do not require parenthesis or punctuation and therefore can be often tricky. Therefore the following:
#assert isodd(4) "What even are numbers?"
if rewritten as
#assert isodd(4)
"What even are numbers?"
does not produce what you expect, and you need to do e.g.:
#assert(isodd(4),
"What even are numbers?")
I create an object and then try to use it in a loop
a = 0
for i in 1:2
a += 1
end
However, this results in the following error:
UndefVarError: a not defined
Have I forgotten how loops work or what is going wrong?
The point is that your a is a global variable and in Julia versions between 1.0 and 1.4.2 the local scoping for variables within loops was introduced. More precisely if you do not explicitly say you want to mutate a global variable (global a += 1) a new variable within a local scope is introduced.
The best thing to do is to upgrade to Julia 1.5. Julia 1.5 has introduced back soft scoping in the REPL.
If you need to stick with the current Julia version you could use global keyword or place your code in a function or use Jupyter notebook instead (that has soft scoping) or surround your code around let...end block such as:
let
a = 0
for i in 1:2
a += 1
end
a
end
I would like to build a Julia application where a user can specify a function using a configuration file (and therefore as a string). The configuration file then needs to be parsed before the function is evaluated in the program.
The problem is that while the function name is known locally, it is not known in the module containing the parser. One solution I have come up with is to pass the local eval function to the parsing function but that does not seem very elegant.
I have tried to come up with a minimal working example here, where instead of parsing a configuration file, the function name is already contained in a string:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
# in a real application, parseconfig would parse the configuration file to extract funstr
function parseconfig(funstr)
return eval(Meta.parse(funstr))
end
function parseconfig(funstr, myeval)
return myeval(Meta.parse(funstr))
end
end
# test 1 -- succeeds
f1 = MyFuns.myfun
println("test1: $(f1(1))")
# test 2 -- succeeds
f2 = MyUtil.parseconfig("MyFuns.myfun", eval)
println("test2: $(f2(1))")
# test 3 -- fails
f3 = MyUtil.parseconfig("MyFuns.myfun")
println("test3: $(f3(1))")
The output is:
test1: 3
test2: 3
ERROR: LoadError: UndefVarError: MyFuns not defined
So, the second approach works but is there a better way to achieve the goal?
Meta.parse() will transform your string to an AST. What MyFuns.myfun refers to depends on the scope provided by the eval() you use.
The issue with your example is that the eval() inside MyUtil will evaluate in the context of that module. If that is the desired behavior, you simply miss using MyFuns inside MyUtil.
But what you really want to do is write a macro. This allows the code to be included when parsing your program, before running it. The macro will have access to a special argument __module__, which is the context where the macro is used. So __module__.eval() will execute an expression in that very scope.
foo = "outside"
module MyMod
foo = "inside"
macro eval(string)
expr = Meta.parse(string)
__module__.eval(expr)
end
end
MyMod.#eval "foo"
# Output is "outside"
See also this explanation on macros:
https://docs.julialang.org/en/v1/manual/metaprogramming/index.html#man-macros-1
And for the sake of transforming the answer of #MauricevanLeeuwen into the framework of my question, this code will work:
module MyFuns
function myfun(a)
return a+2
end
end
module MyUtil
macro parseconfig(funstr)
__module__.eval(Meta.parse(funstr))
end
end
f4 = MyUtil.#parseconfig "MyFuns.myfun"
println("test4: $(f4(1))")
I have a script that defines a function, and later intended to call the function but forgot to add the parentheses, like this:
function myfunc()
println("inside myfunc")
end
myfunc # This line is silently ignored. The function isn't invoked and there's no error message.
After a while I did figure out that I was missing the parentheses, but since Julia didn't give me an error, I'm wondering what that line is actually doing? I'm assuming that it must be doing something with the myfunc statement, but I don't know Julia well enough to understand what is happening.
I tried --depwarn=yes but don't see a julia command line switch to increase the warning level or verbosity. Please let me know if one exists.
For background context, the reason this came up is that I'm trying to translate a Bash script to Julia, and there are numerous locations where an argument-less function is defined and then invoked, and in Bash you don't need parentheses after the function name to invoke it.
The script is being run from command line (julia stub.jl) and I'm using Julia 1.0.3 on macOS.
It doesn't silently ignore the function. Calling myfunc in an interactive session will show you what happens: the call returns the function object to the console, and thus call's the show method for Function, showing how many methods are currently defined for that function in your workspace.
julia> function myfunc()
println("inside myfunc")
end
myfunc (generic function with 1 method)
julia> myfunc
myfunc (generic function with 1 method)
Since you're calling this in a script, show is never called, and thus you don't see any result. But it doesn't error, because the syntax is valid.
Thanks to DNF for the helpful comment on it being in a script.
It does nothing.
As in c, an expression has a value: in c the expression _ a=1+1; _ has the value _ 2 _ In c, this just fell out of the parser: they wanted to be able to evaluate expressions like _ a==b _
In Julia, it's the result of designing a language where the code you write is handled as a data object of the language. In Julia, the expression "a=1+1" has the value "a=1+1".
In c, the fact that
a=1+1;
is an acceptable line of code means that, accidentally,
a;
is also an acceptable line of code. The same is true in Julia: the compiler expects to see a data value there: any data value you put may be acceptable: even for example the data value that represents the calculated value returned by a function:
myfunc()
or the value that represents the function object itself:
myfunc
As in c, the fact that data values are everywhere in your code just indicates that the syntax allows data values everywhere in your code and that the compiler does nothing with data values that are everywhere in your code.
I would like to write a macro #unpack t which takes an object t and copies all its fields into local scope. For example, given
immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
the expression #unpack foo should expand into
i = foo.i
x = foo.x
Unfortunately, such a macro cannot exist since it would have to know the type of the passed object. To circumvent this limitation, I introduce a type-specific macro #unpackFoo foo with the same effect, but since I'm lazy I want the compiler to write #unpackFoo for me. So I change the type definition to
#unpackable immutable Foo
i::Int
x::Float64
end
which should expand into
immutable Foo
i::Int
x::Float64
end
macro unpackFoo(t)
return esc(quote
i = $t.i
x = $t.x
end)
end
Writing #unpackable is not too hard:
macro unpackable(expr)
if expr.head != :type
error("#unpackable must be applied on a type definition")
end
name = isa(expr.args[2], Expr) ? expr.args[2].args[1] : expr.args[2]
fields = Symbol[]
for bodyexpr in expr.args[3].args
if isa(bodyexpr,Expr) && bodyexpr.head == :(::)
push!(fields,bodyexpr.args[1])
elseif isa(bodyexpr,Symbol)
push!(fields,bodyexpr)
end
end
return esc(quote
$expr
macro $(symbol("unpack"*string(name)))(t)
return esc(Expr(:block, [:($f = $t.$f) for f in $fields]...))
end
end)
end
In the REPL, this definition works just fine:
julia> #unpackable immutable Foo
i::Int
x::Float64
end
julia> macroexpand(:(#unpackFoo foo))
quote
i = foo.i
x = foo.x
end
Problems arise if I put the #unpackFoo in the same compilation unit as the #unpackable:
julia> #eval begin
#unpackable immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
#unpackFoo foo
end
ERROR: UndefVarError: #unpackFoo not defined
I assume the problem is that the compiler tries to proceed as follows
Expand #unpackable but do not parse it.
Try to expand #unpackFoo which fails because the expansion of #unpackable has not been parsed yet.
If we wouldn't fail already at step 2, the compiler would now parse the expansion of #unpackable.
This circumstance prevents #unpackable from being used in a source file. Is there any way of telling the compiler to swap steps 2. and 3. in the above list?
The background to this question is that I'm working on an iterator-based implementation of iterative solvers in the spirit of https://gist.github.com/jiahao/9240888. Algorithms like MinRes require quite a number of variables in the corresponding state object (8 currently), and I neither want to write state.variable every time I use a variable in e.g. the next() function, nor do I want to copy all of them manually as this bloats up the code and is hard to maintain. In the end, this is mainly an exercise in meta-programming though.
Firstly, I would suggest writing this as:
immutable Foo
...
end
unpackable(Foo)
where unpackable is a function which takes the type, constructs the appropriate expression and evals it. There are a couple of advantages to this, e.g. that you can apply it to any type without it being fixed at definition time, and the fact that you don't have to do a bunch of parsing of the type declaration (you can just call fieldnames(Foo) == [:f, :i] and work with that).
Secondly, while I don't know your use case in detail (and dislike blanket rules) I will warn that this kind of thing is frowned upon. It makes code harder to read because it introduces a non-local dependency; suddenly, in order to know whether x is a local or global variable, you have to look up the definition of a type in a whole different file. A better, and more general, approach is to explicitly unpack variables, and this is available in MacroTools.jl via the #destruct macro:
#destruct _.(x, i) = myfoo
# now we can use x and i
(You can destruct nested data structures and indexable objects too, which is nice.)
To answer your question: you're essentially right about how Julia runs code (s/parse/evaluate). The whole block is parsed, expanded and evaluated together, which means in your example you're trying to expand #unpackFoo before it's been defined.
However, when loading a .jl file, Julia evaluates blocks in the file one at a time, rather than all at once.
This means that you can happily write a file like this:
macro foo()
:(println("hi"))
end
#foo()
and run julia foo.jl or include("foo.jl") and it will run fine. You just can't have a macro definition and its use in the same block, as in your begin block above.
Try having a look at Parameters package by Mauro (https://github.com/mauro3/Parameters.jl). It has an #unpack macro and the accompanying machinery similar to what you suggest you need.