I wrote a fairly complex macro that manipulates an array, but it didn't work. I get the same error message even after replacing my macro with something ridiculously simple:
macro len(arr::Array)
length(arr)
end
#len [1 2 3]
The error message I get is:
ERROR: LoadError: MethodError: no method matching #len(::Expr)
Why is Julia insisting on interpreting my input as an expression? I'm completely new to Julia, and there is obviously something I don't understand about either macros or the type system. Can someone please explain?
EDIT: Time to explain what my original macro was for. I want to take a table as input and use it to define variables in the local scope. The entries in the first column define individual variable names and the other columns define variable contents. Since a function can't do this, I need a macro. I have a working macro that takes a long (triple-quoted) string as input, but now I want it to take an array as input instead.
After this explanation, maybe I should add another subquestion: why does my other macro accept the appropriate input (it is defined as macro foo(text::String)) whereas the array version doesn't?
User Chris Rackauckas answered this on irc in discussion with me:
#len $a
macros are on expressions, not on the values, so
this sounds like something you'd want a function for
(if it's really needed at all).
I'm not sure this would work there anyway.
This isn't what macros are for..
it's for converting expressions into other expressions, mostly.
If you're trying to use macros on values, then you want a function
(you may be able to #eval or something to make this work here, but you'll be going through great lengths to make a macro act like a function)
..as a function:
julia> function len(x)
length(x)
end
len (generic function with 1 method)
julia> len(a)
3
julia> a
1x3 Array{Int64,2}:
1 2 3
To answer my own question: this is how that "ridiculously simple" macro should have been written.
macro len(expr)
:(length($expr))
end
This operates on an expression, not an array (thank you Chris Rackauckas and Isaiah for pointing that out). I was misled by my own experiments, since a similar macro for strings happens to work:
# MISLEADING CODE, DON'T USE THIS!
macro lenstr(str::String)
length(str)
end
Apparently Julia allows string macros, but this is an exception to the general rule that macros operate on expressions. To illustrate how arguments appear to the macro:
macro dump(arg)
dump(arg)
end
julia> #dump "foo bar"
String "foo bar"
julia> #dump [1,2,3]
Expr
head: Symbol vect
args: Array{Any}((3,))
1: Int64 1
2: Int64 2
3: Int64 3
typ: Any
julia> #dump [1 2 3]
Expr
head: Symbol hcat
args: Array{Any}((3,))
1: Int64 1
2: Int64 2
3: Int64 3
typ: Any
Note how [1,2,3] and [1 2 3] are seen by the macro as expressions, not arrays. This was the source of my error. By applying lessons learned here, my more complex macro that creates variables in a local scope from an array now works. Thank you all!
Related
I have about 50 functions which should consume only even positive numbers. Right now I am checking each time with an "if" whether the number put in is zero or not:
function grof(x::Int)
if (x % 2) == 0
println("good")
else
throw("x is not an even number!!!!!!!!!!!!! Stupid programmer!")
end
end
Ideally, I would like to have a datatype which produces this automatically, i.e.
function grof(x::EvenInt)
println("good")
end
However, I am not able to produce this datatype by my own since I am unable to understand the documentary. Thanks for your help!
Best, v.
I don't think creating a type is warranted in such a situation: I would simply #assert that the condition is verified at the beginning of the function(s). (Funnily enough, checking the whether a number is even is the example that was chosen in the documentation to illustrate the effect of #assert)
For example:
julia> function grof(x::Int)
#assert iseven(x) "Stupid programmer!"
println("good")
end
grof (generic function with 1 method)
julia> grof(2)
good
julia> grof(3)
ERROR: AssertionError: Stupid programmer!
Stacktrace:
[1] grof(::Int64) at ./REPL[5]:2
[2] top-level scope at REPL[7]:1
EDIT: If you really want to create a type enforcing such a constraint, it is possible. The way to do this would be to
create a type (possibly subtyping one of the Number abstract types; maybe Signed)
define an inner constructor ensuring that such a type cannot hold an odd value
A very simple example to build upon would be along the lines of:
# A wrapper around an even integer value
struct EvenInt
val :: Int
# inner constructor
function EvenInt(val)
#assert iseven(val)
new(val)
end
end
# Accessor to the value of an EvenInt
val(x::EvenInt) = x.val
# A method working only on even numbers
grof(x::EvenInt) = println("good: $(val(x)) is even")
You'd use this like so:
julia> x = EvenInt(42)
EvenInt(42)
julia> grof(x)
good: 42 is even
julia> y = EvenInt(1)
ERROR: AssertionError: iseven(val)
Stacktrace:
[1] EvenInt(::Int64) at ./REPL[1]:5
[2] top-level scope at REPL[6]:1
but note that you can't do anything on EvenInts yet: you need to either unwrap them (using val() in this case), or define operations on them (a task which can be vastly simplified if you make EvenInt a subtype of one of the abstract number types and follow the relevant interface).
All integers multiplied by two are even, so redefine your function to take half the number it currently takes.
function grof2(halfx::Int)
x=2*halfx
println("good")
end
I encountered a problem which I've solved, but why the solution works doesnt make sense to me
I had a function similar to this one
function testB(a::Array{AbstractString})
println(a)
end
running it like so gave me
testB(convert(Array{UTF8String},["a","b"]))
ERROR: MethodError: `testB` has no method matching
testB(::Array{UTF8String,1})
Note that Im not manually converting to UTF8 in reality, its for demonstration, in reality I have an AbstractString array, but when I fetch elements from it, they become UFT8
My solution reads in short
function testA{T <: AbstractString}(a::Array{T})
println(a)
end
running this method gives
testA(convert(Array{UTF8String},["a","b"]))
UTF8String["a","b"]
Can anyone tell me why testA works but testB doesnt?
Also, is there a name for this {T <: SomeDataType} notation?
While UTF8String is a subtype of AbstractString, Array{UTF8String} is not a subtype of Array{AbstractString} (no covariance). Hence your testB does not work. (But testB(convert(Array{AbstractString},["a","b"])) should work.)
Rationale for why it has to be like this: a function f(x::Vector{AbstractString}) could e.g. push! a new FooString into x (assuming FooString is a subtype of AbstractString). Now if x was in fact a Vector{UTF8String}, that would fail.
I would like to write a macro #unpack t which takes an object t and copies all its fields into local scope. For example, given
immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
the expression #unpack foo should expand into
i = foo.i
x = foo.x
Unfortunately, such a macro cannot exist since it would have to know the type of the passed object. To circumvent this limitation, I introduce a type-specific macro #unpackFoo foo with the same effect, but since I'm lazy I want the compiler to write #unpackFoo for me. So I change the type definition to
#unpackable immutable Foo
i::Int
x::Float64
end
which should expand into
immutable Foo
i::Int
x::Float64
end
macro unpackFoo(t)
return esc(quote
i = $t.i
x = $t.x
end)
end
Writing #unpackable is not too hard:
macro unpackable(expr)
if expr.head != :type
error("#unpackable must be applied on a type definition")
end
name = isa(expr.args[2], Expr) ? expr.args[2].args[1] : expr.args[2]
fields = Symbol[]
for bodyexpr in expr.args[3].args
if isa(bodyexpr,Expr) && bodyexpr.head == :(::)
push!(fields,bodyexpr.args[1])
elseif isa(bodyexpr,Symbol)
push!(fields,bodyexpr)
end
end
return esc(quote
$expr
macro $(symbol("unpack"*string(name)))(t)
return esc(Expr(:block, [:($f = $t.$f) for f in $fields]...))
end
end)
end
In the REPL, this definition works just fine:
julia> #unpackable immutable Foo
i::Int
x::Float64
end
julia> macroexpand(:(#unpackFoo foo))
quote
i = foo.i
x = foo.x
end
Problems arise if I put the #unpackFoo in the same compilation unit as the #unpackable:
julia> #eval begin
#unpackable immutable Foo
i::Int
x::Float64
end
foo = Foo(42,pi)
#unpackFoo foo
end
ERROR: UndefVarError: #unpackFoo not defined
I assume the problem is that the compiler tries to proceed as follows
Expand #unpackable but do not parse it.
Try to expand #unpackFoo which fails because the expansion of #unpackable has not been parsed yet.
If we wouldn't fail already at step 2, the compiler would now parse the expansion of #unpackable.
This circumstance prevents #unpackable from being used in a source file. Is there any way of telling the compiler to swap steps 2. and 3. in the above list?
The background to this question is that I'm working on an iterator-based implementation of iterative solvers in the spirit of https://gist.github.com/jiahao/9240888. Algorithms like MinRes require quite a number of variables in the corresponding state object (8 currently), and I neither want to write state.variable every time I use a variable in e.g. the next() function, nor do I want to copy all of them manually as this bloats up the code and is hard to maintain. In the end, this is mainly an exercise in meta-programming though.
Firstly, I would suggest writing this as:
immutable Foo
...
end
unpackable(Foo)
where unpackable is a function which takes the type, constructs the appropriate expression and evals it. There are a couple of advantages to this, e.g. that you can apply it to any type without it being fixed at definition time, and the fact that you don't have to do a bunch of parsing of the type declaration (you can just call fieldnames(Foo) == [:f, :i] and work with that).
Secondly, while I don't know your use case in detail (and dislike blanket rules) I will warn that this kind of thing is frowned upon. It makes code harder to read because it introduces a non-local dependency; suddenly, in order to know whether x is a local or global variable, you have to look up the definition of a type in a whole different file. A better, and more general, approach is to explicitly unpack variables, and this is available in MacroTools.jl via the #destruct macro:
#destruct _.(x, i) = myfoo
# now we can use x and i
(You can destruct nested data structures and indexable objects too, which is nice.)
To answer your question: you're essentially right about how Julia runs code (s/parse/evaluate). The whole block is parsed, expanded and evaluated together, which means in your example you're trying to expand #unpackFoo before it's been defined.
However, when loading a .jl file, Julia evaluates blocks in the file one at a time, rather than all at once.
This means that you can happily write a file like this:
macro foo()
:(println("hi"))
end
#foo()
and run julia foo.jl or include("foo.jl") and it will run fine. You just can't have a macro definition and its use in the same block, as in your begin block above.
Try having a look at Parameters package by Mauro (https://github.com/mauro3/Parameters.jl). It has an #unpack macro and the accompanying machinery similar to what you suggest you need.
In Julia, is there any way to get the name of a passed to a function?
x = 10
function myfunc(a)
# do something here
end
assert(myfunc(x) == "x")
Do I need to use macros or is there a native method that provides introspection?
You can grab the variable name with a macro:
julia> macro mymacro(arg)
string(arg)
end
julia> #mymacro(x)
"x"
julia> #assert(#mymacro(x) == "x")
but as others have said, I'm not sure why you'd need that.
Macros operate on the AST (code tree) during compile time, and the x is passed into the macro as the Symbol :x. You can turn a Symbol into a string and vice versa. Macros replace code with code, so the #mymacro(x) is simply pulled out and replaced with string(:x).
Ok, contradicting myself: technically this is possible in a very hacky way, under one (fairly limiting) condition: the function name must have only one method signature. The idea is very similar the answers to such questions for Python. Before the demo, I must emphasize that these are internal compiler details and are subject to change. Briefly:
julia> function foo(x)
bt = backtrace()
fobj = eval(current_module(), symbol(Profile.lookup(bt[3]).func))
Base.arg_decl_parts(fobj.env.defs)[2][1][1]
end
foo (generic function with 1 method)
julia> foo(1)
"x"
Let me re-emphasize that this is a bad idea, and should not be used for anything! (well, except for backtrace display). This is basically "stupid compiler tricks", but I'm showing it because it can be kind of educational to play with these objects, and the explanation does lead to a more useful answer to the clarifying comment by #ejang.
Explanation:
bt = backtrace() generates a ... backtrace ... from the current position. bt is an array of pointers, where each pointer is the address of a frame in the current call stack.
Profile.lookup(bt[3]) returns a LineInfo object with the function name (and several other details about each frame). Note that bt[1] and bt[2] are in the backtrace-generation function itself, so we need to go further up the stack to get the caller.
Profile.lookup(...).func returns the function name (the symbol :foo)
eval(current_module(), Profile.lookup(...)) returns the function object associated with the name :foo in the current_module(). If we modify the definition of function foo to return fobj, then note the equivalence to the foo object in the REPL:
julia> function foo(x)
bt = backtrace()
fobj = eval(current_module(), symbol(Profile.lookup(bt[3]).func))
end
foo (generic function with 1 method)
julia> foo(1) == foo
true
fobj.env.defs returns the first Method entry from the MethodTable for foo/fobj
Base.decl_arg_parts is a helper function (defined in methodshow.jl) that extracts argument information from a given Method.
the rest of the indexing drills down to the name of the argument.
Regarding the restriction that the function have only one method signature, the reason is that multiple signatures will all be listed (see defs.next) in the MethodTable. As far as I know there is no currently exposed interface to get the specific method associated with a given frame address. (as an exercise for the advanced reader: one way to do this would be to modify the address lookup functionality in jl_getFunctionInfo to also return the mangled function name, which could then be re-associated with the specific method invocation; however, I don't think we currently store a reverse mapping from mangled name -> Method).
Note also that (1) backtraces are slow (2) there is no notion of "function-local" eval in Julia, so even if one has the variable name, I believe it would be impossible to actually access the variable (and the compiler may completely elide local variables, unused or otherwise, put them in a register, etc.)
As for the IDE-style introspection use mentioned in the comments: foo.env.defs as shown above is one place to start for "object introspection". From the debugging side, Gallium.jl can inspect DWARF local variable info in a given frame. Finally, JuliaParser.jl is a pure-Julia implementation of the Julia parser that is actively used in several IDEs to introspect code blocks at a high level.
Another method is to use the function's vinfo. Here is an example:
function test(argx::Int64)
vinfo = code_lowered(test,(Int64,))
string(vinfo[1].args[1][1])
end
test (generic function with 1 method)
julia> test(10)
"argx"
The above depends on knowing the signature of the function, but this is a non-issue if it is coded within the function itself (otherwise some macro magic could be needed).
In Julia, it is possible to view the AST of a user defined function:
julia> myFunc(x) = 5*x+3
myFunc (generic function with 1 method)
julia> tmp = dump(quote myFunc end)
Expr
head: Symbol block
args: Array(Any,(2,))
1: Expr
head: Symbol line
args: Array(Any,(2,))
1: Int64 1
2: Symbol none
typ: Any
2: Symbol myFunc
typ: Any
Which is the AST I am interested in. However, the variable tmp doesn't contain the Expr representing the syntactic tree I am expecting:
julia> tmp
julia> typeof(tmp)
Nothing (constructor with 1 method)
Is there another way to get this Expr? (the one that is displayed when running dump(quote myFunc end) )
dump does not give you the result; it is just a way of printing the value. (As you saw, it prints as a side-effect and returns a nothing.)
What you gave dump was an AST containing the name of your function, not the function itself. dump is not printing out a representation of your function: it is saying it has a block of one line containing the symbol myFunc.
If you want the AST, you should run code_typed(myFunc,(Any,)) or code_lowered(myFunc,(Any,)). For other functions, you will need different and/or more specific type signatures as the second argument.
If you are only planning to call myFunc with Ints or Float64s or whatever, use that instead of Any -- it will make a difference to code_typed's output, since the type inference will change.
I wrote a blog post documenting the code_typed/code_lowered set of functions: http://blog.leahhanson.us/julia-introspects.html
(I also spend time in that post looking at their output, the Expr type and explaining it's structure.)
The Metaprogramming section of the official manual will probably be useful to you in working with ASTs, if you haven't already read it.
You can't access the AST of a function, because a function is a a collection of methods (that might be implemented differently) in Julia. If it suits your needs you should use the documented code_typed function, where you specify the types of the arguments to select the right method. There are also some hints in Access the AST for generic functions in Julia, but that is not documented functionality, so it might change without warning.