I would like to have an N long tuple as an optional argument of a function but I do not know how to provide a N long default value:
function create_grid(d::Int64, n::Tuple{Vararg{Int64, N}}; xmin::Tuple{Vararg{Float64, N}}) where N
I understand xmin should be declared with a default value such as xmin::Tuple{Vararg{Float64, N}}::0., but this is evidently wrong as it is defaulting to a Float instead of Tuple. How can I state I want a N long tuple as optional argument defaulting to (eg.) 0. for all the elements if the argument is not provided explicitly?
Here it is - you just provide the default as a one element tuple:
function somefun(d::Int64, n::Tuple{Vararg{Int64, N}}; xmin::Tuple{Vararg{Float64, N}}=(0.0,)) where N
println("d=$d n=$n xmin=$xmin")
end
To understand how it works just note that:
Tuple{Vararg{Int, 2}} == typeof((2,2))
#and
Tuple{Vararg{Int, 1}} == typeof((2,))
so you needed 1-element tuple as a default.
Let's test it:
julia> somefun(4,(4,))
d=4 n=(4,) xmin=(0.0,)
This works as expected.
Finally, note that providing a 2-element tuple as the second argument without the third one will throw an error because the sizes do not match:
julia> somefun(4,(4,5))
ERROR: MethodError: no method matching #somefun#1(::Tuple{Float64}, ::typeof(somefun), ::Int64, ::Tuple{Int64,Int64})
If you want to workaround this you need another constructor:
function somefun(d::Int64, n::Tuple{Vararg{Int64, N}}; xmin::Tuple{Vararg{Float64, N}}= tuple(zeros(length(n))...)) where N
println("d=$d n=$n xmin=$xmin")
end
Testing:
julia> somefun(4,(4,5))
d=4 n=(4, 5) xmin=(0.0, 0.0)
Related
I have a function like
function f(a = 1; first = 5, second = "asdf")
return a
end
Is there any way to programatically return a vector with the names of the keyword arguments. Something like:
kwargs(f)
# returns [:first, :second]
I realise that this might be complicated by having multiple methods for a functionname. But I was hoping this would still be possible if the exact method is specified. For instance:
kwargs(methods(f).ms[1])
# returns [:first, :second]
Just use Base.kwarg_decl()
julia> Base.kwarg_decl.(methods(f))
2-element Vector{Vector{Symbol}}:
[]
[:first, :second]
If you need the first parameter a as well you could also try:
julia> Base.method_argnames.(methods(f))
2-element Vector{Vector{Symbol}}:
[Symbol("#self#")]
[Symbol("#self#"), :a]
Consider an existing function in Base, which takes in a variable number of arguments of some abstract type T. I have defined a subtype S<:T and would like to write a method which dispatches if any of the arguments is my subtype S.
As an example, consider function Base.cat, with T being an AbstractArray and S being some MyCustomArray <: AbstractArray.
Desired behaviour:
julia> v = [1, 2, 3];
julia> cat(v, v, v, dims=2)
3×3 Array{Int64,2}:
1 1 1
2 2 2
3 3 3
julia> w = MyCustomArray([1,2,3])
julia> cat(v, v, w, dims=2)
"do something fancy"
Attempt:
function Base.cat(w::MyCustomArray, a::AbstractArray...; dims)
pritnln("do something fancy")
end
But this only works if the first argument is MyCustomArray.
What is an elegant way of achieving this?
I would say that it is not possible to do it cleanly without type piracy (but if it is possible I would also like to learn how).
For example consider cat that you asked about. It has one very general signature in Base (actually not requiring A to be AbstractArray as you write):
julia> methods(cat)
# 1 method for generic function "cat":
[1] cat(A...; dims) in Base at abstractarray.jl:1654
You could write a specific method:
Base.cat(A::AbstractArray...; dims) = ...
and check if any of elements of A is your special array, but this would be type piracy.
Now the problem is that you cannot even write Union{S, T} as since S <: T it will be resolved as just T.
This would mean that you would have to use S explicitly in the signature, but then even:
f(::S, ::T) = ...
f(::T, ::S) = ...
is problematic and a compiler will ask you to define f(::S, ::S) as the above definitions lead to dispatch ambiguity. So, even if you wanted to limit the number of varargs to some maximum number you would have to annotate types for all divisions of A into subsets to avoid dispatch ambiguity (which is doable using macros, but grows the number of required methods exponentially).
For general usage, I concur with Bogumił, but let me make an additional comment. If you have control over how cat is called, you can at least write some kind of trait-dispatch code:
struct MyCustomArray{T, N} <: AbstractArray{T, N}
x::Array{T, N}
end
HasCustom() = Val(false)
HasCustom(::MyCustomArray, rest...) = Val(true)
HasCustom(::AbstractArray, rest...) = HasCustom(rest...)
# `IsCustom` or something would be more elegant, but `Val` is quicker for now
Base.cat(::Val{true}, args...; dims) = println("something fancy")
Base.cat(::Val{false}, args...; dims) = cat(args...; dims=dims)
And the compiler is cool enough to optimize that away:
julia> args = (v, v, w);
julia> #code_warntype cat(HasCustom(args...), args...; dims=2);
Variables
#self#::Core.Compiler.Const(cat, false)
#unused#::Core.Compiler.Const(Val{true}(), false)
args::Tuple{Array{Int64,1},Array{Int64,1},MyCustomArray{Int64,1}}
Body::Nothing
1 ─ %1 = Main.println("something fancy")::Core.Compiler.Const(nothing, false)
└── return %1
If you don't have control over calls to cat, the only resort I can think of to make the above technique work is to overdub methods containing such call, to replace matching calls by the custom implementation. In which case you don't even need to overload cat, but can directly replace it by some mycat doing your fancy stuff.
I'm studying Standard ML and one of the exercices I have to do is to write a function called opPairs that receives a list of tuples of type int, and returns a list with the sum of each pair.
Example:
input: opPairs [(1, 2), (3, 4)]
output: val it = [3, 7]
These were my attempts, which are not compiling:
ATTEMPT 1
type T0 = int * int;
fun opPairs ((h:TO)::t) = let val aux =(#1 h + #2 h) in
aux::(opPairs(t))
end;
The error message is:
Error: unbound type constructor: TO
Error: operator and operand don't agree [type mismatch]
operator domain: {1:'Y; 'Z}
operand: [E]
in expression:
(fn {1=1,...} => 1) h
ATTEMPT 2
fun opPairs2 l = map (fn x => #1 x + #2 x ) l;
The error message is: Error: unresolved flex record (need to know the names of ALL the fields
in this context)
type: {1:[+ ty], 2:[+ ty]; 'Z}
The first attempt has a typo: type T0 is defined, where 0 is zero, but then type TO is referenced in the pattern, where O is the letter O. This gets rid of the "operand and operator do not agree" error, but there is a further problem. The pattern ((h:T0)::t) does not match an empty list, so there is a "match nonexhaustive" warning with the corrected type identifier. This manifests as an exception when the function is used, because the code needs to match an empty list when it reaches the end of the input.
The second attempt needs to use a type for the tuples. This is because the tuple accessor #n needs to know the type of the tuple it accesses. To fix this problem, provide the type of the tuple argument to the anonymous function:
fun opPairs2 l = map (fn x:T0 => #1 x + #2 x) l;
But, really it is bad practice to use #1, #2, etc. to access tuple fields; use pattern matching instead. Here is a cleaner approach, more like the first attempt, but taking full advantage of pattern matching:
fun opPairs nil = nil
| opPairs ((a, b)::cs) = (a + b)::(opPairs cs);
Here, opPairs returns an empty list when the input is an empty list, otherwise pattern matching provides the field values a and b to be added and consed recursively onto the output. When the last tuple is reached, cs is the empty list, and opPairs cs is then also the empty list: the individual tuple sums are then consed onto this empty list to create the output list.
To extend on exnihilo's answer, once you have achieved familiarity with the type of solution that uses explicit recursion and pattern matching (opPairs ((a, b)::cs) = ...), you can begin to generalise the solution using list combinators:
val opPairs = map op+
I'm writing a genetic program in order to test the fitness of randomly generated expressions. Shown here is the function to generate the expression as well a the main function. DIV and GT are defined elsewhere in the code:
function create_single_full_tree(depth, fs, ts)
"""
Creates a single AST with full depth
Inputs
depth Current depth of tree. Initially called from main() with max depth
fs Function Set - Array of allowed functions
ts Terminal Set - Array of allowed terminal values
Output
Full AST of typeof()==Expr
"""
# If we are at the bottom
if depth == 1
# End of tree, return function with two terminal nodes
return Expr(:call, fs[rand(1:length(fs))], ts[rand(1:length(ts))], ts[rand(1:length(ts))])
else
# Not end of expression, recurively go back through and create functions for each new node
return Expr(:call, fs[rand(1:length(fs))], create_single_full_tree(depth-1, fs, ts), create_single_full_tree(depth-1, fs, ts))
end
end
function main()
"""
Main function
"""
# Define functional and terminal sets
fs = [:+, :-, :DIV, :GT]
ts = [:x, :v, -1]
# Create the tree
ast = create_single_full_tree(4, fs, ts)
#println(typeof(ast))
#println(ast)
#println(dump(ast))
x = 1
v = 1
eval(ast) # Error out unless x and v are globals
end
main()
I am generating a random expression based on certain allowed functions and variables. As seen in the code, the expression can only have symbols x and v, as well as the value -1. I will need to test the expression with a variety of x and v values; here I am just using x=1 and v=1 to test the code.
The expression is being returned correctly, however, eval() can only be used with global variables, so it will error out when run unless I declare x and v to be global (ERROR: LoadError: UndefVarError: x not defined). I would like to avoid globals if possible. Is there a better way to generate and evaluate these generated expressions with locally defined variables?
Here is an example for generating an (anonymous) function. The result of eval can be called as a function and your variable can be passed as parameters:
myfun = eval(Expr(:->,:x, Expr(:block, Expr(:call,:*,3,:x) )))
myfun(14)
# returns 42
The dump function is very useful to inspect the expression that the parsers has created. For two input arguments you would use a tuple for example as args[1]:
julia> dump(parse("(x,y) -> 3x + y"))
Expr
head: Symbol ->
args: Array{Any}((2,))
1: Expr
head: Symbol tuple
args: Array{Any}((2,))
1: Symbol x
2: Symbol y
typ: Any
2: Expr
[...]
Does this help?
In the Metaprogramming part of the Julia documentation, there is a sentence under the eval() and effects section which says
Every module has its own eval() function that evaluates expressions in its global scope.
Similarly, the REPL help ?eval will give you, on Julia 0.6.2, the following help:
Evaluate an expression in the given module and return the result. Every Module (except those defined with baremodule) has its own 1-argument definition of eval, which evaluates expressions in that module.
I assume, you are working in the Main module in your example. That's why you need to have the globals defined there. For your problem, you can use macros and interpolate the values of x and y directly inside the macro.
A minimal working example would be:
macro eval_line(a, b, x)
isa(a, Real) || (warn("$a is not a real number."); return :(throw(DomainError())))
isa(b, Real) || (warn("$b is not a real number."); return :(throw(DomainError())))
return :($a * $x + $b) # interpolate the variables
end
Here, #eval_line macro does the following:
Main> #macroexpand #eval_line(5, 6, 2)
:(5 * 2 + 6)
As you can see, the values of macro's arguments are interpolated inside the macro and the expression is given to the user accordingly. When the user does not behave,
Main> #macroexpand #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
:((Main.throw)((Main.DomainError)()))
a user-friendly warning message is provided to the user at parse-time, and a DomainError is thrown at run-time.
Of course, you can do these things within your functions, again by interpolating the variables --- you do not need to use macros. However, what you would like to achieve in the end is to combine eval with the output of a function that returns Expr. This is what the macro functionality is for. Finally, you would simply call your macros with an # sign preceding the macro name:
Main> #eval_line(5, 6, 2)
16
Main> #eval_line([1,2,3], 7, 8)
WARNING: [1, 2, 3] is not a real number.
ERROR: DomainError:
Stacktrace:
[1] eval(::Module, ::Any) at ./boot.jl:235
EDIT 1. You can take this one step further, and create functions accordingly:
macro define_lines(linedefs)
for (name, a, b) in eval(linedefs)
ex = quote
function $(Symbol(name))(x) # interpolate name
return $a * x + $b # interpolate a and b here
end
end
eval(ex) # evaluate the function definition expression in the module
end
end
Then, you can call this macro to create different line definitions in the form of functions to be called later on:
#define_lines([
("identity_line", 1, 0);
("null_line", 0, 0);
("unit_shift", 0, 1)
])
identity_line(5) # returns 5
null_line(5) # returns 0
unit_shift(5) # returns 1
EDIT 2. You can, I guess, achieve what you would like to achieve by using a macro similar to that below:
macro random_oper(depth, fs, ts)
operations = eval(fs)
oper = operations[rand(1:length(operations))]
terminals = eval(ts)
ts = terminals[rand(1:length(terminals), 2)]
ex = :($oper($ts...))
for d in 2:depth
oper = operations[rand(1:length(operations))]
t = terminals[rand(1:length(terminals))]
ex = :($oper($ex, $t))
end
return ex
end
which will give the following, for instance:
Main> #macroexpand #random_oper(1, [+, -, /], [1,2,3])
:((-)([3, 3]...))
Main> #macroexpand #random_oper(2, [+, -, /], [1,2,3])
:((+)((-)([2, 3]...), 3))
Thanks Arda for the thorough response! This helped, but part of me thinks there may be a better way to do this as it seems too roundabout. Since I am writing a genetic program, I will need to create 500 of these ASTs, all with random functions and terminals from a set of allowed functions and terminals (fs and ts in the code). I will also need to test each function with 20 different values of x and v.
In order to accomplish this with the information you have given, I have come up with the following macro:
macro create_function(defs)
for name in eval(defs)
ex = quote
function $(Symbol(name))(x,v)
fs = [:+, :-, :DIV, :GT]
ts = [x,v,-1]
return create_single_full_tree(4, fs, ts)
end
end
eval(ex)
end
end
I can then supply a list of 500 random function names in my main() function, such as ["func1, func2, func3,.....". Which I can eval with any x and v values in my main function. This has solved my issue, however, this seems to be a very roundabout way of doing this, and may make it difficult to evolve each AST with each iteration.
I am trying to do something like this:
function outer(x::Array{Float64}, y::Array{Float64}=nothing)
if (y == nothing)
function inner(y::Array{Float64})
return x .* y
end
return inner
else
return x .+ y
end
end
outer([2.], [3.]) # to return 5, works
outer([2.])([3.]) # to return 6, fails.
outer([2.], [3.]) works just fine.
The problem is that outer([2.])([3.]) raises a MethodError stating:
MethodError: no method matching outer(::Array{Float64,1}, ::Void)
Closest candidates are:
outer(::Array{Float64,N} where N) at In[1]:2
outer(::Array{Float64,N} where N, ::Array{Float64,N} where N) at In[1]:2
Stacktrace:
[1] outer(::Array{Float64,1}) at ./In[1]:2
[2] include_string(::String, ::String) at ./loading.jl:522
The weird bit is that under Closest candidates, the single-argument outer(::Array{Float64,N} where N) is the first candidate. So why does it not work with the single argument?
Note: outer([2.], )([3.]), outer([2.], nothing)([3.]), outer([2.], [nothing])([3.]) all produce the same (similar) error.
This can be reproduced using a single argument function too:
function outer(y::Array{Float64}=nothing)
if (y == nothing)
function inner(y::Array{Float64})
return y .* y
end
return inner
else
return y .+ y
end
end
outer([2.])
1-element Array{Float64,1}:
4.0
outer()([3.])
MethodError: no method matching outer(::Void)
Closest candidates are:
outer() at In[6]:2
outer(::Array{Float64,N} where N) at In[6]:2
outer(::Array{Float64,N} where N, ::Array{Float64,N} where N) at In[1]:2
Stacktrace:
[1] outer() at ./In[6]:2
[2] include_string(::String, ::String) at ./loading.jl:522
And again, there is a zero-argument function outer() listed first in the Closest candidates list!
Basically, the above example is a MWE representing a log-sum/likelihood evaluation, where x is the data and y is the parameter of a model. I am trying to return a function in the parameter of the model to optimise using MLE, or return the log-sum if the parameter is passed.
In this analogy, outer computes log-sum, given data and parameter, or returns inner as a function of the parameter, which can be optimised.
You can reproduce this with simply:
f(x::Float64=nothing) = x
f()
When you add = nothing that sets nothing as a default argument. And also adds f() as a method. But when you call f() then julia will try to run f(nothing) as nothing is your default argument. That will then error as nothing is of type Void and you asserted that the argument must be Float64.
For example you could (but shouldn't) use f(x::Union{Float64,Void}=nothing) = x to get around this. But it'd be much better to use a Nullable which is exactly for interacting with a value that may or may not exist.
Read more about Nullables here or type ?Nullable in the REPL.
Edit by OP
MWE:
function outer(x::Array{Float64}, y=Nullable{Array{Float64}}())
if isnull(y)
function inner(y::Array{Float64})
return x .* y
end
return inner
else
return x .+ y
end
end
outer([2.])([3.]) # Produces 6,
outer([2.], [3.]) # Produces 5.
Works as expected.
Readability counts so maybe you have to reconsider define 2 methods explicitly (and simple as it is in Julian way):
outer(x::Array{Float64}, y::Array{Float64}) = x .+ y
outer(x::Array{Float64}) = y -> x .* y