I have two methods with the same name and same amount of "normal" arguments. However, they differ in the number of keyword arguments (which are all non-optional).
I think the following minimal example illustrates my point well:
julia> f(a; b) = a+b
f (generic function with 1 method)
julia> f(a; b, c)= a+b+c
f (generic function with 1 method)
julia> f(1; b=1)
ERROR: UndefKeywordError: keyword argument c not assigned
Stacktrace:
[1] top-level scope
# REPL[72]:1
julia> f(1; b=1, c=2)
4
The second line of output ("f (generic function with 1 method)") already shows that Julia doesn't understand what I want - it should be 2 methods (which it would be e.g. if I wrote f(a, b; c) = a+b+c).
I already found this in the Julia discourse forum, which seems to explain why this doesn't work for multiple dispatch. However, in my case not the types but the counts differ, so I hope that there is a (neat) solution.
Regarding why I want this: I have a function that makes some calculations (to be precise it calculates Stark curves) for different molecules. This function needs to act differently for different molecule types (to be precise different symmetries) and also takes different arguments (different quantum numbers).
So that's why I need multiple dispatch - but why keyword arguments, you might ask. That is because I create the quantum numbers using distributions and passing them as NamedTuple in connection with ....
I'd like to not change this behaviour, but maybe you were curious to why I would need something like this.
In your example, you can reuse the keyword arguments as positional arguments to make f(1; b=1) work, but it won't behave like true keyword dispatch. For one, f(1; c=2) would call _f(a,b).
function f(a; b=missing, c=missing)
_f(a, skipmissing((b, c))...)
end
_f(a,b,c) = a+b+c
_f(a,b) = a+b
Not sure if this is applicable to the actual use case you described, though, things can be harder to reorder than b and c, and as a comment noted, a NamedTuple (which is ordered) is already dispatchable.
Related
I am puzzled by the following results of typeof in the Julia 1.0.0 REPL:
# This makes sense.
julia> typeof(10)
Int64
# This surprised me.
julia> typeof(function)
ERROR: syntax: unexpected ")"
# No answer at all for return example and no error either.
julia> typeof(return)
# In the next two examples the REPL returns the input code.
julia> typeof(in)
typeof(in)
julia> typeof(typeof)
typeof(typeof)
# The "for" word returns an error like the "function" word.
julia> typeof(for)
ERROR: syntax: unexpected ")"
The Julia 1.0.0 documentation says for typeof
"Get the concrete type of x."
The typeof(function) example is the one that really surprised me. I expected a function to be a first-class object in Julia and have a type. I guess I need to understand types in Julia.
Any suggestions?
Edit
Per some comment questions below, here is an example based on a small function:
julia> function test() return "test"; end
test (generic function with 1 method)
julia> test()
"test"
julia> typeof(test)
typeof(test)
Based on this example, I would have expected typeof(test) to return generic function, not typeof(test).
To be clear, I am not a hardcore user of the Julia internals. What follows is an answer designed to be (hopefully) an intuitive explanation of what functions are in Julia for the non-hardcore user. I do think this (very good) question could also benefit from a more technical answer provided by one of the more core developers of the language. Also, this answer is longer than I'd like, but I've used multiple examples to try and make things as intuitive as possible.
As has been pointed out in the comments, function itself is a reserved keyword, and is not an actual function istself per se, and so is orthogonal to the actual question. This answer is intended to address your edit to the question.
Since Julia v0.6+, Function is an abstract supertype, much in the same way that Number is an abstract supertype. All functions, e.g. mean, user-defined functions, and anonymous functions, are subtypes of Function, in the same way that Float64 and Int are subtypes of Number.
This structure is deliberate and has several advantages.
Firstly, for reasons I don't fully understand, structuring functions in this way was the key to allowing anonymous functions in Julia to run just as fast as in-built functions from Base. See here and here as starting points if you want to learn more about this.
Secondly, because each function is its own subtype, you can now dispatch on specific functions. For example:
f1(f::T, x) where {T<:typeof(mean)} = f(x)
and:
f1(f::T, x) where {T<:typeof(sum)} = f(x) + 1
are different dispatch methods for the function f1
So, given all this, why does, e.g. typeof(sum) return typeof(sum), especially given that typeof(Float64) returns DataType? The issue here is that, roughly speaking, from a syntactical perspective, sum needs to serves two purposes simultaneously. It needs to be both a value, like e.g. 1.0, albeit one that is used to call the sum function on some input. But, it is also needs to be a type name, like Float64.
Obviously, it can't do both at the same time. So sum on its own behaves like a value. You can write f = sum ; f(randn(5)) to see how it behaves like a value. But we also need some way of representing the type of sum that will work not just for sum, but for any user-defined function, and any anonymous function. The developers decided to go with the (arguably) simplest option and have the type of sum print literally as typeof(sum), hence the behaviour you observe. Similarly if I write f1(x) = x ; typeof(f1), that will also return typeof(f1).
Anonymous functions are a bit more tricky, since they are not named as such. What should we do for typeof(x -> x^2)? What actually happens is that when you build an anonymous function, it is stored as a temporary global variable in the module Main, and given a number that serves as its type for lookup purposes. So if you write f = (x -> x^2), you'll get something back like #3 (generic function with 1 method), and typeof(f) will return something like getfield(Main, Symbol("##3#4")), where you can see that Symbol("##3#4") is the temporary type of this anonymous function stored in Main. (a side effect of this is that if you write code that keeps arbitrarily generating the same anonymous function over and over you will eventually overflow memory, since they are all actually being stored as separate global variables of their own type - however, this does not prevent you from doing something like this for n = 1:largenumber ; findall(y -> y > 1.0, x) ; end inside a function, since in this case the anonymous function is only compiled once at compile-time).
Relating all of this back to the Function supertype, you'll note that typeof(sum) <: Function returns true, showing that the type of sum, aka typeof(sum) is indeed a subtype of Function. And note also that typeof(typeof(sum)) returns DataType, in much the same way that typeof(typeof(1.0)) returns DataType, which shows how sum actually behaves like a value.
Now, given everything I've said, all the examples in your question now make sense. typeof(function) and typeof(for) return errors as they should, since function and for are reserved syntax. typeof(typeof) and typeof(in) correctly return (respectively) typeof(typeof), and typeof(in), since typeof and in are both functions. Note of course that typeof(typeof(typeof)) returns DataType.
I am sorry about the title, but I couldn't find a better one.
Let's define
function test(n)
print("test executed")
return n
end
f(n) = test(n)
Every time we call f we get
f(5)
test executed
5
Is there a way to tell julia to evaluate test once in the definition of f?
I expect that this is probably not going to be possible, in which case I have a slightly different question. If ar=[1,2,:x,-2,2*:x] is there any way to define f(x) to be the sum of ar, i.e. f(x) = 3*x+1?
If you want to compile based on type information, you can use #generated functions. But it seems like you want to compile based on the runtime values of the input. In this case, you might want to do memoization. There is a library Memoize that provides a macro for memoizing functions.
Say I have a function such as f(x,y) but the y parameter is optional. What is the preferred way to set y as an optional argument? One option that works for me:
function f(x, y=nothing)
# do stuff
if y == nothing
# do stuff
else
# do stuff
end
# do stuff
end
But is this the preferred way? I can't set y to a single default value to use in calculation, as there are a number of calculations that are done differently when y is nothing. I could also just have separate functions f(x) and f(x,y) but that seems like too much code duplication.
This is fine. Note that optional arguments cause dispatch. This means that the if y == nothing (or equivalently, if typeof(y) <: Void), will actually compile away. You'll get two different functions which depend on whether the user gives a value or not. So in the end, the if statement compiles away and it's perfectly efficient to do this.
I will note that the same is not currently true for keyword arguments.
Another way to do this is to have two methods:
f(x)
and
f(x,y)
Whether two methods is nicer than 1 method with an if depends on the problem. Since the if will compile away using type information, there's no difference between these two approaches other than code organization.
How can I use kwargs in a Julia function and declare their types for speed?
function f(x::Float64; kwargs...)
kwargs = Dict(kwargs)
if haskey(kwargs, :c)
c::Float64 = kwargs[:c]
else
c::Float64 = 1.0
end
return x^2 + c
end
f(0.0, c=10.0)
yields:
ERROR: LoadError: syntax: multiple type declarations for "c"
Of course I can define the function as f(x::Float64, c::Float64=1.0) to achieve the result, but I have MANY optional arguments with default values to pass, so I'd prefer to use kwargs.
Thanks.
Related post
As noted in another answer, this really only matters if you're going to have a type instability. If you do, the answer is to layer your functions. Have a top layer which does type checking and all sorts of setup, and then call a function which uses dispatch to be fast. For example,
function f(x::Float64; kwargs...)
kwargs = Dict(kwargs)
if haskey(kwargs, :c)
c = kwargs[:c]
else
c = 1.0
end
return _f(x,c)
end
_f(x,c) = x^2 + c
If most of your time is spent in the inner function, then this will be faster (it might not be for very simple functions). This allows for very general usage too, where you have have a keyword argument be by default nothing and do and if nothing ... which could setup a complicated default, and not have to worry about the type stability since it will be shielded from the inner function.
This kind of high-level type-checking wrapper above a performance sensitive inner function is used a lot in DifferentialEquations.jl. Check out the high-level wrapper for the SDE solvers which led to nice speedups by insuring type stability (the inner function is sde_solve) (or check out the solve for ODEProblem, it's much more complex since it handles conversions to different pacakges but it's the same idea).
A simpler answer for small examples like yours may be possible after this PR merges.
To fix some confusion, here's a declaration form:
function f(x::Float64; kwargs...)
local c::Float64 # Ensures the type of `c` will be `Float64`
kwargs = Dict(kwargs)
if haskey(kwargs, :c)
c = float(kwargs[:c])
else
c = 1.0
end
return x^2 + c
end
This will force anything that saves to c to convert to a Float64 or error, resulting in a type-stability, but is not as general of a solution. What form you use really depends on what you're doing.
Lastly, there's also the type assert, as #TotalVerb showed:
function f(x::Float64; c::Float64=1.0, kwargs...)
return x^2 + c
end
That's clean, or you could assert in the function:
function f(x::Float64; kwargs...)
kwargs = Dict(kwargs)
if haskey(kwargs, :c)
c = float(kwargs[:c])::Float64
else
c = 1.0
end
return x^2 + c
end
which will cause convertions only on the lines where the assertion occurs (i.e. the #TotalVerb form won't dispatch, so you can't make another function with c::Int, and it will only assert (convert) when the keyword arg is first read in).
Summary
The first solution will dispatch to be type stable in _f no matter what type the user makes c, and so if _f is a long calculation, this will get pretty much optimal performance, but for really quick calls it will have dispatch overhead.
The second solution will fix any type stability by forcing anything you set c to be a Float64 (it will try to convert, and if it can't, error). Thus this gets speed by forcing type stability, or erroring.
The assert in the keyword spot (#TotalVerb's answer) is the cleanest, but won't auto-convert later (so you could get a type-instability. But if you don't accidentally convert it later, then you have type stability, types can be inferred, and so you'll get optimal performance) and you can't extend it to cases where the function has c passed in as other types (no dispatch).
The last solution is pretty much the same as 3, except not as nice. I wouldn't recommend it. If you're doing something complicated with asserts, you likely are designing something wrong or really want to do something like the first (dispatch in a longer function call which is type stable).
But note that dispatch with version 3 may be fixed in the near future, which would allow you to have a different function with c::Float64 and c::Int (if necessary). Hopefully your solution is in here somewhere.
Note that declaring types does not give you increased performance; you may wish to relax the type constraints on x and c for your code to be more generic. Anyway, this is probably what you want:
function f(x::Float64; c::Float64=1.0, kwargs...)
return x^2 + c
end
See the keyword arguments section of the manual.
How do you define non-commutable variables? I'm new to julia and have been reading the doc and found nothing so far.
By non-commutable I mean if variables (say symbols) a and b are multiplied. ab =\= ba
About commutativity: Julia does not assume that a*b is the same as b*a (example: let a and b be matrices). However, methods of the function *(a,b) for some specific combinations of types act in a commutative manner, such as when a and b are numbers.
Since you are talking about symbols, I guess that want to work with a symbolic representation of expressions. There's at least two ways to go about this:
Work with the AST of an expression.
You can quote any expression by :(expr) (sometimes :expr is enough, depends on operator precedence):
julia> ex = :(x*y*z)
:(x * y * z)
julia> typeof(ex)
Expr
Note that the order of the factors has been preserved (though the associativity has not).
You can inspect the Expr by looking at it's head and args fields:
julia> ex.head
:call
julia> ex.args
4-element Array{Any,1}:
:*
:x
:y
:z
The args may be e.g. symbols (of type Symbol), constant values, and other expressions.
This technique works well if you want to write your own macros that process expressions, since the input to a macro is the AST of its arguments.
Define your own types and overload the * function. Example: with
abstract Symbolic
# might as well make these immutable
immutable Sym <: Symbolic
name::Symbol
end
immutable Prod <: Symbolic
x::Symbolic
y::Symbolic
end
# The * function is imported by default.
# Usually, we need to import a function
# from Base before extending it.
*(x::Symbolic, y::Symbolic) = Prod(x, y)
you can do
julia> x, y = Sym(:x), Sym(:y)
(Sym(:x),Sym(:y))
julia> x*y
Prod(Sym(:x),Sym(:y))
Since our *(x::Symbolic, y::Symbolic) function preserves the order of the arguments, we can see it in the created Prod object.
In most languages, the second alternative is your only option. The first alternative is more direct since you don't have to write a new AST framework for yourself, and don't have to execute the expression just to get at it's representation. Which one is more suitable depends on the application, however. If you want to associate different properties with your variables, the second approach seems easier.
Was it something like this that you were looking for?