The methods function returns the method table of a function as also mentioned here. I am looking for an explanation on how the function works.
Consider the following example in Julia 1.7:
julia> f(a::Int64,b::Int64=1,c::Float64=1.0) = (a+b+c)
f (generic function with 3 methods)
julia> g(a::Int64,b::Int64=1;c::Float64=1.0) = (a+b+c)
g (generic function with 2 methods)
julia> methods(f)
# 3 methods for generic function "f":
[1] f(a::Int64) in Main at REPL[1]:1
[2] f(a::Int64, b::Int64) in Main at REPL[1]:1
[3] f(a::Int64, b::Int64, c::Float64) in Main at REPL[1]:1
julia> methods(g)
# 2 methods for generic function "g":
[1] g(a::Int64) in Main at REPL[1]:1
[2] g(a::Int64, b::Int64; c) in Main at REPL[1]:1
julia> f(1,1.0)
ERROR: MethodError: no method matching f(::Int64, ::Float64)
Closest candidates are:
f(::Int64) at REPL[1]:1
f(::Int64, ::Int64) at REPL[1]:1
f(::Int64, ::Int64, ::Float64) at REPL[1]:1
Stacktrace:
[1] top-level scope
# REPL[4]:1
julia> g(1,c=1.0)
3.0
julia>
It is not quite clear to me why there is no method f(::Int64, ::Float64) (hence the error). I am also wondering why there is no error for g(1,c=1.0) given that g(::Int64, ::Float64) or g(::Int64, c) are not listed as valid methods for g.
Ah, so to be a bit technical this is really more accurately a question about how type annotations, dispatch, optional arguments, and keyword arguments work in Julia; the methods function just gives you some insight into that process, but it's not the methods function that makes those decisions. To answer your individual questions
It is not quite clear to me why there is no method f(::Int64, ::Float64) (hence the error).
There is no method for this because you you can only omit optional normal (non-keyword) arguments contiguously from the last normal (non-keyword) argument. Consider the following case:
julia> f(a=1, b=1, c=1, d=1) = a + 2b +3c +4d
f (generic function with 8 methods)
julia> f(2,4)
If there were not a rule for this, the compiler would have no idea whether the 2 and 4 provided were supposed to be for a and b, or do I mean that actually I wanted the 2 to go to a and the 4 to go to d? or c? Or anything! This would be undecidable. So we have a rule, and the rule is that the first argument goes to a, the second to b, and the omitted ones are c and d. Even though you have specified defaults, you cannot omit middle arguments, you can only omit the last N optional arguments. This is just the rule, and it does not matter whether or not you have applied type annotations or not.
I am also wondering why there is no error for g(1,c=1.0) given that g(::Int64, ::Float64) or g(::Int64, c) are not listed as valid methods for g.
Firstly, there is no method for g(::Int64, ::Float64) or g(::Int64, c) because keyword arguments (in this example, c) do not participate in dispatch. There is no error for g(1,c=1.0) because when you write g(1,c=1.0), you the optional argument for b is falling back to its default, so you are actually calling g(1,1,c=1.0) When you write g(1,c=1.0), you have explicitly specified that the 1.0 is being assigned to c, so it cannot possibly be the value for b. The value for b has to fall back to it's default, 1.
Related
As mentioned in the title, I want to pass a function as an argument for another function in Julia. However, I want the passed function to be partially specified, such as:
func1(a, b) = println(a, b)
func2(a, func::Function) = func(a)
func2(1, func1(b=0)) # returns an error
An alternative is to build a new function func3 such as
func3(a) = func1(a, b=0)
func2(1, func3)
But it does not look so elegant. Is it possible not to define a new function?
You should get the error because func1(a, b) gets two positional arguments, meanwhile, you're passing just the first one to it! So the result is func1 won't work and throws an error!
I didn't understand you here:
Obviously, I can't specify b by just passing a value to b in the func1. How should I do that?
Because it's not obvious actually(!), since you specified b=0.
You can do this:
julia> func1(a, b) = println(a, b)
func1 (generic function with 1 method)
julia> func2(a, b, func::Function) = func(a, b)
func2 (generic function with 1 method)
julia> func2(1, 0, func1)
10
An alternative way
Another way is to use the Varargs functions:
julia> func1(a...) = println(a)
func1 (generic function with 2 methods)
julia> func2(a, func::Function) = func(a...)
func2 (generic function with 2 methods)
julia> func2([1, 0], func1)
10
Let me start with this: I do not honestly know what you're trying to do, but I am just going to look at your source code from just pure Julia.
Your source code contains some runtime and logic errors. So instead of figuring out the "how-to" on your request, let's look at the errors first.
Line 1:
julia> func1(a, b) = println(a, b)
func1 (generic function with 1 method)
This function on it own has no error. So let's move on. Please note that it returns nothing which is of type Nothing.
Line 2:
func2(a, func::Function) = func(a)
func2 (generic function with 1 method)
This function on its own has no "runtime error", but there is a "logic error" here with the statement func(a) because of how you called func2 as we will see later.
Line 3:
func2(1, func1(b=0))
This is where the "runtime" and "logic" error really piles up. So let's look at this line statement closely.
func1(b=0) will throw an error on its own because you're using a keyword argument for a positional argument. I guess you're coming from Python where this won't error:
>>> def add(a):
return a * 1
>>> add(a=1)
1
However, in Julia it does:
julia> function add(a)
return a * 1
end
add (generic function with 1 method)
julia> add(a=1)
ERROR: MethodError: no method matching add(; a=1)
Closest candidates are:
add(::Any) at REPL[1]:1 got unsupported keyword argument "a"
Take a look at the section of Julia documentation about functions to learn more about how Julia functions work.
func2(1, func1(b=0)): Now let's assume we've fixed this statement and now used:
func2(1, func1(0));
however there is still two errors here.
The runtime error is we've not passed the a argument to func1 which is actually what your question is about: "how do I partially specify func1".
And the logic error is you explicitly passed a as 1 to func1, so it demands you passing it explicitly to func1 as well. To pass it implicitly there are two ways: go into some sophistication methods (which I won't even bother going into), or use a global variable (which will slow your performance, but is good for this case-study).
So now let's assume we have a variable a = 1 now. At this point our new source code now looks like this:
a = 1;
func1(a, b) = println(a, b);
func2(a, func::Function) = func(a);
func2(a, func3(0))
Well Julia has Base.Fix1 and Base.Fix2 for partially specifying a two-argument function. We'll use Base.Fix1 (since we're concerned with a which is the first argument):
func3 = Base.Fix1(func1, a);
We now have a third function which is fixed to the variable a. So as long as we only pass a implicitly to func2(a, func3(0)), both func2 and func3 will both use the global variable a. Now lets run the code:
julia> a = 1;
julia> func1(a, b) = println(a, b);
julia> func2(a, func::Function) = func(a);
julia> func3 = Base.Fix1(func1, a);
julia> func2(a, func3(0))
10
ERROR: MethodError: no method matching func2(::Int64, ::Nothing)
Closest candidates are:
func2(::Any, ::Function) at REPL[3]:1
As seen above the func3(0) actually runs correctly printing a and b (which is 1 and 0). However, there is still an error (which I guess you might have figured out now). You're using the returned value of func3 and not the function itself. Your type annotation was func2(a, func::Function), so you have to change the design. I would stop here since I can't make any recommendations as I do not know what you're trying to do.
I am new to Julia and I am trying to create a plot with the following:
xi2 = range(0,sqrt(6),step=1e-3)
collect(xi2)
plot(xi2, 1-xi2^2/6, label="n = 0")
When I try this though, I have the error:
MethodError: no method matching ^(::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, ::Int64)
Closest candidates are:
^(::Union{AbstractChar, AbstractString}, ::Integer) at C:\Users\Acer\AppData\Local\Programs\Julia-1.7.0\share\julia\base\strings\basic.jl:721
^(::Rational, ::Integer) at C:\Users\Acer\AppData\Local\Programs\Julia-1.7.0\share\julia\base\rational.jl:475
^(::Complex{<:AbstractFloat}, ::Integer) at C:\Users\Acer\AppData\Local\Programs\Julia-1.7.0\share\julia\base\complex.jl:839
...
What am I missing here?
You want the elements of xi2 raised to the power of two, so you want element-wise operations using the dot operator:
julia> xi2 = range(0,sqrt(6),step=1e-3);
julia> plot(xi2, 1 .- xi2.^2/6, label="n = 0")
(The collect step was unnecessary, since most array operations can be performed on a range directly. And in case you did want to collect - i.e. allocate memory and make it a full array - you have to assign the result of collect to some variable. In your original code, the elements were being collected into an array, but then thrown away since the result wasn't assigned to anything.)
Is there in Julia a Collection type from which both Set and Array derive ?
I have both:
julia> supertype(Array)
DenseArray{T,N} where N where T
julia> supertype(DenseArray)
AbstractArray{T,N} where N where T
julia> supertype(AbstractArray)
Any
And:
julia> supertype(Set)
AbstractSet{T} where T
julia> supertype(AbstractSet)
Any
What I try to achieve is to write function that can take both Array or Set as argument, because the type of collection doesn't matter as long as I can iterate over it.
function(Collection{SomeOtherType} myCollection)
for elem in myCollection
doSomeStuff(elem)
end
end
No, there is no Collection type, nor is there an Iterable one.
In theory, what you ask can be accomplished through traits, which you can read about elsewhere. However, I would argue that you should not use traits here, and instead simply refrain from restricting the type of your argument to the function. That is, instead of doing
foo(x::Container) = bar(x)
, do
foo(x) = bar(x)
There will be no performance difference.
If you want to restrict your argument types you could create a type union:
julia> ty = Union{AbstractArray,AbstractSet}
Union{AbstractSet, AbstractArray}
julia> f(aarg :: ty) = 5
f (generic function with 1 method)
This will work on both sets and arrays
julia> f(1:10)
5
julia> f(rand(10))
5
julia> f(Set([1,2,5]))
5
But not on numbers, for example
julia> f(5)
ERROR: MethodError: no method matching f(::Int64)
Closest candidates are:
f(::Union{AbstractSet, AbstractArray}) at REPL[2]:1
Can we pass a function as an argument of another function in julia? How does it work? Does this mean the input function is run before the calling function, or does the input function only get called if the calling function calls it specifically?
One of the greatest features of Julia is that you can dig out the answer to these kinda questions by yourself. Instead of making experiments and then observing the behaviors on the surface, directly asking Julia what it did under the hood is a more convenient and concise way to get the answer. Let's borrow these examples from the other two answers and ask Julia what happened via #code_lowered:
julia> f() = println("hello world")
julia> g(any_func::Function) = any_func()
#Me: hi, Julia, what happened here?
julia> #code_lowered g(f)
#Julia: hi there, this is the lowered code, anything else you want to know?
CodeInfo(:(begin
nothing
return (any_func)()
end))
#Me: it looks like function `g` just returned `(any_func)()` and did nothing else,
# is it equvienlent to write `f()`?
julia> #code_typed g(f)
#Julia: yes, the codes looks the same after type inference stage:
CodeInfo(:(begin
$(Expr(:inbounds, false))
# meta: location REPL[1] f 1
# meta: location coreio.jl println 5
SSAValue(0) = (Core.typeassert)(Base.STDOUT, Base.IO)::IO
# meta: pop location
# meta: pop location
$(Expr(:inbounds, :pop))
return (Base.print)(SSAValue(0), "hello world", $(QuoteNode('\n')))::Void
end))=>Void
julia> #code_typed f()
CodeInfo(:(begin
$(Expr(:inbounds, false))
# meta: location coreio.jl println 5
SSAValue(0) = (Core.typeassert)(Base.STDOUT, Base.IO)::IO
# meta: pop location
$(Expr(:inbounds, :pop))
return (Base.print)(SSAValue(0), "hello world", $(QuoteNode('\n')))::Void
end))=>Void
Does this mean the input function is run before the calling function?
In this special case, it's hard to answer, the calling function g was optimized out by the compiler at compile time, so there was no g at runtime. :P Let's add some extra contents to g:
julia> g(any_func::Function) = (println("I don't wanna be optimized out!"); any_func())
g (generic function with 1 method)
julia> #code_lowered g(f)
CodeInfo(:(begin
nothing
(Main.println)("I don't wanna be optimized out!")
return (any_func)()
end))
julia> #code_typed g(f)
CodeInfo(:(begin
$(Expr(:inbounds, false))
# meta: location coreio.jl println 5
SSAValue(0) = (Core.typeassert)(Base.STDOUT, Base.IO)::IO
# meta: pop location
$(Expr(:inbounds, :pop))
(Base.print)(SSAValue(0), "I don't wanna be optimized out!", $(QuoteNode('\n')))::Void
$(Expr(:inbounds, false))
# meta: location REPL[2] f 1
# meta: location coreio.jl println 5
SSAValue(1) = (Core.typeassert)(Base.STDOUT, Base.IO)::IO
# meta: pop location
# meta: pop location
$(Expr(:inbounds, :pop))
return (Base.print)(SSAValue(1), "hello world", $(QuoteNode('\n')))::Void
end))=>Void
This example shows the content of g would be run exactly before f, so the answer is YES. Follow the same pattern, it's easy to check out Liso's example:
julia> f = x->x+1
julia> fun(f::Function, a) = f(a)
# nothing new here, `f(9)` was called before `fun`
# and the return value was passed to `fun` with a binding name `a`.
julia> #code_lowered fun(f, f(9))
CodeInfo(:(begin
nothing
return (f)(a)
end))
# to verify your second question:
julia> foo(f, x) = x
foo (generic function with 1 method)
julia> #code_lowered foo(f, 1)
CodeInfo(:(begin
nothing
return x
end))
Does the input function only get called if the calling function calls it specifically?
So yes, the example above shows that if f is not called by the calling function foo, it will be directly optimized out.
Unlike other languages, Julia is not just a magic black box which is opaque to users, sometimes it's efficient and effective to open the box and be self-taught. BTW, There are two more stages(#code_llvm, #code_native) in Julia, you might need to dump these low-level codes for some advanced investigation, refer to Stefan's great answer in this post: What is the difference between #code_native, #code_typed and #code_llvm in Julia? for further details.
UPDATE:
what is different between these functions: g1() = f() , g2(f::Function) = f(). Results for both of them are the same, so What is different between them?
julia> #code_lowered g1()
CodeInfo(:(begin
nothing
return (Main.f)()
end))
julia> #code_lowered g2(f)
CodeInfo(:(begin
nothing
return (f)()
end))
The lowered code tell that g1() always returns Main.f(), here the Main.f is the f in the Main module, but g2 returns f() where f is the function you passed to it. To make this clear, we could define g2 as:
julia> g2(argumentf::Function) = argumentf()
g2 (generic function with 1 method)
julia> #code_lowered g2(f)
CodeInfo(:(begin
nothing
return (argumentf)()
end))
g2 is "pass a function as an argument of another function", g1 can be considered as an alias for Main.f. Does this make sense to you? #ReD
Julia has always supported first-class as well as higher-order functions. This means that yes, functions can be arguments to other functions. In addition, the language has also always had support for anonymous functions and closures.
Originally, calling functions that were passed into other functions came with a performance penalty. However, as of v0.5, this is no longer an issue, and functions you pass into other functions as arguments will run just as quickly as functions from base julia. There is more reading on this here.
A simple example of passing functions follows:
julia> f() = println("hello world") #Define a simple function f
f (generic function with 1 method)
julia> g(any_func::Function) = any_func() #Define a function g that accepts any function as input and then calls that function
g (generic function with 1 method)
julia> g(f) #Call g with f as the input
hello world
What is happening here is that I build a function f in the first line, which, when called via f(), will print "hello world". In the second line I build a function g which accepts any function as input, and then calls that function. In the third line, I call my function g using my function f as input, and g then runs the function f, which, in this case, will print "hello world".
The example above may seem trite, but in more complicated scenarios, being able to pass functions around like this is incredibly useful.
As the other answerer points out, base julia exploits this feature with many native functions that accept other functions (including anonymous functions) as input, such as map, filter, etc.
Most readers can stop here. What follows is only relevant to those who use eval a lot...
There is one exception to the above, which the average user is unlikely to ever encounter: If a function is built by parsing and evaluating a string at runtime, and this function is called during the same runtime, then the call will fail with an error message that will say something about "world age". What is actually happening here is that because the function has been created at runtime, the compiler does not have an opportunity to reason about the behaviour of the function, e.g. input and output types, and so it "prefers" not to call it. Nonetheless, if you really want to you can get around this by calling it via the Base.invokelatest method, but this means the function call will not be type-stable. Basically, unless you really know what you're doing, you should avoid messing around with this.
map is one example where function is used as parameter
# you could define function
julia> f = x->x+1
# and use it (f mapped to array elements) ->
julia> map(f, [1, 2, 3])
3-element Array{Int64,1}:
2
3
4
f is not called before using it in map!
edit (reaction to comment bellow):
to symbol f is mapped function object (it is method what to do with parameters) and this symbol could be called in fun.
# fun(ny) example
julia> function fun(f::Function, a)
return f(a)
end
fun (generic function with 1 method)
# using f defined above
julia> fun(f, 10)
11
# or anonymous function defined in place:
julia> fun(x->x^2, 10)
100
And maybe more complicated example could be understandable too. f is called one time before fun execution (to calculate parameter a) and one time in fun function body:
julia> fun(f, f(9))
11
This snippet is from the implementation of Rational Numbers in Julia:
# Rational.jl
# ...
Rational{T<:Integer}(n::T, d::T) = Rational{T}(n,d)
Rational(n::Integer, d::Integer) = Rational(promote(n,d)...)
Rational(n::Integer) = Rational(n,one(n))
//(x::Rational, y::Integer) = x.num // (x.den*y) <--- HERE!
# ...
See how the // function is implemented and then used with infix notation? How does this actually return a value?
When I saw this code I interpreted it like this:
The // function is called with a Rational and an Integer.
But then it makes a recursive call with no other arguments.
#2 is the one that really confuses me. Where does the recursion within data structure end? How does // return a value if it is constantly evaluating nothing?
Please help me understand this.
This works because of one of the most fundamental features of Julia: multiple dispatch. In Julia, functions can have many methods which apply to various combinations of argument types, and when you call a function, Julia invokes the most specific method which matches the type of all the arguments that you called it with. The // call in the method definition you posted defines rational-integer // in terms of integer-integer // – so it isn't actually recursive because the method doesn't call itself, it calls a different method that is part of the same "generic function".
To understand how multiple dispatch works in this case, let's consider the evaluation of the expression (3//4)//6. We'll use the #which macro to see which method each function call invokes.
julia> #which (3//4)//6
//(x::Rational{T<:Integer}, y::Integer) at rational.jl:25
Since 3//4 is a Rational{Int} <: Rational and 6 is an Int <: Integer, and no other more specific methods apply, this method is called:
//(x::Rational, y::Integer) = x.num // (x.den*y)
The current version of the method is actually slightly more complicated than what you posted because it's been modified to check for integer overflow – but it's essentially the same, and it's easier to understand the older, simpler version, so I'll use that. Let's assign x and y to the arguments and see what method the definition calls:
julia> x, y = (3//4), 6
(3//4,6)
julia> x.num
3
julia> x.den*y
24
julia> x.num // (x.den*y)
1//8
julia> #which x.num // (x.den*y)
//(n::Integer, d::Integer) at rational.jl:22
As you can see, this expression doesn't call the same method, it calls a different method:
//(n::Integer, d::Integer) = Rational(n,d)
This method simply calls the Rational constructor which puts the ratio of n and d into lowest terms and creates a Rational number object.
It is quite common to define one method of a function in terms of another method of the same function, in Julia. This is how argument defaults work, for example. Consider this definition:
julia> f(x, y=1) = 2x^y
f (generic function with 2 methods)
julia> methods(f)
# 2 methods for generic function "f":
f(x) at none:1
f(x, y) at none:1
julia> f(1)
2
julia> f(2)
4
julia> f(2,2)
8
The default argument syntax simply generates a second method with only onee argument, which calls the two-argument form with the default value. So f(x, y=1) = 2x^y is exactly equivalent to defining two methods, where the unary method just calls the binary method, supplying a default value for the second argument:
julia> f(x, y) = 2x^y
f (generic function with 1 method)
julia> f(x) = f(x, 1)
f (generic function with 2 methods)