I'm trying to understand how parametric types work in Julia.
Suppose I have a function foo, which takes an Integer (i.e. non-parametric type), and I want to assert this function to return a vector, whose elements are subtypes of Real. From documentation I deduced that it should be implemented as follows:
function foo{T<:Real}(n::Integer)
# come code to generate variable vec
return vec::Vector{T}
end
But it is not working. What am I doing wrong?
Parametric types are used to parameterise the inputs to a function, not the outputs. If your function is type-stable (read about what that means in the performance tips), then the output of the function can be automatically inferred by the compiler based on the input types, so there should be no need to specify it.
So your function might look something like this:
function foo{T<:Real}(n::T)
#some code to generate vec
return(vec)
end
For example, you can generate a vector of the appropriate type within the body of your function, e.g. something like this:
function f1{T<:Number}(x::T)
y = Array(T, 0)
#some code to fill out y
return(y)
end
But note that T must appear in the argument definitions of the inputs to the function, because this is the purpose of parametric types. You'll get an error if T does not appear in the description of the inputs. Something like this doesn't work:
f{T<:Number}(x::Vector) = x
WARNING: static parameter T does not occur in signature for f at none:1.
The method will not be callable.
f (generic function with 1 method)
But this does, and is type stable, since if the input types are known, then so are the outputs
f{T<:Number}(x::Vector{T}) = x
A parametric method or a constructor of a parametric type
A function is called using the traditional parenthesis syntax:
julia> f(2,3)
5
The above rule is true for both parametric and non-parametric methods.
So, one should not try to call a parametric method like: f{atype}(2,3). I think the source of this mall-usage could be the syntax of calling a parametric type constructor:
julia> typeof(Array{Int})
DataType
julia> Array{Int}(2)
2-element Array{Int32,1}:
57943068
72474848
But in the case of a parametric method:
julia> same_type{T}(x::T, y::T) = true;
julia> typeof(same_type)
Function
julia> same_type{Int}
ERROR: TypeError: Type{...} expression: expected Type{T}, got Function
So with this introduction it has become clear that something like:
function foo{T<:Real}(n::Integer)
.....
end
would be useless, because the value of T won't be determined from caller the arguments.
Make benefit of parametric methods
julia> same_type{T}(x::T, y::T) = true;
julia> same_type(x,y) = false;
julia> same_type(1, 2)
true
julia> same_type(1, 2.0)
false
The main purpose of parametric methods is to let dispatch find the right method to call with respect to argument types when calling a function, but also it has the following idiomatic side effect:
Method type parameters are not restricted to being used as the types
of parameters: they can be used anywhere a value would be in the
signature of the function or body of the function.
julia> mytypeof{T}(x::T) = T
mytypeof (generic function with 1 method)
julia> mytypeof(1)
Int64
Now, let's go back to the main problem of:
How to write a method with Parametric return types?
If your method is already parametric, you could use the parameter value as return type as in the above mytypeof sample or e.g:
julia> newoftype{T}(x::T,n) = T(n)
newoftype (generic function with 1 methods)
julia> newoftype([1,2],4)
4-element Array{Int32,1}:
57943296
72475184
141142104
1970365810
But making return type of a method, a function of arguments value is a straightforward functionality and could be simply done without need to a parametric method. really many of typical methods do this job e.g:
julia> Array(Int,4)
4-element Array{Int32,1}:
126515600
72368848
72474944
0
julia> Array(Float64,4)
4-element Array{Float64,1}:
-2.122e-314
0.0
5.12099e-292
5.81876e-292
It could be done using a argument type of Type e.g:
julia> myarray(T::Type,n::Int)=Array(T,n);
Related
I'm very new to Julia, and I'm trying to just pass an array of numbers into a function and count the number of zeros in it. I keep getting the error:
ERROR: UndefVarError: array not defined
I really don't understand what I am doing wrong, so I'm sorry if this seems like such an easy task that I can't do.
function number_of_zeros(lst::array[])
count = 0
for e in lst
if e == 0
count + 1
end
end
println(count)
end
lst = [0,1,2,3,0,4]
number_of_zeros(lst)
There are two issues with your function definition:
As noted in Shayan's answer and Dan's comment, the array type in Julia is called Array (capitalized) rather than array. To see:
julia> array
ERROR: UndefVarError: array not defined
julia> Array
Array
Empty square brackets are used to instantiate an array, and if preceded by a type, they specifically instantiate an array holding objects of that type:
julia> x = Int[]
Int64[]
julia> push!(x, 3); x
1-element Vector{Int64}:
3
julia> push!(x, "test"); x
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64
Thus when you do Array[] you are actually instantiating an empty vector of Arrays:
julia> y = Array[]
Array[]
julia> push!(y, rand(2)); y
1-element Vector{Array}:
[0.10298669573927233, 0.04327245960128345]
Now it is important to note that there's a difference between a type and an object of a type, and if you want to restrict the types of input arguments to your functions, you want to do this by specifying the type that the function should accept, not an instance of this type. To see this, consider what would happen if you had fixed your array typo and passed an Array[] instead:
julia> f(x::Array[])
ERROR: TypeError: in typeassert, expected Type, got a value of type Vector{Array}
Here Julia complains that you have provided a value of the type Vector{Array} in the type annotation, when I should have provided a type.
More generally though, you should think about why you are adding any type restrictions to your functions. If you define a function without any input types, Julia will still compile a method instance specialised for the type of input provided when first call the function, and therefore generate (most of the time) machine code that is optimal with respect to the specific types passed.
That is, there is no difference between
number_of_zeros(lst::Vector{Int64})
and
number_of_zeros(lst)
in terms of runtime performance when the second definition is called with an argument of type Vector{Int64}. Some people still like type annotations as a form of error check, but you also need to consider that adding type annotations makes your methods less generic and will often restrict you from using them in combination with code other people have written. The most common example of this are Julia's excellent autodiff capabilities - they rely on running your code with dual numbers, which are a specific numerical type enabling automatic differentiation. If you strictly type your functions as suggested (Vector{Int}) you preclude your functions from being automatically differentiated in this way.
Finally just a note of caution about the Array type - Julia's array's can be multidimensional, which means that Array{Int} is not a concrete type:
julia> isconcretetype(Array{Int})
false
to make it concrete, the dimensionality of the array has to be provided:
julia> isconcretetype(Array{Int, 1})
true
First, it might be better to avoid variable names similar to function names. count is a built-in function of Julia. So if you want to use the count function in the number_of_zeros function, you will undoubtedly face a problem.
Second, consider returning the value instead of printing it (Although you didn't write the print function in the correct place).
Third, You can update the value by += not just a +!
Last but not least, Types in Julia are constantly introduced with the first capital letter! So we don't have an array standard type. It's an Array.
Here is the correction of your code.
function number_of_zeros(lst::Array{Int64})
counter = 0
for e in lst
if e == 0
counter += 1
end
end
return counter
end
lst = [0,1,2,3,0,4]
number_of_zeros(lst)
would result in 2.
Additional explanation
First, it might be better to avoid variable names similar to function names. count is a built-in function of Julia. So if you want to use the count function in the number_of_zeros function, you will undoubtedly face a problem.
Check this example:
function number_of_zeros(lst::Array{Int64})
count = 0
for e in lst
if e == 0
count += 1
end
end
return count, count(==(1), lst)
end
number_of_zeros(lst)
This code will lead to this error:
ERROR: MethodError: objects of type Int64 are not callable
Maybe you forgot to use an operator such as *, ^, %, / etc. ?
Stacktrace:
[1] number_of_zeros(lst::Vector{Int64})
# Main \t.jl:10
[2] top-level scope
# \t.jl:16
Because I overwrote the count variable on the count function! It's possible to avoid such problems by calling the function from its module:
function number_of_zeros(lst::Array{Int64})
count = 0
for e in lst
if e == 0
count += 1
end
end
return count, Base.count(==(1), lst)
The point is I used Base.count, then the compiler knows which count I mean by Base.count.
Trying to understand parametric types and the new function available for inner methods. The manual states "special function available to inner constructors which created a new object of the type". See the section of the manual on new here and the section of the manual on inner constructor methods here.
Consider an inner method designed to calculate the sum of x, where x could be, say, a vector or a tuple, and is given the parametric type T. A natural thing to want is for the type of the elements of x to be inherited by their sum s. I don't seem to need new for that, correct?
struct M{T}
x::T
s
function M(x)
s = sum(x)
x,s
end
end
julia> M([1,2,3])
([1, 2, 3], 6)
julia> M([1.,2.,3.])
([1.0, 2.0, 3.0], 6.0)
julia> typeof(M([1.,2.,3.]))
Tuple{Vector{Float64}, Float64}
Edit: Correction! I intended to have the last line of the inner constructor be M(x,s)... It's still an interesting question, so I won't correct it. How does M(x,s) differ from new{typeof(x)}(x,s)?
One usage of new I have seen is in combination with typeof(), something like:
struct M{T}
x::T
s
function M(x)
s = sum(x)
new{typeof(x)}(x,s)
end
end
julia> M([1,2,3])
M{Vector{Int64}}([1, 2, 3], 6)
julia> M([1.,2.,3.])
M{Vector{Float64}}([1.0, 2.0, 3.0], 6.0)
What if wanted to constrain s to the same type as x? That is, for instance, if x is a vector, then s should be a vector (in this case, a vector of one element). How would I do that? If I replace the last line of the inner constructor with x, new{typeof(x)}(s), I get the understandable error:
MethodError: Cannot `convert` an object of type Int64 to an object of type Vector{Int64}
Here are the rules:
If you are writing an outer constructor for a type M, the constructor should return an instance of M by eventually calling the inner constructor, like this: M(<args>).
If you are writing an inner constructor, this will override the default inner constructor. So you must return an instance of M by calling new(<args>).
The new "special function" exists to allow the construction of a type that doesn't have a constructor yet. Observe the following example:
julia> struct A
x::Int
function A(x)
A(x)
end
end
julia> A(4)
ERROR: StackOverflowError:
Stacktrace:
[1] A(::Int64) at ./REPL[3]:4 (repeats 79984 times)
This is a circular definition of the constructor for A, which results in a stack overflow. You cannot pull yourself up by your bootstraps, so Julia provides the new function as a way to circumvent this problem.
You should provide the new function with a number of arguments equal to the number of fields in your struct. Note that the new function will attempt to convert the types of its inputs to match the declared types of the fields of your struct:
julia> struct B
x::Float64
B(x) = new(x)
end
julia> B(5)
B(5.0)
julia> B('a')
B(97.0)
julia> B("a")
ERROR: MethodError: Cannot `convert` an object of type String to an object
of type Float64
(The inner constructor for B above is exactly the same as the default inner constructor.)
When you're defining parametric types, the new function must be provided with a number of parameters equal to the number of parameters for your type (and in the same order), analogously to the default inner constructor for parametric types. First observe how the default inner constructor for parametric types is used:
julia> struct Foo{T}
x::T
end
julia> Foo{String}("a")
Foo{String}("a")
Now if you were writing an inner constructor for Foo, instead of writing Foo{T}(x) inside the constructor, you would replace the Foo with new, like this: new{T}(x).
You might need typeof to help define the constructor, but often you don't. Here's one way you could define your M type:
struct M{I, T}
x::I
s::T
function M(x::I) where I
s = sum(x)
new{I, typeof(s)}(x, s)
end
end
I'm using typeof here so that I could be any iterable type that returns numbers:
julia> typeof(M(1:3))
M{UnitRange{Int64},Int64}
julia> g = (rand() for _ in 1:10)
Base.Generator{UnitRange{Int64},var"#5#6"}(var"#5#6"(), 1:10)
julia> typeof(M(g))
M{Base.Generator{UnitRange{Int64},var"#5#6"},Float64}
Note that providing the parameters for your type is required when you are using new inside an inner constructor for a parametric type:
julia> struct C{T}
x::Int
C(x) = new(x)
end
ERROR: syntax: too few type parameters specified in "new{...}" around REPL[6]:1
Remember, a constructor is designed to construct something. Specifically, the constructor M is designed to construct a value of type M. Your example constructor
struct M{T}
x::T
s
function M(x)
s = sum(x)
x,s
end
end
means that the result of evaluating the expression M([1 2 3]) is a tuple, not an instance of M. If I encountered such a constructor in the wild, I'd assume it was a bug and report it. new is the internal magic that allows you to actually construct a value of type M.
It's a matter of abstraction. If you just want a tuple in the first place, then forget about the structure called M and just define a function m at module scope that returns a tuple. But if you intend to treat this as a special data type, potentially for use with dynamic dispatch but even just for self-documentation purposes, then your constructor should return a value of type M.
To simplify, I am trying to write a function with two arguments, where:
The base method accepts two Integers as arguments
func(x::Int, y::Int) = something
Additional methods accept either or both arguments as arbitrary types, map these arguments onto integers, and call the base method
Additional methods accept either or both arguments as arrays (or ::Colon types) and generate an array from applying the appropriate prior methods (1) or (2) elementwise.
Unsurprisingly (in hindsight), this approach generates method ambiguities. Given the types of arguments provide to a function, Julia chooses a valid method with the most specific types. But if x is an array and y is an Int, the following methods are equally specific and Julia doesn't know which one to call:
func(x::Any, y::Int)
func(x::Array, y::Any)
I would like to do something like
func(x::T, y::Int) T <: any_so_long_as_not_array = func(map_x_to_Int(x), y)
func(x::Array, y::Any) = (el -> func(el, y)).(x)
Is there such as thing as a not-a-type type? Am I thinking about this the wrong way? Is there a canonical way to approach this sort of problem?
For context, I am trying to implement a Base.getindex for a struct I wrote, and I want the getindex to support many different ways to index the struct when the contents of the struct could be somewhat diverse. Under-the-hood, elements in the struct are indexed by integers, but the user might be using almost-arbitrary non-integer types to index elements in the struct (I don't want to force the user to use particular types to index elements).
You can specify the (Array, Int) case and then add less specific methods:
julia> func(x::Array, i::Int) = 0
func (generic function with 1 method)
julia> func(x, i::Int) = 1
func (generic function with 2 methods)
julia> func(x::Array, i) = 2
func (generic function with 3 methods)
julia> methods(func)
# 3 methods for generic function "func":
[1] func(x::Array, i::Int64) in Main at REPL[1]:1
[2] func(x, i::Int64) in Main at REPL[2]:1
[3] func(x::Array, i) in Main at REPL[3]:1
I want to get the result type of a function call in Julia without evaluating the function, and use that type. The desired usage looks somewhat like this:
foo(x::Int32) = x
foo(x::Float32) = x
y = 0.0f0
# Assert that y has the type of result of foo(Float32)
y::#resultof foo(Float32) # This apparently does not work in Julia
While in the case above, I can simply use y::typeof(foo(1.0f0)) with evaluation of a dummy variable, in more complicated cases, initializing a dummy variable might be inconvenient and expensive. For example, I want to use the iterator type returned by function eachline(filename::AbstractString; keep::Bool=false), but using the typeof really requires opening a file successfully, which looks like an overkill.
From a C++ background, what I am asking is that is there an equivalent of std::result_of in Julia. The question is almost the same as this one but the language is Julia.
After some research I see that Julia allows for return value of different types in one function, where the type inference looks very hard. For example,
foo(x::Int64) = x == 1 ? 1 : 1.0
The return type can now be Int64 or Float64, depending on the input value. Nevertheless, in this case, I am still wondering if there are some macro tricks that can deduce that the return type is of Union{ Int64, Float64 }?
To summarize, my questions are:
Fundamentally, is it possible to get the function return type by only supplying argument types in Julia?
If 1 is not possible, for functions that have one deterministic return type (as in the 1st example), is it possible to get the return type unevaluated?
(Might be unrelated with what I want but I think it can boost my understanding) When Julia codes are compiled, are the return types of the functions known? Or is the type information only determined at run time?
1) Yes, Base.return_types(foo, (Int64,)) will return an array containing the return type you're asking for, i.e. Union{ Int64, Float64 } in this case. If you drop the second argument, a tuple specifying the input argument types, you'll get all possible infered return types.
It should be noted, however, that the compiler might at any point decide to return Any or any other correct but imprecise return type.
2) see 1) ?
3) For given input argument types, the compiler tries to infer the return type at compile-time. If multiple are possible, it will infer a Union type. If it fails completely, or there are too many different return types, it will infer Any as the return type. In the latter cases, the actual return type is only known at runtime.
Demonstration of 1):
julia> foo(x::Float32) = x
foo (generic function with 1 methods)
julia> foo(x::Int32) = x
foo (generic function with 2 methods)
julia> foo(x::Int64) = x == 1 ? 1 : 1.0
foo (generic function with 3 methods)
julia> Base.return_types(foo)
3-element Array{Any,1}:
Union{Float64, Int64}
Int32
Float32
julia> Base.return_types(foo, (Int64,))
1-element Array{Any,1}:
Union{Float64, Int64}
julia> Base.return_types(foo, (Int32,))
1-element Array{Any,1}:
Int32
Best be explained by an example:
I define a type
type myType
α::Float64
β::Float64
end
z = myType( 1., 2. )
Then suppose that I want to pass this type as an argument to a function:
myFunc( x::Vector{Float64}, m::myType ) =
x[1].^m.α+x[2].^m.β
Is there a way to pass myType so that I can actually use it in the body of the function in a "cleaner" fashion as follows:
x[1].^α+x[2].^β
Thanks for any answer.
One way is to use dispatch to a more general function:
myFunc( x::Vector{Float64}, α::Float64, β::Float64) = x[1].^α+x[2].^β
myFunc( x::Vector{Float64}, m::myType = myFunc(x,m.α,m.β)
Or if your functions are longer, you may want to use Parameters.jl's #unpack:
function myFunc( x::Vector{Float64}, m::myType )
#unpack m: α,β #now those are defined
x[1].^α+x[2].^β
end
The overhead of unpacking is small because you're not copying, it's just doing a bunch of α=m.α which is just making an α which points to m.α. For longer equations, this can be a much nicer form if you have many fields and use them in long calculations (for reference, I use this a lot in DifferentialEquations.jl).
Edit
There's another way as noted in the comments. Let me show this. You can define your type (with optional kwargs) using the #with_kw macro from Parameters.jl. For example:
using Parameters
#with_kw type myType
α::Float64 = 1.0 # Give a default value
β::Float64 = 2.0
end
z = myType() # Generate with the default values
Then you can use the #unpack_myType macro which is automatically made by the #with_kw macro:
function myFunc( x::Vector{Float64}, m::myType )
#unpack_myType m
x[1].^α+x[2].^β
end
Again, this only has the overhead of making the references α and β without copying, so it's pretty lightweight.
You could add this to the body of your function:
(α::Float64, β::Float64) = (m.α, m.β)
UPDATE: My original answer was wrong for a subtle reason, but I thought it was a very interesting bit of information so rather than delete it altogether, I'm leaving it with an explanation on why it's wrong. Many thanks to Fengyang for pointing out the global scope of eval! (as well as the use of $ in an Expr context!)
The original answer suggested that:
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
would return a list comprehension which had assignment side-effects, since it conceptually would result in something like [α=1., β=2., etc]. The assumption was that this assignment would be within local scope. However, as pointed out, eval is always assessed at global scope, therefore the above one-liner does not do what it's meant to. Example:
julia> type MyType
α::Float64
β::Float64
end
julia> function myFunc!(x::Vector{Float64}, m::MyType)
α=5.; β=6.;
[eval( parse( string( i,"=",getfield( m,i)))) for i in fieldnames( m)]
x[1] = α; x[2] = β; return x
end;
julia> myFunc!([0.,0.],MyType(1., 2.))
2-element Array{Float64,1}:
5.0
6.0
julia> whos()
MyType 124 bytes DataType
myFunc 0 bytes #myFunc
α 8 bytes Float64
β 8 bytes Float64
I.e. as you can see, the intention was for the local variables α and β to be overwritten, but they didn't; eval placed α and β variables at global scope instead. As a matlab programmer I naively assumed that eval() was conceptually equivalent to Matlab, without actually checking. Turns out it's more similar to the evalin('base',...) command.
Thanks again to Fengyand for giving another example of why the phrase "parse and eval" seems to have about the same effect on Julia programmers as the word "it" on the knights who until recently said "NI". :)