For two objects A and B we could previously get the vector [A*A, A*B] with the code A .* [A, B]. From the deprecation warnings in Julia 0.7, it seems that the new way to do this is to use a reference of the first A. So it becomes Ref(A) .* [A,B].
It doesn't seem that there is a strong link between references and broadcasting operations. What is the link here and why is using a Reference preferred (by the deprecation warnings at least)?
Minimal Working Example
import Base.*
struct example
num::Int
end
function *(lhs::example, rhs::example)
return example(lhs.num * rhs.num)
end
A = example(2)
B = example(4)
# Previously this could be done as follows.
A .* [A, B]
# Now we need to use Refs
Ref(A) .* [A, B]
I will here refer to the main case of Ref, that is when you pass it one argument (there are also other subtypes of Ref but they are not relevant for us here).
Writing Ref(x) creates a mutable wrapper around object x. The wrapper is a very simple RefValue type defined in the following way:
mutable struct RefValue{T} <: Ref{T}
x::T
RefValue{T}() where {T} = new()
RefValue{T}(x) where {T} = new(x)
end
Now why it is useful, because Ref has defined the following utility functions:
eltype(x::Type{<:Ref{T}}) where {T} = #isdefined(T) ? T : Any
size(x::Ref) = ()
axes(x::Ref) = ()
length(x::Ref) = 1
ndims(x::Ref) = 0
ndims(::Type{<:Ref}) = 0
iterate(r::Ref) = (r[], nothing)
iterate(r::Ref, s) = nothing
IteratorSize(::Type{<:Ref}) = HasShape{0}()
which means that it can be used in broadcasting as only objects that have axes defined and support indexing can be used with broadcast.
Fortunately it is easy to avoid writing Ref(A) all the time.
Just define:
Base.broadcastable(e::example) = Ref(e)
and the machinery of broadcast will work again as Base.broadcastable is called on each argument of broadcast.
More details about customizing broadcasting can be found here https://docs.julialang.org/en/v1/manual/interfaces/#man-interfaces-broadcasting.
Related
I am seeing that Julia explicitly does NOT do classes... and I should instead embrace mutable structs.. am I going down the correct path here?? I diffed my trivial example against an official flux library but cannot gather how do I reference self like a python object.. is the cleanest way to simply pass the type as a parameter in the function??
Python
# Dense Layer
class Layer_Dense
def __init__(self, n_inputs, n_neurons):
self.weights = 0.01 * np.random.randn(n_inputs, n_neurons)
self.biases = np.zeros((1, n_neurons))
def forward(self, inputs):
pass
My JuliaLang version so far
mutable struct LayerDense
num_inputs::Int64
num_neurons::Int64
weights
biases
end
function forward(layer::LayerDense, inputs)
layer.weights = 0.01 * randn(layer.num_inputs, layer.num_neurons)
layer.biases = zeros((1, layer.num_neurons))
end
The flux libraries version of a dense layer... which looks very different to me.. and I do not know what they're doing or why.. like where is the forward pass call, is it here in flux just named after the layer Dense???
source : https://github.com/FluxML/Flux.jl/blob/b78a27b01c9629099adb059a98657b995760b617/src/layers/basic.jl#L71-L111
struct Dense{F, M<:AbstractMatrix, B}
weight::M
bias::B
σ::F
function Dense(W::M, bias = true, σ::F = identity) where {M<:AbstractMatrix, F}
b = create_bias(W, bias, size(W,1))
new{F,M,typeof(b)}(W, b, σ)
end
end
function Dense(in::Integer, out::Integer, σ = identity;
initW = nothing, initb = nothing,
init = glorot_uniform, bias=true)
W = if initW !== nothing
Base.depwarn("keyword initW is deprecated, please use init (which similarly accepts a funtion like randn)", :Dense)
initW(out, in)
else
init(out, in)
end
b = if bias === true && initb !== nothing
Base.depwarn("keyword initb is deprecated, please simply supply the bias vector, bias=initb(out)", :Dense)
initb(out)
else
bias
end
return Dense(W, b, σ)
end
This is an equivalent of your Python code in Julia:
mutable struct Layer_Dense
weights::Matrix{Float64}
biases::Matrix{Float64}
Layer_Dense(n_inputs::Integer, n_neurons::Integer) =
new(0.01 * randn(n_inputs, n_neurons),
zeros((1, n_neurons)))
end
forward(ld::Layer_Dense, inputs) = nothing
What is important here:
here I create an inner constructor only, as outer constructor is not needed; as opposed in the Flux.jl code you have linked the Dense type defines both inner and outer constructors
in python forward function does not do anything, so I copied it in Julia (your Julia code worked a bit differently); note that instead of self one should pass an instance of the object to the function as the first argument (and add ::Layer_Dense type signature so that Julia knows how to correctly dispatch it)
similarly in Python you store only weights and biases in the class, I have reflected this in the Julia code; note, however, that for performance reasons it is better to provide an explicit type of these two fields of Layer_Dense struct
like where is the forward pass call
In the code you have shared only constructors of Dense object are defined. However, in the lines below here and here the Dense type is defined to be a functor.
Functors are explained here (in general) and in here (more specifically for your use case)
Trying to understand parametric types and the new function available for inner methods. The manual states "special function available to inner constructors which created a new object of the type". See the section of the manual on new here and the section of the manual on inner constructor methods here.
Consider an inner method designed to calculate the sum of x, where x could be, say, a vector or a tuple, and is given the parametric type T. A natural thing to want is for the type of the elements of x to be inherited by their sum s. I don't seem to need new for that, correct?
struct M{T}
x::T
s
function M(x)
s = sum(x)
x,s
end
end
julia> M([1,2,3])
([1, 2, 3], 6)
julia> M([1.,2.,3.])
([1.0, 2.0, 3.0], 6.0)
julia> typeof(M([1.,2.,3.]))
Tuple{Vector{Float64}, Float64}
Edit: Correction! I intended to have the last line of the inner constructor be M(x,s)... It's still an interesting question, so I won't correct it. How does M(x,s) differ from new{typeof(x)}(x,s)?
One usage of new I have seen is in combination with typeof(), something like:
struct M{T}
x::T
s
function M(x)
s = sum(x)
new{typeof(x)}(x,s)
end
end
julia> M([1,2,3])
M{Vector{Int64}}([1, 2, 3], 6)
julia> M([1.,2.,3.])
M{Vector{Float64}}([1.0, 2.0, 3.0], 6.0)
What if wanted to constrain s to the same type as x? That is, for instance, if x is a vector, then s should be a vector (in this case, a vector of one element). How would I do that? If I replace the last line of the inner constructor with x, new{typeof(x)}(s), I get the understandable error:
MethodError: Cannot `convert` an object of type Int64 to an object of type Vector{Int64}
Here are the rules:
If you are writing an outer constructor for a type M, the constructor should return an instance of M by eventually calling the inner constructor, like this: M(<args>).
If you are writing an inner constructor, this will override the default inner constructor. So you must return an instance of M by calling new(<args>).
The new "special function" exists to allow the construction of a type that doesn't have a constructor yet. Observe the following example:
julia> struct A
x::Int
function A(x)
A(x)
end
end
julia> A(4)
ERROR: StackOverflowError:
Stacktrace:
[1] A(::Int64) at ./REPL[3]:4 (repeats 79984 times)
This is a circular definition of the constructor for A, which results in a stack overflow. You cannot pull yourself up by your bootstraps, so Julia provides the new function as a way to circumvent this problem.
You should provide the new function with a number of arguments equal to the number of fields in your struct. Note that the new function will attempt to convert the types of its inputs to match the declared types of the fields of your struct:
julia> struct B
x::Float64
B(x) = new(x)
end
julia> B(5)
B(5.0)
julia> B('a')
B(97.0)
julia> B("a")
ERROR: MethodError: Cannot `convert` an object of type String to an object
of type Float64
(The inner constructor for B above is exactly the same as the default inner constructor.)
When you're defining parametric types, the new function must be provided with a number of parameters equal to the number of parameters for your type (and in the same order), analogously to the default inner constructor for parametric types. First observe how the default inner constructor for parametric types is used:
julia> struct Foo{T}
x::T
end
julia> Foo{String}("a")
Foo{String}("a")
Now if you were writing an inner constructor for Foo, instead of writing Foo{T}(x) inside the constructor, you would replace the Foo with new, like this: new{T}(x).
You might need typeof to help define the constructor, but often you don't. Here's one way you could define your M type:
struct M{I, T}
x::I
s::T
function M(x::I) where I
s = sum(x)
new{I, typeof(s)}(x, s)
end
end
I'm using typeof here so that I could be any iterable type that returns numbers:
julia> typeof(M(1:3))
M{UnitRange{Int64},Int64}
julia> g = (rand() for _ in 1:10)
Base.Generator{UnitRange{Int64},var"#5#6"}(var"#5#6"(), 1:10)
julia> typeof(M(g))
M{Base.Generator{UnitRange{Int64},var"#5#6"},Float64}
Note that providing the parameters for your type is required when you are using new inside an inner constructor for a parametric type:
julia> struct C{T}
x::Int
C(x) = new(x)
end
ERROR: syntax: too few type parameters specified in "new{...}" around REPL[6]:1
Remember, a constructor is designed to construct something. Specifically, the constructor M is designed to construct a value of type M. Your example constructor
struct M{T}
x::T
s
function M(x)
s = sum(x)
x,s
end
end
means that the result of evaluating the expression M([1 2 3]) is a tuple, not an instance of M. If I encountered such a constructor in the wild, I'd assume it was a bug and report it. new is the internal magic that allows you to actually construct a value of type M.
It's a matter of abstraction. If you just want a tuple in the first place, then forget about the structure called M and just define a function m at module scope that returns a tuple. But if you intend to treat this as a special data type, potentially for use with dynamic dispatch but even just for self-documentation purposes, then your constructor should return a value of type M.
I want to create an analogue of the Data.Either type from Haskell in Julia. The following works in v0.5:
immutable Either{T, S}
left :: Nullable{T}
right :: Nullable{S}
end
either{T, S}(::Type{T}, ::Type{S}, value::T) = Either(Nullable{T}(value), Nullable{S}())
either{T, S}(::Type{T}, ::Type{S}, value::S) = Either(Nullable{T}(), Nullable{S}(value))
a = either(Int64, String, 1)
b = either(Int64, String, "a")
println(a)
println(b)
My question is: is it possible to make the following constructions work:
a = Either{Int64, String}(1)
b = Either{Int64, String}("a")
(this way an additional constructor function is not required).
It seems that there should be enough information to construct the object, but so far I could not persuade the compiler to accept any of the variants I tried; e.g. writing
immutable Either{T, S}
left :: Nullable{T}
right :: Nullable{S}
Either(value::T) = Either(Nullable{T}(value), Nullable{S}())
Either(value::S) = Either(Nullable{T}(), Nullable{S}(value))
end
results in
ERROR: LoadError: MethodError: no method matching Either{T,S}(::Nullable{Int64}, ::Nullable{String})
It seems that I forgot that the default constructor is called with new. This variant works:
immutable Either{T, S}
left :: Nullable{T}
right :: Nullable{S}
Either(value::T) = new(Nullable{T}(value), Nullable{S}())
Either(value::S) = new(Nullable{T}(), Nullable{S}(value))
end
a = Either{Int64, String}(1)
b = Either{Int64, String}("a")
println(a)
println(b)
Plus, since the default constructor is not exposed, you can't create an object with two non-null values, so the invariant is enforced automatically.
How to check that a type implements an interface in Julia?
For exemple iteration interface is implemented by the functions start, next, done.
I need is to have a specialization of a function depending on wether the argument type implements a given interface or not.
EDIT
Here is an example of what I would like to do.
Consider the following code:
a = [7,8,9]
f = 1.0
s = Set()
push!(s,30)
push!(s,40)
function getsummary(obj)
println("Object of type ", typeof(obj))
end
function getsummary{T<:AbstractArray}(obj::T)
println("Iterable Object starting with ", next(obj, start(obj))[1])
end
getsummary(a)
getsummary(f)
getsummary(s)
The output is:
Iterable Object starting with 7
Object of type Float64
Object of type Set{Any}
Which is what we would expect since Set is not an AbstractArray. But clearly my second method only requires the type T to implement the iteration interface.
my issue isn't only related to the iteration interface but to all interfaces defined by a set of functions.
EDIT-2
I think my question is related to
https://github.com/JuliaLang/julia/issues/5
Since we could have imagined something like T<:Iterable
Typically, this is done with traits. See Traits.jl for one implementation; a similar approach is used in Base to dispatch on Base.iteratorsize, Base.linearindexing, etc. For instance, this is how Base implements collect using the iteratorsize trait:
"""
collect(element_type, collection)
Return an `Array` with the given element type of all items in a collection or iterable.
The result has the same shape and number of dimensions as `collection`.
"""
collect{T}(::Type{T}, itr) = _collect(T, itr, iteratorsize(itr))
_collect{T}(::Type{T}, itr, isz::HasLength) = copy!(Array{T,1}(Int(length(itr)::Integer)), itr)
_collect{T}(::Type{T}, itr, isz::HasShape) = copy!(similar(Array{T}, indices(itr)), itr)
function _collect{T}(::Type{T}, itr, isz::SizeUnknown)
a = Array{T,1}(0)
for x in itr
push!(a,x)
end
return a
end
See also Mauro Werder's talk on traits.
I would define a iterability(::T) trait as follows:
immutable Iterable end
immutable NotIterable end
iterability(T) =
if method_exists(length, (T,)) || !isa(Base.iteratorsize(T), Base.HasLength)
Iterable()
else
NotIterable()
end
which seems to work:
julia> iterability(Set)
Iterable()
julia> iterability(Number)
Iterable()
julia> iterability(Symbol)
NotIterable()
you can check whether a type implements an interface via methodswith as follows:
foo(a_type::Type, an_interface::Symbol) = an_interface ∈ [i.name for i in methodswith(a_type, true)]
julia> foo(EachLine, :done)
true
but I don't quite understand the dynamic dispatch approach you mentioned in the comment, what does the generic function looks like? what's the input & output of the function? I guess you want something like this?
function foo(a_type::Type, an_interface::Symbol)
# assume bar baz are predefined
if an_interface ∈ [i.name for i in methodswith(a_type, true)]
# call function bar
else
# call function baz
end
end
or some metaprogramming stuff to generate those functions respectively at compile time?
I am learning Jason Hickey's Introduction to Objective Caml.
Here is an exercise I don't have any clue
First of all, what does it mean to implement a dictionary as a function? How can I image that?
Do we need any array or something like that? Apparently, we can't have array in this exercise, because array hasn't been introduced yet in Chapter 3. But How do I do it without some storage?
So I don't know how to do it, I wish some hints and guides.
I think the point of this exercise is to get you to use closures. For example, consider the following pair of OCaml functions in a file fun-dict.ml:
let empty (_ : string) : int = 0
let add d k v = fun k' -> if k = k' then v else d k'
Then at the OCaml prompt you can do:
# #use "fun-dict.ml";;
val empty : string -> int =
val add : ('a -> 'b) -> 'a -> 'b -> 'a -> 'b =
# let d = add empty "foo" 10;;
val d : string -> int =
# d "bar";; (* Since our dictionary is a function we simply call with a
string to look up a value *)
- : int = 0 (* We never added "bar" so we get 0 *)
# d "foo";;
- : int = 10 (* We added "foo" -> 10 *)
In this example the dictionary is a function on a string key to an int value. The empty function is a dictionary that maps all keys to 0. The add function creates a closure which takes one argument, a key. Remember that our definition of a dictionary here is function from key to values so this closure is a dictionary. It checks to see if k' (the closure parameter) is = k where k is the key just added. If it is it returns the new value, otherwise it calls the old dictionary.
You effectively have a list of closures which are chained not by cons cells by by closing over the next dictionary(function) in the chain).
Extra exercise, how would you remove a key from this dictionary?
Edit: What is a closure?
A closure is a function which references variables (names) from the scope it was created in. So what does that mean?
Consider our add function. It returns a function
fun k' -> if k = k' then v else d k
If you only look at that function there are three names that aren't defined, d, k, and v. To figure out what they are we have to look in the enclosing scope, i.e. the scope of add. Where we find
let add d k v = ...
So even after add has returned a new function that function still references the arguments to add. So a closure is a function which must be closed over by some outer scope in order to be meaningful.
In OCaml you can use an actual function to represent a dictionary. Non-FP languages usually don't support functions as first-class objects, so if you're used to them you might have trouble thinking that way at first.
A dictionary is a map, which is a function. Imagine you have a function d that takes a string and gives back a number. It gives back different numbers for different strings but always the same number for the same string. This is a dictionary. The string is the thing you're looking up, and the number you get back is the associated entry in the dictionary.
You don't need an array (or a list). Your add function can construct a function that does what's necessary without any (explicit) data structure. Note that the add function takes a dictionary (a function) and returns a dictionary (a new function).
To get started thinking about higher-order functions, here's an example. The function bump takes a function (f: int -> int) and an int (k: int). It returns a new function that returns a value that's k bigger than what f returns for the same input.
let bump f k = fun n -> k + f n
(The point is that bump, like add, takes a function and some data and returns a new function based on these values.)
I thought it might be worth to add that functions in OCaml are not just pieces of code (unlike in C, C++, Java etc.). In those non-functional languages, functions don't have any state associated with them, it would be kind of rediculous to talk about such a thing. But this is not the case with functions in functional languages, you should start to think of them as a kind of objects; a weird kind of objects, yes.
So how can we "make" these objects? Let's take Jeffrey's example:
let bump f k =
fun n ->
k + f n
Now what does bump actually do? It might help you to think of bump as a constructor that you may already be familiar with. What does it construct? it constructs a function object (very losely speaking here). So what state does that resulting object has? it has two instance variables (sort of) which are f and k. These two instance variables are bound to the resulting function-object when you invoke bump f k. You can see that the returned function-object:
fun n ->
k + f n
Utilizes these instance variables f and k in it's body. Once this function-object is returned, you can only invoke it, there's no other way for you to access f or k (so this is encapsulation).
It's very uncommon to use the term function-object, they are called just functions, but you have to keep in mind that they can "enclose" state as well. These function-objects (also called closures) are not far separated from the "real" objects in object-oriented programming languages, a very interesting discussion can be found here.
I'm also struggling with this problem. Here's my solution and it works for the cases listed in the textbook...
An empty dictionary simply returns 0:
let empty (k:string) = 0
Find calls the dictionary's function on the key. This function is trivial:
let find (d: string -> int) k = d k
Add extends the function of the dictionary to have another conditional branching. We return a new dictionary that takes a key k' and matches it against k (the key we need to add). If it matches, we return v (the corresponding value). If it doesn't match we return the old (smaller) dictionary:
let add (d: string -> int) k v =
fun k' ->
if k' = k then
v
else
d k'
You could alter add to have a remove function. Also, I added a condition to make sure we don't remove a non-exisiting key. This is just for practice. This implementation of a dictionary is bad anyways:
let remove (d: string -> int) k =
if find d k = 0 then
d
else
fun k' ->
if k' = k then
0
else
d k'
I'm not good with the terminology as I'm still learning functional programming. So, feel free to correct me.