Newbie Julia question here. Given two arrays,
W = [randn(3,2), randn(3,2)]
b = [randn(3), randn(3)]
I would like to do a "nested broadcast" along the lines of,
W .+ b = [W[1].+b[1], W[2].+b[2]]
So far the best I've been able to come up with is,
[Wi.+bi (Wi,bi) for zip(W,b)]
Coming from a Python background this feels sacrilegious. Is there a better way to do this in Julia?
You could do something like the following:
julia> W = [randn(3,2), randn(3,2)]
2-element Array{Array{Float64,2},1}:
[0.39179718902868116 -0.5387622679356612; -0.594274465053327 0.018804631512093436; -2.273706742420988 -0.4638617400026042]
[0.3249960563405678 -0.4877554417492699; 0.5036437919340767 1.3172770503034696; 0.03501532820428975 -0.2675024677340758]
julia> b = [randn(3), randn(3)]
2-element Array{Array{Float64,1},1}:
[1.2571527266220441, 0.21599608118129476, 0.21498843153804936]
[-0.528960345932853, 0.5610435189953311, -0.8636370930615718]
# A helper function which broadcasts once
julia> s_(Wi, bi) = Wi .+ bi
s_ (generic function with 1 method)
# Broadcast the helper function
julia> s_.(W, b)
2-element Array{Array{Float64,2},1}:
[1.6489499156507252 0.718390458686383; -0.3782783838720323 0.2348007126933882; -2.0587183108829388 -0.24887330846455485]
[-0.20396428959228524 -1.016715787682123; 1.0646873109294077 1.8783205692988008; -0.828621764857282 -1.1311395607956476]
As mentioned in comments, you can use one of the available user-definable infix operators to name the helper function, which allows for a nicer syntax (the particular symbol used below can be obtained by typing \oplus then Tab):
julia> ⊕(x, y) = x .+ y
⊕ (generic function with 1 method)
julia> W .⊕ b
2-element Array{Array{Float64,2},1}:
[1.6489499156507252 0.718390458686383; -0.3782783838720323 0.2348007126933882; -2.0587183108829388 -0.24887330846455485]
[-0.20396428959228524 -1.016715787682123; 1.0646873109294077 1.8783205692988008; -0.828621764857282 -1.1311395607956476]
Or - again as mentioned in comments - if you don't feel like explicitly declaring a helper function:
julia> ((Wi, bi)->Wi.+bi).(W, b)
2-element Array{Array{Float64,2},1}:
[1.6489499156507252 0.718390458686383; -0.3782783838720323 0.2348007126933882; -2.0587183108829388 -0.24887330846455485]
[-0.20396428959228524 -1.016715787682123; 1.0646873109294077 1.8783205692988008; -0.828621764857282 -1.1311395607956476]
(I personally have more difficulty reading that last version, but YMMV)
When you want to broadcast broadcast, then broadcast broadcast:
julia> broadcast.(+, W, b)
2-element Array{Array{Float64,2},1}:
[-0.7364111670769904 0.010994354421031916; -0.9010128415786036 0.22868802910609998; 1.2030371118617933 0.21305414210853912]
[0.19183885867446926 0.5362077496502086; 1.5909421118115665 0.1235808501390212; 1.5190965380769597 0.1883638848487652]
julia> [W[1].+b[1], W[2].+b[2]]
2-element Array{Array{Float64,2},1}:
[-0.7364111670769904 0.010994354421031916; -0.9010128415786036 0.22868802910609998; 1.2030371118617933 0.21305414210853912]
[0.19183885867446926 0.5362077496502086; 1.5909421118115665 0.1235808501390212; 1.5190965380769597 0.1883638848487652]
Related
The github repo for ForwardDiff.jl has some examples. I am trying to extend the example to take in addition to a vector of variables, a parameter. I cannot get it to work.
This is the example (it is short so I will show it rather than linking)
using ForwardDiff
x = rand(5)
f(x::Vector) = sum(sin, x) .+ prod(tan, x) * sum(sqrt, x);
g = x -> ForwardDiff.gradient(f, x);
g(x) # this outputs the gradient.
I want to modify this since I use functions with multiple parameters as well as variables. As a simple modification I have tried adding a single parameter.
f(x::Vector, y) = (sum(sin, x) .+ prod(tan, x) * sum(sqrt, x)) * y;
I have tried the following to no avail:
fp = x -> ForwardDiff.gradient(f, x);
fp = x -> ForwardDiff.gradient(f, x, y);
y = 1
println("test grad: ", fp(x, y))
I get the following error message:
ERROR: LoadError: MethodError: no method matching (::var"#73#74")(::Array{Float64,1}, ::Int64)
A similar question was not answered in 2017. A comment led me to here and it seems the function can only accept one input?
The target function must be unary (i.e., only accept a single argument). ForwardDiff.jacobian is an exception to this rule.
Has this changed? It seems very limited to only be able to differentiate unary functions.
A possible workaround would be to concatenate the list of variables and parameters and then just slice the returned gradient to not include the gradients with respect to the parameters, but this seems silly.
I personally think it makes sense to have this unary-only syntax for ForwardDiff. In your case, you could just pack/unpack x and y into a single vector (nominally x2 below):
julia> using ForwardDiff
julia> x = rand(5)
5-element Array{Float64,1}:
0.4304735670747184
0.3939269364431113
0.7912705403776603
0.8942024934250143
0.5724373306715196
julia> f(x::Vector, y) = (sum(sin, x) .+ prod(tan, x) * sum(sqrt, x)) * y;
julia> y = 1
1
julia> f(x2::Vector) = f(x2[1:end-1], x2[end]) % unpacking in f call
f (generic function with 2 methods)
julia> fp = x -> ForwardDiff.gradient(f, x);
julia> println("test grad: ", fp([x; y])) % packing in fp call
test grad: [2.6105844240785796, 2.741442601659502, 1.9913192377198885, 1.9382805843854594, 2.26202717745402, 3.434350946190029]
But my preference would be to explicitly name the partial derivatives differently:
julia> ∂f∂x(x,y) = ForwardDiff.gradient(x -> f(x,y), x)
∂f∂x (generic function with 1 method)
julia> ∂f∂y(x,y) = ForwardDiff.derivative(y -> f(x,y), y)
∂f∂y (generic function with 1 method)
julia> ∂f∂x(x, y)
5-element Array{Float64,1}:
2.6105844240785796
2.741442601659502
1.9913192377198885
1.9382805843854594
2.26202717745402
julia> ∂f∂y(x, y)
3.434350946190029
Here's a quick attempt at a function which takes multiple arguments, the same signature as Zygote.gradient:
julia> using ForwardDiff, Zygote
julia> multigrad(f, xs...) = ntuple(length(xs)) do i
g(y) = f(ntuple(j -> j==i ? y : xs[j], length(xs))...)
xs[i] isa AbstractArray ? ForwardDiff.gradient(g, xs[i]) :
xs[i] isa Number ? ForwardDiff.derivative(g, xs[i]) : nothing
end;
julia> f1(x,y,z) = sum(x.^2)/y;
julia> multigrad(f1, [1,2,3], 4)
([0.5, 1.0, 1.5], -0.875)
julia> Zygote.gradient(f1, [1,2,3], 4)
([0.5, 1.0, 1.5], -0.875)
For a function with several scalar arguments, this evaluates each derivative separately, and perhaps it would be more efficient to use one evaluation with some Dual(x, (dx, dy, dz)). With large-enough array arguments, ForwardDiff.gradient will already perform multiple evaluations, each with some number of perturbations (the chunk size, which you can control).
I want to assign the result of an operation to a concatenation of variables in Julia. Something similar to this (although this doesn't work):
a = zeros(5)
b = zeros(5)
a, b .= rand(10)
Is it possible? Thank you.
You are looking for "vector view concatenation". The idea here is to use SubArrays to build an Array that is actually a view into two arrays. Julia does not support this out of the box. The Julia package ChainedVectors.jl was built for this, but it is heavily outdated and only works with Julia <= 0.4.
Not everything is lost. You have two alternatives:
Use CatViews.jl
As pointed out in the comments, CatViews.jl is like ChainedVectors.jl, but works with Julia 0.6 and 0.7:
Pkg.add("CatViews")
using CatViews
a = zeros(2)
b = zeros(2)
CatView(a, b) .= rand(4)
Build your own solution
With a little work, we can get as good as
a = zeros(2)
b = zeros(2)
MyView(a, b) .= rand(4)
Julia allows you to build your own view-concatenation type. The effort required to build it scales proportional to how general you want it to be. Here is a first attempt that works with vectors:
julia> # Create a type for a view into two vectors.
julia> type MyView{T} <: AbstractVector{T}
a::Vector{T}
b:: Vector{T}
end
julia> import Base: size, getindex, setindex!
julia> # Define methods to make MyView behave properly.
julia> size(c::MyView) = size(c.a) .+ size(c.b)
julia> getindex(c::MyView, i::Int) = i <= length(c.a) ? getindex(a, i) : getindex(b, i-length(a))
julia> setindex!(c::MyView, val, i::CartesianIndex) = i[1] <= length(c.a) ? setindex!(c.a, val, i[1]) : setindex!(c.b, val, i[1]-length(a))
julia> setindex!(c::MyView, val, i::Int) = i <= length(c.a) ? setindex!(c.a, val, i) : setindex!(c.b, val, i-length(a))
julia> # Test MyView. Define two arrays and put them
julia> # into a single view.
julia> a = rand(2)
2-element Array{Float64,1}:
0.701867
0.543514
julia> b = rand(2)
2-element Array{Float64,1}:
0.00355893
0.405809
julia> MyView(a, b) .= rand(4)
4-element MyView{Float64}:
0.922896
0.969057
0.586866
0.457117
julia> # Hooray, it worked! As we see below,
julia> # the individual arrays were updated.
julia> a
2-element Array{Float64,1}:
0.922896
0.969057
julia> b
2-element Array{Float64,1}:
0.586866
0.457117
This?
a .= x[1:5]
b .= x[6:end]
You must tell Julia somehow where to split the vector.
I am trying to port some code and now I've hit a sticky bit. The original code is in C++. I need to port a union that has two 32 bit ints (in an array) and a double.
So far I have:
I1 = UInt32(56) # arbitrary integer values for example
I2 = UInt32(1045195987)
# do transforms on I1 and I2 as per the code I'm porting
A = bits(I1)
B = bits(I2)
return parse(Float64, string(A,B))
Is this the way to do it? The string operation seems expensive. Any advice appreciated.
I also come from mostly C/C++ programming, and this is what I do to handle the problem:
First, create an immutable type with two UInt32 elements:
immutable MyType
a::UInt32
b::UInt32
end
Then you can convert a vector of Float64 to that type with reinterpret.
For example:
julia> x = [1.5, 2.3]
2-element Array{Float64,1}:
1.5
2.3
julia> immutable MyType ; a::UInt32 ; b::UInt32 ; end
julia> y = reinterpret(MyType, x)
2-element Array{MyType,1}:
MyType(0x00000000,0x3ff80000)
MyType(0x66666666,0x40026666)
julia> x[1]
1.5
julia> y[1]
MyType(0x00000000,0x3ff80000)
julia> y[1].a
0x00000000
julia> y[1].b
0x3ff80000
Note: the two vectors still point to the same memory, so you can even update elements, using either type.
julia> x[1] = 10e91
1.0e92
julia> y[1].a
0xbf284e24
julia> y[1].b
0x53088ba3
julia> y[1] = MyType(1,2)
MyType(0x00000001,0x00000002)
julia> x[1]
4.2439915824e-314
Say I enter:
julia> X = randn(3,4)
3x4 Array{Float64,2}:
-0.862092 0.78568 0.140078 -0.0409951
-0.157692 0.0838577 1.38264 -0.296809
1.40242 -0.628556 -0.500275 0.258898
What functions were called to produce the output given?
Note that overloading Base.show does not seem to be sufficient to change this behaviour, so I'm unsure where to go.
julia> Base.show(io::IO, A::Array{Float64, 2}) = println("Humbug")
show (generic function with 120 methods)
julia> X
3x4 Array{Float64,2}:
-0.862092 0.78568 0.140078 -0.0409951
-0.157692 0.0838577 1.38264 -0.296809
1.40242 -0.628556 -0.500275 0.258898
Is it perhaps the case that I would have to change Base/array.jl source code and rebuild julia before such a change would work? Note the difference between this and a user defined type:
julia> type foo
x::Float32
s::ASCIIString
end
julia> ob = foo(1., "boo")
foo(1.0f0,"boo")
julia> Base.show(io::IO, someob::foo) = print("Humbug!")
show (generic function with 123 methods)
julia> ob
Humbug!
well, you should overload display():
julia> Base.display(A::Array{Float64, 2}) = println("Humbug")
display (generic function with 11 methods)
julia> X
Humbug
you can find the definition in REPL.jl.
I have a function, f. I want to add a method that takes any container of Strings. For example, I want to write a method that generates the following when needed:
f(xs::Array{String, 1}) = ...
f(xs::DataArray{String, 1}) = ...
f(xs::ITERABLE{String}) = ...
Is this possible to do in Julia's type system? Right now, I'm using a macro to write a specialized method when I need it.
#make_f(Array{String, 1})
#make_f(DataArray{String, 1})
This keeps things DRY, but it feels...wrong.
Can't you just use duck typing? I.e., just assume that you're feeding the function an object of the right type and throw an error if at some point e.g. you don't have a string in your iterable.
This should improve once you can really talk about iterables using traits; currently there is no iterable type. Scott's answer, for example, will not work with a tuple of strings, even though that is iterable.
E.g.
julia> f(x) = string(x...) # just concatenate the strings
f (generic function with 1 method)
julia> f(("a", "á"))
"aá"
julia> f(["a", "á"])
"aá"
julia> f(["a" "b"; "c" "d"]) # a matrix of strings!
"acbd"
At least in Julia 0.4, the following should work:
julia> abstract Iterable{T} <: AbstractVector{T}
julia> f{T<:Union{Vector{String},Iterable{String}}}(xs::T) = 1
f (generic function with 1 method)
julia> x = String["a", "é"]
2-element Array{AbstractString,1}:
"a"
"é"
julia> f(x)
1