I have a example python batch processing snippet that I am attempting to recreate in JuliaLang
softmax_outputs = np.array([[ 0.7, 0.1, 0.2 ],
[ 0.1, 0.5, 0.4 ],
[ 0.02, 0.9, 0.08]])
class_targets = [0, 1, 1]
print(softmax_outuputs[[0,1,2], class_targets])
>>>
[0.7 0.5 0.9]
Julia
softmax_outputs = [ 0.7 0.1 0.2
0.1 0.5 0.4
0.02 0.9 0.08]
class_targets = [1 2 2]
println(softmax_outputs[[1,2,3],class_targets])
[0.7; 0.1; 0.02]
[0.1; 0.5; 0.9]
[0.1; 0.5; 0.9]
julia>
You can use CartesianIndex like this:
julia> softmax_outputs[CartesianIndex.([1, 2, 3], [1, 2, 2])]
3-element Vector{Float64}:
0.7
0.5
0.9
(note that [1, 2, 2] is vector not a matrix like in your code)
Alternatively (assuming you always want to index the whole range) you can write:
julia> getindex.(eachrow(softmax_outputs), [1, 2, 2])
3-element Vector{Float64}:
0.7
0.5
0.9
Finally you could use a comprehension (which is probably most natural when porting from Python):
julia> [softmax_outputs[i, j] for (i, j) in zip([1, 2, 3], [1, 2, 2])]
3-element Vector{Float64}:
0.7
0.5
0.9
Related
I have a 2D array of colours in Julia
using Images
white = RGB{Float32}(1, 1, 1)
green = RGB{Float32}(0.1, 1, 0.1)
blue = RGB{Float32}(0, 0.1, 1)
A = [white white;
green blue;
blue blue]
I want to turn each RGB colour into a Array{Float32, 3} in the higher dimension. This is what I tried:
B = map(A) do a
[a.r, a.g, a.b]
end
size(B) == (3, 2, 3) # (rows, cols, channels)
# => false
Instead, B is a 2D-matrix of 1D-arrays.
Does Julia have a map-like method for expanding the dimensions of a matrix?
You should use ImageCore's channelview instead:
julia> Av = channelview(A)
3×3×2 reinterpret(reshape, Float32, ::Array{RGB{Float32},2}) with eltype Float32:
[:, :, 1] =
1.0 0.1 0.0
1.0 1.0 0.1
1.0 0.1 1.0
[:, :, 2] =
1.0 0.0 0.0
1.0 0.1 0.1
1.0 1.0 1.0
The color channel is the first dimension (the fastest dimension). You can check that by setting some values and seeing the impact on the original, since Av is a view of A:
julia> Av[1,2,1] = -5
-5
julia> Av[1,2,2] = -10
-10
julia> A
3×2 Array{RGB{Float32},2} with eltype RGB{Float32}:
RGB{Float32}(1.0,1.0,1.0) RGB{Float32}(1.0,1.0,1.0)
RGB{Float32}(-5.0,1.0,0.1) RGB{Float32}(-10.0,0.1,1.0)
RGB{Float32}(0.0,0.1,1.0) RGB{Float32}(0.0,0.1,1.0)
In both cases we tweaked the red channel, because of using 1 as the first index.
map doesn't work because it is elementwise, so it can only allocate an output with the same size as the input. It's not difficult to allocate your output array in this case:
function RGBtoT_loop(x::Array{RGB{T}, N}) where {T,N}
# allocate output array without any (valid) instances
result = Array{T, N+1}(undef, size(x)..., 3)
# write RGB values into output array
for i in CartesianIndices(x)
result[i, 1] = x[i].r
result[i, 2] = x[i].g
result[i, 3] = x[i].b
end
result
end
EDIT: I figured out how to do the same thing by broadcasting the getfield method where getfield(RGBvalue, 1) is equivalent to RGBvalue.r. Interestingly, ndims(A) and thus the Tuple of ones is calculated at compile-time from A's type parameters, so this method ends up type-stable and only allocates 1 thing at run-time: the result array.
RGBtoT(x) = getfield.(x, reshape(1:3, ntuple(i->1, ndims(x))..., 3) )
How do I make the following code into nested list comprehension?
node_x = 5
node_y = 5
node_z = 5
xyz = Matrix(undef, node_x*node_y*node_z,3)
ii = 0
dx = 1.0
for k in 0:node_z-1
for j in 0:node_y-1
for i in 0:node_x-1
x = i * dx
y = j * dx
z = k * dx
ii += 1
#println([x, y, z])
xyz[ii, 1] = x
xyz[ii, 2] = y
xyz[ii, 3] = z
end
end
end
In python and numpy, I can write such as following codes.
xyz = np.array([[i*dx, j*dx, k*dx] for k in range(node_z) for j in range(node_y) for i in range(node_x)])
Comprehensions can be nested just the same, it's just range that is a bit different, but in your case there is the start:end syntactic sugar:
julia> [[i*dx, j*dx, k*dx] for k in 1:node_z for j in 1:node_y for i in 1:node_x]
125-element Vector{Vector{Float64}}:
[1.0, 1.0, 1.0]
[2.0, 1.0, 1.0]
⋮
[4.0, 5.0, 5.0]
[5.0, 5.0, 5.0]
To get the same array as your Python example, you'd have to permute the dimensions of the 3-element vectors and concatenate the list:
julia> vcat(([i*dx j*dx k*dx] for k in 1:node_z for j in 1:node_y for i in 1:node_x)...)
125×3 Matrix{Float64}:
1.0 1.0 1.0
2.0 1.0 1.0
⋮
4.0 5.0 5.0
5.0 5.0 5.0
is there any equivalent to scipy.sparse.linalg.spsolve in Julia? Here's the description of the function in Python.
In [59]: ?spsolve
Signature: spsolve(A, b, permc_spec=None, use_umfpack=True)
Docstring:
Solve the sparse linear system Ax=b, where b may be a vector or a matrix.
I couldn't find this in Julia's LinearAlgebra and SparseArrays. Is there anything I miss or any alternatives?
Thanks
EDIT
For example:
In [71]: A = sparse.csc_matrix([[3, 2, 0], [1, -1, 0], [0, 5, 1]], dtype=float)
In [72]: B = sparse.csc_matrix([[2, 0], [-1, 0], [2, 0]], dtype=float)
In [73]: spsolve(A, B).data
Out[73]: array([ 1., -3.])
In [74]: spsolve(A, B).toarray()
Out[74]:
array([[ 0., 0.],
[ 1., 0.],
[-3., 0.]])
In Julia, with \ operator
julia> A = Float64.(sparse([3 2 0; 1 -1 0; 0 5 1]))
3×3 SparseMatrixCSC{Float64,Int64} with 6 stored entries:
[1, 1] = 3.0
[2, 1] = 1.0
[1, 2] = 2.0
[2, 2] = -1.0
[3, 2] = 5.0
[3, 3] = 1.0
julia> B = Float64.(sparse([2 0; -1 0; 2 0]))
3×2 SparseMatrixCSC{Float64,Int64} with 3 stored entries:
[1, 1] = 2.0
[2, 1] = -1.0
[3, 1] = 2.0
julia> A \ B
ERROR: MethodError: no method matching ldiv!(::SuiteSparse.UMFPACK.UmfpackLU{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64})
Closest candidates are:
ldiv!(::Number, ::AbstractArray) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/generic.jl:236
ldiv!(::SymTridiagonal, ::Union{AbstractArray{T,1}, AbstractArray{T,2}} where T; shift) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/tridiag.jl:208
ldiv!(::LU{T,Tridiagonal{T,V}}, ::Union{AbstractArray{T,1}, AbstractArray{T,2}} where T) where {T, V} at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/lu.jl:588
...
Stacktrace:
[1] \(::SuiteSparse.UMFPACK.UmfpackLU{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/factorization.jl:99
[2] \(::SparseMatrixCSC{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/SparseArrays/src/linalg.jl:1430
[3] top-level scope at REPL[81]:1
Yes, it's the \ function.
julia> using SparseArrays, LinearAlgebra
julia> A = sprand(Float64, 20, 20, 0.01) + I # just adding the identity matrix so A is non-singular.
julia> typeof(A)
SparseMatrixCSC{Float64,Int64}
julia> v = rand(20);
julia> A \ v
20-element Array{Float64,1}:
0.5930744938331236
0.8726507741810358
0.6846427450637211
0.3135234897986168
0.8366321472466727
0.11338490488638651
0.3679058951515244
0.4931583108292607
0.3057947282994271
0.27481281228206955
0.888942874188458
0.905356044150361
0.17546911165214607
0.13636389619386557
0.9607381212005248
0.2518153541168824
0.6237205353883974
0.6588050295549153
0.14748809413104935
0.9806131247053784
Edit in response to question edit:
If you want v here to instead be a sparse matrix B, then we can proceed by using the QR decomposition of B (note that cases where B is truly sparse are rare:
function myspsolve(A, B)
qrB = qr(B)
Q, R = qrB.Q, qrB.R
R = [R; zeros(size(Q, 2) - size(R, 1), size(R, 2))]
A\Q * R
end
now:
julia> A = Float64.(sparse([3 2 0; 1 -1 0; 0 5 1]))
3×3 SparseMatrixCSC{Float64,Int64} with 6 stored entries:
[1, 1] = 3.0
[2, 1] = 1.0
[1, 2] = 2.0
[2, 2] = -1.0
[3, 2] = 5.0
[3, 3] = 1.0
julia> B = Float64.(sparse([2 0; -1 0; 2 0]))
3×2 SparseMatrixCSC{Float64,Int64} with 3 stored entries:
[1, 1] = 2.0
[2, 1] = -1.0
[3, 1] = 2.0
julia> mysolve(A, B)
3×2 Array{Float64,2}:
0.0 0.0
1.0 0.0
-3.0 0.0
and we can test to make sure we did it right:
julia> mysolve(A, B) ≈ A \ collect(B)
true
I would like to produce an n x 3 matrix where n is the number of pixels (width * height).
x = linspace(-1, 1, width)
y = linspace(-1, 1, height)
r = 1.0
viewDirections = [[i j 1.0] for i in x for j in y]
However, when I run this I get a:
16-element Array{Array{Float64,2},1}
and not my desired a 16x3 Array{Float64,2}. I am obviously not using comprehensions properly to construct matrices. I tried using comprehensions to create an array of tuples, but I can't then convert those tuples into a matrix.
The problem here is array comprehension will give us a nested array instead of a Matrix. This is the right behavior of comprehension, it won't do extra guesswork for us, so we need to convert the nested array to matrix manually, which can be done using vcat with splating operator(...):
julia> vcat(viewDirections...)
6×3 Array{Float64,2}:
-1.0 -1.0 1.0
-1.0 1.0 1.0
0.0 -1.0 1.0
0.0 1.0 1.0
1.0 -1.0 1.0
1.0 1.0 1.0
It seems like you're constructing homogeneous coordinates from 2D Euclidean space. Using Base.Iterators.product is a more concise and robust way to create the iterator:
julia> w = linspace(-1,1,3)
-1.0:1.0:1.0
julia> h = linspace(-1,1,2)
-1.0:2.0:1.0
julia> r = 1.0
1.0
julia> viewDirections = [collect(i) for i in Iterators.product(w, h, r)]
3×2 Array{Array{Float64,1},2}:
[-1.0, -1.0, 1.0] [-1.0, 1.0, 1.0]
[0.0, -1.0, 1.0] [0.0, 1.0, 1.0]
[1.0, -1.0, 1.0] [1.0, 1.0, 1.0]
julia> hcat(viewDirections...).'
6×3 Array{Float64,2}:
-1.0 -1.0 1.0
0.0 -1.0 1.0
1.0 -1.0 1.0
-1.0 1.0 1.0
0.0 1.0 1.0
1.0 1.0 1.0
Note that, the order of coordinates is different from your original version, that's because Julia is column-major, Iterators.product will iterate the rightest dimension "outestly" i.e. [[i j r] for j in y for i in x ]. If the order is important in your use case, just pay attention to it.
Here are some benchmark results when width/height goes large:
julia> w = linspace(-1,1,300)
-1.0:0.006688963210702341:1.0
julia> h = linspace(-1,1,200)
-1.0:0.010050251256281407:1.0
julia> foo(w,h,r) = hcat([collect(i) for i in Iterators.product(w, h, r)]...).'
julia> bar(w,h,r) = vcat([[i j r] for i in w for j in h]...)
julia> #btime foo($w,$h,$r);
6.172 ms (60018 allocations: 10.99 MiB)
julia> #btime bar($w,$h,$r);
11.294 ms (360028 allocations: 17.02 MiB)
Imagine you have a 3 dimensonal Julia Array A of type Float64 where size(A) = (2, 3, 3).
How could you assign blocks of this array at a time using 2-dimensional arrays? For example, say I wanted A[1, :, :] to be the identity matrix. I would think of doing something like this:
A = Array(Float64, 2, 3, 3)
A[1, :, :] = eye(3)
When I do this I get the following error:
ERROR: argument dimensions must match
in setindex! at array.jl:592
I know it is because size(A[1, :, :]) = (1, 3, 3), but I can't figure out how to either 1) get this slice to be just (3, 3) so eye(3) fits or 2) make eye(3) also be (1, 3, 3) to conform to the shape of the slice of A.
Any suggestions?
EDIT 12:51 AM PST 8-13-13
I learned two new things:
If I take a slice of A along either of the other two dimensions, the result is a 2-dimensional array instead of a 3-d array with the leading dimension being 1.
I found a temporary fix to my specific problem by doing A[1, :, :] = reshape(eye(3), (1, 3, 3)). This is sub-optimal and I am hoping for a better fix.
You may be looking for slice:
julia> sA = slice(A, 1, :, :)
3x3 SubArray of 2x3x3 Float64 Array:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> sA[:] = eye(3)
3x3 Float64 Array:
1.0 0.0 0.0
0.0 1.0 0.0
0.0 0.0 1.0
julia> A
2x3x3 Float64 Array:
[:, :, 1] =
1.0 0.0 0.0
0.0 0.0 0.0
[:, :, 2] =
0.0 1.0 0.0
0.0 0.0 0.0
[:, :, 3] =
0.0 0.0 1.0
0.0 0.0 0.0