I have a 2D array of colours in Julia
using Images
white = RGB{Float32}(1, 1, 1)
green = RGB{Float32}(0.1, 1, 0.1)
blue = RGB{Float32}(0, 0.1, 1)
A = [white white;
green blue;
blue blue]
I want to turn each RGB colour into a Array{Float32, 3} in the higher dimension. This is what I tried:
B = map(A) do a
[a.r, a.g, a.b]
end
size(B) == (3, 2, 3) # (rows, cols, channels)
# => false
Instead, B is a 2D-matrix of 1D-arrays.
Does Julia have a map-like method for expanding the dimensions of a matrix?
You should use ImageCore's channelview instead:
julia> Av = channelview(A)
3×3×2 reinterpret(reshape, Float32, ::Array{RGB{Float32},2}) with eltype Float32:
[:, :, 1] =
1.0 0.1 0.0
1.0 1.0 0.1
1.0 0.1 1.0
[:, :, 2] =
1.0 0.0 0.0
1.0 0.1 0.1
1.0 1.0 1.0
The color channel is the first dimension (the fastest dimension). You can check that by setting some values and seeing the impact on the original, since Av is a view of A:
julia> Av[1,2,1] = -5
-5
julia> Av[1,2,2] = -10
-10
julia> A
3×2 Array{RGB{Float32},2} with eltype RGB{Float32}:
RGB{Float32}(1.0,1.0,1.0) RGB{Float32}(1.0,1.0,1.0)
RGB{Float32}(-5.0,1.0,0.1) RGB{Float32}(-10.0,0.1,1.0)
RGB{Float32}(0.0,0.1,1.0) RGB{Float32}(0.0,0.1,1.0)
In both cases we tweaked the red channel, because of using 1 as the first index.
map doesn't work because it is elementwise, so it can only allocate an output with the same size as the input. It's not difficult to allocate your output array in this case:
function RGBtoT_loop(x::Array{RGB{T}, N}) where {T,N}
# allocate output array without any (valid) instances
result = Array{T, N+1}(undef, size(x)..., 3)
# write RGB values into output array
for i in CartesianIndices(x)
result[i, 1] = x[i].r
result[i, 2] = x[i].g
result[i, 3] = x[i].b
end
result
end
EDIT: I figured out how to do the same thing by broadcasting the getfield method where getfield(RGBvalue, 1) is equivalent to RGBvalue.r. Interestingly, ndims(A) and thus the Tuple of ones is calculated at compile-time from A's type parameters, so this method ends up type-stable and only allocates 1 thing at run-time: the result array.
RGBtoT(x) = getfield.(x, reshape(1:3, ntuple(i->1, ndims(x))..., 3) )
Related
I have a complex matrix (i.e. Array{Complex{Float64},2}) in julia that I would like to upsample in one dimension.
My equivalent python code is:
data_package['time_series'] = sp.signal.resample(data_package['time_series'] .astype('complex64'), data_package['time_series'].shape[1]*upsample_factor, axis=1)
A resample() function can be found in DSP.jl. But it only works on Vectors, so one has to apply it manually along the desired dimension. One possible way looks like this (resampling along the second dimension, with a new rate of 2):
julia> using DSP
julia> test = reshape([1.0im, 2.0im, 3.0im, 4., 5., 6.], 3, 2)
3×2 Matrix{ComplexF64}:
0.0+1.0im 4.0+0.0im
0.0+2.0im 5.0+0.0im
0.0+3.0im 6.0+0.0im
julia> newRate = 2
2
julia> up = [resample(test[:, i], newRate) for i in 1:size(test, 2)] # gives a vector of vectors
2-element Vector{Vector{ComplexF64}}:
[0.0 + 0.9999042566881922im, 0.0 + 1.2801955476665785im, 0.0 + 1.9998085133763843im, 0.0 + 2.968204475861045im, 0.0 + 2.9997127700645763im]
[3.9996170267527686 + 0.0im, 4.466495565312296 + 0.0im, 4.999521283440961 + 0.0im, 6.154504493506763 + 0.0im, 5.9994255401291525 + 0.0im]
julia> cat(up..., dims = 2) # fuse to matrix
5×2 Matrix{ComplexF64}:
0.0+0.999904im 3.99962+0.0im
0.0+1.2802im 4.4665+0.0im
0.0+1.99981im 4.99952+0.0im
0.0+2.9682im 6.1545+0.0im
0.0+2.99971im 5.99943+0.0im
Please consider the package FFTResampling.jl
The method is based on the FFT, assuming periodic and band-limited input.
I would like to execute the following code, which works perfectly well when I type every line into my Julia console on Windows 10, but throws an error because of the mismatching type LinearAlgebra.Adjoint{Float64,Array{Float64,2}} (my subsequent code expects Array{Float64,2}).
This is the code:
x = [0.2, 0.1, 0.2]
y = [-0.5 0.0 0.5]
fx = x * y
fy = fx'
return fx::Array{Float64,2}, fy::Array{Float64,2}
There is a TypeError, because fy seems to be of type LinearAlgebra.Adjoint{Float64,Array{Float64,2}} instead of Array{Float64,2}.
How can I do a transpose and get a "normal" Array{Float64,2} object ?
And why does this work when I type every line into my Julia console, but does not when I run the file via include("myfile.jl") ?
Use collect to have a copy of actual data rather than a transformed view of the original (note that this rule applies to many other similar situations):
julia> x = [0.2, 0.1, 0.2];
julia> y = [-0.5 0.0 0.5];
julia> fx = x * y
3×3 Array{Float64,2}:
-0.1 0.0 0.1
-0.05 0.0 0.05
-0.1 0.0 0.1
julia> fy = fx'
3×3 LinearAlgebra.Adjoint{Float64,Array{Float64,2}}:
-0.1 -0.05 -0.1
0.0 0.0 0.0
0.1 0.05 0.1
julia> fy = collect(fx')
3×3 Array{Float64,2}:
-0.1 -0.05 -0.1
0.0 0.0 0.0
0.1 0.05 0.1
To get a normal Matrix{Float64} use:
fy = permutedims(fx)
or
fy = Matrix(fx')
Those two are not 100% equivalent in general as fx' is a recursive adjoint operation (conjugate transpose), while permutedims is a non-recursive transpose, but in your case they will give the same result.
What does recursive adjoint mean exactly?
recursive: the conjugate transpose is applied recursively to all entries of the array (in your case you have array of numbers and transpose of a number is the same number so this does not change anything);
adjoint: if you would have complex numbers then the operation would return their complex conjugates (in your case you have real numbers so this does not change anything);
Here is an example when both things matter:
julia> x = [[im, -im], [1-im 1+im]]
2-element Array{Array{Complex{Int64},N} where N,1}:
[0+1im, 0-1im]
[1-1im 1+1im]
julia> permutedims(x)
1×2 Array{Array{Complex{Int64},N} where N,2}:
[0+1im, 0-1im] [1-1im 1+1im]
julia> Matrix(x')
1×2 Array{AbstractArray{Complex{Int64},N} where N,2}:
[0-1im 0+1im] [1+1im; 1-1im]
However, unless you really need to you do not have to do it if you really need to get a conjugate transpose of your data. It is enough to change type assertion to
return fx::Array{Float64,2}, fy::AbstractArray{Float64,2}
or
return fx::Matrix{Float64}, fy::AbstractMatrix{Float64}
Conjugate transpose was designed to avoid unnecessary allocation of data and most of the time this will be more efficient for you (especially with large matrices).
Finally the line:
return fx::Array{Float64,2}, fy::Array{Float64,2}
throws an error also in the Julia command line (not only when run from a script).
I can never remember how to do this this.
How can go
from a Vector (size (n1)) to a Column Matrix (size (n1,1))?
or from a Matrix (size (n1,n2)) to a Array{T,3} (size (n1,n2,1))?
or from a Array{T,3} (size (n1,n2,n3)) to a Array{T,4} (size (n1,n2,n3, 1))?
and so forth.
I want to know to take Array and use it to define a new Array with an extra singleton trailing dimension.
I.e. the opposite of squeeze
You can do this with reshape.
You could define a method for this:
add_dim(x::Array) = reshape(x, (size(x)...,1))
julia> add_dim([3;4])
2×1 Array{Int64,2}:
3
4
julia> add_dim([3;4])
2×1 Array{Int64,2}:
3
4
julia> add_dim([3 30;4 40])
2×2×1 Array{Int64,3}:
[:, :, 1] =
3 30
4 40
julia> add_dim(rand(4,3,2))
4×3×2×1 Array{Float64,4}:
[:, :, 1, 1] =
0.483307 0.826342 0.570934
0.134225 0.596728 0.332433
0.597895 0.298937 0.897801
0.926638 0.0872589 0.454238
[:, :, 2, 1] =
0.531954 0.239571 0.381628
0.589884 0.666565 0.676586
0.842381 0.474274 0.366049
0.409838 0.567561 0.509187
Another easy way other than reshaping to an exact shape, is to use cat and ndims together. This has the added benefit that you can specify "how many extra (singleton) dimensions you would like to add". e.g.
a = [1 2 3; 2 3 4];
cat(ndims(a) + 0, a) # add zero singleton dimensions (i.e. stays the same)
cat(ndims(a) + 1, a) # add one singleton dimension
cat(ndims(a) + 2, a) # add two singleton dimensions
etc.
UPDATE (julia 1.3). The syntax for cat has changed in julia 1.3 from cat(dims, A...) to cat(A...; dims=dims).
Therefore the above example would become:
a = [1 2 3; 2 3 4];
cat(a; dims = ndims(a) + 0 )
cat(a; dims = ndims(a) + 1 )
cat(a; dims = ndims(a) + 2 )
etc.
Obviously, like Dan points out below, this has the advantage that it's nice and clean, but it comes at the cost of allocation, so if speed is your top priority and you know what you're doing, then in-place reshape operations will be faster and are to be preferred.
Some time before the Julia 1.0 release a reshape(x, Val{N}) overload was added which for N > ndim(x) results in the adding of right most singleton dimensions.
So the following works:
julia> add_dim(x::Array{T, N}) where {T,N} = reshape(x, Val(N+1))
add_dim (generic function with 1 method)
julia> add_dim([3;4])
2×1 Array{Int64,2}:
3
4
julia> add_dim([3 30;4 40])
2×2×1 Array{Int64,3}:
[:, :, 1] =
3 30
4 40
julia> add_dim(rand(4,3,2))
4×3×2×1 Array{Float64,4}:
[:, :, 1, 1] =
0.0737563 0.224937 0.6996
0.523615 0.181508 0.903252
0.224004 0.583018 0.400629
0.882174 0.30746 0.176758
[:, :, 2, 1] =
0.694545 0.164272 0.537413
0.221654 0.202876 0.219014
0.418148 0.0637024 0.951688
0.254818 0.624516 0.935076
Try this
function extend_dims(A,which_dim)
s = [size(A)...]
insert!(s,which_dim,1)
return reshape(A, s...)
end
the variable extend_dim specifies which dimension to extend
Thus
extend_dims(randn(3,3),1)
will produce a 1 x 3 x 3 array and so on.
I find this utility helpful when passing data into convolutional neural networks.
I have an empty matrix initially:
m = Matrix(0, 3)
and a row that I want to add:
v = [2,3]
I try to do this:
[m v]
But I get an error
ERROR: ArgumentError: number of rows of each array must match
What's the proper way to do this?
That is because your matrix sizes don't match. Specifically v does not contain enough columns to match m. And its transposed
So this doesnt work
m = Matrix(0, 3)
v = [2,3]
m = cat(1, m, v) # or a = [m; v]
>> ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
whereas this does
m = Matrix(0, 3)
v = [2 3 4]
m = cat(1, m, v) # or m = [m; v]
>> 1x3 Array{Any,2}:
>> 2 3 4
and if you run it again it creates another row
m = cat(1, m, v) # or m = [m; v]
>> 2x3 Array{Any,2}:
>> 2 3 4
>> 2 3 4
Use the vcat (concatenate vertically) function:
help?> vcat
search: vcat hvcat VecOrMat DenseVecOrMat StridedVecOrMat AbstractVecOrMat levicivita is_valid_char #vectorize_2arg
vcat(A...)
Concatenate along dimension 1
Notice you have to transpose the vector v, ie. v', else you get a DimensionMismatch error:
julia> v = zeros(3)
3-element Array{Float64,1}:
0.0
0.0
0.0
julia> m = ones(3, 3)
3x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
julia> vcat(m, v') # '
4x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
0.0 0.0 0.0
julia> v' # '
1x3 Array{Float64,2}:
0.0 0.0 0.0
julia> vcat(m, v)
ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
in cat_t at abstractarray.jl:850
in vcat at abstractarray.jl:887
Note: the comments; # ' are there just to make syntax highlighting work well.
Isn't that Matrix creates a two-dimensional array in Julia? If you try with m =[0, 3], which creates a one-dimensional Vector for you, you can append it by [m; v].
I think using [m v] is create a two-dimensional array as well, from the Julia Document
In Matlab/Octave, I can use logical indexing to assign a value to matrix B in every location that meets a certain requirement in matrix A.
octave:1> A = [.1;.2;.3;.4;.11;.13;.14;.01;.04;.09];
octave:2> C = A < .12
C =
1
0
0
0
1
0
0
1
1
1
octave:3> B = spalloc(10,1);
octave:4> B(C) = 1
B =
Compressed Column Sparse (rows = 10, cols = 1, nnz = 5 [50%])
(1, 1) -> 1
(5, 1) -> 1
(8, 1) -> 1
(9, 1) -> 1
(10, 1) -> 1
However, if I attempt essentially the same code in Julia, the results are incorrect:
julia> A = [.1;.2;.3;.4;.11;.13;.14;.01;.04;.09];
julia> B = spzeros(10,1)
10x1 sparse matrix with 0 Float64 entries:
julia> C = A .< .12
10-element BitArray{1}:
true
false
false
false
true
false
false
true
true
true
julia> B[C] = 1
1
julia> B
10x1 sparse matrix with 5 Float64 entries:
[0 , 1] = 1.0
[0 , 1] = 1.0
[1 , 1] = 1.0
[1 , 1] = 1.0
[1 , 1] = 1.0
Have I made a mistake in the syntax somewhere, am I misunderstanding something, or is this a bug? Note, I get the correct results if I use full matrices in Julia, but since the matrix in my application is really sparse (essential boundary conditions in a finite element simulation), I would much prefer to use the sparse matrices
It looks as if sparse has some problems with BitArray's.
julia> VERSION
v"0.3.5"
julia> A = [.1;.2;.3;.4;.11;.13;.14;.01;.04;.09]
julia> B = spzeros(10,1)
julia> C = A .< .12
julia> B[C] = 1
julia> B
10x1 sparse matrix with 5 Float64 entries:
[0 , 1] = 1.0
[0 , 1] = 1.0
[1 , 1] = 1.0
[1 , 1] = 1.0
[1 , 1] = 1.0
So I get the same thing as the questioner. However when I do things "my way"
julia> B = sparse(C)
ERROR: `sparse` has no method matching sparse(::BitArray{1})
julia> B = sparse(float(C))
10x1 sparse matrix with 5 Float64 entries:
[1 , 1] = 1.0
[5 , 1] = 1.0
[8 , 1] = 1.0
[9 , 1] = 1.0
[10, 1] = 1.0
So this works if you convert the BitArray to Float. I imagine that this workaround will get you going, but it does seem that sparse should work with BitArray.
Some Additional Thoughts (Edit)
As I thought further about this, it occurs to me that one reason why there is no BitArray method for sparse() is that it is not terribly useful to implement sparse storage for an already highly compact type. Considering B and C from above:
julia> sizeof(C)
8
julia> sizeof(B)
40
So for these data, the sparse version is much larger than the original. It's actually worse than this simple (perhaps simplistic) check shows at first glance. sizeof(::BitArray{1}) appears to be the size of the entire array, but sizeof(::SparseMatrixCSC{}) shows the size of each element stored. So the real size disparity is something like 8 versus 200 bytes.
Of course if the data is sparse enough (somewhat less than 1% true), sparse storage begins to win out, despite it's high overhead.
julia> C = rand(10^6) .< 0.01
julia> B = sparse(float(C))
julia> sizeof(C)
125000
julia> sum(C)*sizeof(B)
394520
julia> C = rand(10^6) .< 0.001
julia> B = sparse(float(C))
julia> sizeof(C)
125000
julia> sum(C)*sizeof(B)
40280
So perhaps it is not an oversight that sparse() has no BitArray method. Cases where it would represent a significant space saving may be less common than one might think at first glance.