We can reshape a regular matrix to the third dimension using:
julia> data = rand(4,2)
4×2 Array{Float64,2}:
0.89585 0.328315
0.77878 0.619666
0.232389 0.132091
0.48543 0.829476
julia> reshape(data, 4, 1, 2)
4×1×2 Array{Float64,3}:
[:, :, 1] =
0.895850499952602
0.7787804133322247
0.23238945917674037
0.4854297310447009
[:, :, 2] =
0.3283154491436233
0.6196660556881552
0.13209084702809903
0.8294762758800456
But if using CircularBuffer we get an error:
using DataStructures
julia> data = CircularBuffer{Vector{Float64}}(4)
0-element CircularBuffer{Array{Float64,1}}
julia> push!(data, rand(2))
1-element CircularBuffer{Array{Float64,1}}:
[0.0271144, 0.131345]
julia> push!(data, rand(2))
2-element CircularBuffer{Array{Float64,1}}:
[0.0271144, 0.131345]
[0.0483998, 0.384114]
julia> push!(data, rand(2))
3-element CircularBuffer{Array{Float64,1}}:
[0.0271144, 0.131345]
[0.0483998, 0.384114]
[0.856657, 0.239313]
julia> push!(data, rand(2))
4-element CircularBuffer{Array{Float64,1}}:
[0.0271144, 0.131345]
[0.0483998, 0.384114]
[0.856657, 0.239313]
[0.573953, 0.0423042]
julia> reshape(data, 4, 1, 2)
ERROR: DimensionMismatch("parent has 4 elements, which is incompatible with size (4, 1, 2)")
Stacktrace:
[1] _throw_dmrs(::Int64, ::String, ::Tuple{Int64,Int64,Int64}) at ./reshapedarray.jl:180
[2] _reshape at ./reshapedarray.jl:175 [inlined]
[3] reshape(::CircularBuffer{Array{Float64,1}}, ::Tuple{Int64,Int64,Int64}) at ./reshapedarray.jl:112
[4] reshape(::CircularBuffer{Array{Float64,1}}, ::Int64, ::Int64, ::Vararg{Int64,N} where N) at ./reshapedarray.jl:115
[5] top-level scope at none:0
Your CirclularBuffer is a Vector of size 4x1 so it has 4 elements.
Hence you need a reshaped structure with that element count:
julia> CircularBuffer <: AbstractVector
true
julia> reshape(data,2,1,2)
2×1×2 reshape(::CircularBuffer{Array{Float64,1}}, 2, 1, 2) with eltype Array{Float64,1}:
[:, :, 1] =
[0.8553997132170639, 0.823601762586583]
[0.16126832224468735, 0.11158333622955818]
[:, :, 2] =
[0.3559049470946831, 0.8110608687196386]
[0.7729569142023647, 0.9370118594277839]
Another option is to have a CircularBuffer 8x1 and then reshape it - see the Julia session below:
julia> data2 = CircularBuffer{Float64}(8);
julia> push!.(Ref(data2), rand(8));
julia> reshape(data2, 4, 1, 2)
4×1×2 reshape(::CircularBuffer{Float64}, 4, 1, 2) with eltype Float64:
[:, :, 1] =
0.016467979685045098
0.14558440901833336
0.11701214175525476
0.7006868771470229
[:, :, 2] =
0.03545592972243128
0.19139585572379736
0.5295403356035531
0.5409464215739019
I guess what you want is a CircularArrayBuffer
julia> using ReinforcementLearningCore
julia> b = CircularArrayBuffer{Float64}(4, 2)
4×0 CircularArrayBuffer{Float64,2}
julia> push!(b, rand(4));
julia> push!(b, rand(4));
julia> reshape(b, 4, 1, 2)
4×1×2 reshape(::CircularArrayBuffer{Float64,2}, 4, 1, 2) with eltype Float64:
[:, :, 1] =
0.09621058339946265
0.19652636521722577
0.14816367263437913
0.5415815617368629
[:, :, 2] =
0.38976815167466494
0.9344752986999203
0.43187275186834295
0.7951761882018082
Related
Suppose I have this Matrix:
julia> mat = [
1 2 3 4
5 6 7 8
9 8 7 6
];
Then I want to put slices of this Matrix into a 3D Array with types of SMatrix{Int64}, like below:
julia> using StaticArrays
julia> arr = Array{SMatrix{Int64}, 3}(undef, 3, 2, 3);
julia> col_idx = [1, 2, 3];
julia> foreach(x->arr[:, :, x] = mat[:, x:x+1], col_idx)
ERROR: MethodError: Cannot `convert` an object of type
Int64 to an object of type
SMatrix{Int64}
Closest candidates are:
convert(::Type{T}, ::LinearAlgebra.Factorization) where T<:AbstractArray at C:\Users\JUL\.julia\juliaup\julia-1.8.3+0.x64\share\julia\stdlib\v1.8\LinearAlgebra\src\factorization.jl:58
convert(::Type{SA}, ::Tuple) where SA<:StaticArray at C:\Users\JUL\.julia\packages\StaticArrays\x7lS0\src\convert.jl:179
convert(::Type{SA}, ::SA) where SA<:StaticArray at C:\Users\JUL\.julia\packages\StaticArrays\x7lS0\src\convert.jl:178
...
Stacktrace:
[1] setindex!
# .\array.jl:968 [inlined]
[2] macro expansion
# .\multidimensional.jl:946 [inlined]
[3] macro expansion
# .\cartesian.jl:64 [inlined]
[4] macro expansion
# .\multidimensional.jl:941 [inlined]
[5] _unsafe_setindex!(::IndexLinear, ::Array{SMatrix{Int64}, 3}, ::Matrix{Int64}, ::Base.Slice{Base.OneTo{Int64}}, ::Base.Slice{Base.OneTo{Int64}}, ::Int64)
# Base .\multidimensional.jl:953
[6] _setindex!
# .\multidimensional.jl:930 [inlined]
[7] setindex!(::Array{SMatrix{Int64}, 3}, ::Matrix{Int64}, ::Function, ::Function, ::Int64)
# Base .\abstractarray.jl:1344
[8] (::var"#5#6")(x::Int64)
# Main .\REPL[20]:1
[9] foreach(f::var"#5#6", itr::Vector{Int64})
# Base .\abstractarray.jl:2774
[10] top-level scope
# REPL[20]:1
How can I achieve it?
P.S.:
This is just a minimal and reproducible example. In the practical sense, I have a size of (10, 10, 2000) for arr and a big size for mat as well (10x2000, I guess)!
If I understood correctly, do you want an Array of SMatrices?
mat = [ 1 2 3 4
5 6 7 8
9 8 7 6 ];
using StaticArrays
col_idx = [1, 2, 3];
arr = [SMatrix{3,2}(mat[:, x:x+1]) for x in col_idx]
3-element Vector{SMatrix{3, 2, Int64, 6}}:
[1 2; 5 6; 9 8]
[2 3; 6 7; 8 7]
[3 4; 7 8; 7 6]
Then, what if I say:
julia> using StaticArrays
julia> mat = [
1 2 3 4
5 6 7 8
9 8 7 6
];
julia> arr = Array{Int64, 3}(undef, 3, 2, 3);
julia> foreach(x->arr[:, :, x] = mat[:, x:x+1], [1, 2, 3]);
julia> sarr = SArray{Tuple{3, 2, 3}}(arr)
3×2×3 SArray{Tuple{3, 2, 3}, Int64, 3, 18} with indices SOneTo(3)×SOneTo(2)×SOneTo(3):
[:, :, 1] =
1 2
5 6
9 8
[:, :, 2] =
2 3
6 7
8 7
[:, :, 3] =
3 4
7 8
7 6
julia> typeof(sarr[:, :, 1])
SMatrix{3, 2, Int64, 6} (alias for SArray{Tuple{3, 2}, Int64, 2, 6})
First, I created a regular 3D Array, then constructed a SArray based on it.
However, in the case of your practical situation, I tried the following:
julia> mat = rand(10, 2000);
julia> arr = Array{Float64, 3}(undef, 10, 2, 1999);
julia> foreach(x->arr[:, :, x] = mat[:, x:x+1], 1:1999);
julia> sarr = SArray{Tuple{10, 2, 1999}}(arr);
But it takes too much time to construct such a container. (I already canceled it, and I don't know the runtime of it.). Hence, in these cases, it's better to take #AboAmmar's advice.
Inspired by #Shayan and #AboAmmar, this answer explores using BlockArrays.jl package to construct the desired result. BlockArrays puts existing arrays into a 'meta-array'. The sub-arrays can be of SMatrix type.
In code:
using StaticArrays, BlockArrays
mat = rand(10,2000) # random demo matrix
# make all the slice SArrays
arr = [SArray{Tuple{10,2,1}, Float64, 3}(mat[:,i:i+1])
for i=1:1999]
arr = reshape(arr,1,1,1999)
# glue them into a BlockArray
bricked = mortar(arr)
After now:
julia> size(bricked)
(10, 2, 1999)
julia> bricked[:,:,25]
1×1-blocked 10×2 BlockMatrix{Float64}:
0.265972 0.258414
0.396142 0.863366
0.41708 0.648276
0.960283 0.773064
0.62513 0.268989
0.132796 0.0493077
0.844674 0.791772
0.59638 0.0769661
0.221536 0.388623
0.595742 0.50732
Hopefully this method gets the performance trade-off you wanted (or at least introduces some new ideas).
The following code works fine:
f(x) = 2*x
[1, 2, 3] |> f
However, the following code fails:
g(x) = x^2
[1, 2, 3] |> g
Closest candidates are:
^(::Union{AbstractChar, AbstractString}, ::Integer) at strings/basic.jl:718
^(::Complex{var"#s79"} where var"#s79"<:AbstractFloat, ::Integer) at complex.jl:818
^(::Complex{var"#s79"} where var"#s79"<:Integer, ::Integer) at complex.jl:820
...
Stacktrace:
[1] macro expansion
# ./none:0 [inlined]
[2] literal_pow
# ./none:0 [inlined]
[3] g(x::Vector{Int64})
# Main ./REPL[17]:1
[4] |>(x::Vector{Int64}, f::typeof(g))
# Base ./operators.jl:858
[5] top-level scope
This is not related to the pipe operator at all:
julia> [1, 2, 3] * 2
3-element Vector{Int64}:
2
4
6
julia> [1, 2, 3] ^ 2
ERROR: MethodError: no method matching ^(::Vector{Int64}, ::Int64)
If you want to apply an operation on every element in a container you should use broadcasting (see also https://julialang.org/blog/2017/01/moredots/)
julia> [1, 2, 3] .* 2
3-element Vector{Int64}:
2
4
6
julia> [1, 2, 3] .^ 2
3-element Vector{Int64}:
1
4
9
The fact that [1, 2, 3] * 2 works is because vector times scalar is a mathematical operation whereas vector raised to a scalar ([1, 2, 3] ^ 2) is not.
In addition to #fredrikekre 's answer, I would like to mention that the following works:
g(x) = x^2
[1, 2, 3] .|> g
Adding an extra dot for broadcasting does the trick and allows using the pipe operator thereby feeding collections to pipes and getting collections out.
Another way to get the broadcast operator have effect on the entire pipe is to use the #. macro:
f(x) = 2x
g(x) = x^2
#. [1, 2, 3] |> f |> g
I'm new to Julia, coming more from Matlab and Python.
I cannot understand the Julia Syntax for conditional assignment of an array of vectors. To me, it appears to be inconsistent with the conditional assignment of an array of numbers.
For an array of numbers I can do the following:
a = [1,2,3]
b = [0,1,1]
a[b.>0] .= 5
This replaces the two elements of a with the new value 5.
My actual working example is with an array with elements of type Array{Int64,1}:
a = [[1,0,0], [0,1,0], [0,0,1]]
b = [0,1,1]
a[b.>0] .= Int64[0,0,5]
This does not work, output is:
ERROR: DimensionMismatch("array could not be broadcast to match destination")
However, this line works:
a[b.>0] .= [Int64[0,0,5]]
I cant understand this, as the element-wise assignment (.=) makes even less sense to me in the latter case, as the two arrays on the left and right have different sizes.
Can someone give an explanation?
Thanks in advance.
The operation:
x .= y
tries to iterate over x and y and performs the assignment. A simple case is:
julia> x = [1,2,3,4]
4-element Array{Int64,1}:
1
2
3
4
julia> y = [12,14]
2-element Array{Int64,1}:
12
14
julia> x[[2,4]] .= y
2-element view(::Array{Int64,1}, [2, 4]) with eltype Int64:
12
14
julia> x
4-element Array{Int64,1}:
1
12
3
14
We see that the left and right hand side have 2 elements so that an in-place assignment can be performed.
Then Julia has a special rule that if a container in right hand side has length 1 it can be expanded to match the size of the left hand side (this also works in higher dimensions than 1, but let us focus on a simple case).
You you for example have:
julia> x = [1,2,3,4]
4-element Array{Int64,1}:
1
2
3
4
julia> x[[2,4]] .= 11
2-element view(::Array{Int64,1}, [2, 4]) with eltype Int64:
11
11
julia> x
4-element Array{Int64,1}:
1
11
3
11
julia> length(11)
1
julia> x[[2,4]] .= [11]
2-element view(::Array{Int64,1}, [2, 4]) with eltype Int64:
11
11
julia> x
4-element Array{Int64,1}:
1
11
3
11
julia> length([1])
1
A crucial thing to note here is that [1] and 1 behave exactly the same in this case as a number is considered like a 1-element container holding this number in broadcasting.
Now going to your examples:
a = [[1,0,0], [0,1,0], [0,0,1]]
b = [0,1,1]
a[b.>0] .= Int64[0,0,5]
fails because:
julia> length(a[b.>0])
2
julia> length(Int64[0,0,5])
3
and we see that the dimensions do not match.
However in:
a[b.>0] .= [Int64[0,0,5]]
you have:
julia> length([Int64[0,0,5]])
1
So a container having length one gets expanded.
Note, however, that most likely you do not want to do a[b.>0] .= [Int64[0,0,5]] assignment as then a will hold the same array Int64[0,0,5]. E.g.
julia> a = [[1,0,0], [0,1,0], [0,0,1]]
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[0, 1, 0]
[0, 0, 1]
julia> b = [0,1,1]
3-element Array{Int64,1}:
0
1
1
julia> a[b.>0] .= [Int64[0,0,5]]
2-element view(::Array{Array{Int64,1},1}, [2, 3]) with eltype Array{Int64,1}:
[0, 0, 5]
[0, 0, 5]
julia> a
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[0, 0, 5]
[0, 0, 5]
julia> a[2][1] = 100
100
julia> a
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[100, 0, 5]
[100, 0, 5]
and in most cases this is not what you want. A safer approach would be do to for example a for loop like this:
julia> a = [[1,0,0], [0,1,0], [0,0,1]]
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[0, 1, 0]
[0, 0, 1]
julia> b = [0,1,1]
3-element Array{Int64,1}:
0
1
1
julia> for i in axes(b, 1)
b[i] > 0 && (a[i] = Int64[0,0,5])
end
julia> a
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[0, 0, 5]
[0, 0, 5]
julia> a[2][1] = 100
100
julia> a
3-element Array{Array{Int64,1},1}:
[1, 0, 0]
[100, 0, 5]
[0, 0, 5]
and you can see that each entry of a is a distinct object.
Logical indexing only selects those elements for which the condition is true. In your case of a[b.>0], that selects two elements:
julia> a[b.>0]
2-element Array{Int64,1}:
2
3
You are attempting to assign three elements into those two locations:
julia> a[b.>0] .= [10,20,30]
ERROR: DimensionMismatch("array could not be broadcast to match destination")
What you can do is also subset the values array you're assigning with the same conditional logic to pick which two elements should be assigned:
julia> a[b.>0] .= [10,20,30][b.>0]
2-element view(::Array{Int64,1}, [2, 3]) with eltype Int64:
20
30
julia> a
3-element Array{Int64,1}:
1
20
30
The syntax a[b.>0] .= [Int64[0,0,5]] will only work if a is an Any array, and it means something completely different. It broadcasts the values array itself into all the selected locations — that is, it puts the whole array as repeated elements in a!
julia> a = Any[1,2,3]
3-element Array{Any,1}:
1
2
3
julia> a[b.>0] .= [Int64[0,0,5]]
2-element view(::Array{Any,1}, [2, 3]) with eltype Any:
[0, 0, 5]
[0, 0, 5]
julia> a
3-element Array{Any,1}:
1
[0, 0, 5]
[0, 0, 5]
When creating an array of various sized vectors(e.g. arrays) I am generating an error msg.
julia> A = [[1,2] [1,2,3] [1,4] [1] [1,5,6,7]]
ERROR: DimensionMismatch("vectors must have same lengths")
Stacktrace:
[1] hcat(::Array{Int64,1}, ::Array{Int64,1}, ::Array{Int64,1}, ::Vararg{Array{Int64,1},N} where N) at .\array.jl:1524
[2] top-level scope at none:0
Although, if I initalize an array and assign the vectors 'its okay'...
julia> A = Array{Any}(undef,5)
5-element Array{Any,1}:
#undef
#undef
#undef
#undef
#undef
pseudo code> A[i] = [x,y...]
2-element Array{Int64,1}:
1
2
julia> A
5-element Array{Any,1}:
[1, 2]
[1, 2, 3]
[1]
[1, 5]
[1, 2, 6, 4, 5]
Is there a way to initialize the array with the variously sized arrays or is Julia configured this way to prevent errors.
The space-separated syntax you're using for the outermost array is specifically for horizontal concatenation of matrices, so your code is trying to concatenate all of these vectors into a matrix, which doesn't work since they have different lengths. Use commas in the outer array like the inner one to get an array of arrays:
julia> A = [[1,2], [1,2,3], [1,4], [1], [1,5,6,7]]
5-element Array{Array{Int64,1},1}:
[1, 2]
[1, 2, 3]
[1, 4]
[1]
[1, 5, 6, 7]
Given a vector z = [1, 2, 3], I want to create a vector of vectors with all circular permutations of z (i.e. zp = [[1,2,3], [3,1,2], [2,3,1]]).
I can print all elements of zp with
for i in 1:length(z)
push!(z, shift!(z)) |> println
end
How can I store the resulting permutations? Note that
zp = Vector(length(z))
for i in 1:length(z)
push!(z, shift!(z))
push!(zp, z)
end
doesn't work as it stores the same vector z 3 times in zp.
One way would just be to copy the vector before pushing it:
z = [1, 2, 3];
zp = Vector();
for i in 1:length(z)
push!(z, shift!(z))
push!(zp, copy(z))
end
gives me
julia> zp
3-element Array{Any,1}:
[2,3,1]
[3,1,2]
[1,2,3]
But I tend to prefer avoiding mutating operations when I can. So I'd instead write this as
julia> zp = [circshift(z, i) for i=1:length(z)]
3-element Array{Array{Int64,1},1}:
[3,1,2]
[2,3,1]
[1,2,3]
This seems to execute pretty quick on my machine (faster than a comprehension):
julia> z=[1,2,3]
3-element Array{Int64,1}:
1
2
3
julia> zp=Vector{typeof(z)}(length(z))
3-element Array{Array{Int64,1},1}:
#undef
#undef
#undef
julia> for i=1:length(z)
zp[i]=circshift(z,i-1)
end
julia> zp
3-element Array{Array{Int64,1},1}:
[1,2,3]
[3,1,2]
[2,3,1]
julia>