I would like to check the loop for number of items and then if the items are of not expected size then fill it with 0's. For example, I have created a loop which tries to access an array's elements for a range of 10
x = range(1, 100, length=45) |> collect
n = trunc(Int, length(x)/10) + 1
s = 1
l = 10
for i in 1:n
print(x[s:l])
s += 10
l +=10
end
In the above code, last iteration doesn't print any result as the number of elements are only 5 but it expects it to be 10. Hence, I would like to know, how may i check in this loop for every iteration the number of elements and if they are not expected then fill it with 0's.
Please suggest and advise on achieving the expected operation.
Thanks!
I think that PaddedViews is what you are looking for:
julia> using PaddedViews
julia> PaddedView(0, x, (ceil(Int, length(x)/10)*10,))
50-element PaddedView(0.0, ::Vector{Float64}, (Base.OneTo(50),)) with eltype Float64:
1.0
3.25
5.5
7.75
10.0
12.25
14.5
16.75
⋮
97.75
100.0
0.0
0.0
0.0
0.0
0.0
Related
I have a complex matrix (i.e. Array{Complex{Float64},2}) in julia that I would like to upsample in one dimension.
My equivalent python code is:
data_package['time_series'] = sp.signal.resample(data_package['time_series'] .astype('complex64'), data_package['time_series'].shape[1]*upsample_factor, axis=1)
A resample() function can be found in DSP.jl. But it only works on Vectors, so one has to apply it manually along the desired dimension. One possible way looks like this (resampling along the second dimension, with a new rate of 2):
julia> using DSP
julia> test = reshape([1.0im, 2.0im, 3.0im, 4., 5., 6.], 3, 2)
3×2 Matrix{ComplexF64}:
0.0+1.0im 4.0+0.0im
0.0+2.0im 5.0+0.0im
0.0+3.0im 6.0+0.0im
julia> newRate = 2
2
julia> up = [resample(test[:, i], newRate) for i in 1:size(test, 2)] # gives a vector of vectors
2-element Vector{Vector{ComplexF64}}:
[0.0 + 0.9999042566881922im, 0.0 + 1.2801955476665785im, 0.0 + 1.9998085133763843im, 0.0 + 2.968204475861045im, 0.0 + 2.9997127700645763im]
[3.9996170267527686 + 0.0im, 4.466495565312296 + 0.0im, 4.999521283440961 + 0.0im, 6.154504493506763 + 0.0im, 5.9994255401291525 + 0.0im]
julia> cat(up..., dims = 2) # fuse to matrix
5×2 Matrix{ComplexF64}:
0.0+0.999904im 3.99962+0.0im
0.0+1.2802im 4.4665+0.0im
0.0+1.99981im 4.99952+0.0im
0.0+2.9682im 6.1545+0.0im
0.0+2.99971im 5.99943+0.0im
Please consider the package FFTResampling.jl
The method is based on the FFT, assuming periodic and band-limited input.
I have seen it a few times where someone has a situation where they want to put two for loops on the same line nested in one another.
Just to confirm, is this possible in Julia and if so what does it look like? Thanks!
Correct, Julia allows you to tersely express nested for loops.
As an example, consider filling in a 3x3 matrix in column order:
julia> xs = zeros(3,3)
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> let a = 1
for j in 1:3, i in 1:3
xs[i,j] = a
a += 1
end
end
julia> xs
3×3 Array{Float64,2}:
1.0 4.0 7.0
2.0 5.0 8.0
3.0 6.0 9.0
The above loop is equivalent to this more verbose version:
julia> let a = 1
for j in 1:3
for i in 1:3
xs[i,j] = a
a += 1
end
end
end
This syntax is even supported for higher dimensions(!):
julia> for k in 1:3, j in 1:3, i in 1:3
#show (i, j, k)
end
I have a boolean array (from previous computations) and I would like to select the related rows from several matrices. That is why I need the proper index array (to be reused later). This is easy in Matlab and python but I do not crock the correct julian way of doing it...
I am aware of DataFrames, but would like to find an orthodox matrix and array way of doing this.
In Matlab I would say:
n= 9; temp= 1:n; A= 1.0 + temp;
someTest= mod(temp,2) == 0; % just a substitute of a more complex case
% now I have both someTest and A!
inds= find(someTest); Anew= A(inds,:);
% I got inds (which I need)!
What I have got working is this:
n= 10; data= Array(1:n); A= 1.0 .+ data;
someTest= rem.(data,2) .== 0;
inds= [xy[2] for xy in zip(someTest,1:length(someTest)) if xy[1]]; # (*)
Anew= A[inds,:];
What I assumed is that there is some shorter way to express the above phrase. in v. 0.6 there was find() function, but I have not gotten good sense of the julia documentation yet (I am a very very newbie in this).
You can use the BitArray just directly to select the elements:
julia> A[someTest]
5-element Array{Float64,1}:
3.0
5.0
7.0
9.0
11.0
Fot your case:
julia> A[someTest,:] == A[inds,:]
true
find in 0.6 was renamed to findall in Julia 1.0.
To get inds, you can simply do the following:
inds = findall(someTest)
You do not have to compute the intermediate someTest first, which would allocate an array you do not intend to use. Instead, you can do the test with findall directly passing a predicate function.
inds = findall(x -> rem(x,2) == 0, data)
This will return indices of data for which the predicate rem(x,2) == 0 returns true. This will not allocate an intermediate array to find the indices, and should be faster.
As a side note, most of the time you do not need to materialize a range in Julia. Ranges are already iterable and indexable. They will automatically be converted to an Array when there is a need. Array(1:n) or collect(1:n) are usually redundant, and allocates more memory.
Your Matlab code doesn't work. A is just a row-vector (1x9 matrix), so when you try to do A(inds, :) you get an error:
>> Anew= A(inds,:)
Index in position 1 exceeds array bounds
(must not exceed 1).
But if you just fix that, you can solve the problem in exactly the same way in both Matlab and Julia, using either logical indices or regular ones:
Matlab (I'm making sure it's a matrix this time):
n = 9;
temp = (1:n).';
A = temp * (1:4);
inds = mod(temp,2) == 0;
>> A(inds, :) % using logical indices
ans =
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
>> A(find(inds), :) % using regular indices
ans =
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
And now, Julia:
n = 9;
temp = 1:n;
A = temp .* (1:4)'; # notice that we're transposing the opposite vector from Matlab
inds = mod.(temp, 2) .== 0; # you can use iseven.(temp) instead
julia> A[inds, :] # logical indices (BitArray)
4×4 Array{Int64,2}:
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
julia> A[findall(inds), :] # regular integer indices
4×4 Array{Int64,2}:
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
In this case, I would use the logical indices in both Julia and Matlab. In fact, the Matlab linter (in the editor) will tell that you should use logical indices here because it's faster. In Julia, however, there might be cases where it's more efficient to use inds = findall(iseven, temp), and just skip the logical BitArray, like #hckr says.
I would like to execute the following code, which works perfectly well when I type every line into my Julia console on Windows 10, but throws an error because of the mismatching type LinearAlgebra.Adjoint{Float64,Array{Float64,2}} (my subsequent code expects Array{Float64,2}).
This is the code:
x = [0.2, 0.1, 0.2]
y = [-0.5 0.0 0.5]
fx = x * y
fy = fx'
return fx::Array{Float64,2}, fy::Array{Float64,2}
There is a TypeError, because fy seems to be of type LinearAlgebra.Adjoint{Float64,Array{Float64,2}} instead of Array{Float64,2}.
How can I do a transpose and get a "normal" Array{Float64,2} object ?
And why does this work when I type every line into my Julia console, but does not when I run the file via include("myfile.jl") ?
Use collect to have a copy of actual data rather than a transformed view of the original (note that this rule applies to many other similar situations):
julia> x = [0.2, 0.1, 0.2];
julia> y = [-0.5 0.0 0.5];
julia> fx = x * y
3×3 Array{Float64,2}:
-0.1 0.0 0.1
-0.05 0.0 0.05
-0.1 0.0 0.1
julia> fy = fx'
3×3 LinearAlgebra.Adjoint{Float64,Array{Float64,2}}:
-0.1 -0.05 -0.1
0.0 0.0 0.0
0.1 0.05 0.1
julia> fy = collect(fx')
3×3 Array{Float64,2}:
-0.1 -0.05 -0.1
0.0 0.0 0.0
0.1 0.05 0.1
To get a normal Matrix{Float64} use:
fy = permutedims(fx)
or
fy = Matrix(fx')
Those two are not 100% equivalent in general as fx' is a recursive adjoint operation (conjugate transpose), while permutedims is a non-recursive transpose, but in your case they will give the same result.
What does recursive adjoint mean exactly?
recursive: the conjugate transpose is applied recursively to all entries of the array (in your case you have array of numbers and transpose of a number is the same number so this does not change anything);
adjoint: if you would have complex numbers then the operation would return their complex conjugates (in your case you have real numbers so this does not change anything);
Here is an example when both things matter:
julia> x = [[im, -im], [1-im 1+im]]
2-element Array{Array{Complex{Int64},N} where N,1}:
[0+1im, 0-1im]
[1-1im 1+1im]
julia> permutedims(x)
1×2 Array{Array{Complex{Int64},N} where N,2}:
[0+1im, 0-1im] [1-1im 1+1im]
julia> Matrix(x')
1×2 Array{AbstractArray{Complex{Int64},N} where N,2}:
[0-1im 0+1im] [1+1im; 1-1im]
However, unless you really need to you do not have to do it if you really need to get a conjugate transpose of your data. It is enough to change type assertion to
return fx::Array{Float64,2}, fy::AbstractArray{Float64,2}
or
return fx::Matrix{Float64}, fy::AbstractMatrix{Float64}
Conjugate transpose was designed to avoid unnecessary allocation of data and most of the time this will be more efficient for you (especially with large matrices).
Finally the line:
return fx::Array{Float64,2}, fy::Array{Float64,2}
throws an error also in the Julia command line (not only when run from a script).
Suppose we have the following 3 arrays in Julia:
5.0 3.5
6.0 3.6
7.0 3.0
5.0 4.5
6.0 4.7
8.0 3.0
5.0 4.0
6.0 3.2
8.0 4.0
I want to merge the 3 arrays in one array, by common values of the first column, summing the values of the second column. The result must be the following array:
5.0 12
6.0 11.5
7.0 3.0
8.0 7.0
I tried vcat and reduce but I don't get the pretended result. Is there a relatively simple way to code the instructions, avoiding a time-consuming code? Thank you!
There are probably many ways to do it. If you want to avoid coding you can use DataFrames package. This is not the fastest solution, but it is short.
Assume you have arrays defined as variables:
x = [5.0 3.5
6.0 3.6
7.0 3.0]
y = [5.0 4.5
6.0 4.7
8.0 3.0]
z = [5.0 4.0
6.0 3.2
8.0 4.0]
Then you can do:
using DataFrames
Matrix(aggregate(DataFrame(vcat(x,y,z)), :x1, sum))
The :x1 part is because by default first column of a DataFrame is called :x1 if you do not give an explicit name to it. In this recipe we convert matrices to a DataFrame aggregate them and convert back the result to a matrix.
Without extra package, a possible solution can be something like
function aggregate(m::Array{<:Number,2}...)
result=sortrows(vcat(m...))
n = size(result,1)
if n <= 1
return result
end
key_idx=1
key=result[key_idx,1]
for i in 2:n
if key==result[i,1]
result[key_idx,2:end] += result[i,2:end]
else
key = result[i,1]
key_idx += 1
result[key_idx,1] = key
result[key_idx,2:end] = result[i,2:end]
end
end
return result[1:key_idx,:]
end
Demo:
x = [5.0 3.5
6.0 3.6
7.0 3.0]
y = [5.0 4.5
6.0 4.7
8.0 3.0]
z = [5.0 4.0
6.0 3.2
8.0 4.0]
aggregate(x,y,z)
Prints:
4×2 Array{Float64,2}:
5.0 12.0
6.0 11.5
7.0 3.0
8.0 7.0
Note: this solution also works with any number of columns
Given the following two assumptions:
the first column of each input array is sorted,
the first column of each input array is unique,
then for most input combinations (i.e. number of input arrays, sizes of arrays), the following algorithm should significantly outperform the other answers by taking advantage of the assumptions:
function f_ag(x::Matrix{T}...)::Matrix{T} where {T<:Number}
isempty(x) && error("Empty input")
any([ size(y,2) != 2 for y in x ]) && error("Input matrices must have two columns")
length(x) == 1 && return copy(x[1]) #simple case shortcut
nxmax = [ size(y,1) for y in x ]
nxarrinds = find(nxmax .> 0)
nxrowinds = ones(Int, length(nxarrinds))
z = Tuple{T,T}[]
while !isempty(nxarrinds)
xmin = minimum(T[ x[nxarrinds[j]][nxrowinds[j], 1] for j = 1:length(nxarrinds) ])
minarrinds = Int[ j for j = 1:length(nxarrinds) if x[nxarrinds[j]][nxrowinds[j], 1] == xmin ]
rowsum = sum(T[ x[nxarrinds[k]][nxrowinds[k], 2] for k in minarrinds ])
push!(z, (xmin, rowsum))
for k in minarrinds
nxrowinds[k] += 1
end
for j = length(nxarrinds):-1:1
if nxrowinds[j] > nxmax[nxarrinds[j]]
deleteat!(nxrowinds, j)
deleteat!(nxarrinds, j)
end
end
end
return [ z[n][j] for n = 1:length(z), j = 1:2 ]
end
If assumption 2 is violated, that is, the first column is not guaranteed to be unique, you can still take advantage of the sort order, but the algorithm is going to be more complicated again since you'll need to additionally look forward on each minimum index to check for duplicates. I'm not going to put myself through that pain at this point.
Also note, you could adjust the following line:
rowsum = sum(T[ x[nxarrinds[k]][nxrowinds[k], 2] for k in minarrinds ])
to this:
rowsum = input_func(T[ x[nxarrinds[k]][nxrowinds[k], 2:end] for k in minarrinds ])
and now you can input whatever function you like, and also have any number of additional columns in your input matrices.
There are probably some additional optimizations that could be added here, eg pre-allocating z, specialized routine when there are only two input matrices, etc, but I'm not going to bother with them.