I want to calculate a vector of a quadratic form, extracting the submatrix from 3 by 3 by 5 arrays. However, I cannot make the quadratic form using broadcasting (i.e., macro "#."). When using “for” statement, we can calculate the vector of the quadratic form. I have no idea how to conduct matrix operations using “#.” (I am reluctant to expand the quadratic form to calculate the vector.)
By contrast, the inner product is computable using “#.”.
The example code is as follows:
using LinearAlgebra
a1=[5 7 2; 2 1 5; 6 2 3]
a2=[2 7 1; 3 7 2; 1 2 3]
a3=[8 5 9; 1 1 3; 2 2 3]
a4=[2 5 6; 3 5 1; 1 1 1]
a5=[7 8 1; 5 1 3; 1 5 2]
z=cat(a1,a2,a3,a4,a5,dims=3)
##### case of inner product
x=zeros(5,3)
wz = reshape([],0)
for k in 1:5
w = hcat(z[[1],[1],k], z[2,2,k]) * hcat(z[[1],[1],k], z[[2],[2],k])'
#println(w)
wz=vcat(wz, w)
end
#. wz=convert(Float64,wz)
wz=Matrix{Float64}(wz)
x[:,3]=wz
# [inner product] same result, the 3rd column vector [26.0, 53.0, 65.0, 29.0, 50.0]
display(x)
x=zeros(5,3)
#. x[:,3] = dot(hcat(z[1,1,:],z[2,2,:]), hcat(z[1,1,:],z[2,2,:])) # ok, working
# [inner product] same result, the 3rd column vector [26.0, 53.0, 65.0, 29.0, 50.0]
display(x)
##### case of quadratic form
x=zeros(5,3)
wy = reshape([],0)
for k in 1:5
w = hcat(z[[1],[1],k], z[[2],[2],k]) * z[[1,3],[1,3],k] * hcat(z[[1],[1],k], z[[2],[2],k])'
#println(w)
wy=vcat(wy, w)
end
#. wy=convert(Float64,wy)
wy=Matrix{Float64}(wy)
x[:,3]=wy
# [quadratic form] distinct result, the 3rd column vector [168.0, 183.0, 603.0, 103.0, 359.0]
display(x)
# generating five 2 by 2 matrices, distinct result
#. dot(hcat(z[[1],[1],:],z[[2],[2],:]), z[[1,3],[1,3],:], hcat(z[[1],[1],:],z[[2],[2],:]))
# obtaining ERROR: DimensionMismatch("arrays could not be broadcast to a common size; got a dimension with lengths 2 and 5")
#. dot(hcat(z[1,1,:],z[2,2,:]), z[[1,3],[1,3],:], hcat(z[1,1,:],z[2,2,:]))
Would you mind giving helps and suggestions how to get the calculation of 3rd column vector [168.0, 183.0, 603.0, 103.0, 359.0] (which is made from the quadratic form) in the above code using "#."?
EDIT:
Perhaps the question is about specifically how to make broadcasting work in this case. If so:
#views dot.(vcat.(z[1,1,:],z[2,2,:]),getindex.(Ref(z),Ref([1,3]),Ref([1,3]),axes(z,3)),vcat.(z[1,1,:],z[2,2,:]))
should be a possible clarification. Or with the #. macro (though it doesn't seem simpler):
#. dot(vcat(z[1,1,:],z[2,2,:]),getindex($Ref(z),$Ref([1,3]),$Ref([1,3]),$axes(z,3)),vcat(z[1,1,:],z[2,2,:]))
ORIGINAL:
One way to calculate this:
[
[z[1,1,k] z[2,2,k]]*z[[1,3],[1,3],k]*[z[1,1,k] z[2,2,k]]' |> first
for k ∈ axes(z,3)
]
giving:
5-element Vector{Int64}:
168
183
603
103
359
(the |> first turns 1x1 matrix into scalar)
Option 2:
[let t = z[[1,3],[1,3],k] ; sum(z[i,i,k]*t[i,j]*z[j,j,k] for i ∈ (1,2), j ∈ (1,2)) ; end for k ∈ 1:5]
or:
[let t = z[[1,3],[1,3],k], v = [z[1,1,k],z[2,2,k]] ; dot(v,t,v) ; end for k ∈ 1:5]
or (this is pretty cool):
map((z;t=z[[1,3],[1,3]],v=[z[1,1],z[2,2]])->dot(v,t,v), eachslice(z,dims=3))
I have a boolean array (from previous computations) and I would like to select the related rows from several matrices. That is why I need the proper index array (to be reused later). This is easy in Matlab and python but I do not crock the correct julian way of doing it...
I am aware of DataFrames, but would like to find an orthodox matrix and array way of doing this.
In Matlab I would say:
n= 9; temp= 1:n; A= 1.0 + temp;
someTest= mod(temp,2) == 0; % just a substitute of a more complex case
% now I have both someTest and A!
inds= find(someTest); Anew= A(inds,:);
% I got inds (which I need)!
What I have got working is this:
n= 10; data= Array(1:n); A= 1.0 .+ data;
someTest= rem.(data,2) .== 0;
inds= [xy[2] for xy in zip(someTest,1:length(someTest)) if xy[1]]; # (*)
Anew= A[inds,:];
What I assumed is that there is some shorter way to express the above phrase. in v. 0.6 there was find() function, but I have not gotten good sense of the julia documentation yet (I am a very very newbie in this).
You can use the BitArray just directly to select the elements:
julia> A[someTest]
5-element Array{Float64,1}:
3.0
5.0
7.0
9.0
11.0
Fot your case:
julia> A[someTest,:] == A[inds,:]
true
find in 0.6 was renamed to findall in Julia 1.0.
To get inds, you can simply do the following:
inds = findall(someTest)
You do not have to compute the intermediate someTest first, which would allocate an array you do not intend to use. Instead, you can do the test with findall directly passing a predicate function.
inds = findall(x -> rem(x,2) == 0, data)
This will return indices of data for which the predicate rem(x,2) == 0 returns true. This will not allocate an intermediate array to find the indices, and should be faster.
As a side note, most of the time you do not need to materialize a range in Julia. Ranges are already iterable and indexable. They will automatically be converted to an Array when there is a need. Array(1:n) or collect(1:n) are usually redundant, and allocates more memory.
Your Matlab code doesn't work. A is just a row-vector (1x9 matrix), so when you try to do A(inds, :) you get an error:
>> Anew= A(inds,:)
Index in position 1 exceeds array bounds
(must not exceed 1).
But if you just fix that, you can solve the problem in exactly the same way in both Matlab and Julia, using either logical indices or regular ones:
Matlab (I'm making sure it's a matrix this time):
n = 9;
temp = (1:n).';
A = temp * (1:4);
inds = mod(temp,2) == 0;
>> A(inds, :) % using logical indices
ans =
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
>> A(find(inds), :) % using regular indices
ans =
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
And now, Julia:
n = 9;
temp = 1:n;
A = temp .* (1:4)'; # notice that we're transposing the opposite vector from Matlab
inds = mod.(temp, 2) .== 0; # you can use iseven.(temp) instead
julia> A[inds, :] # logical indices (BitArray)
4×4 Array{Int64,2}:
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
julia> A[findall(inds), :] # regular integer indices
4×4 Array{Int64,2}:
2 4 6 8
4 8 12 16
6 12 18 24
8 16 24 32
In this case, I would use the logical indices in both Julia and Matlab. In fact, the Matlab linter (in the editor) will tell that you should use logical indices here because it's faster. In Julia, however, there might be cases where it's more efficient to use inds = findall(iseven, temp), and just skip the logical BitArray, like #hckr says.
I'm trying to find functionality in Julia similar to MATLAB's meshgrid or ndgrid. I know Julia has defined ndgrid in the examples but when I try to use it I get the following error.
UndefVarError: ndgrid not defined
Anyone know either how to get the builtin ndgrid function to work or possibly another function I haven't found or library that provides these methods (the builtin function would be preferred)? I'd rather not write my own in this case.
Thanks!
We prefer to avoid these functions, since they allocate arrays that usually aren't necessary. The values in these arrays have such a regular structure that they don't need to be stored; they can just be computed during iteration. For example, one alternative approach is to write an array comprehension:
julia> [ 10i + j for i=1:5, j=1:5 ]
5×5 Array{Int64,2}:
11 12 13 14 15
21 22 23 24 25
31 32 33 34 35
41 42 43 44 45
51 52 53 54 55
Or, you can write for loops, or iterate over a product iterator:
julia> collect(Iterators.product(1:2, 3:4))
2×2 Array{Tuple{Int64,Int64},2}:
(1, 3) (1, 4)
(2, 3) (2, 4)
I do find sometimes it's convenient to use some function like meshgrid in numpy. It's easy to do it with list comprehension:
function meshgrid(x, y)
X = [i for i in x, j in 1:length(y)]
Y = [j for i in 1:length(x), j in y]
return X, Y
end
e.g.
x = 1:4
y = 1:3
X, Y = meshgrid(x, y)
now
julia> X
4×3 Array{Int64,2}:
1 1 1
2 2 2
3 3 3
4 4 4
julia> Y
4×3 Array{Int64,2}:
1 2 3
1 2 3
1 2 3
1 2 3
However, I did not find this makes the code run faster than using iteration. Here's what I mean:
After defining
x = 1:1000
y = x
X, Y = meshgrid(x, y)
I did benchmark on the following two functions
using Statistics
function fun1()
return mean(sqrt.(X.*X + Y.*Y))
end
function fun2()
sum = 0.0
for i in 1:1000
for j in 1:1000
sum += sqrt(i*i + j*j)
end
end
return sum / (1000*1000)
end
Here are the benchmark results:
julia> #btime fun1()
8.310 ms (19 allocations: 30.52 MiB)
julia> #btime run2()
1.671 ms (0 allocations: 0 bytes)
The meshgrid method is both significantly slower and taking more memory. Any Julia expert knows why? I understand Julia is a compiling language unlike Python so iterations won't be slower than vectorization, but I don't understand why vector(array) calculation is many times slower than iteration. (For bigger N this difference is even larger.)
Edit: After reading this post, I have the following updated version of the 'meshgrid' method. The idea is to not create a meshgrid beforehand, but to do it in the calculation via Julia's powerful elementwise array operation:
x = collect(1:1000)
y = x'
function fun1v2()
mean(sqrt.(x .* x .+ y .* y))
end
The trick here is the .+ between a size-M column array and a size-N row array which returns a M-by-N array. It does the 'meshgrid' for you. This function is nearly 3 times faster then fun1, albeit not as fast as fun2.
julia> #btime fun1v2()
3.189 ms (24 allocations: 7.63 MiB)
765.8435104896155
Above, #ChrisRackauckas suggests that the "proper way" to do this is with a lazy operator but he hadn't gotten around to it.
There is now a registered packaged with lazy ndgrid in it:
https://github.com/JuliaArrays/LazyGrids.jl
It is more general than the version in
VectorizedRoutines.jl
because it can handle vectors with different types, e.g.,
ndgrid(1:3, Float16[0:2], ["x", "y", "z"]).
There are Literate.jl examples in the docs that show the lazy performance is pretty good.
Of course lazy meshgrid is just one step away:
meshgrid(y,x) = (ndgrid_lazy(x,y)[[2,1]]...,)
I wasn't starting to understand linear recursion and then I thought I practice up on sorting algorithms and then quick sort was where I had trouble with recursion. So I decided to work with a simpler eg, a binary sum that I found online. I understand that recursion, like all function calls, are executed one # a time and not at the same time (which is what multi-threading does but is not of my concern when tracing). So I need to execute all of recursive call A BEFORE recursive call B, but I get lost in the mix. Does anyone mind tracing it completely. The e.g. I have used of size, n = 9 where elems are all 1's to keep it simple.
/**
* Sums an integer array using binary recursion.
* #param arr, an integer array
* #param i starting index
* #param n size of the array
* floor(x) is largest integer <= x
* ceil(x) is smallest integer >= x
*/
public int binarySum(int arr[], int i, int n) {
if (n == 1)
return arr[i];
return binarySum(arr, i, ceil(n/2)) + binarySum(arr,i + ceil(n/2), floor(n/2));
}
What I personally do is start with an array of size 2. There are two elements.
return binarySum(arr, i, ceil(n/2)) + binarySum(arr,i + ceil(n/2), floor(n/2)) will do nothing but split the array into 2 and add the two elements. - case 1
now, this trivial starting point will be the lowest level of the recursion for the higher cases.
now increase n = 4. the array is split into 2 : indices from 0-2 and 2-4.
now the 2 elements inside indices 0 to 2 are added in case 1 and so are the 2 elements added in indices 2-4.
Now these two results are added in this case.
Now we are able to make more sense of the recursion technique, some times understanding bottom up is easier as in this case!
Now to your question consider an array of 9 elements : 1 2 3 4 5 6 7 8 9
n = 9 => ceil(9/2) = 5, floor(9/2) = 4
Now first call (top call) of binarySum(array, 0, 9)
now n = size is not 1
hence the recursive call....
return binarySum(array, 0, 5) + binarySum(array, 5, 4)
now the first binarySum(array, 0 ,5) operates on the first 5 elements of the array and the second binarySum(array,5,4) operates on the last 4 elements of the array
hence the array division can be seen like this: 1 2 3 4 5 | 6 7 8 9
The first function finds the sum of the elements: 1 2 3 4 5
and the second function finds the sum of the elements 6 7 8 9
and these two are added together and returned as the answer to the top call!
now how does this 1+2+3+4+5 and 6+7+8+9 work? we recurse again....
so the tracing will look like
1 2 3 4 5 | 6 7 8 9
1 2 3 | 4 5 6 7 | 8 9
1 2 | 3 4 | 5 6 | 7 8 | 9
[1 | 2]___[3]___[4 5]___[6 7]___[8 9]
Till this we are fine..we are just calling the functions recursively.
But now, we hit the base case!
if (n == 1)
return arr[i];
[1 + 2]____[3]____[4 + 5]____[6 + 7]____[8 + 9]
[3 + 3] ____ [9] ____[13 + 17]
[6 + 9] [30]
[15 + 30]
[45]
which is the sum.
So for understanding see what is done to the major instance of the problem and you can be sure that the same thing is going to happen to the minor instance of the problem.
This example explains binary sum with trace in java
the trace is based on index of array , where 0 - is yours starting index and 8 is length of the array
int sum(int* arr, int p, int k) {
if (p == k)
return arr[k];
int s = (p + k) / 2;
return sum(arr, p, s) + sum(arr, s + 1, k);
}