How do I make the following code into nested list comprehension?
node_x = 5
node_y = 5
node_z = 5
xyz = Matrix(undef, node_x*node_y*node_z,3)
ii = 0
dx = 1.0
for k in 0:node_z-1
for j in 0:node_y-1
for i in 0:node_x-1
x = i * dx
y = j * dx
z = k * dx
ii += 1
#println([x, y, z])
xyz[ii, 1] = x
xyz[ii, 2] = y
xyz[ii, 3] = z
end
end
end
In python and numpy, I can write such as following codes.
xyz = np.array([[i*dx, j*dx, k*dx] for k in range(node_z) for j in range(node_y) for i in range(node_x)])
Comprehensions can be nested just the same, it's just range that is a bit different, but in your case there is the start:end syntactic sugar:
julia> [[i*dx, j*dx, k*dx] for k in 1:node_z for j in 1:node_y for i in 1:node_x]
125-element Vector{Vector{Float64}}:
[1.0, 1.0, 1.0]
[2.0, 1.0, 1.0]
⋮
[4.0, 5.0, 5.0]
[5.0, 5.0, 5.0]
To get the same array as your Python example, you'd have to permute the dimensions of the 3-element vectors and concatenate the list:
julia> vcat(([i*dx j*dx k*dx] for k in 1:node_z for j in 1:node_y for i in 1:node_x)...)
125×3 Matrix{Float64}:
1.0 1.0 1.0
2.0 1.0 1.0
⋮
4.0 5.0 5.0
5.0 5.0 5.0
Related
is there any equivalent to scipy.sparse.linalg.spsolve in Julia? Here's the description of the function in Python.
In [59]: ?spsolve
Signature: spsolve(A, b, permc_spec=None, use_umfpack=True)
Docstring:
Solve the sparse linear system Ax=b, where b may be a vector or a matrix.
I couldn't find this in Julia's LinearAlgebra and SparseArrays. Is there anything I miss or any alternatives?
Thanks
EDIT
For example:
In [71]: A = sparse.csc_matrix([[3, 2, 0], [1, -1, 0], [0, 5, 1]], dtype=float)
In [72]: B = sparse.csc_matrix([[2, 0], [-1, 0], [2, 0]], dtype=float)
In [73]: spsolve(A, B).data
Out[73]: array([ 1., -3.])
In [74]: spsolve(A, B).toarray()
Out[74]:
array([[ 0., 0.],
[ 1., 0.],
[-3., 0.]])
In Julia, with \ operator
julia> A = Float64.(sparse([3 2 0; 1 -1 0; 0 5 1]))
3×3 SparseMatrixCSC{Float64,Int64} with 6 stored entries:
[1, 1] = 3.0
[2, 1] = 1.0
[1, 2] = 2.0
[2, 2] = -1.0
[3, 2] = 5.0
[3, 3] = 1.0
julia> B = Float64.(sparse([2 0; -1 0; 2 0]))
3×2 SparseMatrixCSC{Float64,Int64} with 3 stored entries:
[1, 1] = 2.0
[2, 1] = -1.0
[3, 1] = 2.0
julia> A \ B
ERROR: MethodError: no method matching ldiv!(::SuiteSparse.UMFPACK.UmfpackLU{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64})
Closest candidates are:
ldiv!(::Number, ::AbstractArray) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/generic.jl:236
ldiv!(::SymTridiagonal, ::Union{AbstractArray{T,1}, AbstractArray{T,2}} where T; shift) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/tridiag.jl:208
ldiv!(::LU{T,Tridiagonal{T,V}}, ::Union{AbstractArray{T,1}, AbstractArray{T,2}} where T) where {T, V} at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/lu.jl:588
...
Stacktrace:
[1] \(::SuiteSparse.UMFPACK.UmfpackLU{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/LinearAlgebra/src/factorization.jl:99
[2] \(::SparseMatrixCSC{Float64,Int64}, ::SparseMatrixCSC{Float64,Int64}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/SparseArrays/src/linalg.jl:1430
[3] top-level scope at REPL[81]:1
Yes, it's the \ function.
julia> using SparseArrays, LinearAlgebra
julia> A = sprand(Float64, 20, 20, 0.01) + I # just adding the identity matrix so A is non-singular.
julia> typeof(A)
SparseMatrixCSC{Float64,Int64}
julia> v = rand(20);
julia> A \ v
20-element Array{Float64,1}:
0.5930744938331236
0.8726507741810358
0.6846427450637211
0.3135234897986168
0.8366321472466727
0.11338490488638651
0.3679058951515244
0.4931583108292607
0.3057947282994271
0.27481281228206955
0.888942874188458
0.905356044150361
0.17546911165214607
0.13636389619386557
0.9607381212005248
0.2518153541168824
0.6237205353883974
0.6588050295549153
0.14748809413104935
0.9806131247053784
Edit in response to question edit:
If you want v here to instead be a sparse matrix B, then we can proceed by using the QR decomposition of B (note that cases where B is truly sparse are rare:
function myspsolve(A, B)
qrB = qr(B)
Q, R = qrB.Q, qrB.R
R = [R; zeros(size(Q, 2) - size(R, 1), size(R, 2))]
A\Q * R
end
now:
julia> A = Float64.(sparse([3 2 0; 1 -1 0; 0 5 1]))
3×3 SparseMatrixCSC{Float64,Int64} with 6 stored entries:
[1, 1] = 3.0
[2, 1] = 1.0
[1, 2] = 2.0
[2, 2] = -1.0
[3, 2] = 5.0
[3, 3] = 1.0
julia> B = Float64.(sparse([2 0; -1 0; 2 0]))
3×2 SparseMatrixCSC{Float64,Int64} with 3 stored entries:
[1, 1] = 2.0
[2, 1] = -1.0
[3, 1] = 2.0
julia> mysolve(A, B)
3×2 Array{Float64,2}:
0.0 0.0
1.0 0.0
-3.0 0.0
and we can test to make sure we did it right:
julia> mysolve(A, B) ≈ A \ collect(B)
true
I have seen it a few times where someone has a situation where they want to put two for loops on the same line nested in one another.
Just to confirm, is this possible in Julia and if so what does it look like? Thanks!
Correct, Julia allows you to tersely express nested for loops.
As an example, consider filling in a 3x3 matrix in column order:
julia> xs = zeros(3,3)
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> let a = 1
for j in 1:3, i in 1:3
xs[i,j] = a
a += 1
end
end
julia> xs
3×3 Array{Float64,2}:
1.0 4.0 7.0
2.0 5.0 8.0
3.0 6.0 9.0
The above loop is equivalent to this more verbose version:
julia> let a = 1
for j in 1:3
for i in 1:3
xs[i,j] = a
a += 1
end
end
end
This syntax is even supported for higher dimensions(!):
julia> for k in 1:3, j in 1:3, i in 1:3
#show (i, j, k)
end
I have a xy-grid with two vector fields u and v. I represent vector fields as Array{Float64, 3} with dimensions nx × ny × 2.
I would like to have dot product u.v as a scalar field (Array{Float64,2} with dimensions nx×ny). What is the best way to achieve that?
It would be perfect to have something like dot(u,v,3), where 3 is the dimension over which the dot product is taken.
nx, ny = 3, 4
u = Array{Float64,3}(rand(0:1, nx, ny, 2))
#[0.0 1.0 0.0 1.0; 1.0 0.0 1.0 0.0; 1.0 0.0 1.0 0.0]
#[1.0 0.0 1.0 0.0; 1.0 0.0 0.0 1.0; 1.0 0.0 1.0 1.0]
v = Array{Float64,3}(rand(0:1, nx, ny, 2))
#[1.0 1.0 1.0 1.0; 0.0 1.0 0.0 0.0; 0.0 1.0 1.0 0.0]
#[1.0 0.0 0.0 0.0; 0.0 1.0 0.0 0.0; 1.0 1.0 1.0 0.0]
[dot(u[i,j,:], v[i,j,:]) for i in 1:nx, j in 1:ny]
3×4 Array{Float64,2}:
1.0 1.0 0.0 1.0
0.0 0.0 0.0 0.0
1.0 0.0 2.0 0.0
sum(u.*v,3) is both (reasonably) fast and short.
To explicity get a matrix you can squeeze the third dimension like so squeeze(sum(u.*v,3), 3)
Update: Of course, this has allocations and is not the best answer if speed is everything. In this case, see #DNF's direct loop implementation which is basically as fast as you can get it.
squeeze(sum(u .* v, 3), 3) is clean and simple, but if you need more speed, this is approximately 20x times faster on my pc:
a = fill(zero(eltype(u)), nx, ny)
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
Edit: For easier benchmarking comparison, this try this code:
function dotsum3(u, v)
a = fill(zero(eltype(u)), size(u, 1), size(u, 2))
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
return a
end
Also note that, if using #inbounds, one should probably explicitely check that the sizes of u and v are compatible.
Another version is
a = copy(view(u,:,:,1))
a .*= view(v,:,:,1)
a .+= #views u[:,:,2].*v[:,:,2]
Which is easily extensible to arbitrary 3rd dimension length. The benchmarks from my machine:
julia> using BenchmarkTools
julia> function f1(u,v)
a = fill(zero(eltype(u)), nx, ny)
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
return a
end
f1 (generic function with 1 method)
julia> f2(u,v) = squeeze(sum(u .* v, 3), 3)
f2 (generic function with 1 method)
julia> function f3(u,v)
a = copy(view(u,:,:,1))
a .*= view(v,:,:,1)
a .+= #views u[:,:,2].*v[:,:,2]
return a
end
f3 (generic function with 1 method)
julia> nx, ny = 3, 4
(3, 4)
julia> u = Array{Float64,3}(rand(0:1, nx, ny, 2));
julia> v = Array{Float64,3}(rand(0:1, nx, ny, 2));
Timing results:
julia> #btime f1($u,$v); # DNF suggestion
1.016 μs (1 allocation: 176 bytes)
julia> #btime f2($u,$v); # original answer
1.263 μs (14 allocations: 816 bytes)
julia> #btime f3($u,$v); # this answer
168.591 ns (5 allocations: 432 bytes)
The suggested version is faster (which is logical considering memory ordering of Julia Arrays).
I would like to produce an n x 3 matrix where n is the number of pixels (width * height).
x = linspace(-1, 1, width)
y = linspace(-1, 1, height)
r = 1.0
viewDirections = [[i j 1.0] for i in x for j in y]
However, when I run this I get a:
16-element Array{Array{Float64,2},1}
and not my desired a 16x3 Array{Float64,2}. I am obviously not using comprehensions properly to construct matrices. I tried using comprehensions to create an array of tuples, but I can't then convert those tuples into a matrix.
The problem here is array comprehension will give us a nested array instead of a Matrix. This is the right behavior of comprehension, it won't do extra guesswork for us, so we need to convert the nested array to matrix manually, which can be done using vcat with splating operator(...):
julia> vcat(viewDirections...)
6×3 Array{Float64,2}:
-1.0 -1.0 1.0
-1.0 1.0 1.0
0.0 -1.0 1.0
0.0 1.0 1.0
1.0 -1.0 1.0
1.0 1.0 1.0
It seems like you're constructing homogeneous coordinates from 2D Euclidean space. Using Base.Iterators.product is a more concise and robust way to create the iterator:
julia> w = linspace(-1,1,3)
-1.0:1.0:1.0
julia> h = linspace(-1,1,2)
-1.0:2.0:1.0
julia> r = 1.0
1.0
julia> viewDirections = [collect(i) for i in Iterators.product(w, h, r)]
3×2 Array{Array{Float64,1},2}:
[-1.0, -1.0, 1.0] [-1.0, 1.0, 1.0]
[0.0, -1.0, 1.0] [0.0, 1.0, 1.0]
[1.0, -1.0, 1.0] [1.0, 1.0, 1.0]
julia> hcat(viewDirections...).'
6×3 Array{Float64,2}:
-1.0 -1.0 1.0
0.0 -1.0 1.0
1.0 -1.0 1.0
-1.0 1.0 1.0
0.0 1.0 1.0
1.0 1.0 1.0
Note that, the order of coordinates is different from your original version, that's because Julia is column-major, Iterators.product will iterate the rightest dimension "outestly" i.e. [[i j r] for j in y for i in x ]. If the order is important in your use case, just pay attention to it.
Here are some benchmark results when width/height goes large:
julia> w = linspace(-1,1,300)
-1.0:0.006688963210702341:1.0
julia> h = linspace(-1,1,200)
-1.0:0.010050251256281407:1.0
julia> foo(w,h,r) = hcat([collect(i) for i in Iterators.product(w, h, r)]...).'
julia> bar(w,h,r) = vcat([[i j r] for i in w for j in h]...)
julia> #btime foo($w,$h,$r);
6.172 ms (60018 allocations: 10.99 MiB)
julia> #btime bar($w,$h,$r);
11.294 ms (360028 allocations: 17.02 MiB)
I have an empty matrix initially:
m = Matrix(0, 3)
and a row that I want to add:
v = [2,3]
I try to do this:
[m v]
But I get an error
ERROR: ArgumentError: number of rows of each array must match
What's the proper way to do this?
That is because your matrix sizes don't match. Specifically v does not contain enough columns to match m. And its transposed
So this doesnt work
m = Matrix(0, 3)
v = [2,3]
m = cat(1, m, v) # or a = [m; v]
>> ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
whereas this does
m = Matrix(0, 3)
v = [2 3 4]
m = cat(1, m, v) # or m = [m; v]
>> 1x3 Array{Any,2}:
>> 2 3 4
and if you run it again it creates another row
m = cat(1, m, v) # or m = [m; v]
>> 2x3 Array{Any,2}:
>> 2 3 4
>> 2 3 4
Use the vcat (concatenate vertically) function:
help?> vcat
search: vcat hvcat VecOrMat DenseVecOrMat StridedVecOrMat AbstractVecOrMat levicivita is_valid_char #vectorize_2arg
vcat(A...)
Concatenate along dimension 1
Notice you have to transpose the vector v, ie. v', else you get a DimensionMismatch error:
julia> v = zeros(3)
3-element Array{Float64,1}:
0.0
0.0
0.0
julia> m = ones(3, 3)
3x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
julia> vcat(m, v') # '
4x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
0.0 0.0 0.0
julia> v' # '
1x3 Array{Float64,2}:
0.0 0.0 0.0
julia> vcat(m, v)
ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
in cat_t at abstractarray.jl:850
in vcat at abstractarray.jl:887
Note: the comments; # ' are there just to make syntax highlighting work well.
Isn't that Matrix creates a two-dimensional array in Julia? If you try with m =[0, 3], which creates a one-dimensional Vector for you, you can append it by [m; v].
I think using [m v] is create a two-dimensional array as well, from the Julia Document