Given some A: [sps] over a sparse subdomain of a dom: domain(2), a slice A[A.domain.dim(1), k] yields the kth column as a dense 1D-array. How do I retrieve the kth n−1 dimensional slice of a sparse nD-array as a sparse (n-1)D-array?
var nv: int = 8,
D: domain(2) = {1..nv, 1..nv},
SD: sparse subdomain(D),
X: [SD] real;
SD += (1,2); X[1,2] = 1;
SD += (2,3); X[2,3] = 1;
SD += (3,1); X[3,1] = 1;
SD += (3,4); X[3,4] = 1;
SD += (4,5); X[4,5] = 1;
SD += (3,6); X[3,6] = 1;
SD += (6,8); X[6,8] = 1;
writeln(X);
writeln(X[X.domain.dim(1),2]);
returns
1.0
1.0
1.0 1.0 1.0
1.0
1.0
1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
The expectation in the case that I succeed in sparse slicing would be a single 1.0 returned with the ability to retrieve this position of that entry by calling writeln() on slice.domain.
I think that, unfortunately, you are doing the right sort of thing and that you're just running afoul of the current (as of Chapel 1.16) limitations with respect to slicing sparse domains.
Related
Please consider the following image:
I know when I use a:b in #constraint it means an array from a to b. I need to code the arrays like {a_j,b_j} in #constraint of the mentioned code. Can you please help me to code that?
I understand that you are asking about a custom indice iteration over a single constraint. This can be done in JuMP as:
using JuMP, Cbc
m = Model(Cbc.Optimizer)
#variable(m,x[1:3,1:3]>=0)
#constraint(m, con[ (i,j) in [(1,2),(2,3),(3,3)] ], x[i,j] >= 5)
Let us have a look what we got:
julia> println(m)
Feasibility
Subject to
con[(1, 2)] : x[1,2] >= 5.0
con[(2, 3)] : x[2,3] >= 5.0
con[(3, 3)] : x[3,3] >= 5.0
x[1,1] >= 0.0
x[2,1] >= 0.0
x[3,1] >= 0.0
x[1,2] >= 0.0
x[2,2] >= 0.0
x[3,2] >= 0.0
x[1,3] >= 0.0
x[2,3] >= 0.0
x[3,3] >= 0.0
A key point to understand is that--unlike AMPL and GAMS--there is no specialized
syntax for constructing and managing sets in JuMP. Instead, you can use any
available Julia syntax and datastructures.
Examples:
using JuMP
N = 10
model = Model();
#variable(model, x[1:N]);
#constraint(model, [s=1:3], sum(x[j] for j in 1:N if mod(j, s) == 0) == 1)
using JuMP
N = 10
model = Model();
#variable(model, x[1:N]);
d = [Set([1, 2, 3]), Set([2, 4, 6])]
a = [Set([3, 4]), Set([5])]
J(s) = [j for j in 1:2 if s in union(d[j], a[j])]
#constraint(model, [s=1:2], sum(x[j] for j in J(s)) == 1)
I have seen it a few times where someone has a situation where they want to put two for loops on the same line nested in one another.
Just to confirm, is this possible in Julia and if so what does it look like? Thanks!
Correct, Julia allows you to tersely express nested for loops.
As an example, consider filling in a 3x3 matrix in column order:
julia> xs = zeros(3,3)
3×3 Array{Float64,2}:
0.0 0.0 0.0
0.0 0.0 0.0
0.0 0.0 0.0
julia> let a = 1
for j in 1:3, i in 1:3
xs[i,j] = a
a += 1
end
end
julia> xs
3×3 Array{Float64,2}:
1.0 4.0 7.0
2.0 5.0 8.0
3.0 6.0 9.0
The above loop is equivalent to this more verbose version:
julia> let a = 1
for j in 1:3
for i in 1:3
xs[i,j] = a
a += 1
end
end
end
This syntax is even supported for higher dimensions(!):
julia> for k in 1:3, j in 1:3, i in 1:3
#show (i, j, k)
end
Expanding a previous question I put here before, suppose we have a large number of arrays (say 500 arrays), like the following 3 first ones
5.0 3.5
6.0 3.6
7.0 3.0
5.0 4.5
6.0 4.7
8.0 3.0
5.0 4.0
6.0 3.2
8.0 4.0
and so on, stored in one array, so that we have an array of 500 arrays of the type above. I want to merge the 500 arrays into one array, by common values of the first column, calculating the mean values of the corresponding elements of the second column. The result must be the following array:
5.0 mean of all 5's values
6.0 mean of all 6's values
7.0 mean of all 7's values
8.0 mean of all 8's values
How can I achieve that? Thank you!
Also back with a slight modification of https://stackoverflow.com/a/50842721/2001017
function aggregate(m::Array{<:Array{<:Number,2},1})
result=sortrows(vcat(m...))
n = size(result,1)
if n <= 1
return result
end
key_idx = 1
key = result[key_idx,1]
count = 1
for i in 2:n
if key == result[i,1]
result[key_idx,2:end] += result[i,2:end]
count += 1
else
result[key_idx,2:end] /= count
count = 1
key = result[i,1]
key_idx += 1
result[key_idx,1] = key
result[key_idx,2:end] = result[i,2:end]
end
end
result[key_idx,2:end] /= count
return result[1:key_idx,:]
end
Demo:
x = [5.0 3.5
6.0 3.6
7.0 3.0]
y = [5.0 4.5
6.0 4.7
8.0 3.0]
z = [5.0 4.0
6.0 3.2
8.0 4.0]
a=[x,y,z]
julia> a
3-element Array{Array{Float64,2},1}:
[5.0 3.5; 6.0 3.6; 7.0 3.0]
[5.0 4.5; 6.0 4.7; 8.0 3.0]
[5.0 4.0; 6.0 3.2; 8.0 4.0]
julia> aggregate(a)
4×2 Array{Float64,2}:
5.0 4.0
6.0 3.83333
7.0 3.0
8.0 3.5
Here's a version that is ~6 times faster than the answer from #PicaudVincent (based on his input data), but which doesn't sort the keys, so the rows of the return matrix is in arbitrary order:
function accumarrays(A::Vector{Matrix{T}}) where {T}
d = Dict{T, Tuple{T, Int}}()
for a in A
for i in indices(a, 1)
ai = a[i, 1]
d[ai] = get(d, ai, (zero(T), 0)) .+ (a[i, 2], 1)
end
end
Aout = Matrix{typeof(one(T)/1)}(length(d), 2)
i = 0
for (key, val) in d
Aout[i+=1, 1] = key
Aout[i, 2] = val[1] / val[2]
end
return Aout
end
If you need the rows to be sorted, this works, but is just 4-5 times faster:
function accumarrays_(A::Vector{Matrix{T}}) where {T}
d = Dict{T, Tuple{T, Int}}()
for a in A
for i in indices(a, 1)
ai = a[i, 1]
d[ai] = get(d, ai, (zero(T), 0)) .+ (a[i, 2], 1)
end
end
dkeys = sort!(collect(keys(d)))
Aout = Matrix{typeof(one(T)/1)}(length(dkeys), 2)
for i in eachindex(dkeys)
val = d[dkeys[i]]
Aout[i, 1] = dkeys[i]
Aout[i, 2] = val[1] / val[2]
end
return Aout
end
I have a xy-grid with two vector fields u and v. I represent vector fields as Array{Float64, 3} with dimensions nx × ny × 2.
I would like to have dot product u.v as a scalar field (Array{Float64,2} with dimensions nx×ny). What is the best way to achieve that?
It would be perfect to have something like dot(u,v,3), where 3 is the dimension over which the dot product is taken.
nx, ny = 3, 4
u = Array{Float64,3}(rand(0:1, nx, ny, 2))
#[0.0 1.0 0.0 1.0; 1.0 0.0 1.0 0.0; 1.0 0.0 1.0 0.0]
#[1.0 0.0 1.0 0.0; 1.0 0.0 0.0 1.0; 1.0 0.0 1.0 1.0]
v = Array{Float64,3}(rand(0:1, nx, ny, 2))
#[1.0 1.0 1.0 1.0; 0.0 1.0 0.0 0.0; 0.0 1.0 1.0 0.0]
#[1.0 0.0 0.0 0.0; 0.0 1.0 0.0 0.0; 1.0 1.0 1.0 0.0]
[dot(u[i,j,:], v[i,j,:]) for i in 1:nx, j in 1:ny]
3×4 Array{Float64,2}:
1.0 1.0 0.0 1.0
0.0 0.0 0.0 0.0
1.0 0.0 2.0 0.0
sum(u.*v,3) is both (reasonably) fast and short.
To explicity get a matrix you can squeeze the third dimension like so squeeze(sum(u.*v,3), 3)
Update: Of course, this has allocations and is not the best answer if speed is everything. In this case, see #DNF's direct loop implementation which is basically as fast as you can get it.
squeeze(sum(u .* v, 3), 3) is clean and simple, but if you need more speed, this is approximately 20x times faster on my pc:
a = fill(zero(eltype(u)), nx, ny)
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
Edit: For easier benchmarking comparison, this try this code:
function dotsum3(u, v)
a = fill(zero(eltype(u)), size(u, 1), size(u, 2))
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
return a
end
Also note that, if using #inbounds, one should probably explicitely check that the sizes of u and v are compatible.
Another version is
a = copy(view(u,:,:,1))
a .*= view(v,:,:,1)
a .+= #views u[:,:,2].*v[:,:,2]
Which is easily extensible to arbitrary 3rd dimension length. The benchmarks from my machine:
julia> using BenchmarkTools
julia> function f1(u,v)
a = fill(zero(eltype(u)), nx, ny)
#inbounds for k in indices(u, 3)
for j in indices(u, 2)
for i in indices(u, 1)
a[i, j] += u[i, j, k] * v[i, j, k]
end
end
end
return a
end
f1 (generic function with 1 method)
julia> f2(u,v) = squeeze(sum(u .* v, 3), 3)
f2 (generic function with 1 method)
julia> function f3(u,v)
a = copy(view(u,:,:,1))
a .*= view(v,:,:,1)
a .+= #views u[:,:,2].*v[:,:,2]
return a
end
f3 (generic function with 1 method)
julia> nx, ny = 3, 4
(3, 4)
julia> u = Array{Float64,3}(rand(0:1, nx, ny, 2));
julia> v = Array{Float64,3}(rand(0:1, nx, ny, 2));
Timing results:
julia> #btime f1($u,$v); # DNF suggestion
1.016 μs (1 allocation: 176 bytes)
julia> #btime f2($u,$v); # original answer
1.263 μs (14 allocations: 816 bytes)
julia> #btime f3($u,$v); # this answer
168.591 ns (5 allocations: 432 bytes)
The suggested version is faster (which is logical considering memory ordering of Julia Arrays).
I have an empty matrix initially:
m = Matrix(0, 3)
and a row that I want to add:
v = [2,3]
I try to do this:
[m v]
But I get an error
ERROR: ArgumentError: number of rows of each array must match
What's the proper way to do this?
That is because your matrix sizes don't match. Specifically v does not contain enough columns to match m. And its transposed
So this doesnt work
m = Matrix(0, 3)
v = [2,3]
m = cat(1, m, v) # or a = [m; v]
>> ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
whereas this does
m = Matrix(0, 3)
v = [2 3 4]
m = cat(1, m, v) # or m = [m; v]
>> 1x3 Array{Any,2}:
>> 2 3 4
and if you run it again it creates another row
m = cat(1, m, v) # or m = [m; v]
>> 2x3 Array{Any,2}:
>> 2 3 4
>> 2 3 4
Use the vcat (concatenate vertically) function:
help?> vcat
search: vcat hvcat VecOrMat DenseVecOrMat StridedVecOrMat AbstractVecOrMat levicivita is_valid_char #vectorize_2arg
vcat(A...)
Concatenate along dimension 1
Notice you have to transpose the vector v, ie. v', else you get a DimensionMismatch error:
julia> v = zeros(3)
3-element Array{Float64,1}:
0.0
0.0
0.0
julia> m = ones(3, 3)
3x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
julia> vcat(m, v') # '
4x3 Array{Float64,2}:
1.0 1.0 1.0
1.0 1.0 1.0
1.0 1.0 1.0
0.0 0.0 0.0
julia> v' # '
1x3 Array{Float64,2}:
0.0 0.0 0.0
julia> vcat(m, v)
ERROR: DimensionMismatch("mismatch in dimension 2 (expected 3 got 1)")
in cat_t at abstractarray.jl:850
in vcat at abstractarray.jl:887
Note: the comments; # ' are there just to make syntax highlighting work well.
Isn't that Matrix creates a two-dimensional array in Julia? If you try with m =[0, 3], which creates a one-dimensional Vector for you, you can append it by [m; v].
I think using [m v] is create a two-dimensional array as well, from the Julia Document