I'm working with a 121x137 (i,j) array (mortality table) and am trying to create a 121x1 vector from this made up of the (i+1,j+1) values (i.e. index (50,50), (51,51) and so forth.
I'm using the following code - BaseTable is my 121x137 array:
age=50
survivalcurve = for i in age:nrow(BaseTable)-1
for j in age:ncol(BaseTable)-1
println(getindex(FemaleBaseTable, i+1, j+1))
end
end
However, when I do this, its returning all values of i and j - if I picture the values I want as a diagonal line running top-bottom, L-R of my table, its giving me all the values on the top right of my imaginary diagonal line.
If I fix i and loop through j, it works and returns the entire 50th row:
age=50
survivalcurve =
for j in age:ncol(FemaleBaseTable)
println(getindex(FemaleBaseTable, 50, j+1))
end
and likewise if I fix j and loop through i, this works and returns the entire jth column:
age=50
survivalcurve =
for i in age:nrow(FemaleBaseTable)-1
println(getindex(FemaleBaseTable, i+1, 50))
end
I've surmised its returning all values because I'm using age:nrow / age:ncol, but not sure what a suitable replacement would be to only return the (i+1),(j+1) value would be. Any help would be appreciated!
You can collect the diagonal elements starting at any index (i,j) by using either diag from LinearAlgebra.jl or manually using a single loop.
Assuming you have:
nrows, ncols = 121, 137
FemaleBaseTable = rand(nrows, ncols)
# And the diagonal should start at any (i,j)
i, j = 50, 50
You can either use
diag(#view FemaleBaseTable[i:end,j:end])
or,
[FemaleBaseTable[i,i] for (i,j) in zip(i:nrows,j:ncols)]
Related
I need to identify the rows (/columns) that have defined values in a large sparse Boolean Matrix. I want to use this to 1. slice (actually view) the Matrix by those rows/columns; and 2. slice (/view) vectors and matrices that have the same dimensions as the margins of a Matrix. I.e. the result should probably be a Vector of indices / Bools or (preferably) an iterator.
I've tried the obvious:
a = sprand(10000, 10000, 0.01)
cols = unique(a.colptr)
rows = unique(a.rowvals)
but each of these take like 20ms on my machine, probably because they allocate about 1MB (at least they allocate cols and rows). This is inside a performance-critical function, so I'd like the code to be optimized. The Base code seems to have an nzrange iterator for sparse matrices, but it is not easy for me to see how to apply that to my case.
Is there a suggested way of doing this?
Second question: I'd need to also perform this operation on views of my sparse Matrix - would that be something like x = view(a,:,:); cols = unique(x.parent.colptr[x.indices[:,2]]) or is there specialized functionality for this? Views of sparse matrices appear to be tricky (cf https://discourse.julialang.org/t/slow-arithmetic-on-views-of-sparse-matrices/3644 – not a cross-post)
Thanks a lot!
Regarding getting the non-zero rows and columns of a sparse matrix, the following functions should be pretty efficient:
nzcols(a::SparseMatrixCSC) = collect(i
for i in 1:a.n if a.colptr[i]<a.colptr[i+1])
function nzrows(a::SparseMatrixCSC)
active = falses(a.m)
for r in a.rowval
active[r] = true
end
return find(active)
end
For a 10_000x10_000 matrix with 0.1 density it takes 0.2ms and 2.9ms for cols and rows, respectively. It should also be quicker than method in question (apart from the correctness issue as well).
Regarding views of sparse matrices, a quick solution would be to turn view into a sparse matrix (e.g. using b = sparse(view(a,100:199,100:199))) and use functions above. In code:
nzcols(b::SubArray{T,2,P}) where {T,P<:AbstractSparseArray} = nzcols(sparse(b))
nzrows(b::SubArray{T,2,P}) where {T,P<:AbstractSparseArray} = nzrows(sparse(b))
A better solution would be to customize the functions according to view. For example, when the view uses UnitRanges for both rows and columns:
# utility predicate returning true if element of sorted v in range r
inrange(v,r) = searchsortedlast(v,last(r))>=searchsortedfirst(v,first(r))
function nzcols(b::SubArray{T,2,P,Tuple{UnitRange{Int64},UnitRange{Int64}}}
) where {T,P<:SparseMatrixCSC}
return collect(i+1-start(b.indexes[2])
for i in b.indexes[2]
if b.parent.colptr[i]<b.parent.colptr[i+1] &&
inrange(b.parent.rowval[nzrange(b.parent,i)],b.indexes[1]))
end
function nzrows(b::SubArray{T,2,P,Tuple{UnitRange{Int64},UnitRange{Int64}}}
) where {T,P<:SparseMatrixCSC}
active = falses(length(b.indexes[1]))
for c in b.indexes[2]
for r in nzrange(b.parent,c)
if b.parent.rowval[r] in b.indexes[1]
active[b.parent.rowval[r]+1-start(b.indexes[1])] = true
end
end
end
return find(active)
end
which work faster than the versions for the full matrices (for 100x100 submatrix of above 10,000x10,000 matrix cols and rows take 16μs and 12μs, respectively on my machine, but these are unstable results).
A proper benchmark would use fixed matrices (or at least fix the random seed). I'll edit this line with such a benchmark if I do it.
In case the indices are not ranges, the fallback to converting to a sparse matrix works, but here are versions for indices which are Vectors. If the indices are mixed, yet another set of versions needs to be made. Quite repetitive, but this is the strength of Julia, when the versions are done, the code will choose optimized methods correctly using the types in the caller without too much effort.
function sortedintersecting(v1, v2)
i,j = start(v1), start(v2)
while i <= length(v1) && j <= length(v2)
if v1[i] == v2[j] return true
elseif v1[i] > v2[j] j += 1
else i += 1
end
end
return false
end
function nzcols(b::SubArray{T,2,P,Tuple{Vector{Int64},Vector{Int64}}}
) where {T,P<:SparseMatrixCSC}
brows = sort(unique(b.indexes[1]))
return [k
for (k,i) in enumerate(b.indexes[2])
if b.parent.colptr[i]<b.parent.colptr[i+1] &&
sortedintersecting(brows,b.parent.rowval[nzrange(b.parent,i)])]
end
function nzrows(b::SubArray{T,2,P,Tuple{Vector{Int64},Vector{Int64}}}
) where {T,P<:SparseMatrixCSC}
active = falses(length(b.indexes[1]))
for c in b.indexes[2]
active[findin(b.indexes[1],b.parent.rowval[nzrange(b.parent,c)])] = true
end
return find(active)
end
-- ADDENDUM --
Since it was noted nzrows for Vector{Int} indices is a bit slow, this is an attempt to improve its speed by replacing findin with a version exploiting sortedness:
function findin2(inds,v,w)
i,j = start(v),start(w)
res = Vector{Int}()
while i<=length(v) && j<=length(w)
if v[i]==w[j]
push!(res,inds[i])
i += 1
elseif (v[i]<w[j]) i += 1
else j += 1
end
end
return res
end
function nzrows(b::SubArray{T,2,P,Tuple{Vector{Int64},Vector{Int64}}}
) where {T,P<:SparseMatrixCSC}
active = falses(length(b.indexes[1]))
inds = sortperm(b.indexes[1])
brows = (b.indexes[1])[inds]
for c in b.indexes[2]
active[findin2(inds,brows,b.parent.rowval[nzrange(b.parent,c)])] = true
end
return find(active)
end
I'm using Julia 0.3.4
I'm trying to write LU-decomposition using Gaussian elimination. So I have to swap rows. And here's my problem:
If I'm using a,b = b,a I get an error,
but if I'm using:
function swapRows(row1, row2)
temp = row1
row1 = row2
row2 = temp
end
then everything works just fine.
Am I doing something wrong or it's a bug?
Here's my source code:
function lu_t(A::Matrix)
# input value: (A), where A is a matrix
# return value: (L,U), where L,U are matrices
function swapRows(row1, row2)
temp = row1
row1 = row2
row2 = temp
return null
end
if size(A)[1] != size(A)[2]
throw(DimException())
end
n = size(A)[1] # matrix dimension
U = copy(A) # upper triangular matrix
L = eye(n) # lower triangular matrix
for k = 1:n-1 # direct Gaussian elimination for each column `k`
(val,id) = findmax(U[k:end,k]) # find max pivot element and it's row `id`
if val == 0 # check matrix for singularity
throw(SingularException())
end
swapRows(U[k,k:end],U[id,k:end]) # swap row `k` and `id`
# U[k,k:end],U[id,k:end] = U[id,k:end],U[k,k:end] - error
for i = k+1:n # for each row `i` > `k`
μ = U[i,k] / U[k,k] # find elimination coefficient `μ`
L[i,k] = μ # save to an appropriate position in lower triangular matrix `L`
for j = k:n # update each value of the row `i`
U[i,j] = U[i,j] - μ⋅U[k,j]
end
end
end
return (L,U)
end
###### main code ######
A = rand(4,4)
#time (L,U) = lu_t(A)
#test_approx_eq(L*U, A)
The swapRows function is a no-op and has no effect whatsoever – all it does is swap around some local variable names. See various discussions of the difference between assignment and mutation:
https://groups.google.com/d/msg/julia-users/oSW5hH8vxAo/llAHRvvFVhMJ
http://julia.readthedocs.org/en/latest/manual/faq/#i-passed-an-argument-x-to-a-function-modified-it-inside-that-function-but-on-the-outside-the-variable-x-is-still-unchanged-why
http://julia.readthedocs.org/en/latest/manual/faq/#why-does-x-y-allocate-memory-when-x-and-y-are-arrays
The constant null doesn't mean what you think it does – in Julia v0.3 it's a function that computes the null space of a linear transformation; in Julia v0.4 it still means this but has been deprecated and renamed to nullspace. The "uninteresting" value in Julia is called nothing.
I'm not sure what's wrong with your commented out row swapping code, but this general approach does work:
julia> X = rand(3,4)
3x4 Array{Float64,2}:
0.149066 0.706264 0.983477 0.203822
0.478816 0.0901912 0.810107 0.675179
0.73195 0.756805 0.345936 0.821917
julia> X[1,:], X[2,:] = X[2,:], X[1,:]
(
1x4 Array{Float64,2}:
0.478816 0.0901912 0.810107 0.675179,
1x4 Array{Float64,2}:
0.149066 0.706264 0.983477 0.203822)
julia> X
3x4 Array{Float64,2}:
0.478816 0.0901912 0.810107 0.675179
0.149066 0.706264 0.983477 0.203822
0.73195 0.756805 0.345936 0.821917
Since this creates a pair of temporary arrays that we can't yet eliminate the allocation of, this isn't the most efficient approach. If you want the most efficient code here, looping over the two rows and swapping pairs of scalar values will be faster:
function swapRows!(X, i, j)
for k = 1:size(X,2)
X[i,k], X[j,k] = X[j,k], X[i,k]
end
end
Note that it is conventional in Julia to name functions that mutate one or more of their arguments with a trailing !. Currently, closures (i.e. inner functions) have some performance issues, so you'll want such a helper function to be defined at the top-level scope instead of inside of another function the way you've got it.
Finally, I assume this is an exercise since Julia ships with carefully tuned generic (i.e. it works for arbitrary numeric types) LU decomposition: http://docs.julialang.org/en/release-0.3/stdlib/linalg/#Base.lu.
-
It's quite simple
julia> A = rand(3,4)
3×4 Array{Float64,2}:
0.241426 0.283391 0.201864 0.116797
0.457109 0.138233 0.346372 0.458742
0.0940065 0.358259 0.260923 0.578814
julia> A[[1,2],:] = A[[2,1],:]
2×4 Array{Float64,2}:
0.457109 0.138233 0.346372 0.458742
0.241426 0.283391 0.201864 0.116797
julia> A
3×4 Array{Float64,2}:
0.457109 0.138233 0.346372 0.458742
0.241426 0.283391 0.201864 0.116797
0.0940065 0.358259 0.260923 0.578814
A (m rows, n columns) is a (0,1)-Matrix (or logical matrix).
How to get a sub matrix B (p rows, p columns) from A, satisfying that B is a permutation matrix and p is the maximum? For instance,
PS: A permutation matrix is a square binary matrix that has exactly one entry 1 in each row and each column and 0s elsewhere.
One possibility is to exploit that every permutation matrix can be built up one row and column at a time.
So we can take every permutation matrix of a certain size, try to extend it by all possible rows or columns,
and see what results in a permutation matrix that is one size larger.
The running time isn't that great. I think it's something like O(2^(m+n)). (And I've used Python, FWIW.)
#!/usr/local/bin/python3
import itertools
A = ((0,1,0,0),
(0,0,1,0),
(0,1,1,0),
(1,0,0,1))
maximalSubmatrices = { ( (), () ), }
# each tuple is a tuple of rows and then columns
maxP = 0
def isPerm(rows,cols):
if ( len(rows) != len(cols) ):
return False
for row in rows:
if not exactlyOne( A[row][col] for col in cols ):
return False
for col in cols:
if not exactlyOne( A[row][col] for row in rows ):
return False
return True
def exactlyOne(sequence):
return sum( 1 for elt in sequence if elt ) == 1
while True:
moreMaxl = set()
for submatrix in maximalSubmatrices:
for row,col in itertools.product(range(len(A)),range(len(A[0]))):
if ( row not in submatrix[0] and col not in submatrix[1] ):
moreMaxl.add( ( tuple(sorted(submatrix[0]+(row,))) , tuple(sorted(submatrix[1]+(col,))) ) )
moreMaxl = set( ( maxl for maxl in moreMaxl if isPerm(*maxl) ) )
if ( len(moreMaxl) ):
maxP += 1
maximalSubmatrices = moreMaxl
else:
break
for maxl in maximalSubmatrices:
print("maximal rows: ",maxl[0],"\nmaximal cols: ",maxl[1],end="\n\n")
print("maximum permutation size is: ",maxP)
The output is:
maximal rows: (0, 1, 3)
maximal cols: (0, 1, 2)
maximal rows: (0, 1, 3)
maximal cols: (1, 2, 3)
maximum permutation size is: 3
Explanation:
In Python, a tuple is an immutable array of objects. Because it’s immutable, it can be hashed and made an element of a set. So maximalSubmatrices is a set of the rows and columns needed to make a submatrix. In Java, we’d do something like:
class Submatrix {
List<Integer> rows;
List<Integer> columns;
public int hashCode();
public boolean equals(Object);
}
Set<Submatrix> maximalSubmatrices;
but Python can take care of all that by itself.
We start with the rows and columns needed to make a submatrix of size 0: both are the empty tuple (). Each time through the while loop, we take all possible row,column pairs and see if that row,column could extend a current permutation matrix (in other words, they’re not already in the matrix). If so, we add the extended matrix to the set moreMaxl. Then we go through moreMaxl and keep only the permutation matrices. If there’s still elements in moreMaxl, then they’re permutation matrices that are one size larger than the matrices in maximalSubmatrices. Since we could extend, the while loop continues.
I am stuck at creating a matrix of a matrix (vector in this case)
What I have so far
index = zeros(size(A)) // This is some matrix but isn't important to the question
indexIndex = 1;
for rows=1:length(R)
for columns=1:length(K)
if(A(rows,columns)==x)
V=[rows columns]; // I create a vector holding the row + column
index(indexIndex) = V(1,2) // I want to store all these vectors
indexIndex = indexIndex + 1
end
end
end
I have tried various ways of getting the information out of V (such as V(1:2)) but nothing seems to work correctly.
In other words, I'm trying to get an array of points.
Thanks in advance
I do not understand your question exactly. What is the size of A? What is x, K and R? But under some assumptions,
Using list
You could use a list
// Create some matrix A
A = zeros(8,8)
//initialize the list
index = list();
// Get the dimensions of A
rows = size(A,1);
cols = size(A,2);
x = 0;
for row=1:rows
for col=1:cols
if(A(row,col)==x)
// Create a vector holding row and col
V=[row col];
// Append it to list using $ (last index) + 1
index($+1) = V
end
end
end
Single indexed matrices
Another approach would be to make use of the fact an multi-dimensional matrix can also be indexed by a single value.
For instance create a random matrix named a:
-->a = rand(3,3)
a =
0.6212882 0.5211472 0.0881335
0.3454984 0.2870401 0.4498763
0.7064868 0.6502795 0.7227253
Access the first value:
-->a(1)
ans =
0.6212882
-->a(1,1)
ans =
0.6212882
Access the second value:
-->a(2)
ans =
0.3454984
-->a(2,1)
ans =
0.3454984
So that proves how the single indexing works. Now to apply it to your problem and knocking out a for-loop.
// Create some matrix A
A = zeros(8,8)
//initialize the array of indices
index = [];
// Get the dimensions of A
rows = size(A,1);
cols = size(A,2);
x = 0;
for i=1:length(A)
if(A(i)==x)
// Append it to list using $ (last index) + 1
index($+1) = i;
end
end
Without for-loop
If you just need the values that adhere to a certain condition you could also do something like this
values = A(A==x);
Be carefull when comparing doubles, these are not always (un)equal when you expect.
I have the image and the vector
a = imread('Lena.tiff');
v = [0,2,5,8,10,12,15,20,25];
and this M-file
function y = Funks(I, gama, c)
[m n] = size(I);
for i=1:m
for j=1:n
J(i, j) = (I(i, j) ^ gama) * c;
end
end
y = J;
imshow(y);
when I'm trying to do this:
f = Funks(a,v,2)
I am getting this error:
??? Error using ==> mpower
Integers can only be combined with integers of the same class, or scalar doubles.
Error in ==> Funks at 5
J(i, j) = (I(i, j) ^ gama) * c;
Can anybody help me, with this please?
The error is caused because you're trying to raise a number to a vector power. Translated (i.e. replacing formal arguments with actual arguments in the function call), it would be something like:
J(i, j) = (a(i, j) ^ [0,2,5,8,10,12,15,20,25]) * 2
Element-wise power .^ won't work either, because you'll try to "stuck" a vector into a scalar container.
Later edit: If you want to apply each gamma to your image, maybe this loop is more intuitive (though not the most efficient):
a = imread('Lena.tiff'); % Pics or GTFO
v = [0,2,5,8,10,12,15,20,25]; % Gamma (ar)ray -- this will burn any picture
f = cell(1, numel(v)); % Prepare container for your results
for k=1:numel(v)
f{k} = Funks(a, v(k), 2); % Save result from your function
end;
% (Afterwards you use cell array f for further processing)
Or you may take a look at the other (more efficient if maybe not clearer) solutions posted here.
Later(er?) edit: If your tiff file is CYMK, then the result of imread is a MxNx4 color matrix, which must be handled differently than usual (because it 3-dimensional).
There are two ways I would follow:
1) arrayfun
results = arrayfun(#(i) I(:).^gama(i)*c,1:numel(gama),'UniformOutput',false);
J = cellfun(#(x) reshape(x,size(I)),results,'UniformOutput',false);
2) bsxfun
results = bsxfun(#power,I(:),gama)*c;
results = num2cell(results,1);
J = cellfun(#(x) reshape(x,size(I)),results,'UniformOutput',false);
What you're trying to do makes no sense mathematically. You're trying to assign a vector to a number. Your problem is not the MATLAB programming, it's in the definition of what you're trying to do.
If you're trying to produce several images J, each of which corresponds to a certain gamma applied to the image, you should do it as follows:
function J = Funks(I, gama, c)
[m n] = size(I);
% get the number of images to produce
k = length(gama);
% Pre-allocate the output
J = zeros(m,n,k);
for i=1:m
for j=1:n
J(i, j, :) = (I(i, j) .^ gama) * c;
end
end
In the end you will get images J(:,:,1), J(:,:,2), etc.
If this is not what you want to do, then figure out your equations first.