I'm looking at the numerics of some matrices that depends on a parameter x. The matrix has real eigenvalues for certain values x, but for other values I have a degeneracy in both eigenvalues and eigenvectors (the occurrence of exceptional points).
One of the simplest examples to get an exceptional point is with the matrix:
julia> h=[1 1im; 1im -1]
2×2 Array{Complex{Int64},2}:
1+0im 0+1im
0+1im -1+0im
The eigenvalues are zero, as they should be
2-element Array{Complex{Float64},1}:
-2.22045e-16+0.0im
0.0+0.0im
However, I would like to know why Julia give me the eigenvectors:
julia> b[2][:,1]
2-element Array{Complex{Float64},1}:
-0.0-0.707107im
0.707107+0.0im
julia> b[2][:,2]
2-element Array{Complex{Float64},1}:
0.707107+0.0im
0.0+0.707107im
Since in this case the eigenvalue is zero, I think it doesn't really matter what is the associated eigenvector. But if the eigenvalues coalesce somewhere in the
complex plane, do I really get two equal eigenvectors?
Is there an specific way to treat this cases in Julia?
The kernel of your matrix consists of multiples of (1,i)', which is what you get. As the matrix is not the zero matrix, it has rank 1 and thus also co-rank 1, the eigenspace has dimension 1. The generalized eigenspace is the full space, you get A*(1,0)' = (1,i)' so that in that basis ((1,i)',(1,0)') the linear operator has the matrix [[0,1],[0,0]], its Jordan normal form.
Related
I am currently in an online class in genomics, coming in as a wetlab physician, so my statistical knowledge is not the best. Right now we are working on PCA and SVD in R. I got a big matrix:
head(mat)
ALL_GSM330151.CEL ALL_GSM330153.CEL ALL_GSM330154.CEL ALL_GSM330157.CEL ALL_GSM330171.CEL ALL_GSM330174.CEL ALL_GSM330178.CEL ALL_GSM330182.CEL
ENSG00000224137 5.326553 3.512053 3.455480 3.472999 3.639132 3.391880 3.282522 3.682531
ENSG00000153253 6.436815 9.563955 7.186604 2.946697 6.949510 9.095092 3.795587 11.987291
ENSG00000096006 6.943404 8.840839 4.600026 4.735104 4.183136 3.049792 9.736803 3.338362
ENSG00000229807 3.322499 3.263655 3.406379 9.525888 3.595898 9.281170 8.946498 3.473750
ENSG00000138772 7.195113 8.741458 6.109578 5.631912 5.224844 3.260912 8.889246 3.052587
ENSG00000169575 7.853829 10.428492 10.512497 13.041571 10.836815 11.964498 10.786381 11.953912
Those are just the first few columns and rows, it has 60 columns and 1000 rows. Columns are cancer samples, rows are genes
The task is to:
removing the eigenvectors and reconstructing the matrix using SVD, then we need to calculate the reconstruction error as the difference between the original and the reconstructed matrix. HINT: You have to use the svd() function and equalize the eigenvalue to $0$ for the component you want to remove.
I have been all over google, but can't find a way to solve this task, which might be because I don't really get the question itself.
so i performed SVD on my matrix m:
d <- svd(mat)
Which gives me 3 matrices (Eigenassays, Eigenvalues and Eigenvectors), which i can access using d$u and so on.
How do I equalize the eigenvalue and ultimately calculate the error?
https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/svd
the decomposition expresses your matrix mat as a product of 3 matrices
mat = d$u x diag(d$d) x t(d$v)
so first confirm you are able to do the matrix multiplications to get back mat
once you are able to do this, set the last couple of elements of d$d to zero before doing the matrix multiplication
It helps to create a function that handles the singular values.
Here, for instance, is one that zeros out any singular value that is too small compared to the largest singular value:
zap <- function(d, digits = 3) ifelse(d < 10^(-digits) * max(abs(d))), 0, d)
Although mathematically all singular values are guaranteed non-negative, numerical issues with floating point algorithms can--and do--create negative singular values, so I have prophylactically wrapped the singular values in a call to abs.
Apply this function to the diagonal matrix in the SVD of a matrix X and reconstruct the matrix by multiplying the components:
X. <- with(svd(X), u %*% diag(zap(d)) %*% t(v))
There are many ways to assess the reconstruction error. One is the Frobenius norm of the difference,
sqrt(sum((X - X.)^2))
I wanna fill my array in Julia by positive real numbers. But I found information only how to do it with integers or real numbers (including negatives). Is it possible?
Thanks!
You can use any mathematical formula that maps the [0, 1) range into the [0, +inf]:
For example, if x is your random variable in the [0, 1) range (obtained with e.g. rand() for float data types):
tan(x * pi / 2)
atanh(x)
log(x) ^ 2
-log(x) / x ^ p (for p non-negative integer -- it will change the number distribution)
There are many other functions.
Of course the numbers are no longer uniformly distributed, but that is impossible to achieve.
Technically, the built-in randexp fulfils your requirement: the exponential distribution has the positive reals as its support. The scale of the numbers you practically get is much lower, though. The same holds for abs ∘ randn, the half-normal distribution. (In both cases, you could multiply the results with a large positive number to increase the variance to your requirements.)
Here's a funny alternative: you can generate uniformly random bits, and reinterpret them as floats (and just set the sign always to positive):
julia> bitrand(3*64).chunks
3-element Vector{UInt64}:
0xe7c7c52703987e68
0xc221b9864e7bab7e
0xa45b39faa65b446e
julia> reinterpret(Float64, bitrand(3*64).chunks)
3-element reinterpret(Float64, ::Vector{UInt64}):
2.8135484124856866e-108
-4.521596431965459e53
-5.836451011310255e78
julia> abs.(reinterpret(Float64, bitrand(3*64).chunks))
3-element Vector{Float64}:
1.6467305137006711e236
3.3503597018864875e-260
1.2211675821672628e77
julia> bitstring.(abs.(reinterpret(Float64, bitrand(3*64).chunks)))
3-element Vector{String}:
"0000110000011000001110000110001111010000011110111101000101101101"
"0011000010110101111100111011110100111100011011000101001100010011"
"0110111000001000101011010100011011010010100111111011001000001100"
This is still not a uniform distribution on the values, though, as the precision of floats gets smaller the larger the exponent gets.
So I solve the eigenvectors for a matrix in Maxima.
a:matrix([10,10],[-4,-3]);
\\outputs matrix
vec:eigenvectors(a);
[[[5,2],[1,1]],[[[1,-1/2]],[[1,-4/5]]]]
I've hand calculated the eigenvalues, and vectors as (1x2) 5: [-2,1]. 2:[-5,4], which are correct. What is Maxima outputting?
Eigenvectors are only determined up to a multiplicative constant. That is, if x is an eigenvector, then so is a*x where a is a scalar. I think if you look at your result and Maxima's result, you'll see that they are equivalent in that sense.
There are different normalization schemes. Looks like Maxima makes the first element 1. Another common scheme is to make the norm of the eigenvector equal to 1. Or one can just leave them unnormalized.
I am doing a project which requires me to normalize a sparse NxNmatrix. I read somewhere that we can normalize a matrix so that its eigen values lie between [-1,1] by multiplying it with a diagonal matrix D such that N = D^{-1/2}*A*D^{-1/2}.
But I am not sure what D is here. Also, is there a function in Matlab that can do this normalization for sparse matrices?
It's possible that I am misunderstanding your question, but as it reads it makes no sense to me.
A matrix is just a representation of a linear transformation. Given that a matrix A corresponds to a linear transformation T, any matrix of the form B^{-1} A B (called the conjugate of A by B) for an invertible matrix B corresponds to the same transformation, represented in a difference basis. In particular, the eigen values of a matrix correspond to the eigen values of the linear transformation, so conjugating by an invertible matrix cannot change the eigen values.
It's possible that you meant that you want to scale the eigen vectors so that each has unit length. This is a common thing to do since then the eigen values tell you how far a vector of unit length is magnified by the transformation.
How does it actually reduce noise..can you suggest some nice tutorials?
SVD can be understood from a geometric sense for square matrices as a transformation on a vector.
Consider a square n x n matrix M multiplying a vector v to produce an output vector w:
w = M*v
The singular value decomposition M is the product of three matrices M=U*S*V, so w=U*S*V*v. U and V are orthonormal matrices. From a geometric transformation point of view (acting upon a vector by multiplying it), they are combinations of rotations and reflections that do not change the length of the vector they are multiplying. S is a diagonal matrix which represents scaling or squashing with different scaling factors (the diagonal terms) along each of the n axes.
So the effect of left-multiplying a vector v by a matrix M is to rotate/reflect v by M's orthonormal factor V, then scale/squash the result by a diagonal factor S, then rotate/reflect the result by M's orthonormal factor U.
One reason SVD is desirable from a numerical standpoint is that multiplication by orthonormal matrices is an invertible and extremely stable operation (condition number is 1). SVD captures any ill-conditioned-ness in the diagonal scaling matrix S.
One way to use SVD to reduce noise is to do the decomposition, set components that are near zero to be exactly zero, then re-compose.
Here's an online tutorial on SVD.
You might want to take a look at Numerical Recipes.
Singular value decomposition is a method for taking an nxm matrix M and "decomposing" it into three matrices such that M=USV. S is a diagonal square (the only nonzero entries are on the diagonal from top-left to bottom-right) matrix containing the "singular values" of M. U and V are orthogonal, which leads to the geometric understanding of SVD, but that isn't necessary for noise reduction.
With M=USV, we still have the original matrix M with all its noise intact. However, if we only keep the k largest singular values (which is easy, since many SVD algorithms compute a decomposition where the entries of S are sorted in nonincreasing order), then we have an approximation of the original matrix. This works because we assume that the small values are the noise, and that the more significant patterns in the data will be expressed through the vectors associated with larger singular values.
In fact, the resulting approximation is the most accurate rank-k approximation of the original matrix (has the least squared error).
To answer to the tittle question: SVD is a generalization of eigenvalues/eigenvectors to non-square matrices.
Say,
$X \in N \times p$, then the SVD decomposition of X yields X=UDV^T where D is diagonal and U and V are orthogonal matrices.
Now X^TX is a square matrice, and the SVD decomposition of X^TX=VD^2V where V is equivalent to the eigenvectors of X^TX and D^2 contains the eigenvalues of X^TX.
SVD can also be used to greatly ease global (i.e. to all observations simultaneously) fitting of an arbitrary model (expressed in an formula) to data (with respect to two variables and expressed in a matrix).
For example, data matrix A = D * MT where D represents the possible states of a system and M represents its evolution wrt some variable (e.g. time).
By SVD, A(x,y) = U(x) * S * VT(y) and therefore D * MT = U * S * VT
then D = U * S * VT * MT+ where the "+" indicates a pseudoinverse.
One can then take a mathematical model for the evolution and fit it to the columns of V, each of which are a linear combination the components of the model (this is easy, as each column is a 1D curve). This obtains model parameters which generate M? (the ? indicates it is based on fitting).
M * M?+ * V = V? which allows residuals R * S2 = V - V? to be minimized, thus determining D and M.
Pretty cool, eh?
The columns of U and V can also be inspected to glean information about the data; for example each inflection point in the columns of V typically indicates a different component of the model.
Finally, and actually addressing your question, it is import to note that although each successive singular value (element of the diagonal matrix S) with its attendant vectors U and V does have lower signal to noise, the separation of the components of the model in these "less important" vectors is actually more pronounced. In other words, if the data is described by a bunch of state changes that follow a sum of exponentials or whatever, the relative weights of each exponential get closer together in the smaller singular values. In other other words the later singular values have vectors which are less smooth (noisier) but in which the change represented by each component are more distinct.