I want to solve the following simultaneous equations using R:
# -0.9x+0.01y+0.001z+0.001n=0
# 0.9x-0.82y+0.027z+0.027n=0
# 0x+0.81y+(0.243-1)z+0.243n=0
# 0x+0y+0.729z+(0.729-1)n=0
# x+y+z+n=1
I tried:
A <- matrix(data=c(-0.9,0.01,0.001,0.001,0.9,-0.82,0.027,0.027,0,0.81,0.243-1,0.243,0,0,0.729,0.729-1,1,1,1,1), nrow=5, ncol=4, byrow=TRUE)
b <- matrix(data=c(0, 0, 0, 0, 1), nrow=5, ncol=1, byrow=FALSE)
round(solve(A, b), 4)
and it caught error:
Error in solve.default(A, b) : 'a' (5 x 4) must be square
What does the error mean? Is this function only applicable to square matrix?
Edit:
I removed the last equation and tried:
A <- matrix(data=c(-0.9,0.01,0.001,0.001,0.9,-0.82,0.027,0.027,0,0.81,0.243-1,0.243,0,0,0.729,0.729-1), nrow=4, ncol=4, byrow=TRUE)
b <- matrix(data=c(0, 0, 0, 0), nrow=4, ncol=1, byrow=FALSE)
A;b;
round(solve(A, b), 4)
which caught error:
Error in solve.default(A, b) : Lapack routine dgesv: system is exactly singular: U[4,4] = 0
Since you have 5 linear equations (rows of your matrix) and only 4 columns (variables), then your problem is overdetermined and in general it can't be solved. Consider a smaller example: x+y=1; x-2*y=2; 3*x-5=0. Once you use the first two equations:
(1) x+y=1 → y=1-x
(2) x-2*(1-x)=2 → 3*x-2=2 → x=4/3 ## substitute (1) into second eq.
(3) → y=-1/3
you have a solution for x and y (x=4/3, y=-1/3), so the last equation is either redundant or makes your solution impossible (in this case, the latter: 3*x-5 = -1 != 5).
On the other hand, if you have fewer rows than columns, then your system is underdetermined and you can't get a unique answer.
So the number of matrix rows must equal the number of columns → you can only do this with a square matrix. This is not a limitation of the R function, it's about the underlying math of linear equations ...
In your specific case the problem is that the sum of the first four rows of your matrix add up to zero, so these are redundant. As you found out, just dropping the last equation doesn't help. But if you drop one of the redundant rows you can get an answer.
solve(A[-4,],b[-4])
Note you get the same answer whether you drop row 1, 2, 3, or 4.
Related
I am trying to grasp the basic concept of invertible and non-invertible matrices.
I created a random non-singular square matrix
S <- matrix(rnorm(100, 0, 1), ncol = 10, nrow = 10)
I know that this matrix is positive definite (thus invertible) because when I decompose the matrix S into its eigenvalues, their product is positive.
eig_S <- eigen(S)
eig_S$values
[1] 3.0883683+0.000000i -2.0577317+1.558181i -2.0577317-1.558181i 1.6884120+1.353997i 1.6884120-1.353997i
[6] -2.1295086+0.000000i 0.1805059+1.942696i 0.1805059-1.942696i -0.8874465+0.000000i 0.8528495+0.000000i
solve(S)
According to this paper, we can compute the inverse of a non-singular matrix by its SVD too.
Where
(where U and V are eigenvectors and D eigenvalues, please do correct me if I am wrong).
The inverse then is, .
Indeed, I can run the formula in R:
s <- svd(S)
s$v%*%solve(diag(s$d))%*%t(s$u)
Which produces exactly the same result as solve(S).
My first question is:
1) Are s$d indeed represent the eigenvalues of S? Because s$d and eig_S$values are quite different.
Now the second part,
If I create a singular matrix
I <- matrix(rnorm(100, 0, 1), ncol = 5, nrow = 20)
I <- I%*%t(I)
eig_I <- eigen(I)
eig_I$values
[1] 3.750029e+01 2.489995e+01 1.554184e+01 1.120580e+01 8.674039e+00 3.082593e-15 5.529794e-16 3.227684e-16
[9] 2.834454e-16 5.876634e-17 -1.139421e-18 -2.304783e-17 -6.636508e-17 -7.309336e-17 -1.744084e-16 -2.561197e-16
[17] -3.075499e-16 -4.150320e-16 -7.164553e-16 -3.727682e-15
The solve function will produce an error
solve(I)
system is computationally singular: reciprocal condition number =
1.61045e-19
So, again according to the same paper we can use the SVD
i <- svd(I)
solve(i$u %*% diag(i$d) %*% t(i$v))
which produces the same error.
Then I tried to use the Cholesky decomposition for matrix inversion
Conj(t(I))%*%solve(I%*%Conj(t(I)))
and again I get the same error.
Could someone please explain where am I using the equations wrong?
I know that for matrix I%*%Conj(t(I)), the determinant of the eigenvalue matrix is positive but the matrix is not a full rank due to the initial multiplication that I did.
j <- eigen(I%*%Conj(t(I)))
det(diag(j$values))
[1] 3.17708e-196
qr(I %*% Conj(t(I)))$rank
[1] 5
UPDATE 1: Following the comments bellow, and after going through the paper/Wikipedia page again. I used these two codes, which they produce some results but I am not sure about their validity. The first example seems more believable. The SVD solution
i$v%*%diag(1/i$d)%*%t(i$u)
and the Cholesky
Conj(t(I))%*%(I%*%Conj(t(I)))^(-1)
I am not sure if I interpreted the two sources correctly though.
I need to compute the matrix A on the power of -1/2, which basically means the square root of the initial matrix's inverse.
If A is singular then the Moore-Penrose generalized inverse is computed with the ginv function from the MASS package, otherwise the regular inverse is computed using the solve function.
Matrix A is defined below:
A <- structure(c(604135780529.807, 0, 58508487574887.2, 67671936726183.9,
0, 0, 0, 1, 0, 0, 0, 0, 58508487574887.2, 0, 10663900590720128,
10874631465443760, 0, 0, 67671936726183.9, 0, 10874631465443760,
11315986615387788, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1), .Dim = c(6L,
6L))
I check singularity with the comparison of the rank and the dimension.
rankMatrix(A) == nrow(A)
The above code returns FALSE, So I have to use ginv to get the inverse. The inverse of A is as follows:
A_inv <- ginv(A)
The square-root of the inverse matrix is computed with the sqrtm function from the expm package.
library(expm)
sqrtm(A_inv)
The function returns the following error:
Error in solve.default(X[ii, ii] + X[ij, ij], S[ii, ij] - sumU) :
Lapack routine zgesv: system is exactly singular
So how can we compute the square root in this case? Please note that matrix A is not always singular so we have to provide a general solution for the problem.
Your question relates to two distinct problems:
Inverse of a matrix
Square root of a matrix
Inverse
The inverse does not exist for singular matrices. In some applications, the Moore-Penrose or some other generalised inverse may be taken as a suitable substitute for the inverse. However, note that computer numerics will incur rounding errors in most cases; and these errors may make a singular matrix appear regular to the computer or vice versa.
If A always exhibits the the block structure of the matrix you give, I suggest to consider only its non-diagonal block
A3 = A[ c( 1, 3, 4 ), c( 1, 3, 4 ) ]
A3
[,1] [,2] [,3]
[1,] 6.041358e+11 5.850849e+13 6.767194e+13
[2,] 5.850849e+13 1.066390e+16 1.087463e+16
[3,] 6.767194e+13 1.087463e+16 1.131599e+16
instead of all of A for better efficiency and less rounding issues. The remaining 1-diagonal entries would remain 1 in the inverse of the square root, so no need to clutter the calculation with them. To get an impression of the impact of this simplification, note that R can calculate
A3inv = solve(A3)
while it could not calculate
Ainv = solve(A)
But we will not need A3inverse, as will become evident below.
Square root
As a general rule, the square root of a matrix A will only exist if the matrix has a diagonal Jordan normal form (https://en.wikipedia.org/wiki/Jordan_normal_form). Hence, there is no truly general solution of the problem as you require.
Fortunately, like “most” (real or complex) matrices are invertible, “most” (real or complex) matrices have a diagonal complex Jordan normal form. In this case, the Jordan normal form
A3 = T·J·T⁻¹
can be calculated in R as such:
X = eigen(A3)
T = X$vectors
J = Diagonal( x=X$values )
To test this recipe, compare
Tinv = solve(T)
T %*% J %*% Tinv
with A3. They should match (up to rounding errors) if A3 has a diagonal Jordan normal form.
Since J is diagonal, its squareroot is simply the diagonal matrix of the square roots
Jsqrt = Diagonal( x=sqrt( X$values ) )
so that Jsqrt·Jsqrt = J. Moreover, this implies
(T·Jsqrt·T⁻¹)² = T·Jsqrt·T⁻¹·T·Jsqrt·T⁻¹ = T·Jsqrt·Jsqrt·T⁻¹ = T·J·T⁻¹ = A3
so that in fact we obtain
√A3 = T·Jsqrt·T⁻¹
or in R code
A3sqrt = T %*% Jsqrt %*% Tinv
To test this, calculate
A3sqrt %*% A3sqrt
and compare with A3.
Square root of the inverse
The square root of the inverse (or, equally, the inverse of the sqare root) can be calculated easily once a diagonal Jordan normal form has been calculated. Instead of J use
Jinvsqrt = Diagonal( x=1/sqrt( X$values ) )
and calculate, analogously to above,
A3invsqrt = T %*% Jinvsqrt %*% Tinv
and observe
A3·A3invsqrt² = … = T·(J/√J/√J)·T⁻¹ = 1
the unit matrix so that A3invsqrt is the desired result.
In case A3 is not invertible, a generalised inverse (not necessarily the Moore-Penrose one) can be calculated by replacing all undefined entries in Jinvsqrt by 0, but as I said above, this should be done with suitable care in the light of the overall application and its stability against rounding errors.
In case A3 does not have a diagonal Jordan normal form, there is no square root, so the above formulas will yield some other result. In order not to run into this case at times of bad luck, best implement a test whether
A3invsqrt %*% A3 %*% A3invsqrt
is close enough to what you would consider a 1 matrix (this only applies if A3 was invertible in the first place).
PS: Note that you can prefix a sign ± for each diagonal entry of Jinvsqrt to your liking.
y <- matrix(c(7, 9, -5, 0, 2, 6), ncol = 1)
try <- t(y)
tryy <- try %*% y
i <- solve(tryy)
h <- y %*% i %*% try
uniroot(as.vector(solve(((1-x) * diag(6)) + h)), c(-Inf, Inf))
Error in (1 - x) * diag(6) : non-conformable arrays
The purpose of this command uniroot(as.vector(solve(((1-x) * diag(6)) + h)), c(-Inf, Inf)) is to solve the characteristics equation det[(1-λ)I+h] = 0
where, λ=eigenvalues , I=identity matrix , h=hat matrix=y(y'y)^(-1)y'
here λ is unknown ,we have to solve for it.
I am not understanding where is the problem here? I have tried as:
as.vector(solve(6*diag(6)+h))
This is not non-conformable. But why is not working inside the uniroot function?
Your question is a bit confusing, so I have to make a couple of assumptions. If you want the eigenvalues of h, then the characteristic equation is:
det(h - I*λ) = 0
not
det[(1-λ)I+h] = 0
So I used the former.
Given the above, the short answer is: do it this way.
f <- function(lambda) det(h -lambda*diag(6))
F <- Vectorize(f)
library(rootSolve)
uniroot.all(F,c(-1000,1000),n=2000)
# [1] 0 1
# or, much more simply
eigen(h)$values
# [1] 1.000000e+00 2.220446e-16 0.000000e+00 -2.731318e-18 -6.876381e-18 -7.365903e-17
So h has 2 eigenvalues, 0 and 1. Note that the built-in function eigen(...) finds 6 roots, but 5 of them are within the machine tolerance of 0.
The question about why your code fails is a bit more involved.
First, your code:
tryy <- try %*% y
is the dot product of y with itself (so, a scalar), returned as a matrix with one element. When you "invert" that using solve(...)
i <- solve(tryy)
you simply take the reciprocal, so i is also a matrix with 1 element. I'm not sure if this is what you had in mind.
Second, uniroot(...) does not work this way. The first argument must be a function; you've passed an expression which depends on x, which in turn is undefined. You could try:
f <- function(x) det(h-x*diag(6))
uniroot(f,c(-Inf,Inf))
but this wouldn't work either because (a) uniroot(...) works on a finite interval, (b) it requires that the function f(...) have different sign at the ends of the interval, and (c) in any event it would return only one root (the smaller one).
So you could use uniroot.all(...) in package rootSolve. uniroot.all(...) also requires a function as it's first argument, but there's a twist: the function must be "vectorized". This means that if you pass a vector of lambda values, f(...) should return a vector of the same length. Fortunately in R there is an easy way to "vectorize" a given function, as in:
F <- Vectorize(f).
Even this has it's limits. uniroot.all(...) also requires a finite interval, so we have to guess what that is, and also it evaluates F on n sub-intervals. So if your interval does not contain all the roots, or if the sub-intervals are not small enough, you will not find all the roots.
Using the built-in eigen(...) function is definitely the best option.
I have a large right stochastic matrix(row sums to 1).size~20000x20000. How can I find the stationary distribution of it?
I tried to calculate the eigenvalues and vectors, and get complex eigenvalues, eg.1+0i(more than one).
And try to use the following method:
pi=u[I-P+U]^-1
while when I do the inversion with solve() I got the error message Error in solve.default(A):system is computationally singular: reciprocal condition number = 3.16663e-19
As far as I understand, the Perron–Frobenius theorem ensures that every stochastic matrix as a stationary probability vector pi that the largest absolute value of an eigenvalue is always 1, so pi=piP,and my matrix has all positive entries,I can get a uniq pi,am I correct?
Or if there any method I can calculate the row vector pi?
Every stochastic matrix indeed has a stationary distribution. Since P has all row sums = 1,
(P-I) has row sums = 0 => (P-I)*(1, ...., 1) always gives you zero. So rank(P-I) <= n-1, and so is rank of transpose to P-I. Hence, there exists q such that (t(P)-I)*q = 0 => t(P)q = q.
Complex value 1+0i seems to be quite real for me. But if you get only complex values, i.e. coefficient before i is not 0, then the algorithm produces an error somewhere -- it solves the problem numerically and does not have to be true all the time. Also it does not matter how many eigenvalues and vectors it produces, what matters is that it finds the right eigenvector for eigenvalue 1 and that's what you need.
Make sure that your stationary distribution is indeed your limit distribution, otherwise there is no point in computing it. You could try to find it out by multiplying different vectors with your matrix^1000, but I don't know how much time it will take in your case.
Last but not least, here is an example:
# first we need a function that calculates matrix^n
mpot = function (A, p) {
# calculates A^p (matrix multiplied p times with itself)
# inputes: A - real-valued square matrix, p - natural number.
# output: A^p
B = A
if (p>1)
for (i in 2:p)
B = B%*%A
return (B)
}
# example matrix
P = matrix( nrow = 3, ncol = 3, byrow = T,
data = c(
0.1, 0.9, 0,
0.4, 0, 0.6,
0, 1, 0
)
)
# this converges to stationary distribution independent of start distribution
t(mpot(P,1000)) %*% c(1/3, 1/3, 1/3)
t(mpot(P,1000)) %*% c(1, 0, 0)
# is it stationary?
xx = t(mpot(P,1000)) %*% c(1, 0, 0)
t(P) %*% xx
# find stationary distribution using eigenvalues
eigen(t(P)) # here it is!
eigen_vect = eigen(t(P))$vectors[,1]
stat_dist = eigen_vect/sum(eigen_vect) # as there is subspace of them,
# but we need the one with sum = 1
stat_dist
How is R able to find eigenvectors for the following matrix? Eigenvalues are 2,2 so eigenvectors require solving solve(matrix(c(0,1,0,0),2,2)) which is singular matrix with no solution.
> eigen(matrix(c(2,1,0,2),2,2))
$values
[1] 2 2
$vectors
[,1] [,2]
[1,] 0 4.440892e-16
[2,] 1 -1.000000e+00
> solve(matrix(c(0,1,0,0),2,2))
Error in solve.default(matrix(c(0, 1, 0, 0), 2, 2)) :
Lapack routine dgesv: system is exactly singular
Both the routines essentially do the same thing. They find x such that (A-lambdaI)x = 0 without finding the inverse of A-lambdaI. Clearly (0 1) is a solution but how I can't understand why solve did not come up with it and how do I manually solve it.
You asked for an eigen decomposition, you got an eigen decomposition.
Had you asked for rcond(), the condition number of the matrix, or for kappa(), you would have gotten the appropriate response.
For your second example:
> mat <- matrix(c(0,1,0,0), 2, 2)
> kappa(mat)
[1] Inf
>
> rcond(mat)
[1] 0
>
For your first example, there is actually no problem:
> mat <- matrix(c(2,1,0,2), 2, 2)
> kappa(mat)
[1] 1.772727
>
> rcond(mat)
[1] 0.5714286
>
>
See e.g. this previous question on SO for more.
Maybe it's using one of the algorithms listed here:
http://en.wikipedia.org/wiki/List_of_numerical_analysis_topics#Eigenvalue_algorithms
?
According to http://stat.ethz.ch/R-manual/R-devel/library/base/html/eigen.html, eigen seems to use the LAPACK routine at http://netlib.org/lapack/double/dgeev.f (if you have a square matrix which is not symmetric).
Note: you're right that A - lambda * I is singular if lambda is an eigenvalue but in order to find eigenvectors, one does need invert A - lambda * I or solve an equation y = (A - lambda * I) * x (with y not being the null vector). It is sufficient to find non-zero vectors x which satisfy
(A - lambda * I) * x = 0
One strategy is to find a non-singular transformation matrix T such that (A - lambda * I) * T is an upper triangular matrix (i.e. all elements below the diagonal are zero). Because A-lambda*I is singular, T can be constructed such that the last element on the diagonal (or even more diagonal elements if the multiplicity of the eigenvalue is larger than one) is zero.
A vector z which only has it's last element equal to a non-zero value (i.e. z = (0,....,0,1) ) will then give the zero vector when multiplied with (A-lambda *I) * T. So one has:
0 = ((A - lambda * I) * T) * z
or in other words, T*z is an eigenvector of A.
They are not doing the same.
Just take a look at the wikipedia link.
And look in code for dgeev.f and you will see that a simple solve of (A-lambda*I)x=0 is not performed.