How to replace an element of a symmetric matrix randomly? - r

Suppose I have a matrix like so:
data=matrix(c(1,0,0,0,0,0,1,0,0.6583,0,0,0,1,0,0,0,0.6583,0,1,0,0,0,0,0,1),nrow=5,ncol=5)
[,1] [,2] [,3] [,4] [,5]
[1,] 1 0.0000 0 0.0000 0
[2,] 0 1.0000 0 0.6583 0
[3,] 0 0.0000 1 0.0000 0
[4,] 0 0.6583 0 1.0000 0
[5,] 0 0.0000 0 0.0000 1
How do I create another matrix, say "data2", such that it has the same number of off-diagonal nonzero elements as "data" but in another location other than the one in data? The randomly simulated data will be uniform (so runif).

Here is a somewhat clumsy way to do this. It works well for small matrices but would be too slow if you're going to use this for some very high-dimensional problems.
# Current matrix:
data=matrix(c(1,0,0,0,0,0,1,0,0.6583,0,0,0,1,0,0,0,0.6583,0,1,0,0,0,0,0,1),nrow=5,ncol=5)
# Number of nonzero elements in upper triangle:
no.nonzero<-sum(upper.tri(data)*data>0)
# Generate same number of new nonzero correlations:
new.cor<-runif(no.nonzero,-1,1)
# Create new diagonal matrix:
p<-dim(data)[1]
data2<-diag(1,p,p)
### Insert nonzero correlations: ###
# Step 1. Identify the places where the nonzero elements can be placed:
pairs<-(p^2-p)/2 # Number of element in upper triangle
combinations<-matrix(NA,pairs,2) # Matrix containing indices for those elements (i.e. (1,2), (1,3), ... (2,3), ... and so on)
k<-0
for(i in 1:(p-1))
{
for(j in {i+1}:p)
{
k<-k+1
combinations[k,]<-c(i,j)
}
}
# Step 2. Randomly pick indices:
places<-sample(1:k,no.nonzero)
# Step 3. Insert nonzero correlations:
for(i in 1:no.nonzero)
{
data2[combinations[places[i],1],combinations[places[i],2]]<-data2[combinations[places[i],2],combinations[places[i],1]]<-new.cor[i]
}

Not really understood the question. There are two off-diagonal and non-zero elements (0.6583) in the example, right? Is matrix with two elements the result you want in this case?
data=matrix(c(1,0,0,0,0,0,1,0,0.6583,0,0,0,1,0,0,0,0.6583,0,1,0,0,0,0,0,1),nrow=5,ncol=5)
# Convert to vector
data2 <- as.numeric(data)
# Remove diagonal
data2 <- data2[as.logical(upper.tri(data) | lower.tri(data))]
# Remove 0 elements
data2 <- data2[data2 != 0]
data2

Related

Multiplying factor to certain cells in matrix in R

I have the following correlation matrix, cor.mat below.
I want to multiply all numbers by 15% except the cell [1,1] [2,2] [3,3] [4,4].
Does anyone have a good code to implent this in R?
1 2 3 4
1 1.0000000 0.1938155 0.1738809 0.2465276
2 0.1938155 1.0000000 0.4045694 0.2729958
3 0.1738809 0.4045694 1.0000000 0.3340883
4 0.2465276 0.2729958 0.3340883 1.0000000
You can use diag which returns the diagonal of a matrix
matrix = matrix*0.15
diag(matrix) = 1
Create a logical matrix excluding the diag and do the multiplication
i1 <- row(m1) != col(m1)
m1[i1] <- m1[i1] * 0.15

How to implement q-learning in R?

I am learning about q-learning and found a Wikipedia post and this website.
According to the tutorials and pseudo code I wrote this much in R
#q-learning example
#http://mnemstudio.org/path-finding-q-learning-tutorial.htm
#https://en.wikipedia.org/wiki/Q-learning
set.seed(2016)
iter=100
dimension=5;
alpha=0.1 #learning rate
gamma=0.8 #exploration/ discount factor
# n x n matrix
Q=matrix( rep( 0, len=dimension*dimension), nrow = dimension)
Q
# R -1 is fire pit,0 safe path and 100 Goal state########
R=matrix( sample( -1:0, dimension*dimension,replace=T,prob=c(1,2)), nrow = dimension)
R[dimension,dimension]=100
R #reward matrix
################
for(i in 1:iter){
row=sample(1:dimension,1)
col=sample(1:dimension,1)
I=Q[row,col] #randomly choosing initial state
Q[row,col]=Q[row,col]+alpha*(R[row,col]+gamma*max(Qdash-Q[row,col])
#equation from wikipedia
}
But I have problem in max(Qdash-Q[row,col] which according to the website is Max[Q(next state, all actions)] How to I programmatically search all actions for next state?
The second problem is this pseudo code
Do While the goal state hasn't been reached.
Select one among all possible actions for the current state.
Using this possible action, consider going to the next state.
Get maximum Q value for this next state based on all possible actions.
Compute: Q(state, action) = R(state, action) + Gamma * Max[Q(next state, all actions)]
Set the next state as the current state.
End Do
Is it this
while(Q<100){
Q[row,col]=Q[row,col]+alpha*(R[row,col]+gamma*max(Qdash-Q[row,col])
}
This post is by no means a complete implementation of Q-learning in R. It is an attempt to answer the OP with regards to the description of the algorithm in the website linked in the post and in Wikipedia.
The assumption here is that the reward matrix R is as described in the website. Namely that it encodes reward values for possible actions as non-negative numbers, and -1's in the matrix represent null values (i.e., where there is no possible action to transition to that state).
With this setup, an R implementation of the Q update is:
Q[cs,ns] <- Q[cs,ns] + alpha*(R[cs,ns] + gamma*max(Q[ns, which(R[ns,] > -1)]) - Q[cs,ns])
where
cs is the current state at the current point in the path.
ns is the new state based on a (randomly) chosen action at the current state. This action is chosen from the collection of possible actions at the current state (i.e., for which R[cs,] > -1). Since the state transition itself is deterministic here, the action is the transition to the new state.
For this action resulting in ns, we want to add its maximum (future) value over all possible actions that can be taken at ns. This is the so-called Max[Q(next state, all actions)] term in the linked website and the "estimate of optimal future value" in Wikipedia. To compute this, we want to maximize over the ns-th row of Q but consider only columns of Q for which columns of R at the corresponding ns-th row are valid actions (i.e., for which R[ns,] > -1). Therefore, this is:
max(Q[ns, which(R[ns,] > -1)])
An interpretation of this value is a one-step look ahead value or an estimate of the cost-to-go in dynamic programming.
The equation in the linked website is the special case in which alpha, the learning rate, is 1. We can view the equation in Wikipedia as:
Q[cs,ns] <- (1-alpha)*Q[cs,ns] + alpha*(R[cs,ns] + gamma*max(Q[ns, which(R[ns,] > -1)]))
where alpha "interpolates" between the old value Q[cs,ns] and the learned value R[cs,ns] + gamma*max(Q[ns, which(R[ns,] > -1)]). As noted in Wikipedia,
In fully deterministic environments, a learning rate of alpha=1 is optimal
Putting it all together into a function:
q.learn <- function(R, N, alpha, gamma, tgt.state) {
## initialize Q to be zero matrix, same size as R
Q <- matrix(rep(0,length(R)), nrow=nrow(R))
## loop over episodes
for (i in 1:N) {
## for each episode, choose an initial state at random
cs <- sample(1:nrow(R), 1)
## iterate until we get to the tgt.state
while (1) {
## choose next state from possible actions at current state
## Note: if only one possible action, then choose it;
## otherwise, choose one at random
next.states <- which(R[cs,] > -1)
if (length(next.states)==1)
ns <- next.states
else
ns <- sample(next.states,1)
## this is the update
Q[cs,ns] <- Q[cs,ns] + alpha*(R[cs,ns] + gamma*max(Q[ns, which(R[ns,] > -1)]) - Q[cs,ns])
## break out of while loop if target state is reached
## otherwise, set next.state as current.state and repeat
if (ns == tgt.state) break
cs <- ns
}
}
## return resulting Q normalized by max value
return(100*Q/max(Q))
}
where the input parameters are:
R is the rewards matrix as defined in the blog
N is the number of episodes to iterate
alpha is the learning rate
gamma is the discount factor
tgt.state is the target state of the problem.
Using the example in the linked website as a test:
N <- 1000
alpha <- 1
gamma <- 0.8
tgt.state <- 6
R <- matrix(c(-1,-1,-1,-1,0,-1,-1,-1,-1,0,-1,0,-1,-1,-1,0,-1,-1,-1,0,0,-1,0,-1,0,-1,-1,0,-1,0,-1,100,-1,-1,100,100),nrow=6)
print(R)
## [,1] [,2] [,3] [,4] [,5] [,6]
##[1,] -1 -1 -1 -1 0 -1
##[2,] -1 -1 -1 0 -1 100
##[3,] -1 -1 -1 0 -1 -1
##[4,] -1 0 0 -1 0 -1
##[5,] 0 -1 -1 0 -1 100
##[6,] -1 0 -1 -1 0 100
Q <- q.learn(R,iter,alpha,gamma,tgt.state)
print(Q)
## [,1] [,2] [,3] [,4] [,5] [,6]
##[1,] 0 0 0.0 0 80 0.00000
##[2,] 0 0 0.0 64 0 100.00000
##[3,] 0 0 0.0 64 0 0.00000
##[4,] 0 80 51.2 0 80 0.00000
##[5,] 64 0 0.0 64 0 100.00000
##[6,] 0 80 0.0 0 80 99.99994

In R: calculate Area Under Precision/Recall Curve (AUPR)?

Suppose I have two matrices: A for Label matrix and B for corresponding predicted probability matrix of A. Now I would like to calculate the the AUPR (Area Under Precision/Recall Curve) according to matrices A and B. For common AUC (Area Under ROC Curve), there are many packages in R, such as ROCR, pROC, can directly calculate the AUC value, but currently, what packages in R can calculate the AUPR? or Can you help give the method the compute the AUPR?
Here is the two example matrics:
> pp
[,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,] 0.01792 0.00155 -0.00140 0.00522 0.01320 0.22506 0.00454
[2,] 0.05883 0.11256 0.82862 0.12406 0.08298 -0.00392 0.30724
[3,] 0.00743 0.06357 0.14500 0.00213 0.00545 0.03452 0.11189
[4,] 0.02571 0.01460 0.01108 0.00494 0.01246 0.11880 0.05504
[5,] 0.02407 0.00961 0.00720 0.00382 0.01039 0.10974 0.04512
> ll
D00040 D00066 D00067 D00075 D00088 D00094 D00105
hsa190 0 0 0 0 0 1 0
hsa2099 0 1 1 0 0 0 1
hsa2100 0 0 0 0 0 0 1
hsa2101 0 0 0 0 0 0 0
hsa2103 0 0 0 0 0 0 0
pp is the predicted probability matrix for the true label ll matrix, and ll is just the label matrix.
Thanks in advance.
I would first convert the prediction scores and classes into vectors from matrix.
There is a "PRROC" package that provides the similar function of generating ROC and PRC as "ROCR", and it also gives the AUC of the PRC.
Specifically, I'm using the data ROCR.simple from "ROCR" package as an example.
library(PRROC)
library(ROCR)
data("ROCR.simple")
scores <- data.frame(ROCR.simple$predictions, ROCR.simple$labels)
pr <- pr.curve(scores.class0=scores[scores$ROCR.simple.labels=="1",]$ROCR.simple.predictions,
scores.class1=scores[scores$ROCR.simple.labels=="0",]$ROCR.simple.predictions,
curve=T)
Note that here in this function, the "scores.class0" needs to be the scores for the positive class (which is a little confusing, because personally I consider 0 as negative and 1 as positive). So I switched the order of 0 and 1.
This way, the PR curve and AUC are all saved in the pr variable.
pr
Precision-recall curve
Area under curve (Integral):
0.7815038
Area under curve (Davis & Goadrich):
0.7814246
Curve for scores from 0.005422562 to 0.9910964
( can be plotted with plot(x) )
Then, you can plot the PRC with plot(pr) or with ggplot:
y <- as.data.frame(pr$curve)
ggplot(y, aes(y$V1, y$V2))+geom_path()+ylim(0,1)
The resulting curve is the same with the curve made by ROCR package.

matrix with row and column constraints

I need to solve for a n x n (n usually <12) matrix subject to a few constraints:
1.Predetermined row and column sums are satisfied.
2.Each element in the matrix having a row number greater than column number must be zero (so basically the only nonzero elements must be in the top right portion).
3.For a given row, every element more than three columns to the right first nonzero element must also be zero.
So, a 4x4 matrix might look something like this (the row and column constraints will be much larger in practice, usually around 1-3 million):
|3 2 1 0| = 6
|0 2 1 1| = 4
|0 0 2 1| = 3
|0 0 0 4| = 4
3 4 4 6
I have been trying to use some solver approaches to do this in excel and also have tried some R based optimization packages but have been so unsuccessful so far.
Any suggestions on how else I might approach this would be much appreciated.
Thanks!
Test data:
x <- c(2,2,2,1,1,1,1)
rowVals <- c(6,4,3,4)
colVals <- c(3,4,4,6)
Function to construct the appropriate test matrix from (3N-5) parameters:
makeMat <- function(x,n) {
## first and last element of diag are constrained by row/col sums
diagVals <- c(colVals[1],x[1:(n-2)],rowVals[n])
## set up off-diagonals 2,3
sup2Vals <- x[(n-1):(2*n-3)]
sup3Vals <- x[(2*n-2):(3*n-5)]
## set up matrix
m <- diag(diagVals)
m[row(m)==col(m)-1] <- sup2Vals
m[row(m)==col(m)-2] <- sup3Vals
m
}
Objective function (sum of squares of row & column deviations):
objFun <- function(x,n) {
m <- makeMat(x,n)
## compute SSQ deviation from row/col constraints
sum((rowVals-rowSums(m))^2+(colVals-colSums(m))^2)
}
Optimizing:
opt1 <- optim(fn=objFun,par=x,n=4)
## recovers original values, although it takes a lot of steps
opt2 <- optim(fn=objFun,par=rep(0,length(x)),n=4)
makeMat(opt2$par,n=4)
## [,1] [,2] [,3] [,4]
## [1,] 3 2.658991 0.3410682 0.0000000
## [2,] 0 1.341934 1.1546649 1.5038747
## [3,] 0 0.000000 2.5042858 0.4963472
## [4,] 0 0.000000 0.0000000 4.0000000
##
## conjugate gradients might be better
opt3 <- optim(fn=objFun,par=rep(0,length(x)),n=4,
method="CG")
It seems that there are multiple solutions to this problem, which
isn't surprising (since there are 2N constraints on (N-2)+(N-1)+(N-2)=
3N-5 parameters).
You didn't say whether you need integer solutions or not -- if
so you will need more specialized tools ...

Can't get a positive definite variance matrix when very small eigen values

To run a Canonical correspondence analysis (cca package ade4) I need a positive definite variance matrix. (Which in theory is always the case)
but:
matrix(c(2,59,4,7,10,0,7,0,0,0,475,18714,4070,97,298,0,1,0,17,7,4,1,4,18,36),nrow=5)
> a
[,1] [,2] [,3] [,4] [,5]
[1,] 2 0 475 0 4
[2,] 59 7 18714 1 1
[3,] 4 0 4070 0 4
[4,] 7 0 97 17 18
[5,] 10 0 298 7 36
> eigen(var(a))
$values
[1] 6.380066e+07 1.973658e+02 3.551492e+01 1.033096e+01
[5] -1.377693e-09
The last eigen value is -1.377693e-09 which is < 0. But the theorical value is > 0.
I can't run the function if one of the eigen value is < 0
I really don't know how to fix this without changing the code of the function cca()
Thanks for help
You can change the input, just a little bit, to make the matrix positive definite.
If you have the variance matrix, you can truncate the eigenvalues:
correct_variance <- function(V, minimum_eigenvalue = 0) {
V <- ( V + t(V) ) / 2
e <- eigen(V)
e$vectors %*% diag(pmax(minimum_eigenvalue,e$values)) %*% t(e$vectors)
}
v <- correct_variance( var(a) )
eigen(v)$values
# [1] 6.380066e+07 1.973658e+02 3.551492e+01 1.033096e+01 1.326768e-08
Using the singular value decomposition, you can do the same thing directly with a.
truncate_singular_values <- function(a, minimum = 0) {
s <- svd(a)
s$u %*% diag( ifelse( s$d > minimum, s$d, minimum ) ) %*% t(s$v)
}
svd(a)$d
# [1] 1.916001e+04 4.435562e+01 1.196984e+01 8.822299e+00 1.035624e-01
eigen(var( truncate_singular_values(a,.2) ))$values
# [1] 6.380066e+07 1.973680e+02 3.551494e+01 1.033452e+01 6.079487e-09
However, this changes your matrix a by up to 0.1, which is a lot
(I suspect it is that high because the matrix a is square: as a result,
one of the eigenvalues of var(a) is exactly 0.)
b <- truncate_singular_values(a,.2)
max( abs(b-a) )
# [1] 0.09410187
We can actually do better simply by adding some noise.
b <- a + 1e-6*runif(length(a),-1,1) # Repeat if needed
eigen(var(b))$values
# [1] 6.380066e+07 1.973658e+02 3.551492e+01 1.033096e+01 2.492604e-09
Here are two approaches:
V <- var(a)
# 1
library(Matrix)
nearPD(V)$mat
# 2 perturb diagonals
eps <- 0.01
V + eps * diag(ncol(V))

Resources