I am following some code using a matrix transformation with an input of type num:
> input
1 101.3862407 63.8025351
2 -39.2936491 0.3279252
3 -2.5526003 1.8761791
4 36.1441720 45.3625071
....
And a transformation matrix:
transfmatrix <- matrix(c(0.9,-0.1,0.1,0.9),nrow=2)
> transfmatrix
[,1] [,2]
[1,] 0.9 0.1
[2,] -0.1 0.9
I can then do a data transformation using:
# transformation
result <-input%*%transfmatrix
This works fine, but I would like "input" to be a raster file.
Is it possible to apply a similar transformation to raster layers? I tried the following:
library(raster)
r <- raster(ncol=40, nrow=20)
r[] <- rnorm(n=ncell(r))
test1<-stack(r,r*2)
test2 <- test1%*%transfmatrix
but received an error:
Error in test1 %*% transf.m :
requires numeric/complex matrix/vector arguments
You need to make a 2-column matrix from your data to multiply by the 2x2 matrix. Perhaps this:
> z = as.matrix(test1) %*% transf.m
> dim(z)
[1] 800 2
That makes an 800x2 matrix from the 800 cells of test1.
If you want a raster stack with the values, then, overwriting test1, do:
> test1[[1]][]=z[,1]
> test1[[2]][]=z[,2]
Related
I am using R to apply a self-written function, that takes as an input two numeric vectors plus a numeric parameter, over column margins of data frame. Each column in data frame is a numeric vector and I want to perform pairwise computations and create a matrix which has all possible combinations of the columns with indicated result of the computation. So essentially I want to generate a behaviour similar to the one yielded by cor() function.
# Data
> head(d)
1 2 3 4
1 -1.01035342 1.2490665 0.7202516 0.101467379
2 -0.50700743 1.4356733 0.9032172 -0.001583743
3 -0.09055243 0.4695046 2.4487632 -1.082570048
4 1.11230416 0.2885735 0.3534247 -0.728574628
5 -1.96115691 0.4831158 1.5650052 0.648675605
6 1.20434218 1.7668086 0.2170858 -0.161570792
> cor(d)
1 2 3 4
1 1.00000000 0.08320968 -0.06432155 0.04909430
2 0.08320968 1.00000000 -0.04557743 -0.01092765
3 -0.06432155 -0.04557743 1.00000000 -0.01654762
4 0.04909430 -0.01092765 -0.01654762 1.00000000
I found this useful answer: Perform pairwise comparison of matrix
Based on this I wrote this function which makes use of another self-written function compareFunctions()
createProbOfNonEqMatrix <- function(df,threshold){
combinations <- combn(ncol(df),2)
predDF <- matrix(nrow = length(density(df[,1])$y)) # df creation for predicted values from density function
for(i in 1:ncol(df)){
predCol <- density(df[,i])$y # convert df of original values to df of predicted values from density function
predDF <- cbind(predDF,predCol)
}
predDF <- predDF[,2:ncol(predDF)]
colnames(predDF) <- colnames(df) # give the predicted values column names as in the original df
predDF <- as.matrix(predDF)
out.mx <- apply( X=combinations,MARGIN = 2,FUN = "compareFunctions",
predicted_by_first = predDF[,combinations[1]],
predicted_by_second = predDF[,combinations[2]],
threshold = threshold)
return(out.mx)
}
The predicted_by_first, predicted_by_second and threshold are inputs for compareFunctions. However I get the following error:
Error in FUN(newX[, i], ...) : unused argument (newX[, i])
In desperation I tried this:
createProbOfNonEqMatrix <- function(df,threshold){
combinations <- combn(ncol(df),2)
predDF <- matrix(nrow = length(density(df[,1])$y))
for(i in 1:ncol(df)){
predCol <- density(df[,i])$y
predDF <- cbind(predDF,predCol)
}
predDF <- predDF[,2:ncol(predDF)]
colnames(predDF) <- colnames(df)
predDF <- as.matrix(predDF)
out.mx <- apply(
X=combinations,MARGIN = 2,FUN = function(x) {
diff <- abs(predDF[,x[1]]-predDF[,x[2]])
boolean <- diff<threshold
acceptCount <- length(boolean[boolean==TRUE])
probability <- acceptCount/length(diff)
return(probability)
}
)
return(out.mx)
}
It does seem to be working but instead of returning the pairwise matrix it gives me a vector:
> createProbOfNonEqMatrix(d,0.001)
[1] 0.10351562 0.08203125 0.13476562 0.13085938 0.14843750 0.10937500
Will you be able to guide me on how to make the desired pairwise matrix even if it implies writing the function code again within apply()? Also, if you could give me an idea on how to keep track of what pairwise comparisons are performed it will be greatly appreciated.
Thank you,
Alex
Your output gives you the result of the calculation in the order of the pairs in combinations: (1,2), (1,3), (1,4), (2,3), (2,4), (3,4). If you want to organise this into a symmetric square matrix you can do a basic manipulation on the result, e.g. as follows:
out.mx<-c(0.10351562, 0.08203125, 0.13476562, 0.13085938, 0.14843750, 0.10937500)
out.mtx<-matrix(nrow=ncol(df1),ncol=ncol(df1))
out.mtx[,]<-1
for (i in 1:length(combinations[1,])){
a<-combinations[1,i]
b<-combinations[2,i]
out.mtx[a,b]<-out.mtx[b,a]<-out.mx[i]
}
out.mtx
which gives you
[,1] [,2] [,3] [,4]
[1,] 1.00000000 0.1035156 0.08203125 0.1347656
[2,] 0.10351562 1.0000000 0.13085938 0.1484375
[3,] 0.08203125 0.1308594 1.00000000 0.1093750
[4,] 0.13476562 0.1484375 0.10937500 1.0000000
I am using nleqslv package in R to solve nonlinear system of equations. The R codes are given below;
require(nleqslv)
x <- c(6,12,18,24,30)
NMfun1 <- function(k,n) {
y <- rep(NA, length(k))
y[1] <- -(5/k[1])+sum(x^k[2]*exp(k[3]*x))+2*sum(k[4]*x^k[2]*exp(-k[1]*x^k[2]*exp(k[3]*x)+k[3]*x)/(1-k[4]*exp(-k[1]*x^k[2]*exp(k[3]*x))))
y[2] <- -sum(log(x))-sum(1/(k[2]+k[3]*x))+sum(k[1]*x^k[2]*exp(k[3]*x)*log(x))+2*sum(k[1]*k[4]*exp(-k[1]*x^k[2]*exp(k[3]*x)+k[3]*x)*log(x)/(1-k[4]*exp(-k[1]*x^k[2]*exp(k[3]*x))))
y[3] <- -sum(x/(k[2]+k[3]*x))+sum(k[1]*x^(k[2]+1)*exp(k[3]*x))-sum(x)+2*sum(k[4]*x^k[2]*exp(-k[1]*x^k[2]*exp(k[3]*x)+k[3]*x)/(1-k[4]*exp(-k[1]*x^k[2]*exp(k[3]*x))))
y[4] <- -(5/(1-k[4]))+2*sum(exp(-k[1]*x^k[2]*exp(k[3]*x))/(1-k[4]*exp(-k[1]*x^k[2]*exp(k[3]*x))))
return(y)
}
kstart <- c(0.05, 0, 0.35, 0.9)
NMfun1(kstart)
nleqslv(kstart, NMfun1, control=list(btol=.0001),method="Newton")
The estimated values for k obtained are; 0.04223362 -0.08360564 0.14216026 0.37854908
But the estimated values of k are to be
greater than zero.
Ok. So you want real larger than 0 solutions if they exist of course.
Make a new function which squares the input argument before passing it to NMfun1. And then use the searchZeros function in the package nleqslv to search for solutions. Like this
NMfun1.alt <- function(k0,n) NMfun1(k0^2,n)
3 use set.seed for reproducibility
set.seed(413)
# generate 100 random starting values
xstart <- matrix(runif(4*100,min=0,max=1), nrow=100,ncol=4)
z <- searchZeros(xstart,NMfun1.alt)
z
ksol <- z$x^2
ksol
# in this case there are two solutions
NMfun1(ksol[1,])
NMfun1(ksol[2,])
The output of the last 4 non comment lines of this code are
> ksol <- z$x^2
> ksol
[,1] [,2] [,3] [,4]
[1,] 0.002951051 1.669142 0.03589502 0.001167185
[2,] 0.002951051 1.669142 0.03589502 0.001167185
> NMfun1(ksol[1,])
[1] 3.231138e-11 3.602561e-13 -4.665268e-12 -1.119105e-13
> NMfun1(ksol[2,])
[1] 1.532663e-12 1.085046e-14 6.894485e-14 -2.664535e-15
You will see that the solution contained in object z has a negative element. And that is squared.
From this experiment it appears that your system has a single positive solution.
I have an R loop that has been giving me error. Here are the dimensions of the matrices..
> dim(A)
[1] 2 2
> dim(backward)
[1] 6848 2
I am trying to run this loop and get the following error:
for (i in t:1){
backward[i,]=A%*%t(backward[i,])}
Error in A %*% t(backward[i, ]) : non-conformable arguments
Where t equals 6848. Thanks for your time.
EDIT with bgoldst code:
> A
[,1] [,2]
[1,] 0.8 0.2
[2,] 0.2 0.8
> backward <- matrix(1:(t*2),t,2);
> dim(backward)
[1] 6848 2
> for (i in t:1) backward[i,] <- A%*%t(backward[i,,drop=F]);
Error in A %*% t(backward[i, , drop = F]) : non-conformable arguments
I'm guessing that your expectation of
backward[i,]
is that it will return a 1x2 matrix, which you would be able to use as the operand of a matrix multiplication. This is incorrect. In R, when you specify a single index within a dimension of a matrix, then by default, R will "drop" that dimension. In the case of the above piece of code, the row dimension is dropped, and you end up with a vector, whose contents are taken from all columns along the indexed row. A vector is not a valid operand to a matrix multiplication.
You can solve this problem by providing the drop argument to the [ operation:
A <- matrix(1:(2*2),2,2);
backward <- matrix(1:(6848*2),6848,2);
t <- nrow(backward); for (i in t:1) backward[i,] <- A%*%t(backward[i,,drop=F]); ## no error
Here's a demo of the effect of drop=F:
backward[1,]
## [1] 20548 27398
backward[1,,drop=F]
## [,1] [,2]
## [1,] 20548 27398
See ?`[` for more info.
Here's a solution that doesn't depend on the drop=F argument:
for (i in t:1) backward[i,] <- A%*%t(matrix(backward[i,],1));
Suppose I have 3 matrices C,W, and S
C <- matrix(1:3)
W <- matrix(2:4)
S <- matrix(3:5)
I want to make a matrix with those matrices as elements. Say matrix K, but each element of matrix K being a matrix itself. Just as a list of matrices works, but instead in a matrix form. I.e.:
> K
[,1] [,2] [,3]
[1,] C 0 0
[2,] 0 W S
C, W and S would each be matrix objects stored inside the larger matrix K.
Ultimately, I would like to be able to then use matrix multiplication like K %*% K or similar.
There are not a lot of classes than can be an element in an R matrix. In particular objects that rely on attributes for their behavior cannot be objects that will retain their essential features. And ironically that includes matrices themselves since their behavior is governed by the dim(ension) attribute. That exclusion applies to dates, factors and specialized lists such as dataframes. You can include lists as index-able items in a matrix, but as #thelatemail's comment points out this will be somewhat clunky.
> C <- matrix(0, 3,2)
> W <- matrix(1, 4,5)
> S <- matrix(2, 6,7)
> bigM <- matrix( list(), 2, 3)
> bigM[1,1] <- list(C)
> bigM[2,2] <- list(W)
> bigM[2,3] <- list(S)
> bigM
[,1] [,2] [,3]
[1,] Numeric,6 NULL NULL
[2,] NULL Numeric,20 Numeric,42
> bigM[2,3][[1]][42]
[1] 2
Notice the need to extract the matrix itself with [[1]] after extracting it as a list with [2,3]. It's only after that additonal step thay you can get the 42nd item in the matrix, whould alos have been the [6,7]th item if you chose to reference it by row,column indices.
I have a sparse matrix (dgCMatrix) as the result of fitting a glmnet. I want to write this result to a .csv but can't use write.table() the matrix because it can't coerced into a data.frame.
Is there a way to coerce the sparse matrix to either a data.frame or a regular matrix? Or is there a way to write it to a file while keeping the coefficient names which are probably row names?
That will be dangerous to transform the sparse matrix to a normal one, if the sparse matrix size is too large. In my case (text classification task), I got a matrix of size 22490 by 120,000. If you try get the dense matrix, that will be more than 20 GB, I think. Then R will break down !
So my suggestion, you may simply store the sparse matrix in an efficient and memory friendly way, such as Matrix Market Format, which keeps all non-zero values and their coordinates (row & col number). In the R you can use the method writeMM
as.matrix() will convert to the full dense representation:
> as.matrix(Matrix(0, 3, 2))
[,1] [,2]
[1,] 0 0
[2,] 0 0
[3,] 0 0
You can write the resulting object out using write.csv or write.table.
Converting directly to a dense matrix is likely to waste a lot of memory. The R package Matrix allows converting the sparse matrix into a memory-efficient coordinate triplet format data frame using the summary() function, which could then be written easily to csv. This is probably simpler and easier than the matrix market approach. See the answer to this related question: Sparse matrix to a data frame in R
Also, here is an illustration from the Matrix package documentation:
## very simple export - in triplet format - to text file:
data(CAex)
s.CA <- summary(CAex)
s.CA # shows (i, j, x) [columns of a data frame]
message("writing to ", outf <- tempfile())
write.table(s.CA, file = outf, row.names=FALSE)
## and read it back -- showing off sparseMatrix():
str(dd <- read.table(outf, header=TRUE))
## has columns (i, j, x) -> we can use via do.call() as arguments to sparseMatrix():
mm <- do.call(sparseMatrix, dd)
stopifnot(all.equal(mm, CAex, tolerance=1e-15))
# input: a sparse matrix with named rows and columns (dimnames)
# returns: a data frame representing triplets (r, c, x) suitable for writing to a CSV file
sparse2triples <- function(m) {
SM = summary(m)
D1 = m#Dimnames[[1]][SM[,1]]
D2 = m#Dimnames[[2]][SM[,2]]
data.frame(row=D1, col=D2, x=m#x)
}
Example
> library(Matrix)
> dn <- list(LETTERS[1:3], letters[1:5])
> m <- sparseMatrix(i = c(3,1,3,2,2,1), p= c(0:2, 4,4,6), x = 1:6, dimnames = dn)
> m
3 x 5 sparse Matrix of class "dgCMatrix"
a b c d e
A . 2 . . 6
B . . 4 . 5
C 1 . 3 . .
> sparse2triples(m)
row col x
1 C a 1
2 A b 2
3 B c 4
4 C c 3
5 A e 6
6 B e 5
[EDIT: use data.frame]