I have a matrix of N*200 values
For each row I am calculating the 5 acf values using
for(i in 1:N){
xx[[i]] <- acf(x[i,], plot=F)$acf[1:5]
}
I was wondering is there an alternative for xx[i] other than using a list? i.e.
is it possible to have a matrix of N*5 containing each of the acf values?
I know I can get the list and then unlit this but is there a quicker way?
Use apply for cleaner code:
iN = 1000
mX = matrix(rnorm(iN*200), iN, 200)
mACF = t(apply(mX, MARGIN = 1,
FUN = function(vX) acf(vX, plot = FALSE, lag.max = 4)$acf))
Output:
> head(mACF)
[,1] [,2] [,3] [,4] [,5]
[1,] 1 -0.01301076 -0.02077288 -0.09442797 -0.010610654
[2,] 1 -0.03060448 -0.06019641 -0.04674656 -0.086555364
[3,] 1 0.09513999 -0.05021542 -0.02757927 -0.002984605
[4,] 1 -0.08135746 0.11003419 -0.06550000 0.033755892
[5,] 1 0.09014033 0.09981602 0.11100782 0.057275603
[6,] 1 -0.08462636 -0.10192390 0.05601853 -0.019114467
Related
I have a matrix of 2134 by 2134 of correlation values and I would like to count the total number of values that are above 0.8 and below -0.8. I have tried
length(TFcoTF[TFcoTF>.8])
but this does not seem to be correct as I am getting about 50 percent of values above .8 which does not correspond to the histogram I have for the data. Also when I do
length(TFcoTF[TFcoTF<-.8])
I got 0 as the output. Any help is appreciated.
The data table package has a function called between. This returns TRUE/FALSE value for each value in your matrix whether the value is between two values.
In my example below, I randomly created a 10x10 matrix with random values [-1,+1]. Using the length function and subsetting where the values are in your range of [-0.8,+0.8].
library(data.table)
data <- matrix(runif(100,-1,1), nrow = 10, ncol=10)
data
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 0.05585901 -0.7497720 -0.8371569 -0.401079424 -0.4130752 -0.788961736 0.2909987 0.48965177 0.4076504 -0.0682856
[2,] -0.42442920 0.7476111 0.8238973 -0.912507391 -0.4450897 -0.001308901 0.5151425 -0.16838841 -0.1648151 0.8370660
[3,] -0.73295874 0.5271986 0.5822628 -0.008554908 -0.2785803 -0.499058508 -0.5661172 0.35957967 0.5807055 0.2350893
[4,] 0.18949338 0.3827603 -0.6112584 0.209209240 -0.5883962 -0.087900052 0.1272227 0.58165922 -0.9950324 -0.9118599
[5,] 0.40862973 0.9496163 0.4996253 0.079538601 0.9839763 -0.119883751 0.3667418 -0.02751815 -0.6724141 0.3217434
[6,] 0.77338548 -0.7698167 -0.5632436 0.223301216 -0.9936610 0.650110638 -0.9400395 -0.47808065 -0.1579283 -0.6896787
[7,] 0.93210326 0.5360980 0.7677325 0.815231731 -0.4320206 0.647954028 0.5180600 -0.09574138 -0.3848389 0.9726445
[8,] -0.66411834 0.1125759 -0.4021577 -0.711363103 0.7161801 -0.071971464 0.7953436 0.40326575 0.6895480 0.7496597
[9,] 0.14118154 0.4775983 0.8966069 0.852880293 0.4715885 -0.542526148 0.5200246 -0.62649677 -0.3677738 0.1961003
[10,] -0.59353193 -0.2358892 0.5769562 -0.287113142 -0.7100862 -0.107092848 -0.8101459 -0.46754146 -0.4082147 -0.4475972
length(data[between(data,-0.8,0.8)])
[1] 84
It's difficult to answer without having your dataset, please provide a minimal reproducible example later.
For the first line of code, this looks correct.
For the second, the error comes from a syntax error. In R you can assign value with = and <-. So x<-1 assign the value whereas x < -1 return a boolean.
You can then combine logical values and run the code below :
set.seed(42)
m <- matrix(runif(25, min = -1, max = 1), nrow = 5, ncol = 5)
m
length(m[ m > .8]) + length(m[ m < -.8]) # long version from what you did.
length(m[ m < -.8 | m > .8]) # | mean or. TRUE | FALSE will return TRUE.
sum(m > .8 | m < -.8)
# The sum of logical is the length, since sum(c(TRUE, FALSE)) is sum(c(0, 1))
sum(abs(m) > .8) # is the shortest version
I'm trying to achieve the following: I want to generate 7 values from a normal distribution. Then I want to take these values, and using them as a mean generate 3 more (for each initial value) values from a normal distribution to replace them. I'd like to write this in a loop.
Let's use sd = 1.5 and sd = 0.7, and start with a mean of 0.
set.seed(1234)
mu.mat<-rnorm(7,mean=0,sd=1.5)
Gives me 7 nice values.
Then I want to create a number num [1:21] that generates 3 norm. distr. values using mean = first value of the just created list with sd = 0.7, three more using the second value and so on.
Of the form:
rnorm(3,mean=mu.mat[1],sd=0.7)
Just for all entries in a loop.
What I've tried:
mu.mat2<-NULL
for(i in 1:7) {
mu.mat2[i]<-rnorm(3,mean=mu.mat[i],sd=0.7)
}
Results in error: no. of items to replace is not a multiple of replacement length.
Any help on how to put this into a loop is very appreciated. Thanks in advance!
You don't need a loop. You can do:
rnorm(21, mean = rep(mu.mat, each = 3), sd = 0.7)
#> [1] -0.4811184 -1.2327778 -1.8603816 -3.3073277 -2.5190560 -3.2298056
#> [7] -2.3695570 -2.0228732 -1.1692489 2.0342910 1.0186855 1.0838678
#> [13] 0.5486730 -0.2439510 -0.1831147 2.2026024 0.1925301 -0.2153864
#> [19] 2.8944894 1.9213206 1.3804706
But the problem with your code is that you are trying to write three values (rnorm(3,mean=mu.mat[i],sd=0.7)) into a single atomic index mu.mat2[i]. It's not clear whether you were expecting a matrix as a result, but if so your loop would be:
mu.mat2 <- matrix(ncol = 3, nrow = 7)
for(i in 1:7) {
mu.mat2[i,] <- rnorm(3, mean = mu.mat[i], sd = 0.7)
}
If you were wanting the result as a 7 x 3 matrix, you can do:
matrix(rnorm(21, mean = rep(mu.mat, each = 3), sd = 0.7), ncol = 3, byrow = TRUE)
#> [,1] [,2] [,3]
#> [1,] -0.96624036 -1.4808460 -2.6824842
#> [2,] -2.88942108 -1.7299094 -3.0446737
#> [3,] -2.82034688 -0.9570087 -2.1822797
#> [4,] 0.58997289 1.0384926 1.8111506
#> [5,] -0.07705959 -0.1024418 0.7249310
#> [6,] 0.48851487 1.4729882 0.6496858
#> [7,] 1.47961292 1.5653253 2.0629409
Try replicate like below
> replicate(3,rnorm(length(mu.mat),mu.mat,0.7))
[,1] [,2] [,3]
[1,] -2.19324092 -1.13895278 -2.1540788
[2,] 0.02102746 0.33894402 0.1077604
[3,] 1.00363528 1.26895511 1.9483744
[4,] -3.85258144 -4.15638335 -4.0041507
[5,] -0.05518348 0.05766686 -0.3700564
[6,] 0.21570611 2.45016846 1.1614128
[7,] -0.81698877 -0.76824819 -1.5786689
I am a beginner in R and I have written a double-for loop for calculating chi2 values for selecting features among 6610 terms and 10 classes.
Here is my for loops:
library(raster)
#for x^2 [n,r] = term n, class r. n starts from col #7 and r starts from col #6617
chi2vals <- matrix(0:0,6610,10)
chi2avgs <- vector("numeric",6610L)
for(r in 1:10){
for(n in 1:6610){
A = sum(data1.sub.added[,6+n]==1 & data1.sub.added[,6616+r]==1)
M = sum(data1.sub.added[,6+n]==1)
P = sum(data1.sub.added[,6616+r]==1)
N = nrow(data1.sub.added)
E = ((A*N)-(M*P))**2
F = (N-P)*(N-M)
chi2vals[n,r] = (N/(P*M))*(E/F) # for term n
}
Prcj = sum(data1.sub.added[,6616+r]==1)/sum(data1.sub.added[,6616:6626]==1) #probability of class c_r
pchi <- Prcj * chi2vals
chi2avgs[n] = rowSums(pchi)[n]
}
The code correctly calculates everything up to the line pchi <- Prcj * chi2vals. The result is a nice matrix of p*chi2 values:
> head(pchi)
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]
[1,] 128.36551442 0.239308113 0.683517530 1.5038665 0.6145058 3.656857e-01 1.3311564 2.6977448 0.410702803
[2,] 0.06632758 0.067970859 0.019178551 0.2900692 1.5300639 4.430705e-08 0.2599859 0.6362953 0.098745147
[3,] 1.85641330 1.411925435 3.590747764 7.3018416 38.8044465 4.102248e-01 6.4118078 13.0164994 1.709506238
[4,] 0.11063892 0.005039029 0.244964758 0.1622654 0.1156411 8.274468e+00 0.2564959 0.0577651 0.242946022
[5,] 0.04788648 0.049072885 0.001420669 0.2094211 1.7200152 2.045923e-01 0.1877019 0.1468187 0.005493183
[6,] 5.39946188 6.899336618 60.735646913 7.4351538 10.7005784 9.946261e+00 35.8868899 178.7112406 11.382740754
[,10]
[1,] 0.26436516
[2,] 0.14414444
[3,] 0.90292073
[4,] 0.01168997
[5,] 0.06641298
[6,] 19.68599142
But the final chi2avgs values mostly turn out to be zeros:
> head(chi2avgs)
[1] 0.000000 0.000000 0.000000 0.000000 2.638835 0.000000
However, when aside from the loop I replace n with any number, the last line works well:
chi2avgs[1] = rowSums(pchi)[1]
chi2avgs[2] = rowSums(pchi)[2]
chi2avgs[3] = rowSums(pchi)[3]
chi2avgs[4] = rowSums(pchi)[4]
chi2avgs[5] = rowSums(pchi)[5]
> head(chi2avgs)
[1] 136.476367 3.112781 75.416334 9.481914 2.638835 0.000000
I wonder what causes this problem. Do you have an idea how I can fix it?
You can try directly rowsums without [n]
chi2avgs = rowSums(pchi)
I have a matrix of n variables and I want to make an new matrix that is a pairwise difference of each vector, but not of itself. Here is an example of the data.
Transportation.services Recreational.goods.and.vehicles Recreation.services Other.services
2.958003 -0.25983789 5.526694 2.8912009
2.857370 -0.03425164 5.312857 2.9698044
2.352275 0.30536569 4.596742 2.9190123
2.093233 0.65920773 4.192716 3.2567390
1.991406 0.92246531 3.963058 3.6298314
2.065791 1.06120930 3.692287 3.4422340
I tried running a for loop below, but I'm aware that R is very slow with loops.
Difference.Matrix<- function(data){
n<-2
new.cols="New Columns"
list = list()
for (i in 1:ncol(data)){
for (j in n:ncol(data)){
name <- paste("diff",i,j,data[,i],data[,j],sep=".")
new<- data[,i]-data[,j]
list[[new.cols]]<-c(name)
data<-merge(data,new)
}
n= n+1
}
results<-list(data=data)
return(results)
}
As I said before the code is running very slow and has not even finished a single run through yet. Also I apologize for the beginner level coding. Also I am aware this code leaves the original data on the matrix, but I can delete it later.
Is it possible for me to use an apply function or foreach on this data?
You can find the pairs with combn and use apply to create the result:
apply(combn(ncol(d), 2), 2, function(x) d[,x[1]] - d[,x[2]])
## [,1] [,2] [,3] [,4] [,5] [,6]
## [1,] 3.217841 -2.568691 0.0668021 -5.786532 -3.151039 2.6354931
## [2,] 2.891622 -2.455487 -0.1124344 -5.347109 -3.004056 2.3430526
## [3,] 2.046909 -2.244467 -0.5667373 -4.291376 -2.613647 1.6777297
## [4,] 1.434025 -2.099483 -1.1635060 -3.533508 -2.597531 0.9359770
## [5,] 1.068941 -1.971652 -1.6384254 -3.040593 -2.707366 0.3332266
## [6,] 1.004582 -1.626496 -1.3764430 -2.631078 -2.381025 0.2500530
You can add appropriate names with another apply. Here the column names are very long, which impairs the formatting, but the labels tell what differences are in each column:
x <- apply(combn(ncol(d), 2), 2, function(x) d[,x[1]] - d[,x[2]])
colnames(x) <- apply(combn(ncol(d), 2), 2, function(x) paste(names(d)[x], collapse=' - '))
> x
Transportation.services - Recreational.goods.and.vehicles Transportation.services - Recreation.services
[1,] 3.217841 -2.568691
[2,] 2.891622 -2.455487
[3,] 2.046909 -2.244467
[4,] 1.434025 -2.099483
[5,] 1.068941 -1.971652
[6,] 1.004582 -1.626496
Transportation.services - Other.services Recreational.goods.and.vehicles - Recreation.services
[1,] 0.0668021 -5.786532
[2,] -0.1124344 -5.347109
[3,] -0.5667373 -4.291376
[4,] -1.1635060 -3.533508
[5,] -1.6384254 -3.040593
[6,] -1.3764430 -2.631078
Recreational.goods.and.vehicles - Other.services Recreation.services - Other.services
[1,] -3.151039 2.6354931
[2,] -3.004056 2.3430526
[3,] -2.613647 1.6777297
[4,] -2.597531 0.9359770
[5,] -2.707366 0.3332266
[6,] -2.381025 0.2500530
I have a list prob with 50 elements. Each element is a 601x3 matrix of probabilities, each row of which represents a complete sample space (i.e., each row of each matrix sums to 1). For instance, here are the first five rows of the first element of prob:
> prob[[1]][1:5,]
[,1] [,2] [,3]
[1,] 0.6027004 0.3655563 0.03174335
[2,] 0.6013667 0.3665756 0.03205767
[3,] 0.6000306 0.3675946 0.03237481
[4,] 0.5986921 0.3686131 0.03269480
[5,] 0.5973513 0.3696311 0.03301765
Now, what I want to do is to create the following matrix for each row of each matrix/element in the list prob. Taking the first row, let a = .603, b = .366, and c = .032 (rounding to three decimal places). Then,
> w
[,1] [,2] [,3]
[1,] a*(1-a) -a*b -a*c
[2,] -b*a b*(1-b) -b*c
[3,] -c*a -c*b c*(1-c)
Such that:
> w
[,1] [,2] [,3]
[1,] 0.239391 -0.220698 -0.019296
[2,] -0.220698 0.232044 -0.011712
[3,] -0.019296 -0.011712 0.030976
I want to obtain a similar 3x3 matrix 600 more times (for the rest of the rows of this matrix) and then to repeat this entire process 49 more times for the rest of the elements of prob. The only thing I can think of is to call apply within lapply so that I am accessing each row of each matrix one-at-a-time. I'm sure that is not an elegant way to do this (not to mention I can't get it to work), but I can't think of anything else. Can anyone help me out with this? I'd also love to hear suggestions for using a different structure (e.g., is it bad to use matrices within lists?).
Running this process with lapply on a list of similarly dimensioned matrices should be very simple. If it represents a challenge, then you should post the dput(.) output for a two element list with similar matrices. The challenge is really to do the processing row by row which is illustrated here with the output being a 3x3xN array:
w <- apply(M, 1, function(rw) diag( rw*(1-rw) ) +
rbind( rw*c(0, -rw[1], -rw[1] ),
rw*c(-rw[2],0, -rw[2] ),
rw*c(-rw[3], -rw[3], 0)
)
)
w
[,1] [,2] [,3] [,4] [,5]
[1,] 0.23945263 0.23972479 0.23999388 0.24025987 0.24052272
[2,] -0.22032093 -0.22044636 -0.22056801 -0.22068575 -0.22079962
[3,] -0.01913173 -0.01927842 -0.01942588 -0.01957412 -0.01972314
[4,] -0.22032093 -0.22044636 -0.22056801 -0.22068575 -0.22079962
[5,] 0.23192489 0.23219793 0.23246881 0.23273748 0.23300395
[6,] -0.01160398 -0.01175156 -0.01190081 -0.01205173 -0.01220435
[7,] -0.01913173 -0.01927842 -0.01942588 -0.01957412 -0.01972314
[8,] -0.01160398 -0.01175156 -0.01190081 -0.01205173 -0.01220435
[9,] 0.03073571 0.03102998 0.03132668 0.03162585 0.03192748
w <- array(w, c(3,3,5) )
w
, , 1
[,1] [,2] [,3]
[1,] 0.23945263 -0.22032093 -0.01913173
[2,] -0.22032093 0.23192489 -0.01160398
[3,] -0.01913173 -0.01160398 0.03073571
, , 2
[,1] [,2] [,3]
[1,] 0.23972479 -0.22044636 -0.01927842
[2,] -0.22044636 0.23219793 -0.01175156
[3,] -0.01927842 -0.01175156 0.03102998
.... snipped remaining output