I'm trying to perform basic excel-like formula-filling in R. I want to populate the value of a "cell" based on the values of other cells in the same matrix or data.frame. The function is pretty straightforward to do with a single cell, but seems to be more difficult to scale across both rows and columns.
Say I have a simple matrix:
simple <- matrix(c(0,1,2,3,0,4,5,6,7,NA,NA,NA,8,NA,NA,NA), nrow = 4, ncol = 4)
[,1] [,2] [,3] [,4]
[1,] 0 0 7 8
[2,] 1 4 NA NA
[3,] 2 5 NA NA
[4,] 3 6 NA NA
I want to populate the NAs with the sum of columns 1 and 2 in the same row and row 1 in the same column. In Excel, for cell C2 it would be
=$A2 + $B2 + C$1
in R
simple[2,3] <- simple[2,1] + simple[2,2] + simple[1,3]
In Excel, you can simply drag the formula over the remaining cells, and voila. In R, not so easy.
Since r is vectorized, I can fill a whole column pretty easily by giving ranges instead of single cells, like so:
simple[2:4,3] <- simple[2:4,1] + simple[2:4,2] + simple[1,3]
[,1] [,2] [,3] [,4]
[1,] 0 0 7 8
[2,] 1 4 12 NA
[3,] 2 5 14 NA
[4,] 3 6 16 NA
But when I try to vectorize over both rows and columns, it doesn't work because it interprets the last value as the vector c(7,8), and tries to add that in a row-wise fashion, rather than adding it column-wise.
simple[2:4,3:4] <- simple[2:4,1] + simple[2:4,2] + simple[1,3:4]
Warning message:
In simple[2:4, 1] + simple[2:4, 2] + simple[1, 3:4] :
longer object length is not a multiple of shorter object length
[,1] [,2] [,3] [,4]
[1,] 0 0 7 8
[2,] 1 4 12 12
[3,] 2 5 15 15
[4,] 3 6 16 16
As an alternative solution, one could do nested for loops, as below:
for (i in 2:4){
for (j in 3:4){
simple[i,j] <- simple[i,1] + simple[i,2] + simple[1,j]
}
}
[,1] [,2] [,3] [,4]
[1,] 0 0 7 8
[2,] 1 4 12 13
[3,] 2 5 14 15
[4,] 3 6 16 17
This actually works and is pretty easy, but it involves nested for loops, so, enough said.
I feel like the "right" solution would be one using correct vectorization, apply(), or dplyr, but I can't seem to figure out how to make them work, short of rearranging the data from a crosstab format to a flat format, but that can explode your file size pretty quickly.
Any ideas on how to make this work in a more R-ish fashion?
Here's a more R like way to do it, let's convert simple to a data.frame first.
library(tidyverse)
df1 <- as.data.frame(simple)
df1 %>% mutate(V3 = V1 + V2 + first(V3), V4 = V1 + V2 + first(V4))
V1 V2 V3 V4
1 0 0 7 8
2 1 4 12 13
3 2 5 14 15
4 3 6 16 17
first from dplyr is handy because it lets you lock to the first value in the column, like you would in Excel with C$1
In matrix arithmetic, each component must be same dimension or any being a single-item vector. Therefore, consider aligning by replicating 7 and 8 for each needed row 2-4 (i.e., 3 times). Then transpose for 2 X 3 dimension:
simple[2:4,3:4] <- simple[2:4,1] + simple[2:4,2] + t(replicate(length(2:4), simple[1,3:4]))
Alternatively, consider sapply iterating through 7 and 8 values respectively:
simple[2:4,3:4] <- sapply(3:4, function(i) simple[2:4,1] + simple[2:4,2] + simple[1,i])
Slightly more concise with rowSums and leaving out row indexing:
simple[,3:4] <- sapply(3:4, function(i) rowSums(simple[,1:2]) + simple[1,i])
I may be late to the game but here is a data.table and base R solution which for large data sets is much faster than tidyverse. The syntax may look more confusing at first but breaking it down piece by piece is very logical and straight-forward once you have a good handle on lapply.
To make the cell and the vectors you are adding compatible you should convert the cell to a vector by simply replicating that value as many times as the number of observations or rows of the dataframe. So in your example, V3 = rep(7,4) will yield a vector with all 7s. R will then let you do V3=V1+V2+V3, where V3 on the right-hand side is the rep(7,4).
The data.table has some handy built-in special read-only symbols that will also give you the ability to extend the solution beyond the two columns you provided in the example. The two I use most frequently are .SD and .N. In this example, you can think of .SD as a way to refer to all columns except the first two and .N is always a constant number equal to the number of rows in the data.table. These symbols can be used in the j slot of a data.table which is equivalent to the columns of a matrix or data.frame object. So your code would look like this:
simple <- data.table(simple)
NAcols <- colnames(simple)[-c(1,2)] ##Can modify this to get names of columns you wish to change if its not the first two using match or grep. I can add that if you want?
simple[,NAcols:=lapply(.SD,function(i) V1+V2+rep(i[1],.N)),.SDcols=NAcols]
Note that each iteration in the lapply loop is simply the ith column and i[1] selects only the first element of that column and replicates it as many times as the number of rows (.N) before adding the three vectors together. The .SDcols is used to prevent this function from being applied to the first two columns. Though there was no need in this problem to group, data.table also allows you to specify 'by = ' as an argument if you want to group by a particular column or columns in the data.table before applying the function. Finally note that I did not need to assign the last line of code to another R object because data.table updates the old columns of 'simple' using pointers which is why it is so much faster than base R and tidyverse data frame objects. However you can use the copy function of data.table like this instead if you wish to save the original data.table for some reason:
final_result <- copy(simple)[,NAcols:=lapply(.SD,function(i) V1+V2+rep(i[1],.N)),.SDcols=NAcols]
Anyway I hope that explanation helps and if you need me to clarify anything please let me know! Best of luck!
say I have a spreadsheet that I imported or am working with a regular dataset. how do I retrieve a list of the top minimum values?
for example.
x<-c(6,7,8,9,10,1,2,3,4,5,11,12,13,14)
how do I get the position of the top 5 minimum values in the x vector (6,7,8,9,10)?
You can use order or sort.list to obtain the permutation of indices that can sort the object in ascending order (or descending if you pass descending = TRUE). Then you can obtain the necessary values using head or passing the indices ([1:5] in this example for the first five) directly
order(x)[1:5]
#[1] 6 7 8 9 10
sort.list(x)[1:5]
#[1] 6 7 8 9 10
We can use sort with index.return = TRUE
head(sort(x, index.return = TRUE)$ix,5)
Or with order
head(order(x),5)
#[1] 6 7 8 9 10
I have a vector of numbers stored in R.
my_vector <- c(1,2,3,4,5)
I want to add two to each number.
my_vector + 2
[1] 3 4 5 6 7
However, I want there to only be a twenty percent chance of adding two to the numbers in my vector each time I run the code. Is there a way to code this in R?
What I mean is, if I run the code, the output could be:
[1] 3 4 5 6 9
Or perhaps
[1] 5 4 5 6 7
i.e. there is only a 20% chance that any one number in the vector will get two added to it.
myvector + 2*sample(c(TRUE,FALSE), length(myvector), prob=c(0.2,0.8), repl=TRUE)
That will give a variable number of 2's to be added (which is what you were asking) but sometimes people want to know that exactly 20% will have a 2 added in whoch case it would be:
myvector + 2*sample(c(TRUE,rep(FALSE,4)))
How can I reduce the size of a vector to a lower dimension?
Say for example X:=(1,2,3,4,5,6,7,8,9,10) is a 10-D vector. Suppose
I want to reduce it to a 5 dimensional space. Is there any way to do this?
I have a situation where I need to compare an N-d vector with a corresponding vector of a lower dimension.
There are an infinite number of ways to convert a 10d vector into a 5d vector.
This is like saying "I want a function that takes two integer parameters and returns an integer, can I make such a function". There an infinite many such functions.
It really depends on what you want to do with the vector. What are the meanings of your 10d and 5d vectors?
If my assumption is right, the OP would like to convert a vector of 10 values to a matrix with 2 columns.
This could be done easily in R:
# make up the demo data
> v <- c(1,2,3,4,5,6,7,8,9,10)
# modify the dimensions of 'v' to have 2 columns
> dim(v) <- c(5,2)
# and check the result
> v
[,1] [,2]
[1,] 1 6
[2,] 2 7
[3,] 3 8
[4,] 4 9
[5,] 5 10