Generate list of datasets with randomly selected features in R - r

Question

I have a dataset with 20 features. I wish to create a list of datasets with random subsets of features from the original data set.
For example - [dataset[, c(1,3,4)], dataset[, c(2,3,5,11,20)]].
I am trying the following code for the same
selectors = array(runif(2000), dim=c(100, 20, 1))
list_datasets = vector("list", 100)
i = 1
while(i < 100)
list_datasets[[i]] = dataset[, selectors[i,,1] > 0.5]
i = i + 1
Here, my dataset has 20 features and I need 100 datasets with random features out of these 20 features. So I have created selectors array, by initially creating a vector of size 2000 with random values and then making it 100*20. In the while loop, I have then tried to add a feature to list_datasets[[i]] only if random value generated for it was greater than 0.5 . Hopefully, I was able to explain myself
But this is very slow. I am new to R and want to know what is the best way to achieve what I am trying to do.

Answer 1

I'm not totally sure if I understand your setup, so correct me if I missed something. My understanding is that you have a dataset (I created a fake dataset of size 100 rows x 20 features) and want to create 100 new datasets using a random subset of features. You generate a random subset of features by generating random uniform values and checking if each is > 0.5.
I have two options here, one using lapply and one using a for loop.
The apply functions are generally faster than loops (and I think you want to use a for loop, not a while loop here anyway).
Other changes:
1) Use a boolean mask as #Krash suggested, since you can check if each value is greater than 0.5 outside the loop, as whether or not it is does not depend on i.
2) selectors can be 2d
set.seed(123)
# Original dataset: assume it's 100 x 20 features
dataset <- array(rnorm(2000), dim = c(100, 20))
## Original (Option 0: while loop)
system.time({
# Select features: 100 x 20 x 1 (one row per dataset)
selectors = array(runif(2000), dim = c(100, 20, 1));
# Initialize list
list_datasets = vector("list", 100);
# Fill in list
i = 1;
while(i < 100) {
list_datasets[[i]] = dataset[, selectors[i,,1] > 0.5];
i = i + 1 # This causes an off-by-one error, as list_datasets[[100]] is never filled in
}
})
## user system elapsed
## 0.006 0.000 0.006
# Option 1: for loop
system.time({
# Select: boolean mask: 100 x 20 (need one row to create each dataset)
selectors = array(runif(2000), dim = c(100, 20));
selectors = selectors < 0.5
# Initialize list
list_datasets = vector("list", 100);
# Fill in list
for (i in 1:100) {
list_datasets[[i]] = dataset[ , selectors[i, ]]
}
})
## user system elapsed
## 0.004 0.000 0.005
# Option 2: lapply
system.time({
# Select: boolean mask: 100 x 20 (need one row to create each dataset)
selectors = array(runif(2000), dim = c(100, 20));
selectors = selectors < 0.5
# Fill in list
list_datasets <- lapply(1:100, FUN = function(x) dataset[ , selectors[x, ]])
})
## user system elapsed
## 0.003 0.000 0.003
Obviously the amount of time statements take to run vary each time you run the statements, but hopefully some of these suggested changes will improve the speed.
Just as a check to make sure the code did what I wanted it to do:
# Check number of cols per dataset
list_datasets %>%
purrr::map_int(~ncol(.))
## [1] 8 7 9 12 11 13 11 10 10 14 14 7 8 10 10 9 14 10 6 11 13 8 7 8 10 12 9 11 9 9 13
## [32] 12 8 14 11 11 8 10 11 8 10 13 12 10 6 10 10 12 9 9 10 11 7 8 11 9 11 9 7 9 9 11
## [63] 14 9 9 9 9 13 13 14 12 9 10 9 12 8 11 14 9 7 12 7 6 11 11 7 9 8 12 10 12 9 11
## [94] 13 12 16 9 8 11 10
Other thoughts: Rather than creating a selectors array via random uniforms with each row corresponding to a new dataset, you could add a line like this inside your loop (or lapply FUN).
include_feature <- sample(0:1, size = 20, replace = TRUE)
include_feature
## [1] 0 0 1 0 0 0 1 1 1 0 1 1 1 0 1 0 0 0 0 1