R refer to multiple dataframe columns to exclude data - r

Question

I have a large data frame that I am working with from which I need to exclude rows that contain all but a small number of characters.
Currently I am using the following code to do so and it is working fine, but I can only seem to apply this to a single column at a time which is not only inefficient but time consuming on my part as I have many columns to work through.
df <- df[(df$column_name_01 %in% c("a", "b", "c", "d")),]
So far I have tried referring to multiple columns like so (as this approach works for single columns):
df <- df[(df[, 1:10] %in% c("a", "b", "c", "d")),]
But this is obviously not working as intended. Is there a concise way to exclude rows from a data frame that contain certain characters (or that do not match certain characters, either way)?

Answer 1

I think you want a regular old apply here:
df[apply(df[,1:10], 1, function(x) all(x %in% c("a", "b", "c", "d"))),]
Or for non-matching rows
df[apply(df[,1:10], 1, function(x) all(! x %in% c("a", "b", "c", "d"))),]

Answer 2

You can compute whether you want each row across the 10 columns, combining into a single vector with Reduce and "&":
df[Reduce("&", lapply(df[,1:10], function(x) x %in% c("a", "b", "c", "d"))),]
# NA NA NA NA NA NA NA NA NA NA
# 14 a c a d d c d c c c
# 25 b a a a b a c a a c
# 29 d d d a a a b c c a
# 31 c b b d b c a b b c
# 33 b a c b a d c a a c
# 36 c d c b d a c a a a
# 42 b b a a b c d b d d
# 45 c c b b d a b a d b
You could also do this by converting the data frame to a matrix and using rowSums to make sure all values in the row fall in the desired set:
df[rowSums(matrix(unlist(df[,1:10]) %in% c("a", "b", "c", "d"), nrow(df))) == 10,]
# NA NA NA NA NA NA NA NA NA NA
# 14 a c a d d c d c c c
# 25 b a a a b a c a a c
# 29 d d d a a a b c c a
# 31 c b b d b c a b b c
# 33 b a c b a d c a a c
# 36 c d c b d a c a a a
# 42 b b a a b c d b d d
# 45 c c b b d a b a d b
Both of these solutions should be faster than an apply-based solution for large matrices (I benchmark a 100k-row data frame here) because they operate on a small number of columns instead of a large number of rows, better taking advantage of vectorization:
josilber.lapply <- function(df) df[Reduce("&", lapply(df[,1:10], function(x) x %in% c("a", "b", "c", "d"))),]
josilber.rowSums <- function(df) df[rowSums(matrix(unlist(df[,1:10]) %in% c("a", "b", "c", "d"), nrow(df))) == 10,]
crimson.apply <- function(df) df[apply(df[,1:10], 1, function(x) all(x %in% c("a", "b", "c", "d"))),]
library(microbenchmark)
microbenchmark(josilber.lapply(big.df), josilber.rowSums(big.df), crimson.apply(big.df))
# Unit: milliseconds
# expr min lq mean median uq max neval
# josilber.lapply(big.df) 67.17092 71.0628 83.36787 74.74011 86.00722 231.6794 100
# josilber.rowSums(big.df) 98.75142 116.3975 136.28880 128.28851 149.55155 301.9346 100
# crimson.apply(big.df) 676.66290 725.6616 789.45954 762.74171 805.72437 2681.8203 100
Data:
set.seed(144)
df <- unname(do.call(data.frame, replicate(10, sample(letters[1:5], 50, replace=TRUE), simplify=FALSE)))
set.seed(144)
big.df <- unname(do.call(data.frame, replicate(10, sample(letters[1:5], 100000, replace=TRUE), simplify=FALSE)))