Function to remove columns with max value less than a given value, - r

Question

I'm doing initial data clean up with 34,000 columns in a dataframe and in order to do that, i have to remove columns whose max value is less than 2.
I'm clueless as to how to remove columns with maxvalue less than 2 but for just getting max values, I tried creating a function as below without converting data with is.numeric:
protein <- is.numeric(protein)
#a:
colMax <- function(data) sapply(data, max, na.rm = TRUE)
colMax(protein)
I got the max not meaningful for factors error, which is why i used the is.numeric function to convert all data to numeric form. despite doing that I still am not getting the desired result. When running the function I got 0 as a result rather than a list of max values for each column.
Why am i getting 0 for my max function?How do I setup a function that can generate max values for each column and remove any columns whose max values are less than 2? Would I need 2 separate functions?

Answer 1

Here is another way using dplyr to select columns where max value is greater than equal to 2. Assuming, we want to test for all the columns and all those columns are of class factor. Using #Maurits data
library(dplyr)
df %>%
#Convert column from factor to numeric
mutate_all(~as.numeric(as.character(.))) %>%
#Select column whose max value is greater than equal to 2
select_if(~max(., na.rm = TRUE) >= 2)
# V3 V4 V5 V6 V7 V8 V9 V10
#1 3 4 5 6 7 8 9 10
#2 3 4 5 6 7 8 9 10
#3 3 4 5 6 7 8 9 10
#4 3 4 5 6 7 8 9 10
#5 3 4 5 6 7 8 9 10
#6 3 4 5 6 7 8 9 10
#7 3 4 5 6 7 8 9 10
#8 3 4 5 6 7 8 9 10
#9 3 4 5 6 7 8 9 10
#10 3 4 5 6 7 8 9 10
Instead of max, we can also use any
df %>%
mutate_all(~as.numeric(as.character(.))) %>%
select_if(~any(. >= 2))
You say that you have 34000 columns. Do you want to check for greater than 2 condition for all the columns? Are all the columns factors ? The above code checks for all the columns and selects the one which do not satisfy the condition. If you want to do this on selected columns (not all), you might need to subset data, select those column and then apply the code.
In base R, we can also use colSums after converting the data from factor to numeric
df[] <- lapply(df, function(x) as.numeric(as.character(x)))
df[, colSums(df >= 2) > 0]

Answer 2

You were nearly there.
Since you don't provide reproducible sample data let's first create some minimal sample data
df <- as.data.frame(matrix(rep(1:10, each = 10), ncol = 10))
df
# V1 V2 V3 V4 V5 V6 V7 V8 V9 V10
#1 1 2 3 4 5 6 7 8 9 10
#2 1 2 3 4 5 6 7 8 9 10
#3 1 2 3 4 5 6 7 8 9 10
#4 1 2 3 4 5 6 7 8 9 10
#5 1 2 3 4 5 6 7 8 9 10
#6 1 2 3 4 5 6 7 8 9 10
#7 1 2 3 4 5 6 7 8 9 10
#8 1 2 3 4 5 6 7 8 9 10
#9 1 2 3 4 5 6 7 8 9 10
#10 1 2 3 4 5 6 7 8 9 10
We now would like to keep only those columns where the max value is >2; we can do this using sapply
df[sapply(df, function(x) max(x, na.rm = T) > 2)]
# V3 V4 V5 V6 V7 V8 V9 V10
#1 3 4 5 6 7 8 9 10
#2 3 4 5 6 7 8 9 10
#3 3 4 5 6 7 8 9 10
#4 3 4 5 6 7 8 9 10
#5 3 4 5 6 7 8 9 10
#6 3 4 5 6 7 8 9 10
#7 3 4 5 6 7 8 9 10
#8 3 4 5 6 7 8 9 10
#9 3 4 5 6 7 8 9 10
#10 3 4 5 6 7 8 9 10
Explanation: sapply loops over the columns of the data.frame df and returns a logical vector (with as many entries as there are columns in df).
Or we can use pmax with apply
df[apply(pmax(df) > 2, 2, all)]
giving the same result. The difference to the first method is that pmax returns a matrix on which we operate column-wise with apply(..., MARGIN = 2, ...).