R how tocheck if a data table column is all zeros - r

Question

I would like to remove the columns that have all zeros. But, some of the columns appear to have non numeric values. How can I remove the non numeric columns, and the columns with all zeros. It would be helpful if the non numeric column name was printed, or the column number, so I can determine if it was ok to remove the column.
Here's what I'm trying, but it doesn't work when the data table has non numeric values.
removeColsAllZeros = function(ddt) {
m <- as.matrix(ddt)
# isNumericColList <- lapply(1:ncol(m), function(ii,mm){is.numeric(mm[,ii])}, mm=m)
# indexNonNumericCols <- which(!unlist(isNumericColList))
mnz <- m[, colSums(abs(m),na.rm = TRUE) != 0]
return(mnz)
}

Answer 1

Here's a simple function that can be applied to all columns in your data frame, returning just the ones that are numeric and not all zero:
# Fake data
dat = data.frame(x=rnorm(5),
y=rep(0,5),
z=sample(c(1,0),5,replace=TRUE),
w=sample(LETTERS[1:3],5,replace=TRUE),
stringsAsFactors=FALSE)
dat
x y z w
1 0.5450570 0 0 B
2 0.5292899 0 0 B
3 -0.2142306 0 1 C
4 -0.7246841 0 0 C
5 -0.7567683 0 1 A
# Remove columns with all zeros or that are not numeric
dat[, !sapply(names(dat), function(col) {all(dat[,col]==0) |
!is.numeric(dat[,col])})]
x z
1 0.5450570 0
2 0.5292899 0
3 -0.2142306 1
4 -0.7246841 0
5 -0.7567683 1
To unpack this, the function checks, for a single column of dat, whether it has all zeros or is not numeric. sapply then "applies" this function to every column in the data frame, returning a logical vector with TRUE for columns of dat with all zeros or that are non-numeric, and FALSE for columns that are numeric and not all zeros. The ! ("NOT") before sapply just reverses the FALSE and TRUE values:
!sapply(names(dat), function(col) {
all(dat[, col]==0) | !is.numeric(dat[, col])
})
x y z w
TRUE FALSE TRUE FALSE
Then we use this logical vector to return only those columns of dat that are TRUE.
dat[ , c(TRUE, FALSE, TRUE, FALSE)]
x z
1 0.5450570 0
2 0.5292899 0
3 -0.2142306 1
4 -0.7246841 0
5 -0.7567683 1
Finally, to check the non-numeric columns that were removed, do the following, which will return all non-numeric columns:
dat[, sapply(names(dat), function(col) {!is.numeric(dat[,col])})]

Answer 2

This is not compact but works on data table after modifying #eipi10's code.
# toy data
set.seed(1)
dat = data.frame(x=rnorm(5),
y=rep(0,5),
z=sample(c(1,0),5,replace=TRUE),
w=sample(LETTERS[1:3],5,replace=TRUE),
stringsAsFactors=FALSE)
# code for a data table
library(data.table)
setDT(dat)
idx = sapply(dat, function(x){ !(all(x==0) | !is.numeric(x)) })
dat[, .SD, .SDcols = idx]
# x z
# 1: -0.6264538 1
# 2: 0.1836433 1
# 3: -0.8356286 0
# 4: 1.5952808 1
# 5: 0.3295078 0

Answer 3

Both of the other answers were helpful, but they didn't totally answer the question. Here's a function with to identify and remove the non-numeric and all zero columns from a data table. This was helpful and provided additional insight into the data set.
removeColsAllZeros = function(ddt) {
# Identify and remove nonnumeric cols and cols with all zeros
idx_all_zeros = ddt[, lapply(.SD, function(x){ (is.numeric(x) & all(x==0)) })]
idx_not_numeric = ddt[, lapply(.SD, function(x){ (!is.numeric(x)) })]
idx_all_zeros = which(unlist(idx_all_zeros))
idx_not_numeric = which(unlist(idx_not_numeric))
# Print bad column names
if (length(idx_all_zeros)>0) {
cat('Numeric columns with all zeros are\n',paste(names(ddt)[idx_all_zeros],collapse='\n'),'\n')
flush.console()
}
if (length(idx_not_numeric)>0) {
cat('Nonnumeric columns are\n',paste(names(ddt)[idx_not_numeric],collapse='\n'),'\n')
flush.console()
}
# Determine the numeric columns that have nonzero values
idx_bad = union(idx_all_zeros, idx_not_numeric)
idx_good = setdiff(seq(1,ncol(ddt)), idx_bad)
# Return nonzero numeric data
ddt[, .SD, .SDcols = idx_good]
}