problems with deleting rows in big data sets in r - r

Question

I wrote a script that delete rows that 20% of their cells ara smaller then 10.
it's work great on small data Sets but for big it's useless.
can sombody help me please.
here is my script:
DataSets<-choose.files()
DataSet<-read.delim(DataSets,header = TRUE,
row.names = 1,sep="\t",blank.lines.skip=TRUE)
delete<-0
for(i in 1:length(DataSet[,1]))
{
count<-0
for(j in 1:length(DataSet[i,]))
{
if(DataSet[i,j]<10 || is.na(DataSet[i,j]))
{
count=count+1
}
}
if(count>0.2*length(DataSet[i,]))
{
DataSet=DataSet[-i,]
delete<-delete+1
}
}

Answer 1

This is essentially instantaneous on my machine:
m <- matrix(runif(100000),10000,10)
system.time(m1 <- m[rowSums((m <= 0.25 | is.na(m)) < 2,])
I only approximated your exact situation, but your version would be analogous. The idea here would be to:
Use a matrix, rather than a data frame, if your data is indeed all numeric.
Use vectorized comparison to determine which elements are less than some value (0.25 in my example).
Then use rowSums to count how many values are less than 0.25 in each row.
Subset the matrix according to which rows have fewer than two values less than (or equal to) 0.25.
Edit Added check for NAs to count them too.

Answer 2

This would solve your problem. You can leave your data as a DataFrame.
dat<-data.frame(matrix(rnorm(100,10,1),10))
bad<-apply(dat,1,function(x){
return((sum(x<10,na.rm=TRUE)+sum(is.na(x)))>length(x)*0.2)
})
dat<-dat[!bad,]

Answer 3

This works pretty quickly for me. Like the solution #joran used, I use a matrix:
data <- matrix(rnorm(1000, 15, 5), 100, 10)
tf <- apply(data, 1, function(x) x < 10) # your value of 10
data[-which(colSums(tf) > ncol(data)*0.2),] # here is where the 20% comes in
TRUE = 1 and FALSE = 0, which is why one can use colSums here
Update to handle NAs
If one follows OP's comment to include "just 20% of the numeric values" and not the original code that counts NA values as values < 10, (i.e. delete rows where 20 % of numeric entries are less than 10), then this will work:
data[-which(colSums(tf, na.rm=T) > (ncol(data) - colSums(apply(tf,2,is.na)))*0.2),]
colSums(apply(tf,2,is.na)) counts the number of entries in a row of data that are NA.
(ncol(data) - colSums(apply(tf,2,is.na))) subtracts that number from the number of columns so that only the total number of numeric columns is returned.
(ncol(data) - colSums(apply(tf,2,is.na)))*0.2 is 20% of the number of numeric entries per row