Creating iterator for a data frame in R - r

Question

I'm trying to create a function in R using an iterator from the iterator package: https://cran.r-project.org/web/packages/iterators/iterators.pdf to help iterator through each row of a data frame.
Given a table like this:
data<-data.frame(c(1,0,0,NA,NA,1,1,NA,0), ncol=3, byrow=TRUE)
>data
[,1] [,2] [,3]
[1,] 1 0 0
[2,] NA NA 1
[3,] 1 NA 0
I want it to go through each row and return the first non NA value from left to right and return NA if all values are NA. So with the above data frame, it should return 1, 1, 1.
The general idea I have right now is to use the iter() function from the package like so:
vec<-vector()
iterRow<-iter(data[x,]) #Creates iterator object for row x of data
i<-1
while(i<iterRow$length){ #iterRow$length gives # of columns essentially
temp<-nextElem(iterRow) #Set temp to the next element of the iterator
if(!is.na(temp)){ #If the value is not NA, set value in vec to the value
vec<-c(vec, temp)
}
i<-i+1
}
vec<-c(vec, NA) #Otherwise set it to NA
return(vec)
The data i'm working with will be up to millions of rows long so ideally I would like to vectorize the function. I'm stuck on how to apply that idea across the whole data frame.
Would it work to make the function like this:
iterateRows<- function(dataFrame){
...
}
with the data frame i'm working with as the argument.
I also do know c++ so if using c++ to write a similar function would be easier, I could also do that. Any help will be appreciated!

Answer 1

Start with a simple approach. Here's a function to do what you want to each row:
first_not_na = function(x) if(all(is.na(x))) NA else x[!is.na(x)][1]
Here's a couple simple ways to apply that to every row of data.
# apply
results = apply(data, 1, foo)
# for loop
results = numeric(nrow(data))
for (i in 1:row) results[i] = foo(data[i, ])
Here's a benchmark comparing timings on pretty large data:
row = 1e6
col = 5
data = matrix(sample(c(1, 0, NA), size = row * col, replace = T), nrow = row)
microbenchmark::microbenchmark(
apply = {results = apply(data, 1, foo)},
loop = {
results = numeric(row)
for (i in 1:row) results[i] = foo(data[i, ])
},
times = 5
)
# Unit: seconds
# expr min lq mean median uq max neval cld
# apply 2.140379 2.249405 2.399239 2.480180 2.524667 2.601563 5 a
# loop 1.970481 1.982853 2.160342 2.090484 2.264797 2.493095 5 a
A simple for loop takes about 2 seconds for 1M rows and 5 columns. If you want to speed up more, you could certainly use foreach to parallelize. Only if that's still not fast enough should you look for more complex solutions like iterators or an implementation in C++.