lapply to find max value across rows not working as expected - r

Question

I am trying to find the max date across rows of a data.table using lapply. I have some rows where all values in the row are NA and in this case I want to return a specific date. I wrote a function to do this but I am not getting the results that I expected.
library(data.table)
my.max = function(x){
if(all(is.na(x))){
return(as.Date("9999-12-01")) #we can use this to identify which BPIDs have no end date
}else{
return(max(x, na.rm = T))
}
}
DT = data.table("Date1" = c(as.Date("2015-12-30"),NA, NA), "Date2" = c(as.Date("2013-02-04"), as.Date("2014-01-01"), NA))
DT[ , "Row" := 1:.N]
DT[ , "Max_Date" := lapply(.SD, my.max), by = .(Row), .SDcols = c("Date1", "Date2")]
This returns
> DT
Date1 Date2 Row Max_Date
1: 2015-12-30 2013-02-04 1 2015-12-30
2: <NA> 2014-01-01 2 9999-12-01
3: <NA> <NA> 3 9999-12-01
So, it does work if all values are NA, but if one of the values is NA it also returns 9999-12-01. I put print functions into my.max to find out what was happening and it looks like it passes in one value of x at a time. This explains why the all(is.na(x)) would be true, but I expected it to pass in a vector of both dates in the row. Otherwise, how would it know what values to take the max of?
How can I change my function so it returns 9999-12-01 only if both of the other dates are NA?

Answer 1

Here is one method that will work. It encapsulates multiple statements in {} to form a single code block:
DT[, "this" := {temp=pmax(Date1, Date2, na.rm=TRUE);
temp[is.na(temp)] = as.Date("9999-12-01"); temp}]
which returns
DT
Date1 Date2 this
1: 2015-12-30 2013-02-04 2015-12-30
2: <NA> 2014-01-01 2014-01-01
3: <NA> <NA> 9999-12-01
data
DT = data.table("Date1" = c(as.Date("2015-12-30"),NA, NA),
"Date2" = c(as.Date("2013-02-04"), as.Date("2014-01-01"), NA))
This way, you don't have to loop through each row which can be quite slow.
While I don't recommend by-row processing...
DT[ , "Row" := 1:.N]
DT[ , "Max_Date" := my.max(unlist(.SD)), by = .(Row), .SDcols = c("Date1", "Date2")]
will produce the same output for this example.

Answer 2

Try this out:
library(data.table)
my.max <- function(x){
if(all(is.na(x))){
return("9999-12-01")
}else{
return(max(x, na.rm = T))
}
}
DT <- data.table("Date1" = c(as.Date("2015-12-30"),NA, NA), "Date2" = c(as.Date("2013-02-04"), as.Date("2014-01-01"), NA))
print(DT)
DT[ , "Max_Date" ] <- apply(DT, 1, my.max)
print(DT)
> DT <- data.table("Date1" = c(as.Date("2015-12-30"),NA, NA), "Date2" = c(as.Date("2013-02-04"), as.Date("2014-01-01"), NA))
> print(DT)
Date1 Date2
1: 2015-12-30 2013-02-04
2: <NA> 2014-01-01
3: <NA> <NA>
> DT[ , "Max_Date" ] <- apply(DT, 1, my.max)
> print(DT)
Date1 Date2 Max_Date
1: 2015-12-30 2013-02-04 2015-12-30
2: <NA> 2014-01-01 2014-01-01
3: <NA> <NA> 9999-12-01