Replace Date Column's NA with New Date in R [duplicate] - r

Question

I am using the function ifelse() to manipulate a date vector. I expected the result to be of class Date, and was surprised to get a numeric vector instead. Here is an example:
dates <- as.Date(c('2011-01-01', '2011-01-02', '2011-01-03', '2011-01-04', '2011-01-05'))
dates <- ifelse(dates == '2011-01-01', dates - 1, dates)
str(dates)
This is especially surprising because performing the operation across the entire vector returns a Date object.
dates <- as.Date(c('2011-01-01', '2011-01-02', '2011-01-03', '2011-01-04','2011-01-05'))
dates <- dates - 1
str(dates)
Should I be using some other function to operate on Date vectors? If so, what function? If not, how do I force ifelse to return a vector of the same type as the input?
The help page for ifelse indicates that this is a feature, not a bug, but I'm still struggling to find an explanation for what I found to be surprising behavior.

Answer 1

You may use data.table::fifelse (data.table >= 1.12.3) or dplyr::if_else.
data.table::fifelse
Unlike ifelse, fifelse preserves the type and class of the inputs.
library(data.table)
dates <- fifelse(dates == '2011-01-01', dates - 1, dates)
str(dates)
# Date[1:5], format: "2010-12-31" "2011-01-02" "2011-01-03" "2011-01-04" "2011-01-05"
dplyr::if_else
From dplyr 0.5.0 release notes:
[if_else] have stricter semantics that ifelse(): the true and false arguments must be the same type. This gives a less surprising return type, and preserves S3 vectors like dates" .
library(dplyr)
dates <- if_else(dates == '2011-01-01', dates - 1, dates)
str(dates)
# Date[1:5], format: "2010-12-31" "2011-01-02" "2011-01-03" "2011-01-04" "2011-01-05"

Answer 2

It relates to the documented Value of ifelse:
A vector of the same length and attributes (including dimensions and "class") as test and data values from the values of yes or no. The mode of the answer will be coerced from logical to accommodate first any values taken from yes and then any values taken from no.
Boiled down to its implications, ifelse makes factors lose their levels and Dates lose their class and only their mode ("numeric") is restored. Try this instead:
dates[dates == '2011-01-01'] <- dates[dates == '2011-01-01'] - 1
str(dates)
# Date[1:5], format: "2010-12-31" "2011-01-02" "2011-01-03" "2011-01-04" "2011-01-05"
You could create a safe.ifelse:
safe.ifelse <- function(cond, yes, no){ class.y <- class(yes)
X <- ifelse(cond, yes, no)
class(X) <- class.y; return(X)}
safe.ifelse(dates == '2011-01-01', dates - 1, dates)
# [1] "2010-12-31" "2011-01-02" "2011-01-03" "2011-01-04" "2011-01-05"
A later note: I see that Hadley has built an if_else into the the magrittr/dplyr/tidyr complex of data-shaping packages.

Answer 3

DWin's explanation is spot on. I fiddled and fought with this for a while before I realized I could simply force the class after the ifelse statement:
dates <- as.Date(c('2011-01-01','2011-01-02','2011-01-03','2011-01-04','2011-01-05'))
dates <- ifelse(dates=='2011-01-01',dates-1,dates)
str(dates)
class(dates)<- "Date"
str(dates)
At first this felt a little "hackish" to me. But now I just think of it as a small price to pay for the performance returns that I get from ifelse(). Plus it's still a lot more concise than a loop.

Answer 4

The reason why this won't work is because, ifelse() function converts the values to factors. A nice workaround would be to convert it to characters before evaluating it.
dates <- as.Date(c('2011-01-01','2011-01-02','2011-01-03','2011-01-04','2011-01-05'))
dates_new <- dates - 1
dates <- as.Date(ifelse(dates =='2011-01-01',as.character(dates_new),as.character(dates)))
This wouldn't require any library apart from base R.

Answer 5

The suggested method does not work with factor columns. Id like to suggest this improvement:
safe.ifelse <- function(cond, yes, no) {
class.y <- class(yes)
if (class.y == "factor") {
levels.y = levels(yes)
}
X <- ifelse(cond,yes,no)
if (class.y == "factor") {
X = as.factor(X)
levels(X) = levels.y
} else {
class(X) <- class.y
}
return(X)
}
By the way: ifelse sucks... with great power comes great responsibility, i.e. type conversions of 1x1 matrices and/or numerics [when they should be added for example] is ok to me but this type conversion in ifelse is clearly unwanted. I bumped into the very same 'bug' of ifelse multiple times now and it just keeps on stealing my time :-(
FW

Answer 6

The answer provided by #fabian-werner is great, but objects can have multiple classes, and "factor" may not necessarily be the first one returned by class(yes), so I suggest this small modification to check all class attributes:
safe.ifelse <- function(cond, yes, no) {
class.y <- class(yes)
if ("factor" %in% class.y) { # Note the small condition change here
levels.y = levels(yes)
}
X <- ifelse(cond,yes,no)
if ("factor" %in% class.y) { # Note the small condition change here
X = as.factor(X)
levels(X) = levels.y
} else {
class(X) <- class.y
}
return(X)
}
I have also submitted a request with the R Development team to add a documented option to have base::ifelse() preserve attributes based on user selection of which attributes to preserve. The request is here: https://bugs.r-project.org/bugzilla/show_bug.cgi?id=16609 - It has already been flagged as "WONTFIX" on the grounds that it has always been the way it is now, but I have provided a follow-up argument on why a simple addition might save a lot of R users headaches. Perhaps your "+1" in that bug thread will encourage the R Core team to take a second look.
EDIT: Here's a better version that allows the user to specify which attributes to preserve, either "cond" (default ifelse() behaviour), "yes", the behaviour as per the code above, or "no", for cases where the attributes of the "no" value are better:
safe_ifelse <- function(cond, yes, no, preserved_attributes = "yes") {
# Capture the user's choice for which attributes to preserve in return value
preserved <- switch(EXPR = preserved_attributes, "cond" = cond,
"yes" = yes,
"no" = no);
# Preserve the desired values and check if object is a factor
preserved_class <- class(preserved);
preserved_levels <- levels(preserved);
preserved_is_factor <- "factor" %in% preserved_class;
# We have to use base::ifelse() for its vectorized properties
# If we do our own if() {} else {}, then it will only work on first variable in a list
return_obj <- ifelse(cond, yes, no);
# If the object whose attributes we want to retain is a factor
# Typecast the return object as.factor()
# Set its levels()
# Then check to see if it's also one or more classes in addition to "factor"
# If so, set the classes, which will preserve "factor" too
if (preserved_is_factor) {
return_obj <- as.factor(return_obj);
levels(return_obj) <- preserved_levels;
if (length(preserved_class) > 1) {
class(return_obj) <- preserved_class;
}
}
# In all cases we want to preserve the class of the chosen object, so set it here
else {
class(return_obj) <- preserved_class;
}
return(return_obj);
} # End safe_ifelse function

Answer 7

Why not use indexing here?
> dates <- as.Date(c('2011-01-01', '2011-01-02', '2011-01-03', '2011-01-04', '2011-01-05'))
> dates[dates == '2011-01-01'] <- NA
> str(dates)
Date[1:5], format: NA "2011-01-02" "2011-01-03" "2011-01-04" "2011-01-05"