Count unique IDs for each group within x days - r

Question

This is a follow-up question to one I already posted here: Count occurrence of IDs within the last x days in R
I'm trying to do another rolling count. I have the following data:
date = c("2014-04-01", "2014-04-12", "2014-04-07", "2014-05-03", "2014-04-14", "2014-05-04", "2014-03-31", "2014-04-18", "2014-04-23", "2014-04-01")
group = c("G","G","F","G","E","E","H","H","H","A")
ID = c(2, 3, 4, 2, 3, 1, 2, 4, 2, 1)
group date ID
1: G 2014-04-01 2
2: G 2014-04-12 3
3: F 2014-04-07 4
4: G 2014-05-03 2
5: E 2014-04-14 3
6: E 2014-05-04 1
7: H 2014-03-31 2
8: H 2014-04-18 4
9: H 2014-04-23 2
10: A 2014-04-01 1
For each group, I would like to count the number of unique ID within the past 30 days of the current date. The desired count column would look like this:
group date ID count
1: G 2014-04-01 2 1
2: G 2014-04-12 3 2
3: F 2014-04-07 4 1
4: G 2014-05-03 1 2
5: E 2014-04-14 3 1
6: E 2014-05-04 1 2
7: H 2014-03-31 2 1
8: H 2014-04-18 4 2
9: H 2014-04-23 2 3
10: A 2014-04-01 1 1
In my previous thread, a solution by #ThomasIsCoding was provided. I tried to modify his code to perform the task I am now trying to do by doing the following:
dt[date <= first(date) + 30, date := as.Date(date)][, count := uniqueN(ID), group]
group date ID count
1: G 2014-04-01 2 2
2: G 2014-04-12 3 2
3: F 2014-04-07 4 1
4: G 2014-05-03 2 2
5: E 2014-04-14 3 2
6: E 2014-05-04 1 2
7: H 2014-03-31 2 2
8: H 2014-04-18 4 2
9: H 2014-04-23 2 2
10: A 2014-04-01 1 1
But it doesn't seem to take into account the time range condition. Any suggestions is greatly appreciated!

Answer 1

An option using non-equi join:
DT[, onemthago := date - 30L]
DT[, count :=
DT[.SD, on=.(group, date>=onemthago, date<=date),
by=.EACHI, length(unique(ID))]$V1
]
output:
group date ID onemthago count
1: G 2014-04-01 2 2014-03-02 1
2: G 2014-04-12 3 2014-03-13 2
3: F 2014-04-07 4 2014-03-08 1
4: G 2014-05-03 2 2014-04-03 2
5: E 2014-04-14 3 2014-03-15 1
6: E 2014-05-04 1 2014-04-04 2
7: H 2014-03-31 2 2014-03-01 1
8: H 2014-04-18 4 2014-03-19 2
9: H 2014-04-23 2 2014-03-24 2
10: A 2014-04-01 1 2014-03-02 1
data:
date = as.Date(c("2014-04-01", "2014-04-12", "2014-04-07", "2014-05-03", "2014-04-14", "2014-05-04", "2014-03-31", "2014-04-18", "2014-04-23", "2014-04-01"))
group = c("G","G","F","G","E","E","H","H","H","A")
ID = c(2, 3, 4, 2, 3, 1, 2, 4, 2, 1)
library(data.table)
DT <- data.table(group, date, ID)
edit to address comment on multiple lookback periods. You can try something like:
for (x in c(30L, 90L)) {
DT[, daysago := date - x]
DT[, paste0("count", x) :=
.SD[.SD, on=.(group, date>=daysago, date<=date),
by=.EACHI, length(unique(ID))]$V1
][]
}
DT

Answer 2

If I understood your problem correctly one alternative way within the tidyverse would be this:
library(tidyverse)
tb <- dplyr::tibble(date = c("2014-04-01", "2014-04-12", "2014-04-07", "2014-05-03", "2014-04-14", "2014-05-04", "2014-03-31", "2014-04-18", "2014-04-23", "2014-04-01"),
group = c("G","G","F","G","E","E","H","H","H","A"),
ID = c(2, 3, 4, 2, 3, 1, 2, 4, 2, 1))
tb %>%
dplyr::group_by(group) %>%
dplyr::mutate(as.numeric(difftime(Sys.Date(), date)) < 31) %>%
dplyr::distinct(ID) %>%
dplyr::count(group) %>%
dplyr::right_join(tb) %>%
dplyr::select(group, date, ID, Count = n)
group date ID Count
<chr> <chr> <dbl> <int>
1 A 2014-04-01 1 1
2 E 2014-04-14 3 2
3 E 2014-05-04 1 2
4 F 2014-04-07 4 1
5 G 2014-04-01 2 2
6 G 2014-04-12 3 2
7 G 2014-05-03 2 2
8 H 2014-03-31 2 2
9 H 2014-04-18 4 2
10 H 2014-04-23 2 2
for the rolling window type of function this should be a solution:
tb %>%
dplyr::full_join(tb, by = "group") %>%
dplyr::filter(as.numeric(difftime(as.Date(date.x), as.Date(date.y), units = "days")) >= 0 & as.numeric(difftime(date.x, date.y, units = "days")) < 31) %>%
dplyr::distinct(group, date.x, ID.y) %>%
dplyr::count(group, date.x) %>%
# you might want to cut the pipe here and look at the result (do not forget to delete the %>% in the line above when removing the part below
dplyr::right_join(tb, by = c("group", "date.x" = "date")) %>%
dplyr::select(group, date = date.x, ID, count = n)
group date ID count
<chr> <chr> <dbl> <int>
1 A 2014-04-01 1 1
2 E 2014-04-14 3 1
3 E 2014-05-04 1 2
4 F 2014-04-07 4 1
5 G 2014-04-01 2 1
6 G 2014-04-12 3 2
7 G 2014-05-03 2 2
8 H 2014-03-31 2 1
9 H 2014-04-18 4 2
10 H 2014-04-23 2 2

Answer 3

See how well this proposed solution works for your case.
date = c("2014-04-01", "2014-04-12", "2014-04-07", "2014-05-03", "2014-04-14", "2014-05-04", "2014-03-31", "2014-04-18", "2014-04-23", "2014-04-01")
group = c("G","G","F","G","E","E","H","H","H","A")
ID = c(2, 3, 4, 2, 3, 1, 2, 4, 2, 1)
dt <- data.table( date=as.Date(date), group, ID )
setkey( dt, group, date )
calc.id <- function(ID,date) {
## there is always going to be at least 1
id.count <- 1
if( length(ID) > 1 ) {
v <- sapply( 2:length(ID), function(i) {
j <- date[ 1:i ] >= date[i] - 30
uniqueN( ID[j] )
})
id.count <- c( id.count, v )
}
return( id.count )
}
dt[ , count := calc.id(ID,date), by=group ]
dt
I didn't find another way than to use sapply for the rolling calculation. I doubt it will outperform the solution above.
It produces this:
date group ID count
1: 2014-04-01 A 1 1
2: 2014-04-14 E 3 1
3: 2014-05-04 E 1 2
4: 2014-04-07 F 4 1
5: 2014-04-01 G 2 1
6: 2014-04-12 G 3 2
7: 2014-05-03 G 2 2
8: 2014-03-31 H 2 1
9: 2014-04-18 H 4 2
10: 2014-04-23 H 2 2