Combine data sets based on date comparisons within groups - r

Question

I have two data sets, 'Df_A' and 'Df_B':
Df_A
Date Info A Info B
9/19/18 23:00 36 48
9/18/18 23:00 47 30
9/17/18 23:00 51 3
8/14/18 23:00 45 16
8/6/18 23:00 37 13
8/5/18 23:00 42 66
7/11/18 23:00 42 53
7/4/18 23:00 38 10
Df_B
Released Info Event Value X
9/6/2018 22:30 Event A 51.8
8/6/2018 22:30 Event A 52
7/5/2018 22:30 Event A 50.6
6/6/2018 22:30 Event A 54
9/2/2018 22:30 Event C 48
7/31/2018 22:30 Event C 45
9/4/2018 22:30 Event D 58.7
8/2/2018 22:30 Event D 56.2
7/3/2018 22:30 Event D 57.3
6/4/2018 22:30 Event D 51.1
5/2/2018 22:30 Event D 54.2
4/4/2018 22:30 Event D 59.8
9/3/2018 1:30 Event E 61.8
8/6/2018 1:30 Event E 63
7/2/2018 1:30 Event E 65.2
Both 'Date' and 'Released.info' are factors.
I have a vector 'Events' which contains the Events in 'Df_B' that I need to parse, e.g.
Events <- c("Event A", "Event D")
For each 'Event' in 'Df_B', I would like to check if 'Date' in 'Df_A' is greater than 'Released Info' in 'Df_B'. If so, I want to add the corresponding value of 'Event A' and 'Event B' to 'Df_A'.
The desired output:
Date Info A Info B Event A Event D
9/19/18 23:00 36 48 51.8 58.7
9/18/18 23:00 47 30 51.8 58.7
9/17/18 23:00 51 3 51.8 58.7
8/14/18 23:00 45 16 52 56.2
8/6/18 23:00 37 13 52 56.2
8/5/18 23:00 42 66 50.6 56.2
7/11/18 23:00 42 53 50.6 57.3
7/4/18 23:00 38 10 54 57.3
For example, for 9/19/18 23:00, 9/18/18 23:00 and 9/17/18 23:00 in 'Df_A', the closest prior date in 'Df_B' for the group 'Event A' is 9/6/2018 22:30. Thus, for these rows we pick the value 51.8 from 'Df_B'. And so on for all dates in Df_A, and for both 'Event A' and 'Event B' in 'Df_B'.
I would like to add new n columns to 'Df_A', in this example 'Event A' and 'Event D', but it could be more.
For this, I have been trying creating some dynamic variables for the dynamic amount of events with something like this (as the Events come from a csv as a matrix) :
#To Create a variable for each Event
ListEvents <- as.list(as.vector(Events))
names(ListEvents) <- paste("Variable", 1:length(ListEvents), sep = "")
list2env(ListEvents,envir = .GlobalEnv)
After creating a variable for each Event, I was thinking in creating a loop so I can create a subset for each event and then compare the Date (Df_A) with the release Date(Df_B) and add it as a column in Df_A. But I know this is an unnecessary complex and inefficient approach. Could someone help me?

Answer 1

The following reproduces your expected output:
events <- c("Event A", "Event D")
library(tidyverse)
library(lubridate)
map(events, ~Df_A %>%
mutate(Event := .x) %>%
left_join(Df_B) %>%
mutate(
Date = mdy_hm(Date),
Released.Info = mdy_hm(Released.Info)) %>%
group_by(Date) %>%
mutate(diff = difftime(Released.Info, Date, units = "days")) %>%
filter(diff < 0) %>%
filter(diff == max(diff)) %>%
select(-Released.Info, -diff) %>%
spread(Event, Value.X)) %>%
reduce(left_join) %>%
arrange(desc(Date))
## A tibble: 8 x 5
## Groups: Date [8]
# Date Info.A Info.B `Event A` `Event D`
# <dttm> <int> <int> <dbl> <dbl>
#1 2018-09-19 23:00:00 36 48 51.8 58.7
#2 2018-09-18 23:00:00 47 30 51.8 58.7
#3 2018-09-17 23:00:00 51 3 51.8 58.7
#4 2018-08-14 23:00:00 45 16 52 56.2
#5 2018-08-06 23:00:00 37 13 52 56.2
#6 2018-08-05 23:00:00 42 66 50.6 56.2
#7 2018-07-11 23:00:00 42 53 50.6 57.3
#8 2018-07-04 23:00:00 38 10 54 57.3
The idea is to do add an Events column to Df_A with entries given in a vector events; we then do a left join of Df_A and Df_B, and select only those rows with the shortest negative time difference between Released.Info and Date (that's the filter(diff < 0) and filter(diff == max(diff)) part). The rest is reshaping and re-arranging to reproduce your expected output.
Sample data
Df_A <-read.table(text =
" Date 'Info A' 'Info B'
'9/19/18 23:00' 36 48
'9/18/18 23:00' 47 30
'9/17/18 23:00' 51 3
'8/14/18 23:00' 45 16
'8/6/18 23:00' 37 13
'8/5/18 23:00' 42 66
'7/11/18 23:00' 42 53
'7/4/18 23:00' 38 10", header = T)
Df_B <- read.table(text =
"'Released Info' Event 'Value X'
'9/6/2018 22:30' 'Event A' 51.8
'8/6/2018 22:30' 'Event A' 52
'7/5/2018 22:30' 'Event A' 50.6
'6/6/2018 22:30' 'Event A' 54
'9/2/2018 22:30' 'Event C' 48
'7/31/2018 22:30' 'Event C' 45
'9/4/2018 22:30' 'Event D' 58.7
'8/2/2018 22:30' 'Event D' 56.2
'7/3/2018 22:30' 'Event D' 57.3
'6/4/2018 22:30' 'Event D' 51.1
'5/2/2018 22:30' 'Event D' 54.2
'4/4/2018 22:30' 'Event D' 59.8
'9/3/2018 1:30' 'Event E' 61.8
'8/6/2018 1:30' 'Event E' 63
'7/2/2018 1:30' 'Event E' 65.2", header = T)

Answer 2

This can be done with a rolling join by group in data.table.
library(data.table)
# convert data to data.table
setDT(Df_A)
setDT(Df_B)
# convert times to POSIXct
Df_A[ , Date := as.POSIXct(Date, format = "%m/%d/%y %H:%M")]
Df_B[ , Released.Info := as.POSIXct(Released.Info, format = "%m/%d/%Y %H:%M")]
# select rows
db <- Df_B[Event %in% Events]
# rolling join: for each Event in db, join to Df_A by nearest preceeding time
d2 <- db[ , .SD[Df_A, on = c(Released.Info = "Date"), roll = Inf], by = Event]
# Event Released.Info Value.X Info.A Info.B
# 1: Event A 2018-09-19 23:00:00 51.8 36 48
# 2: Event A 2018-09-18 23:00:00 51.8 47 30
# [snip]
# 7: Event A 2018-07-11 23:00:00 50.6 42 53
# 8: Event A 2018-07-04 23:00:00 54.0 38 10
# 9: Event D 2018-09-19 23:00:00 58.7 36 48
# 10: Event D 2018-09-18 23:00:00 58.7 47 30
# [snip]
# 15: Event D 2018-07-11 23:00:00 57.3 42 53
# 16: Event D 2018-07-04 23:00:00 57.3 38 10
That's basically it. If desired, cast the 'Event' column to wide and join to 'Df_A':
dcast(d2[ , .(Event, Released.Info, Value.X)],
Released.Info ~ Event, value.var = "Value.X")[
Df_A, on = c(Released.Info = "Date")]
# Released.Info Event A Event D Info.A Info.B
# 1: 2018-09-19 23:00:00 51.8 58.7 36 48
# 2: 2018-09-18 23:00:00 51.8 58.7 47 30
# 3: 2018-09-17 23:00:00 51.8 58.7 51 3
# 4: 2018-08-14 23:00:00 52.0 56.2 45 16
# 5: 2018-08-06 23:00:00 52.0 56.2 37 13
# 6: 2018-08-05 23:00:00 50.6 56.2 42 66
# 7: 2018-07-11 23:00:00 50.6 57.3 42 53
# 8: 2018-07-04 23:00:00 54.0 57.3 38 10