relating data frames more efficiently [duplicate] - r

Question

I have two data frames, one of hospital stays, and the other of lab results. I need to identify which hospital stay a lab result takes place in, and copy the admission and discharge dates from the hospital data frame into the row for the relevant lab result.
I am doing this with a for loop to walk through the lab results, and then if statements and subsets that look for matching entries (by patient SSN and surrounding dates) in the hospital records.
This is quite a large data set and using the for loop is very slow. Is there a way to speed this kind of problem up? (I have several similar problems, so would love an answer.)
Sample data added, note that there are multiple hospital records for each patient with the goal being to get the dates from the record where the dates overlap the lab date. In this example, the resulting data frame should only have admission and discharge dates for patient 1, as patient 2 has no hospital data, and patient 3's records do not overlap the lab date.
testDate <- as.Date(c("2017-01-15", "2017-01-15", "2017-01-15"))
patientSSN <- c("1","2","3")
labs <- data.frame(patientSSN, testDate)
# patientSSN testDate
# 1 1 2017-01-15
# 2 2 2017-01-15
# 3 3 2017-01-15
patientSSN <- c("1","1","3","3")
admissionDate <- as.Date(c("2017-01-07", "2017-02-01", "2016-12-01", "2017-01-16"))
dischargeDate <- as.Date(c("2017-01-16", "2017-02-10", "2016-12-15", "2017-02-01"))
hospitalRec <- data.frame(patientSSN, admissionDate, dischargeDate)
for (I in 1:nrow(labs)) {
labs[I,]$admissionDate <- hospitalRec[hospitalRec$patientSSN == labs[I,]$patientSSN & hospitalRec$admissionDate <= labs[I,]$testDate & hospitalRec$dischargeDate >= labs[I,]$testDate,]$admissionDate
labs[I,]$admissionDate <- hospitalRec[hospitalRec$PatientSSN == labs[I,]$PatientSSN & hospitalRec$admissionDate <= labs[I,]$testDate & hospitalRec$dischargeDate >= labs[I,]$testDate,]$dischargeDate
}
The desired data frame would look like:
labs:
patientSSN testDate admissionDate dischargeDate
1 2017-01-15 2017-01-07 2017-01-16
2 2017-01-15 NA NA
3 2017-01-15 NA NA
Notice, in the real data, there is also the problem of multiple hospital records qualifying (discharges between departments) these records would have the same admission date, but different discharge times with the latest one being important. But first things first...

Answer 1

A non equi join works, for example with data.table:
library(data.table)
setDT(labs); setDT(hospitalRec)
labs[hospitalRec, on=.(patientSSN, testDate >= admissionDate, testDate <= dischargeDate),
`:=`(aDate = i.admissionDate, dDate = i.dischargeDate)]
patientSSN testDate aDate dDate
1: 1 2017-01-15 2017-01-07 2017-01-16
2: 2 2017-01-15 <NA> <NA>
3: 3 2017-01-15 <NA> <NA>
in the real data, there is also the problem of multiple hospital records qualifying (discharges between departments) these records would have the same admission date, but different discharge times with the latest one being important.
If hospitalRec is sorted, adding mult="last" above should work. See ?data.table for full documentation. Alternately, you could just create a version of the hospital records that excludes these "duplicates", like ... sort and then
lastRec = unique(hospitalRec, by=c("patientSSN", "admissionDate"), fromLast=TRUE))
The setorder function is the standard tool for sorting data.tables.

Answer 2

Assuming this is similar to what your df looks like, use dplyr::left_join:
hospital_data <- data.frame(PatientSSN = c('1234567890','9876543210'),
admit = c('8/1/17','8/5/17'),
discharge = c('8/10/17','8/15/17'))
lab_data <- data.frame(specimen_id = c('foo1','foo2','foo3','foo4','foo5','foo6','foo7'),
PatientSSN = c('1234567890','1234567890','1234567890','9876543210','9876543210','9876543210','8527419600'),
test = c('hemoglobin','inr','platelette','hemoglobin','inr','platelette','inr'))
lab_data %>% left_join(hospital_data)
specimen_id PatientSSN test admit discharge
1 foo1 1234567890 hemoglobin 8/1/17 8/10/17
2 foo2 1234567890 inr 8/1/17 8/10/17
3 foo3 1234567890 platelette 8/1/17 8/10/17
4 foo4 9876543210 hemoglobin 8/5/17 8/15/17
5 foo5 9876543210 inr 8/5/17 8/15/17
6 foo6 9876543210 platelette 8/5/17 8/15/17
7 foo7 8527419600 inr <NA> <NA>
Note that your id variable (PatientSSN) are the same in each table.

Answer 3

OK- here's a method. Just a quick heads up, though; it's pretty unlikely that you'll ever work with EMR data that doesn't have an ID variable specific to a visit/account. I would look to use that as the unique identifier before I used SSN. Nevertheless; this should work. I used the data you provided above.
for(i in 1:nrow(labs)){
#finding the ID (ssn)
ssn_match_df <- hospitalRec[which(as.character(labs$patientSSN[i]) == as.character(hospitalRec$patientSSN)),]
#finding record in table where the test date fall between the admit/discharge
ssn_match_df <- ssn_match_df[which(labs$testDate[i] >= ssn_match_df$admissionDate &
labs$testDate[i] <= ssn_match_df$dischargeDate),]
if(nrow(ssn_match_df)>0){
labs[i,3] <- as.character(ssn_match_df[1,2])
labs[i,4] <- as.character(ssn_match_df[1,3])
} else {
labs[i,3] <- NA
labs[i,4] <- NA
}
}
colnames(labs)[3] <- 'admitDate'
colnames(labs)[4] <- 'dischargeDate'