Mark each row in a large dataframe via two variables - r

Question

I have a dataframe like this (the real one is much larger):
time<-c(as.POSIXct('2011-11-11 06:00:00'),as.POSIXct('2011-11-11 06:05:00'),as.POSIXct('2011-11-11 07:05:00'),
as.POSIXct('2011-11-11 07:10:00'),as.POSIXct('2011-11-11 07:13:00'),as.POSIXct('2011-11-11 07:33:00'),
as.POSIXct('2011-11-11 05:05:00'),as.POSIXct('2011-11-11 06:05:00'),as.POSIXct('2011-11-11 06:20:00'),
as.POSIXct('2011-11-11 09:05:00'))
plate<-c('a','a','a','b','c','d','e','e','e','e')
df<-data.frame(time,plate)
The time variable represents the time that the vehicle be identified by the video device. The plate variable represents the vehicle's plate. The dataframe has been well ordered by firstly plate and secondly time.
Given this, I want to devide each vehicle's trip by marking the rows. Different vehicles (plates) certainly represent different trips. For one vehicle, the identified time difference within one trip should be shorter than 30 minutes, if not, the rows should belong to different trips.
In my way, I will do this by the following code:
trip<-vector()
trip[1]<-1
time_diff<-as.POSIXct('2011-11-11 07:00:00')-as.POSIXct('2011-11-11 06:30:00')
for (x in 2:nrow(df)) {
if (!df$plate[x]==df$plate[x-1]) (trip[x]<-trip[x-1]+1
) else{if (df$time[x]-df$time[x-1]<time_diff) (trip[x]<-trip[x-1]
) else (trip[x]<-trip[x-1]+1)}
}
df<-cbind(df,trip)
However, my df contains more than seven million rows thus my method will be very slow. So I'm asking if there are some more efficient ways to do this.

Answer 1

I'll suggest using dplyr for this, though with 7M rows you might consider data.table solution if this doesn't work well for you.
library(dplyr)
time_diff<-as.POSIXct('2011-11-11 07:00:00')-as.POSIXct('2011-11-11 06:30:00')
df %>%
arrange(time) %>% # it's important, so I reinforce it here
group_by(plate) %>%
mutate(
trip = cumsum( c(TRUE, diff(time) > time_diff) )
) %>%
ungroup()
# # A tibble: 10 × 3
# time plate trip
# <dttm> <fctr> <int>
# 1 2011-11-11 06:00:00 a 1
# 2 2011-11-11 06:05:00 a 1
# 3 2011-11-11 07:05:00 a 2
# 4 2011-11-11 07:10:00 b 1
# 5 2011-11-11 07:13:00 c 1
# 6 2011-11-11 07:33:00 d 1
# 7 2011-11-11 05:05:00 e 1
# 8 2011-11-11 06:05:00 e 2
# 9 2011-11-11 06:20:00 e 2
# 10 2011-11-11 09:05:00 e 3
I much prefer the above solution using group_by, but if you want the trip to be unique across plates, one technique is to handle the grouping yourself (requiring strict ordering):
df %>%
arrange(plate, time) %>%
mutate(
trip = cumsum( plate != lag(plate, default = plate[1]) | c(TRUE, diff(time) > time_diff) )
)
# time plate trip
# 1 2011-11-11 06:00:00 a 1
# 2 2011-11-11 06:05:00 a 1
# 3 2011-11-11 07:05:00 a 2
# 4 2011-11-11 07:10:00 b 3
# 5 2011-11-11 07:13:00 c 4
# 6 2011-11-11 07:33:00 d 5
# 7 2011-11-11 05:05:00 e 6
# 8 2011-11-11 06:05:00 e 7
# 9 2011-11-11 06:20:00 e 7
# 10 2011-11-11 09:05:00 e 8