For-loop, calculate predicted movement, delete row if outside range - r

Question

I have a data-set containing vehicle movement, where vehicle time and position is logged with GPS. The issue is that there exist duplicate values (spoofing) for several of the vehicles, and I am not able to identify the true vehicle, outside the guess that the first time the vehicle is introduced it is the true vehicle. My intent is to create a fore-loop that calculates predicted movement from one position to the next , and if the next value is outside this value the row will be deleted. This will also remove outliers, where one position is extremely off for some reason.
Dataset
Vehicle ID Time Lat Long Max Speed (kts)
1 01.01.2013 12:00:00 9.535 18.536 20
1 01.01.2013 12:10:00 9.539 18.539 20
1 01.01.2013 12:20:00 65.535 35.545 20
1 01.01.2013 12:30:00 65.835 35.545 20
1 01.01.2013 12:40:00 9.541 18.542 20
1 01.01.2013 12:50:00 66.135 35.536 20
1 01.01.2013 13:00:00 9.543 18.545 20
2 05.01.2013 17:00:00 13.535 15.536 30
The idea is to run a loop that takes the position from row 1 if the Vehicle ID = Vehicle Id + 1, and calculates the maximum distance possibly traveled by calculating the time between time and time for next row (time + 1) and multiple this by the maximum speed. Then calculate a maximum and minimum latitude and longitude for where the vehicle can be theoretically be at (time + 1) and if the position is outside these maximum values, the row will be deleted and the loop will run the same statement on next row.
Something like this:
if vehicle ID = vehicle ID[n+1],
then (create latmax and latmin) ( time[n+1] - time ) * maximum speed +- latitude &
then (create lonmax and lonmin) ( time[n+1] - time ) * maximum speed +- longitude
then if lat[n+1] > latmax & lat[n+1] < latmin & lon[n+1] > lonmax & lon[n+1] < lonmax (deleterow) if not, do the same at next line
This should result in deleting row 3, 4 and 6 in my sample. For row 8 there is a new vehicle and a large deviation in position is allowed.
This method does not take account in the fact that the earth is circular and distance between latitudes decrease the closer we get to the north or south pole. The best solution would take this into account by solving for this mathematically in the formula, or using distm or similar to calculate the true distance. Implementing cosinus in the formula is properly the easiest method. However the deviation between the outliers and the true position is normalt so large, that the curvature of the earth does not matter in practice for this data-set.

Answer 1

Ok you have 2 problems here, you have a research problem where you need to define the appropriate distance to compare the spoofing pairs in function of coordinates, but foremost you need to define the spoofing pairs and the id of their previous known non-spoofing coordinates. The first problem is a research problem, and I won't go deep into it, but perhaps an internet search on how to calculate distances based on coordinates would help you. A solution to the second problem, the coding part, is proposed below, assuming you want to choose the minimal distance to the latest known non-spoofing position.
First you can take my same example by running this:
dput(df)
structure(list(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L), structure(c(1L,
1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L), .Label = c("01.01.2013", "05.01.2013"
), class = "factor"), structure(c(1L, 2L, 3L, 4L, 5L, 6L, 7L,
8L, 3L), .Label = c("12:00:00", "12:10:00", "12:20:00", "12:30:00",
"12:40:00", "12:50:00", "13:00:00", "17:00:00"), class = "factor"),
c(9.535, 9.635, 65.535, 65.835, 9.935, 66.135, 10.235, 13.535,
40.535), c(18.536, 18.636, 35.536, 35.536, 18.936, 35.536,
19.236, 15.536, 40.545), c(20L, 20L, 20L, 20L, 20L, 20L,
20L, 30L, 20L)), .Names = c("Vehicle ID", "date", "Time",
"Lat", "Long", "Max Speed (kts)"), class = "data.frame", row.names = c(NA,
-9L))
My method is to use a series of apply functions. I am also interested if someone knows a more elegant way of doing it other than explicit loops, which might perhaps do the job in fewer steps, but I tend to avoid those.
spoofingtestdb <- df[,1:3]
df$spoofing <- duplicated(spoofingtestdb)|duplicated(spoofingtestdb, fromLast = T)
df$datetime <- dmy_hms(paste0(df$date,"-", df$Time))
df$candidatespreviousposition <- apply(df, 1, function(x) which(df$`Vehicle ID`== x["Vehicle ID"] & !df$spoofing & (as_datetime(df$datetime) < as_datetime(x["datetime"])) ) )
df$latestpreviousposition <- NA
for(i in 1: nrow(df)){
if(length(df$candidatespreviousposition[[i]]>0)) df$latestpreviousposition[[i]] <- df$candidatespreviousposition[[i]][which.max(df$datetime[df$candidatespreviousposition[[i]]])]
}
df$spoofingkey <- paste0(df$`Vehicle ID`, df$datetime)
df$spoofingid <- ifelse(df$spoofing, apply(df, 1, function(x) which(df$spoofingkey==x["spoofingkey"])), NA)
df$lat1 <- apply(df, 1, function(x) df$Lat[x[["spoofingid"]][1]][which(!is.na(df$Lat[x[["spoofingid"]][1]]))] )
df$long1 <- apply(df, 1, function(x) df$Long[x[["spoofingid"]][1]][which(!is.na(df$Long[x[["spoofingid"]][1]]))] )
df$latinit <- apply(df, 1, function(x) df$Lat[x["latestpreviousposition"]])
df$latinit <- ifelse(df$spoofing, df$Lat[df$latestpreviousposition], NA)
df$longinit <- ifelse(df$spoofing, df$Long[df$latestpreviousposition], NA)
getdistance <- function(latinit, longinit, lat, long) {
distance1 <- abs(lat-latinit)+abs(long-longinit)
}
df$distance <- ifelse(df$spoofing, getdistance(df$latinit, df$longinit, df$Lat, df$Long), NA )
df$spoofingnumber <- apply(df, 1, function(x) paste0(x["spoofingid"], collapse=""))
#apply(df, 1, function(x) which(df$spoofingnumber==x["spoofingnumber"]))
df$ismindistance <- apply(df, 1, function(x) x["distance"] == min(df$distance[which(df$spoofingnumber==x["spoofingnumber"])]))
df$tokeep <- ifelse(is.na(df$ismindistance)|df$ismindistance, T, F)
result <- df[df$tokeep,]
result
Here just using a basic distance calculation function. The result is below, as you can see the second row has been deleted in my example, only the minimum distance was kept with respect to the previous known position.
Vehicle ID date Time Lat Long Max Speed (kts) spoofing datetime candidatespreviousposition
1 1 01.01.2013 12:00:00 9.535 18.536 20 FALSE 2013-01-01 12:00:00
2 1 01.01.2013 12:10:00 9.635 18.636 20 FALSE 2013-01-01 12:10:00 1
4 1 01.01.2013 12:30:00 65.835 35.536 20 FALSE 2013-01-01 12:30:00 1, 2
5 1 01.01.2013 12:40:00 9.935 18.936 20 FALSE 2013-01-01 12:40:00 1, 2, 4
6 1 01.01.2013 12:50:00 66.135 35.536 20 FALSE 2013-01-01 12:50:00 1, 2, 4, 5
7 1 01.01.2013 13:00:00 10.235 19.236 20 FALSE 2013-01-01 13:00:00 1, 2, 4, 5, 6
8 2 05.01.2013 17:00:00 13.535 15.536 30 FALSE 2013-01-05 17:00:00
9 1 01.01.2013 12:20:00 40.535 40.545 20 TRUE 2013-01-01 12:20:00 1, 2
latestpreviousposition spoofingkey spoofingid lat1 long1 latinit longinit distance spoofingnumber ismindistance tokeep
1 NA 12013-01-01 12:00:00 NA NA NA NA NA NA TRUE
2 1 12013-01-01 12:10:00 NA NA NA NA NA NA TRUE
4 2 12013-01-01 12:30:00 NA NA NA NA NA NA TRUE
5 4 12013-01-01 12:40:00 NA NA NA NA NA NA TRUE
6 5 12013-01-01 12:50:00 NA NA NA NA NA NA TRUE
7 6 12013-01-01 13:00:00 NA NA NA NA NA NA TRUE
8 NA 22013-01-05 17:00:00 NA NA NA NA NA NA TRUE
9 2 12013-01-01 12:20:00 3, 9 65.535 35.536 9.635 18.636 52.809 c(3, 9) TRUE TRUE
After you choose which distance function is appropriate for you, you can just replace the getdistance() function above.