join two data frames based on user defined function - r

Question

I'm trying to (inner) join two data frames based on a similarity function that I have.
for example:
data1<-data.frame(a=c(1,2,3),lat=c(38.862976,37.878146,36.825658), lon=c(-99.336782,-99.326054,-98.475976))
data2<-data.frame(b=c(10,20),lat=c(38.863412,37.877333), lon=c(-99.336701,-99.325151))
and given a similarity function:
are.close(lat1,long1,lat2,long2)
something like
data3<-join(a=data1,b=data2,by=c(lat,lon),FUN=are.close(a.lat,a.lon,b.lat,b.lon))
The output I wish to receive is:
a b lat lon
1 1 10 38.862976 -99.336782
2 2 20 37.878146 -99.326054
Where the lat/lon belongs to one of the tables (it doesn't matter which, say the first).
All the join/merge methods I checked doesn't let you define how the join is taking place. It only lets you specify things like col1=col2.
Is there a way to do this computationally efficient (not by running with two loops on the two sets)?

Answer 1

I'd suggest using outer to identify (a,b) pairs that meet the criterion:
neighbormat <- outer(
1:nrow(data1),
1:nrow(data2),
function(i1,i2){
are.close(
data1$lat[i1],
data1$lon[i1],
data2$lat[i2],
data2$lon[i2]
)
}
)
dimnames(neighbormat) <- list(data1$a,data2$b)
Using the names only makes sense if a and b are unique, but I'll assume they are since the OP is using them that way. For #konvas's are.close function, this gives
10 20
1 TRUE TRUE
2 TRUE TRUE
3 FALSE FALSE
To get the (a,b) pairs that meet the criterion, use
ns <- which(neighbormat,arr.ind=TRUE,use.names=TRUE)
dimnames(ns) <- list(NULL,c("a","b"))
a b
[1,] 1 1
[2,] 2 1
[3,] 1 2
[4,] 2 2
It's straightforward to merge these back to the original data. (Taking an arbitrary (lat,lon) as the OP does, seems like a very bad idea, though.)

Answer 2

Here is an approach using dplyr. I have assumed that are.close() is vectorised and returns TRUE/FALSE, for example this will work with a function like are.close <- function(a, b, c, d) (a-c)^2 + (b-d)^2 < 1
library(dplyr)
expand.grid(a = data1$a, b = data2$b) %>%
left_join(data1, by = "a") %>%
left_join(data2, by = "b") %>%
mutate(close = are.close(lat.x, lon.x, lat.y, lon.y)) %>%
filter(close)

Answer 3

I wouldn't know of a function that does this (but there of course might be...), so I would try writing some code myself. Which might be difficult depending on the data. But assuming that couples are really clear (e.g. the latitude of point 1 could be closest to b 10, whereas the longitude might be closer to b 20, etc.) this might be the beginning of something to work with:
data1<-data.frame(a=c(1,2,3),lat=c(38.862976,37.878146,36.825658), lon=c(-99.336782,-99.326054,-98.475976))
data2<-data.frame(b=c(10,20),lat=c(38.863412,37.877333), lon=c(-99.336701,-99.325151))
# calculate which is the closest value
names(data1)=c("a","lat_original","lon_original")
closest=function(x,to=to) to[which.min(abs(to - x))]
data1$lat=sapply(data1$lat_original,function(x) closest(x,to=data2$lat))
data1$lon=sapply(data1$lon_original,function(x) closest(x,to=data2$lon))
# if dataframes are not equally big: remove biggest assigned "closest values" (or doubles?)
if(nrow(data1)!=nrow(data2)) {
data1$diff=abs(data1$lat-data1$lat_original)+abs(data1$lon-data1$lon_original)
maxN <- function(x, N=N){
x=x[!is.na(x)]
len=length(x)
if(N>len){
warning('N greater than length(x). Setting N=length(x)')
N=length(x)
}
sort(x,partial=len-N+1)[as.numeric(len-N+1):len]
}
data1=data1[!data1$diff %in% maxN(data1$diff,N=nrow(data1)-nrow(data2)),]}
# perhaps check if doubles (two different points of data 1 assigned to the same point in data2)
which(duplicated(paste(data1$lat,data1$lon))==T)
#merge based on those closest values
merge(data1,data2,by=c("lat","lon"))