How do I get all pairs of values in a variable based on shared values in a different variable - r

Question

My problem is perhaps a little difficult to formulate, hence I haven't found any solutions yet, but I'll try:
I wan't to find all pairs of values in a variable based on whether they share any value in another variable. Maybe the following example can illustrate it more clearly.
In a 2 variable data frame like this:
data.frame(scaffold = c("A", "A", "B", "B", "B", "C", "C", "D"),
geneID = c("162", "276", "64", "276", "281", "64", "162", "162"))
#> scaffold geneID
#> A 162
#> A 276
#> B 64
#> B 276
#> B 281
#> C 64
#> C 162
#> D 162
... I want to find all pairs of "scaffolds" A, B, C, and D, that share any of the "geneID"s 64, 162, 176, and 281, so that the above would become a data frame with all pairs of scaffolds in 2 new columns like this:
data.frame(V1 = c("A", "A", "A", "B", "C"), V2 =c("B", "C", "D", "C", "D"))
#> V1 V2
#> A B
#> A C
#> A D
#> B C
#> C D
Obviously A and B is the same pair as B and A, so these should be removed somehow, but that's probably easy. Afterwards, this data frame needs to be combined with a data frame containing x/y coordinates of the scaffolds for drawing a line between the pairs on top of a plot with the scaffolds.
I do have a working for-loop to do the job, but I need to replace that with a much faster alternative. I'll spare you the code, it's complicated and doesn't always do it right. Running it on just 20 scaffolds can take seconds, but I need to do it on thousands. I was hoping a series of dplyr or data.table functions could do the job as they probably are as fast as it gets, but I haven't been able to get my head around how.
I hope you can help me, or perhaps something similar is already in another threat I just wasn't able to find.
A performance comparison of the two solutions by #Florian and #Roman can be found at http://rpubs.com/kasperskytte/SO_question_48407650

Answer 1

Here is a possible solution. Note that I modified your example df so A and C share both 162 and 64, and we have to make sure that this group does not occur twice in the output.
df = data.frame(scaffold = c("A", "A", "B", "B", "B", "C", "C", "D","A"),
geneID = c("162", "276", "64", "276", "281", "64", "162", "162","64"),stringsAsFactors = F)
y = split(df$scaffold,df$geneID)
unique(do.call(rbind,(lapply(y[which(sapply(y, length) > 1)],function(x){t(combn(sort(x),2))}))))
Output:
[,1] [,2]
[1,] "A" "C"
[2,] "A" "D"
[3,] "C" "D"
[4,] "A" "B"
[5,] "B" "C"
How it works: First we split the data into groups based on df$geneID, the result we call y. Then we lapply over every element of y that has more than 1 element in it a function that gives us all n possible combinations of 2 as a nx2 matrix. By calling sort() on x inside this function we make removing duplicates easier later on, because we then rbind this list into a large matrix, and call unique() on the result to remove duplicates.
Hope this helps!

Answer 2

See the commends in the code.
xy <- data.frame(scaffold = c("A", "A", "B", "B", "B", "C", "C", "D"),
geneID = c("162", "276", "64", "276", "281", "64", "162", "162"))
# split by gene
xy1 <- split(xy, f = xy$geneID)
# find all combinations
out <- sapply(xy1, FUN = function(x) {
x$scaffold <- as.character(x$scaffold)
# add NA so that we can remove any cases that have a single scaffold
tryCatch(t(combn(x$scaffold, 2)), error = function(e) NA)
}, simplify = FALSE)
# remove NAs and some fiddling to get the desired format
out <- out[!is.na(out)]
out <- do.call(rbind, out)
# sort the data
out <- t(apply(out, MARGIN = 1, FUN = function(x) sort(x)))
# remove duplicates
out <- out[!duplicated(out), ]
out
[,1] [,2]
[1,] "A" "C"
[2,] "A" "D"
[3,] "C" "D"
[4,] "A" "B"
[5,] "B" "C"