Overlap in row values from previous rows - r

Question

I have a dataframe like this:
set.seed(123)
a <- c("A", "B", "C", "D", "E", "F", "G", "H", "I")
df <- data.frame(
V1 = sample(a,4, replace=TRUE),
V2 = sample(a,4, replace=TRUE),
V3 = sample(a,4, replace=TRUE),
V4 = sample(a,4, replace=TRUE)
)
which looks like
V1 V2 V3 V4
1 C I E G
2 H A E F
3 D E I A
4 H I E I
I'd like to count the number of unique values in a row in comparison to the previous rows, so the result would look like:
V1 V2 V3 V4 V5
1 C I E G 4
2 H A E F 3
3 D E I A 2
4 H I E I 1
V5 equals 4 for row 1 since it's the 1st row and all are unique
V5 equals 3 for row 2 since H, A, and F were not in row 1
V5 equals 2 for row 3 since 1) D and I were not in row 2. and 2) D and A were not in row 1.
V5 equals 1 for row 4 since 1) H was not in row 1, 2) I was not in row 2, and 3) H was not in row 4.
if row 4 were H I E A, then V5 for row 4 would have been still been 1 since it only has 1 value not in row 3, even though it would have 2 values not in row 2 and 2 values not in row 1.

Answer 1

Here is a multi-step method in base R.
# Create a list of the elements by row, using mike H's method
myList <- strsplit(Reduce(paste0, df), "")
# previous method, could create new object first t(df) if large df
# myList <- split(t(df), col(t(df)))
# get pairwise combinations of rows
combos <- t(combn(nrow(df):1, 2))[choose(nrow(df), 2):1,]
# get desired values, sapply runs through pairs of rows, tapply calculates min with row
df$cnts <- c(length(unique(myList[[1]])), # value for first row
tapply(sapply(1:nrow(combos), # sapply through pairs, taking set diffs
function(x) length(setdiff(myList[[combos[x,1]]],
myList[[combos[x,2]]]))),
combos[,1], min)) # split set diff lengths by row, get min length
This returns
df
V1 V2 V3 V4 cnts
1 C I E G 4
2 H A E F 3
3 D E I A 2
4 H I E I 1

Answer 2

For such tasks, storing the rows/sets of data like "df" in a tabulation format can be helpful to solve problems:
tab = table(as.matrix(df), row(df)) > 0
#> tab
#
# 1 2 3 4
# A FALSE TRUE TRUE FALSE
# C TRUE FALSE FALSE FALSE
# D FALSE FALSE TRUE FALSE
# E TRUE TRUE TRUE TRUE
# F FALSE TRUE FALSE FALSE
# G TRUE FALSE FALSE FALSE
# H FALSE TRUE FALSE TRUE
# I TRUE FALSE TRUE TRUE
crossprod can be used to retrieve (in a very efficient manner) the number of items that belong to a row but not to any other:
ct = crossprod(tab, !tab)
#> ct
#
# 1 2 3 4
# 1 0 3 2 2
# 2 3 0 2 2
# 3 2 2 0 2
# 4 1 1 1 0
Above we can see that, e.g., row 4 contains 1 element that row 1 does not contain, while row 1 contains 2 elements that are not in row 4, etc.
Since here we only care about the previous rows of each row and, specifically, the minimum of each set of one-to-all comparisons, an idea to get the result is:
ct[upper.tri(ct, TRUE)] = Inf ## to ignore 'upper.tri' values in 'max.col'
j_min = max.col(-ct, "first") ## row-index of the minimum difference per row
c(sum(tab[, 1]),
ct[cbind(2:nrow(df), j_min[-1])])
#[1] 4 3 2 1

Answer 3

Here's an approach that uses Reduce and mapply:
df$cols_paste <- strsplit(Reduce(paste0, df), split = "")
df$V5 <- lapply(1:length(df$cols_paste), function(x){
if(x==1) compare = NA
else compare = df$cols_paste[seq(1:(x-1))]
min(mapply(function(x, y) length(setdiff(x,y)), df$cols_paste[x], compare))
})
df[,setdiff(names(df), "cols_paste")]
V1 V2 V3 V4 V5
1 C I E G 4
2 H A E F 3
3 D E I A 2
4 H I E I 1