aggregate (R) behaving differently for apparently identical tasks - r

Question

I've been banging my head against a brick wall for days on this issue; I wonder if anyone can see what is wrong with my code, or tell me if I am overlooking something obvious.
I have this data.frame, where most columns are vectors, either numerical or character, and one column is a list of character vectors:
t0g2 <- structure(list(P = c(1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4,
4, 4, 5, 5, 5, 5), ID = c(8, 10, 7, 9, 5, 2, 3, 4, 8, 9, 1, 2,
8, 1, 4, 10, 4, 10, 2, 7), SC = c("A", "D", "A", "B", "B", "A",
"A", "E", "A", "B", "D", "A", "A", "D", "E", "D", "E", "D", "A",
"A"), FP = list(`40,41,37,8,11` = c("40", "41", "37", "8", "11"
), `49,28,16,41` = c("49", "28", "16", "41"), `15,49` = c("15",
"49"), `27,12,20,35,45` = c("27", "12", "20", "35", "45"), `1,34,43,37` = c("1",
"34", "43", "37"), `41,7,30,2,34,43` = c("41", "7", "30", "2",
"34", "43"), `22,35,31,10,3` = c("22", "35", "31", "10", "3"),
`29,6,15` = c("29", "6", "15"), `40,41,37,8,11` = c("40",
"41", "37", "8", "11"), `27,12,20,35,45` = c("27", "12",
"20", "35", "45"), `10,49,28` = c("10", "49", "28"), `41,7,30,2,34,43` = c("41",
"7", "30", "2", "34", "43"), `40,41,37,8,11` = c("40", "41",
"37", "8", "11"), `10,49,28` = c("10", "49", "28"), `29,6,15` = c("29",
"6", "15"), `49,28,16,41` = c("49", "28", "16", "41"), `29,6,15` = c("29",
"6", "15"), `49,28,16,41` = c("49", "28", "16", "41"), `41,7,30,2,34,43` = c("41",
"7", "30", "2", "34", "43"), `15,49` = c("15", "49"))), class = "data.frame", row.names = c("8",
"10", "7", "9", "5", "2", "3", "4", "81", "91", "1", "21", "82",
"11", "41", "101", "42", "102", "22", "71"))
I want to aggregate it by one of the columns, with the function for the other columns being simply the concatenation of unique values. [Yes, I know this can be done with many ad hoc packages, but I need to do it with base R].
This works perfectly well if I choose numeric column "ID" as the column to aggregate on:
aggregate(x=t0g2[, !(colnames(t0g2) %in% c("ID"))], by=list(ID=t0g2[["ID"]]),
FUN=function(y) unique(unlist(y)))
# ID P SC FP
#1 1 3, 4 D 10, 49, 28
#2 2 2, 3, 5 A 41, 7, 30, 2, 34, 43
#3 3 2 A 22, 35, 31, 10, 3
#4 4 2, 4, 5 E 29, 6, 15
#5 5 2 B 1, 34, 43, 37
#6 7 1, 5 A 15, 49
#7 8 1, 3, 4 A 40, 41, 37, 8, 11
#8 9 1, 3 B 27, 12, 20, 35, 45
#9 10 1, 4, 5 D 49, 28, 16, 41
or with character column "SC":
aggregate(x=t0g2[, !(colnames(t0g2) %in% c("SC"))], by=list(SC=t0g2[["SC"]]),
FUN=function(y) unique(unlist(y)))
# SC P ID FP
#1 A 1, 2, 3, 4, 5 8, 7, 2, 3 40, 41, 37, 8, 11, 15, 49, 7, 30, 2, 34, 43, 22, 35, 31, 10, 3
#2 B 1, 2, 3 9, 5 27, 12, 20, 35, 45, 1, 34, 43, 37
#3 D 1, 3, 4, 5 10, 1 49, 28, 16, 41, 10
#4 E 2, 4, 5 4 29, 6, 15
However, if I try with "P", which as far as I know is just another numerical column, this is what I get:
aggregate(x=t0g2[, !(colnames(t0g2) %in% c("P"))], by=list(P=t0g2[["P"]]),
FUN=function(y) unique(unlist(y)))
# P ID.1 ID.2 ID.3 ID.4 SC.1 SC.2 SC.3 FP
#1 1 8 10 7 9 A D B 40, 41, 37, 8, 11, 49, 28, 16, 15, 27, 12, 20, 35, 45
#2 2 5 2 3 4 B A E 1, 34, 43, 37, 41, 7, 30, 2, 22, 35, 31, 10, 3, 29, 6, 15
#3 3 8 9 1 2 A B D 40, 41, 37, 8, 11, 27, 12, 20, 35, 45, 10, 49, 28, 7, 30, 2, 34, 43
#4 4 8 1 4 10 A D E 40, 41, 37, 8, 11, 10, 49, 28, 29, 6, 15, 16
#5 5 4 10 2 7 E D A 29, 6, 15, 49, 28, 16, 41, 7, 30, 2, 34, 43
Does anybody know what is going on, why this happens?
Literally going mental with this stuff...
EDIT: adding an example of the desired output from aggregating on "P", as requested by jay.sf.
# P ID SC FP
#1 1 8, 10, 7, 9 A, D, B 40, 41, 37, 8, 11, 49, 28, 16, 15, 27, 12, 20, 35, 45
#2 2 5, 2, 3, 4 B, A, E 1, 34, 43, 37, 41, 7, 30, 2, 22, 35, 31, 10, 3, 29, 6, 15
#3 3 8, 9, 1, 2 A, B, D 40, 41, 37, 8, 11, 27, 12, 20, 35, 45, 10, 49, 28, 7, 30, 2, 34, 43
#4 4 8, 1, 4, 10 A, D, E 40, 41, 37, 8, 11, 10, 49, 28, 29, 6, 15, 16
#5 5 4, 10, 2, 7 E, D, A 29, 6, 15, 49, 28, 16, 41, 7, 30, 2, 34, 43
In fact, I found out that by setting simplify=F in aggregate, it works as I want.
I hope this won't backfire.
EDIT 2: it did backfire...
I don't want all my columns to become lists even when they can be vectors, but with simplify = F they do become lists:
sapply(aggregate(x=t0g2[,!(colnames(t0g2) %in% c("P"))],by=list(P=t0g2[["P"]]),FUN=function(y) unique(unlist(y)), simplify = F),class)
# P ID SC FP
#"numeric" "list" "list" "list"
sapply(aggregate(x=t0g2[,!(colnames(t0g2) %in% c("ID"))],by=list(ID=t0g2[["ID"]]),FUN=function(y) unique(unlist(y)), simplify = T),class)
# ID P SC FP
# "numeric" "list" "character" "list"
sapply(aggregate(x=t0g2[,!(colnames(t0g2) %in% c("ID"))],by=list(ID=t0g2[["ID"]]),FUN=function(y) unique(unlist(y)), simplify = F),class)
# ID P SC FP
#"numeric" "list" "list" "list"
So I still don't have a solution... :(
EDIT 3: maybe a viable (if rather clumsy) solution?
t0g2_by_ID <- aggregate(x=t0g2[,!(colnames(t0g2) %in% c("ID"))],by=list(ID=t0g2[["ID"]]),FUN=function(y) unique(unlist(y)), simplify = F)
sapply(t0g2_by_ID,class)
# ID P SC FP
#"numeric" "list" "list" "list"
for (i in 1:NCOL(t0g2_by_ID)) {y = t0g2_by_ID[,i]; if ((class(y) == "list") & (length(y) == length(unlist(y)))) {t0g2_by_ID[,i] <- unlist(y)} }
sapply(t0g2_by_ID,class)
# ID P SC FP
#"numeric" "list" "character" "list"
I tried to obviate to the inelegant loop using sapply, but then any cbind operation goes back to a data.frame of lists.
This is the best I can come up with.
If anyone can suggest how to do this better using only base R, that'd be great.

Answer 1

aggregate obviously tries to give a matrix where this is possible. See This example:
# data
n <- 10
df <- data.frame(id= rep(1:2, each= n/2),
value= 1:n)
length(unique(df$value[df$id == 1])) == length(unique(df$value[df$id == 2]))
TRUE
Here the length of unique is same for every id value, thus aggregate provides a matrix
aggregate(x= df[, "value"], by=list(id=df[, "id"]),
FUN=function(y) unique(unlist(y)))
id x.1 x.2 x.3 x.4 x.5
1 1 1 2 3 4 5
2 2 6 7 8 9 10
Now we change data so that length of unique per id is not equal
df$value[2] <- 1
length(unique(df$value[df$id == 1])) == length(unique(df$value[df$id == 2]))
FALSE
In this case we get an output with values separated by ,:
aggregate(x= df[, "value"], by=list(id=df[, "id"]),
FUN=function(y) unique(unlist(y)))
id x
1 1 1, 3, 4, 5
2 2 6, 7, 8, 9, 10
In your case you have for every P value exactly 4 unique ID values and exactly 3 unique SC values, hence, aggregate shows those results as a matrix. This is not true for FP: here aggregate can't provide a matrix, hence, we get the values separated by ,

Answer 2

aggregate has an argument simplify that is TRUE by default, which means it tries to simplify to a vector or matrix when possible. All groups in P have n = 4, so your aggregated data is being simplified to a matrix. Just set simpflify = FALSE to change this behavior:
aggregate(x=t0g2[, !(colnames(t0g2) %in% c("P"))], by=list(P=t0g2[["P"]]),
FUN=function(y) unique(unlist(y)), simplify = F)
#### OUTPUT ####
P ID SC FP
1 1 8, 10, 7, 9 A, D, B 40, 41, 37, 8, 11, 49, 28, 16, 15, 27, 12, 20, 35, 45
2 2 5, 2, 3, 4 B, A, E 1, 34, 43, 37, 41, 7, 30, 2, 22, 35, 31, 10, 3, 29, 6, 15
3 3 8, 9, 1, 2 A, B, D 40, 41, 37, 8, 11, 27, 12, 20, 35, 45, 10, 49, 28, 7, 30, 2, 34, 43
4 4 8, 1, 4, 10 A, D, E 40, 41, 37, 8, 11, 10, 49, 28, 29, 6, 15, 16
5 5 4, 10, 2, 7 E, D, A 29, 6, 15, 49, 28, 16, 41, 7, 30, 2, 34, 43