I have a dataframe such as:
COL1 VALUE1 VALUE2
1 A,A 1 5
2 A,A,B 1 3
3 C 1 1
4 D 1 2
5 D 1 2
6 A,A 1 10
7 A,B,A 1 2
and I can succeed to remove duplicate within the COL1 and count the number of different duplicated in COL1 by using:
as.data.frame(table(tab$COL1)) %>%
group_by(Var1 = sapply(strsplit(as.character(Var1), ","), function(x) toString(unique(x)))) %>%
summarise(Freq = sum(Freq))
And then I get:
# A tibble: 4 × 2
Var1 Freq
<chr> <int>
1 A 2
2 A, B 2
3 C 1
4 D 2
But I wondered if someone had an idea in order to add a new column called Mean which would be for each COL1 groups, the mean of the VALUE2 values and then get:
Var1 Freq Mean
1 A 2 7.5 < because (5+10)/2 =7.5
2 A, B 2 2.5 < because (3+2)/2 =2.5
3 C 1 1 < because 1/1 = 1
4 D 2 2 < because (2+2)/2 = 2
Here is the dataframe if it can helps:
structure(list(COL1 = structure(c(1L, 2L, 4L, 5L, 5L, 1L, 3L), .Label = c("A,A",
"A,A,B", "A,B,A", "C", "D"), class = "factor"), VALUE1 = c(1L,
1L, 1L, 1L, 1L, 1L, 1L), VALUE2 = c(5L, 3L, 1L, 2L, 2L, 10L,
2L)), class = "data.frame", row.names = c(NA, -7L))
You can calculate the frequency table directly in the dplyr chain, and then just add a Mean = mean(VALUE2) in the summarise() call.
I.e.
tab %>%
group_by(Var1 = sapply(strsplit(as.character(COL1), ","), function(x) toString(unique(x)))) %>%
summarise(Freq = sum(VALUE1), Mean = mean(VALUE2))
# # A tibble: 4 x 3
# Var1 Freq Mean
# <chr> <int> <dbl>
# 1 A 2 7.5
# 2 A, B 2 2.5
# 3 C 1 1
# 4 D 2 2
Is this what you want:
library(dplyr)
tab %>%
mutate(COL1 = sapply(strsplit(as.character(COL1), ","), function(x) toString(unique(x)))) %>%
group_by(COL1) %>%
summarise(Freq = sum(VALUE1),
Mean = mean(VALUE2))
# A tibble: 4 x 3
COL1 Freq Mean
* <chr> <int> <dbl>
1 A 2 7.5
2 A, B 2 2.5
3 C 1 1
4 D 2 2
Related
So, for example, I have the following dataframe, data:
col1
col2
1
5
1
5
1
3
2
10
2
11
3
11
Now, I want to make a new column, col3, which gives me the number of unique values in col2 for every grouping in col1.
So far, I have the following code:
length(unique(data$col2[data$col1 == 1]))
Which would here return the number 2.
However, I'm having a hard time making a loop that goes through all the values in col1 to create the new column, col3.
We can use n_distinct after grouping
library(dplyr)
data <- data %>%
group_by(col1) %>%
mutate(col3 = n_distinct(col2)) %>%
ungroup
-output
data
# A tibble: 6 × 3
col1 col2 col3
<int> <int> <int>
1 1 5 2
2 1 5 2
3 1 3 2
4 2 10 2
5 2 11 2
6 3 11 1
Or with data.table
library(data.table)
setDT(data)[, col3 := uniqueN(col2), col1]
data
data <- structure(list(col1 = c(1L, 1L, 1L, 2L, 2L, 3L), col2 = c(5L,
5L, 3L, 10L, 11L, 11L)), class = "data.frame", row.names = c(NA,
-6L))
You want the counts for every row, so using a for loop you would do
data$col3 <- NA_real_
for (i in seq_len(nrow(data))) {
data$col3[i] <- length(unique(data$col2[data$col1 == data$col1[i]]))
}
data
# col1 col2 col3
# 1 1 5 2
# 2 1 5 2
# 3 1 3 2
# 4 2 10 2
# 5 2 11 2
# 6 3 11 1
However, using for loops in R is mostly inefficient, and in this case we can use the grouping function ave which comes with R.
data <- transform(data, col3=ave(col2, col1, FUN=\(x) length(unique(x))))
data
# col1 col2 col3
# 1 1 5 2
# 2 1 5 2
# 3 1 3 2
# 4 2 10 2
# 5 2 11 2
# 6 3 11 1
Data:
data <- structure(list(col1 = c(1L, 1L, 1L, 2L, 2L, 3L), col2 = c(5L,
5L, 3L, 10L, 11L, 11L)), class = "data.frame", row.names = c(NA,
-6L))
This question already has answers here:
How can I remove all duplicates so that NONE are left in a data frame?
(3 answers)
Closed 12 months ago.
Here an example of a matrix,
A
B
C
1
1
1
1
1
4
1
2
4
2
1
1
3
1
1
3
1
2
I would like extract only rows which are unique in A and B.
I can't use unique, duplicate etc. because they retain always one of my duplicated row.
In final result I wish obtain:
A
B
C
1
2
4
2
1
1
How can I do it?
Thank you
Here are couple of options -
Base R -
cols <- c('A', 'B')
res <- df[!(duplicated(df[cols]) | duplicated(df[cols], fromLast = TRUE)), ]
res
# A B C
#3 1 2 4
#4 2 1 1
dplyr -
library(dplyr)
df %>% group_by(A, B) %>% filter(n() == 1) %>% ungroup
# A tibble: 2 x 3
# A B C
# <int> <int> <int>
#1 1 2 4
#2 2 1 1
data.table
df <- data.frame(
A = c(1L, 1L, 1L, 2L, 3L, 3L),
B = c(1L, 1L, 2L, 1L, 1L, 1L),
C = c(1L, 4L, 4L, 1L, 1L, 2L)
)
library(data.table)
setDT(df)[, .SD[.N == 1], by = list(A, B)]
#> A B C
#> 1: 1 2 4
#> 2: 2 1 1
Created on 2022-02-28 by the reprex package (v2.0.1)
I have a dataset like this below
W X Y Z
A 2 3 4
A 2 3 6
B 1 2 3
C 3 2 1
B 1 3 4
B 1 2 2
I am want to combine/collapse the values in column Z only if the values in column W, X, Y are similar.
The final dataset will be like this.
W X Y Z
A 2 3 4,6
B 1 2 3,2
C 3 2 1
B 1 3 4
Not sure how to do this, any suggestions is much appreciated.
We can group by 'W', 'X', 'Y' and paste the values of 'Z' (toString is paste(..., collapse=", "))
library(dplyr)
df1 %>%
group_by(W, X, Y) %>%
summarise(Z = toString(unique(Z)))
# A tibble: 4 x 4
# Groups: W, X [3]
# W X Y Z
# <chr> <int> <int> <chr>
#1 A 2 3 4, 6
#2 B 1 2 3, 2
#3 B 1 3 4
#4 C 3 2 1
Or with aggregate from base R
aggregate(Z ~ ., unique(df1), toString)
# W X Y Z
#1 B 1 2 3, 2
#2 C 3 2 1
#3 B 1 3 4
#4 A 2 3 4, 6
data
df1 <- structure(list(W = c("A", "A", "B", "C", "B", "B"), X = c(2L,
2L, 1L, 3L, 1L, 1L), Y = c(3L, 3L, 2L, 2L, 3L, 2L), Z = c(4L,
6L, 3L, 1L, 4L, 2L)), class = "data.frame", row.names = c(NA,
-6L))
I have the following dataset
clust T2 n
1 a 1
1 b 3
1 c 3
2 d 5
3 a 4
3 b 3
4 b 5
4 c 8
4 t 6
4 e 7
etc..
using the following function:
library(dplyr)
table <- data %>% group_by(clust) %>% summarise(max = max(n), name1 = T2[which.max(n)])
I get this output
clust max name1
1 3 b
2 5 d
3 4 a
4 8 c
etc
however there are cases where there are two or more T2 values corresponding to max(n). how can I record those value too?
i.e.
clust max name1
1 3 b,c
2 5 d
3 4 a
4 8 c
etc
or
clust max name1
1 3 b
1 3 c
2 5 d
3 4 a
4 8 c
etc
We can do a == instead of which.max (that returns only the first index of max value) and paste together with toString
library(dplyr)
library(tidyr)
data %>%
group_by(clust) %>%
summarise(max = max(n), name1 = toString(T2[n == max(n)]))
# A tibble: 4 x 3
# clust max name1
# <int> <int> <chr>
#1 1 3 b, c
#2 2 5 d
#3 3 4 a
#4 4 8 c
and this can be expanded with separate_rows in the next step
data %>%
group_by(clust) %>%
summarise(max = max(n), name1 = toString(T2[n == max(n)])) %>%
separate_rows(name1, sep=",\\s+")
# A tibble: 5 x 3
# clust max name1
# <int> <int> <chr>
#1 1 3 b
#2 1 3 c
#3 2 5 d
#4 3 4 a
#5 4 8 c
Or have a list column and then unnest
data %>%
group_by(clust) %>%
summarise(max = max(n), name1 = list(T2[n == max(n)])) %>%
unnest(c(name1))
# A tibble: 5 x 3
# clust max name1
# <int> <int> <chr>
#1 1 3 b
#2 1 3 c
#3 2 5 d
#4 3 4 a
#5 4 8 c
data
data <- structure(list(clust = c(1L, 1L, 1L, 2L, 3L, 3L, 4L, 4L, 4L,
4L), T2 = c("a", "b", "c", "d", "a", "b", "b", "c", "t", "e"),
n = c(1L, 3L, 3L, 5L, 4L, 3L, 5L, 8L, 6L, 7L)),
class = "data.frame", row.names = c(NA,
-10L))
suppose I want to find duplicate rows for columns:
cols<-c("col1", "col2")
I know for data f4 duplicate rows are:
Jo<-df4[duplicated(df4[cols]) | duplicated(df4[cols], fromLast = TRUE), ]
and removing these duplicate rows from data set is given:
No<-df4[!(duplicated(df4[cols]) | duplicated(df4[cols], fromLast = TRUE)), ]
I want to modify the above codes. Suppose there is a column called mode. It takes integers between 1 to 4. I don't want all of duplicate rows have the same mode==2.
example
col1 col2 mode
1 3 5
5 3 9
1 2 1
1 2 1
3 2 2
3 2 2
4 1 3
4 1 2
4 1 2
output
Jo:
col1 col2 mode
1 2 1
1 2 1
4 1 3
4 1 2
4 1 2
No:
col1 col2 mode
1 3 5
5 3 9
3 2 2
3 2 2
in the above example in 3 and 4-th rows since mode==2 for both it is not duplicate but for three last row since one of them is not 2 , the are duplicate
Based on the updated dataset,
library(dplyr)
out1 <- df2 %>%
group_by_at(vars(cols)) %>%
filter(n() > 1, !all(mode ==2))
out2 <- anti_join(df2, out1)
out1
# A tibble: 5 x 3
# Groups: col1, col2 [2]
# col1 col2 mode
# <int> <int> <int>
#1 1 2 1
#2 1 2 1
#3 4 1 3
#4 4 1 2
#5 4 1 2
out2
# col1 col2 mode
#1 1 3 5
#2 5 3 9
#3 3 2 2
#4 3 2 2
Or with data.table
library(data.table)
i1 <- setDT(df2)[ , .I[.N > 1 & !all(mode == 2)], by = cols]$V1
df2[i1]
# col1 col2 mode
#1: 1 2 1
#2: 1 2 1
#3: 4 1 3
#4: 4 1 2
#5: 4 1 2
df2[!i1]
# col1 col2 mode
#1: 1 3 5
#2: 5 3 9
#3: 3 2 2
#4: 3 2 2
Or using base R
i1 <- duplicated(df2[1:2])|duplicated(df2[1:2], fromLast = TRUE)
out11 <- df2[i1 & with(df2, !ave(mode==2, col1, col2, FUN = all)),]
out22 <- df2[setdiff(row.names(df2), row.names(out11)),]
data
df2 <- structure(list(col1 = c(1L, 5L, 1L, 1L, 3L, 3L, 4L, 4L, 4L),
col2 = c(3L, 3L, 2L, 2L, 2L, 2L, 1L, 1L, 1L), mode = c(5L,
9L, 1L, 1L, 2L, 2L, 3L, 2L, 2L)), class = "data.frame", row.names = c(NA,
-9L))