This can be a very dumb question regarding application of 'across' function with multiple columns and different funs applied to each. For eg, below is an example from 'iris' data where column names starting with 'Sepal' are averaged. What if I along with this I want to take median of columns starting with 'Petal'. How can I give the two different column types and funs in same summarise?
iris %>%
group_by(Species) %>%
summarise(across(starts_with("Sepal"), mean))
You can add multiple across statements in one summarise/mutate :
library(dplyr)
iris %>%
group_by(Species) %>%
summarise(across(starts_with("Sepal"), mean),
across(starts_with("Petal"), median))
# Species Sepal.Length Sepal.Width Petal.Length Petal.Width
#* <fct> <dbl> <dbl> <dbl> <dbl>
#1 setosa 5.01 3.43 1.5 0.2
#2 versicolor 5.94 2.77 4.35 1.3
#3 virginica 6.59 2.97 5.55 2
Related
I have a question about for loop in r. I have used the following for loop
for (i in 1:length(unique(iris$Species))) {
datu <- data.frame(ID = unique(i),
Sl = mean(iris$Sepal.Length),
Sw = mean(iris$Sepal.Width))
}
to get the mean of each unique species in iris. But my final data only has one observation. However my desired output is separate for setosa versicolor virginica. What should i change in this code? Thanks
We don't need a loop. It can be done with group by approach
setNames(aggregate(.~ Species, iris[c(1, 2, 5)], mean), c("ID", "Sl", "Sw"))
-output
ID Sl Sw
1 setosa 5.006 3.428
2 versicolor 5.936 2.770
3 virginica 6.588 2.974
Or with tidyverse
library(dplyr)
library(stringr)
iris %>%
group_by(ID = Species) %>%
summarise(across(starts_with("Sepal"), ~ mean(.x, na.rm = TRUE),
.names = "{str_to_title(str_remove_all(.col, '[a-z.]+'))}"))
-output
# A tibble: 3 × 3
ID Sl Sw
<fct> <dbl> <dbl>
1 setosa 5.01 3.43
2 versicolor 5.94 2.77
3 virginica 6.59 2.97
In the loop, the unique(i) is just i, instead if we meant unique(iris$Species)[i]. In addition, the datu will get updated in each iteration, returning only the last output from the iteration. Instead, it can be stored in a list and rbind later or use
datu <- data.frame()
for (i in 1:length(unique(iris$Species))) {
unqSp <- unique(iris$Species)[i]
i1 <- iris$Species == unqSp
datu <- rbind(datu, data.frame(ID = unqSp,
Sl = mean(iris$Sepal.Length[i1]),
Sw = mean(iris$Sepal.Width[i1])))
}
-output
> datu
ID Sl Sw
1 setosa 5.006 3.428
2 versicolor 5.936 2.770
3 virginica 6.588 2.974
A tidyverse approach using dplyr.
dplyr::summarize
‘summarise()’ creates a new data frame. It will have one (or more)
rows for each combination of grouping variables; if there are no
grouping variables, the output will have a single row summarising
all observations in the input.
library(dplyr)
iris %>%
group_by(Species) %>%
summarize(Sl = mean(Sepal.Length), Sw = mean(Sepal.Width))
# A tibble: 3 × 3
Species Sl Sw
<fct> <dbl> <dbl>
1 setosa 5.01 3.43
2 versicolor 5.94 2.77
3 virginica 6.59 2.97
I want to group by keeping the continuous columns as rows and the categorical factors as the column headers with the aggregated record being the mean or min or max. This is a fundamental question, the answer to which I am not being able to figure out. Take the iris data as an example. I want to get the mean of sepal.width and sepal.length with respect to every species category.
library(dplyr)
mydata2 <-iris
# Groupby function for dataframe in R
summarise_at(group_by(mydata2,Species),vars(Sepal.Length),funs(mean(.,na.rm=TRUE)))
OUTPUT
Species Sepal.Length
<fct> <dbl>
1 setosa 5.01
2 versicolor 5.94
3 virginica 6.59
I want to get the same output with Sepal.Length as my rows instead of Species and the various factors of Species as my columns. I also want Sepal.Width, Petal.Length, Petal.Width as well How will I do that?
This is what I am looking for -
Species setosa versicolor virginica
1 Sepal.Length 5.01 5.94 6.59
Below this there should be Sepal.Width and other continuous columns as well.
I have tried transposing but that is changing everything to character data type.
One option to achieve your desired result would be to reshape your data after summarise via e.g. pivot_longer and pivot_wider. If you do that often you could put the code into a convenience function to do that in one step:
Note: I also dropped the summarise_at and switched to the new API using across and where.
library(dplyr)
library(tidyr)
summarise(group_by(iris, Species), across(where(is.numeric), mean, na.rm=TRUE)) %>%
pivot_longer(-Species, names_to = "var") %>%
pivot_wider(names_from = Species, values_from = value)
#> # A tibble: 4 × 4
#> var setosa versicolor virginica
#> <chr> <dbl> <dbl> <dbl>
#> 1 Sepal.Length 5.01 5.94 6.59
#> 2 Sepal.Width 3.43 2.77 2.97
#> 3 Petal.Length 1.46 4.26 5.55
#> 4 Petal.Width 0.246 1.33 2.03
You can use tapply insinde lapply:
do.call(rbind, lapply(iris[sapply(iris, is.numeric)],
function(x) tapply(x, iris$Species, mean)))
# setosa versicolor virginica
#Sepal.Length 5.006 5.936 6.588
#Sepal.Width 3.428 2.770 2.974
#Petal.Length 1.462 4.260 5.552
#Petal.Width 0.246 1.326 2.026
Suppose we want to group_by() and summarise a massive data.frame with very many columns, but that there are some large groups of consecutive columns that will have the same summarise condition (e.g. max, mean etc)
Is there a way to avoid having to specify the summarise condition for each and every column, and instead do it for ranges of columns?
Example
Suppose we want to do this:
iris %>%
group_by(Species) %>%
summarise(max(Sepal.Length), mean(Sepal.Width), mean(Petal.Length), mean(Petal.Width))
but note that 3 consecutive columns have the same summarise condition, mean(Sepal.Width), mean(Petal.Length), mean(Petal.Width)
Is there a way to use some method like mean(Sepal.Width:Petal.Width) to specify the condition for the range of columns, and hence a avoiding having to type out the summarise condition multiple times for all the columns in between)
Note
The iris example above is a small and manageable example that has a range of 3 consecutive columns, but actual use case has ~hundreds.
The upcoming version 1.0.0 of dplyr will have across() function that does what you wish for
Basic usage
across() has two primary arguments:
The first argument, .cols, selects the columns you want to operate on.
It uses tidy selection (like select()) so you can pick variables by
position, name, and type.
The second argument, .fns, is a function or list of functions to apply to
each column. This can also be a purrr style formula (or list of formulas)
like ~ .x / 2. (This argument is optional, and you can omit it if you just want
to get the underlying data; you'll see that technique used in
vignette("rowwise").)
### Install development version on GitHub first
# install.packages("devtools")
# devtools::install_github("tidyverse/dplyr")
library(dplyr, warn.conflicts = FALSE)
Control how the names are created with the .names argument which takes a glue spec:
iris %>%
group_by(Species) %>%
summarise(
across(c(Sepal.Width:Petal.Width), ~ mean(.x, na.rm = TRUE), .names = "mean_{col}"),
across(c(Sepal.Length), ~ max(.x, na.rm = TRUE), .names = "max_{col}")
)
#> # A tibble: 3 x 5
#> Species mean_Sepal.Width mean_Petal.Leng~ mean_Petal.Width max_Sepal.Length
#> * <fct> <dbl> <dbl> <dbl> <dbl>
#> 1 setosa 3.43 1.46 0.246 5.8
#> 2 versicolor 2.77 4.26 1.33 7
#> 3 virginica 2.97 5.55 2.03 7.9
Using multiple functions
my_func <- list(
mean = ~ mean(., na.rm = TRUE),
max = ~ max(., na.rm = TRUE)
)
iris %>%
group_by(Species) %>%
summarise(across(where(is.numeric), my_func, .names = "{fn}.{col}"))
#> # A tibble: 3 x 9
#> Species mean.Sepal.Length max.Sepal.Length mean.Sepal.Width max.Sepal.Width
#> * <fct> <dbl> <dbl> <dbl> <dbl>
#> 1 setosa 5.01 5.8 3.43 4.4
#> 2 versicolor 5.94 7 2.77 3.4
#> 3 virginica 6.59 7.9 2.97 3.8
#> mean.Petal.Length max.Petal.Length mean.Petal.Width max.Petal.Width
#> * <dbl> <dbl> <dbl> <dbl>
#> 1 1.46 1.9 0.246 0.6
#> 2 4.26 5.1 1.33 1.8
#> 3 5.55 6.9 2.03 2.5
Created on 2020-03-06 by the reprex package (v0.3.0)
Since summarise collapses the rows and hence we cannot further apply any functions to it, we can use mutate_at instead, select range of columns to apply function and then select 1st row from every group.
library(dplyr)
iris %>%
group_by(Species) %>%
mutate_at(vars(Sepal.Width:Petal.Width), mean) %>%
mutate_at(vars(Sepal.Length), max) %>%
slice(1L)
# Sepal.Length Sepal.Width Petal.Length Petal.Width Species
# <dbl> <dbl> <dbl> <dbl> <fct>
#1 5.8 3.43 1.46 0.246 setosa
#2 7 2.77 4.26 1.33 versicolor
#3 7.9 2.97 5.55 2.03 virginica
We can use pmap from purrr to apply various functions to various columns and then join back together at the end. Note the use of lst from purrr so we can refer to previously named objects in the list construction. This allows us to analyze the same column with multiple functions, such as Sepal.Length below.
library(tidyverse)
lst(a = list("Sepal.Length", names(select(iris, Sepal.Length:Petal.Width))),
b = list("max" = max, "mean" = mean),
c = names(b)) %>%
pmap(function(a, b, c) {
iris %>%
group_by(Species) %>%
summarize_at(a, b) %>%
rename_at(a, paste0, "_", c)
}) %>%
reduce(inner_join, by = "Species")
#> # A tibble: 3 x 6
#> Species Sepal.Length_max Sepal.Length_me~ Sepal.Width_mean Petal.Length_me~
#> <fct> <dbl> <dbl> <dbl> <dbl>
#> 1 setosa 5.8 5.01 3.43 1.46
#> 2 versic~ 7 5.94 2.77 4.26
#> 3 virgin~ 7.9 6.59 2.97 5.55
#> # ... with 1 more variable: Petal.Width_mean <dbl>
Suppose I have the following function
SlowFunction = function(vector){
return(list(
mean =mean(vector),
sd = sd(vector)
))
}
And I would like to use dplyr:summarise to write the results to a dataframe:
iris %>%
dplyr::group_by(Species) %>%
dplyr::summarise(
mean = SlowFunction(Sepal.Length)$mean,
sd = SlowFunction(Sepal.Length)$sd
)
Does anyone have a suggestion how I can do this by calling "SlowFunction" once instead of twice? (In my code "SlowFunction" is a slow function that I have to call many times.) Without splitting "SlowFunction" in two parts of course. So actually I would like to somehow fill multiple columns of a dataframe in one statement.
Without changing your current SlowFunction one way is to use do
library(dplyr)
iris %>%
group_by(Species) %>%
do(data.frame(SlowFunction(.$Sepal.Length)))
# Species mean sd
# <fct> <dbl> <dbl>
#1 setosa 5.01 0.352
#2 versicolor 5.94 0.516
#3 virginica 6.59 0.636
Or with group_split + purrr::map_dfr
bind_cols(Species = unique(iris$Species), iris %>%
group_split(Species) %>%
map_dfr(~SlowFunction(.$Sepal.Length)))
An option is to use to store the output of SlowFunction in a list column of data.frames and then to use unnest
iris %>%
group_by(Species) %>%
summarise(res = list(as.data.frame(SlowFunction(Sepal.Length)))) %>%
unnest()
## A tibble: 3 x 3
# Species mean sd
# <fct> <dbl> <dbl>
#1 setosa 5.01 0.352
#2 versicolor 5.94 0.516
#3 virginica 6.59 0.636
We can use group_map if you are using dplyr 0.8.0 or later. The output from SlowFunction needs to be converted to a data frame.
library(dplyr)
iris %>%
group_by(Species) %>%
group_map(~SlowFunction(.x$Sepal.Length) %>% as.data.frame())
# # A tibble: 3 x 3
# # Groups: Species [3]
# Species mean sd
# <fct> <dbl> <dbl>
# 1 setosa 5.01 0.352
# 2 versicolor 5.94 0.516
# 3 virginica 6.59 0.636
We can change the SlowFunction to return a tibble and
SlowFunction = function(vector){
tibble(
mean =mean(vector),
sd = sd(vector)
)
}
and then unnest the summarise output in a list
iris %>%
group_by(Species) %>%
summarise(out = list(SlowFunction(Sepal.Length))) %>%
unnest
# A tibble: 3 x 3
# Species mean sd
# <fct> <dbl> <dbl>
#1 setosa 5.01 0.352
#2 versicolor 5.94 0.516
#3 virginica 6.59 0.636
I would like to use summarise() from dplyr after grouping data to compute a new variable. But, I would like it to use one equation for some of the data and a second equation for the rest of the data.
I have tried using group_by() and and summarise() with if_else() but it isn't working.
Here's an example. Let's say--for some reason--I wanted to find a special value for sepal length. For the species 'setosa' this special value is twice the mean of the sepal length. For all of the other species it is simply the mean of sepal length. This is the code I've tried, but it doesn't work with summarise()
library(dplyr)
iris %>%
group_by(Species) %>%
summarise(sepal_special = if_else(Species == "setosa", mean(Sepal.Length)*2, mean(Sepal.Length)))
This idea works with mutate() but I would need to re-format the tibble to be the dataset I am looking for.
library(dplyr)
iris %>%
group_by(Species) %>%
mutate(sepal_special = if_else(Species == "setosa", mean(Sepal.Length)*2, mean(Sepal.Length)))
This is how I want the resulting tibble to be laid out:
library(dplyr)
iris %>%
group_by(Species)%>%
summarise(sepal_mean = mean(Sepal.Length))
# A tibble: 3 x 2
# Species sepal_special
# <fctr> <dbl>
#1 setosa 5.01
#2 versicolor 5.94
#3 virginica 6.59
#>
But my result would show the value for setosa x 2
# A tibble: 3 x 2
# Species sepal_special
# <fctr> <dbl>
#1 setosa **10.02**
#2 versicolor 5.94
#3 virginica 6.59
#>
Suggestions? I feel like I've really searched for ways to use if_else() with summarise() but can't find it anywhere, which means there must be a better way.
Thanks!
After the mutate step, use summarise to get the first element of 'sepal_special' for each 'Species'
iris %>%
group_by(Species) %>%
mutate(sepal_special = if_else(Species == "setosa",
mean(Sepal.Length)*2, mean(Sepal.Length))) %>%
summarise(sepal_special = first(sepal_special))
# A tibble: 3 x 2
# Species sepal_special
# <fctr> <dbl>
#1 setosa 10.0
#2 versicolor 5.94
#3 virginica 6.59
Or instead of calling the mutate, after the if_else is applied, get the first value in summarise
iris %>%
group_by(Species) %>%
summarise(sepal_special = if_else(Species == "setosa",
mean(Sepal.Length)*2, mean(Sepal.Length))[1])
# A tibble: 3 x 2
# Species sepal_special
# <fctr> <dbl>
#1 setosa 10.0
#2 versicolor 5.94
#3 virginica 6.59
Another option: since twice the mean is the same as the mean of twice the values, you can double the sepal lengths for setosa and then summarise:
iris %>%
mutate(Sepal.Length = ifelse(Species == "setosa", 2*Sepal.Length, Sepal.Length)) %>%
group_by(Species) %>%
summarise(sepal_special = mean(Sepal.Length))
# A tibble: 3 x 2
Species sepal_special
<fct> <dbl>
1 setosa 10.0
2 versicolor 5.94
3 virginica 6.59