I have multiple graphs I'm generating with a data set. I preform many operations on the data (filtering rows, aggregating rows, calculations over columns, etc.) before passing on the result to ggplot(). I want to access the data I passed on to ggplot() in subsequent ggplot layers and facets so I can have more control over the resulting graph and to include some characteristics of the data in the plot itself, like for example the number of observations.
Here is a reproducible example:
library(tidyverse)
cars <- mtcars
# Normal scatter plot
cars %>%
filter(
# Many complicated operations
) %>%
group_by(
# More complicated operations
across()
) %>%
summarise(
# Even more complicated operations
n = n()
) %>%
ggplot(aes(x = mpg, y = qsec)) +
geom_point() +
# Join the dots but only if mpg < 20
geom_line(data = .data %>% filter(mpg < 20)) +
# Include the total number of observations in the graph
labs(caption = paste("N. obs =", NROW(.data)))
one could of course create a a separate data set before passing that onto ggplot and then reference that data set throughout (as in the example bellow). However, this is much more cumbersome as you need to save (and later remove) a data set for each graph and run two separate commands for just one graph.
I want to know if there is something that can be done that's more akin to the first example using .data (which obviously doesn't actually work).
library(tidyverse)
cars <- mtcars
tmp <- cars %>%
filter(
# Many complicated operations
) %>%
group_by(
# More complicated operations
across()
) %>%
summarise(
# Even more complicated operations
n = n()
)
tmp %>%
ggplot(aes(x = mpg, y = qsec)) +
geom_point() +
# Join the dots but only if mpg < 20
geom_line(data = tmp %>% filter(mpg < 20)) +
# Include the total number of observations in the graph
labs(caption = paste("N. obs =", NROW(tmp)))
Thanks for your help!
In the help page for each geom_ it helpfully gives a standard way:
A function will be called with a single argument, the plot data. The return value must be a data.frame, and will be used as the layer data. A function can be created from a formula (e.g. ~ head(.x, 10)).
For labs on the other hand you can use the . placeholders in piping, but you have to a) give the . as the data argument in the first place and b) wrap the whole thing in curly braces to recognise the later ..
So for example:
library(tidyverse)
cars <- mtcars
# Normal scatter plot
cars %>%
filter() %>%
group_by(across()) %>%
summarise(n = n()) %>%
{
ggplot(., aes(x = mpg, y = qsec)) +
geom_point() +
geom_line(data = ~ filter(.x, mpg < 20)) +
labs(caption = paste("N. obs =", NROW(.)))
}
Or if you don't like the purrr formula syntax, then the flashy new R anonymous functions work too:
geom_line(data = \(x) filter(x, mpg < 20)) +
Unfortunately the labs function doesn't seem to have an explicit way of testing whether data is shuffling invisibly through the ggplot stack as by-and-large it usually can get on with its job without touching the main data. These are some ways around this.
Related
#Preparing the data and loading packages
library(modelsummary);library(tidyverse);library(gt)
as_tibble(mtcars)
df <- mtcars %>% mutate(cyl_ = factor(cyl)) %>%
dplyr::select(cyl_, mpg, vs, am, hp, wt)
#Gets table of descriptive statistics about different subsets of the data
print(t1 <- datasummary_balance(~cyl_,
data = df,
output = "gt"))
#This hides the "Std. Dev." columns
t1 %>% cols_hide(c(3,5,7))
#Now I want to hide the "Mean" column labels, but I want to keep the "cyl_" value column labels. Any ideas how?
I want something like this:
Using the gt package, you can pipe your table to tab_options(column_labels.hidden = TRUE) to remove column labels. Unfortunately, this will remove both levels: the column headers, and the spanning labels that include the cyl info you want to keep.
Note that datasummary_balance() produces a highly customized table which is intended to be used as a ready-made output. In cases like these, it might be easier to just build the custom table you want using datasummary() instead of trying to customize datasummary_balance() (square peg, round hole, etc). For example:
library(modelsummary)
library(tidyverse)
df <- mtcars %>%
select(cyl, mpg, vs, am, hp, wt) %>%
mutate(cyl = factor(sprintf("%s (N = %s)", cyl, n()))) %>%
as.data.frame() # The `All()` function does not accept tibbles
datasummary(
All(df) ~ Mean * cyl,
data = df,
output = "gt")
Within a custom function, how can I avoid repeating the same code for each group while allowing an unknown number of groups?
Here's a simpler example but assume the function has tons of operations, like calculating different statistics for each group and sticking them on each ggplot facet. Sorry, I find it difficult to make a simpler function to demonstrate this specific challenge.
test.function <- function(variable, group, data) {
if(!require(dplyr)){install.packages("dplyr")}
if(!require(ggplot2)){install.packages("ggplot2")}
if(!require(ggrepel)){install.packages("ggrepel")}
library(dplyr)
library(ggplot2)
require(ggrepel)
data$variable <- data[,variable]
data$group <- factor(data[,group])
# Compute individual group stats
data %>%
filter(data$group==levels(data$group)[1]) %>%
select(variable) %>%
unlist %>%
shapiro.test() -> shap
shapiro.1 <- round(shap$p.value,3)
data %>%
filter(data$group==levels(data$group)[2]) %>%
select(variable) %>%
unlist %>%
shapiro.test() -> shap
shapiro.2 <- round(shap$p.value,3)
data %>%
filter(data$group==levels(data$group)[3]) %>%
select(variable) %>%
unlist %>%
shapiro.test() -> shap
shapiro.3 <- round(shap$p.value,3)
# Make the stats dataframe for ggplot
dat_text <- data.frame(
group = levels(data$group),
text = c(shapiro.1, shapiro.2, shapiro.3))
# Make the plot
ggplot(data, aes(x=variable, fill=group)) +
geom_density() +
facet_grid(group ~ .) +
geom_text_repel(data = dat_text,
mapping = aes(x = Inf,
y = Inf,
label = text))
}
Works if there's three groups
test.function("mpg", "cyl", mtcars)
Doesn't work if there's two groups
test.function("mpg", "vs", mtcars)
Error in shapiro.test(.) : sample size must be between 3 and 5000
Doesn't work if there's more than three groups
test <- mtcars %>% mutate(new = rep(1:4, 8))
test.function("mpg", "new", test)
Error in data.frame(group = levels(data$group), text = c(shapiro.1, shapiro.2, :
arguments imply differing number of rows: 4, 3
What is the trick programmers usually use to accommodate any number of groups in such functions?
I was asked in the comments to explain the thinking here, so I thought I would expand on the original answer, which shows up below the horizontal rule below.
The main question is how to do some operation on an unknown number of groups. There are lots of different ways to do that. In any of the ways, you need the function to be able to identify the number of groups and adapt to that number. For example, you could do something like the code below. There, I identify the unique groups in the data, initialize the required result and then loop over all of the groups. I didn't use this strategy because the for loop feels a bit clunky compared to the dplyr code.
un_group <- na.omit(unique(data[[group]]))
dat_text <- data.frame(group = un_group,
text = NA)
for(i in 1:length(un_group)){
tmp <- data[which(data[[group]] == ungroup[i]), ]
dat_text$text[i] <- as.character(round(shaprio.test(tmp[[variable]])$p.value, 3))
}
The other thing to keep in mind is what's going to scale well. You mentioned that you've got lots of operations the code will ultimately do. In what's below, I just had summarise print a single number. However, you could write a little function that would produce a dataset and then summarise can return a number of results. For example, consider:
myfun <- function(x){
s = shapiro.test(x)
data.frame(p = s$p.value, stat=s$statistic,
mean = mean(x, na.rm=TRUE),
sd = sd(x, na.rm=TRUE),
skew = DescTools::Skew(x, na.rm=TRUE),
kurtosis = DescTools::Kurt(x, na.rm=TRUE))
}
mtcars %>% group_by(cyl) %>% summarise(myfun(mpg))
# # A tibble: 3 x 7
# cyl p stat mean sd skew kurtosis
# * <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
# 1 4 0.261 0.912 26.7 4.51 0.259 -1.65
# 2 6 0.325 0.899 19.7 1.45 -0.158 -1.91
# 3 8 0.323 0.932 15.1 2.56 -0.363 -0.566
In the function above, I had the function return a data frame with several different variables. A single call to summarise returns all of those results for the variable for each group. This would certainly have been possible using a for loop or something like sapply(), but I like how the dplyr code reads a bit better. And, depending on how many groups you have, the dplyr code scales a bit better than some of the base R stuff.
I really like trying to reflect the inputs (i.e., input variable names) in the outputs - so I wanted to find a way to get around making variables called group and variable in the data. The aes_string() specification is one way of doing that and then building a formula using the variable names is another. I recently just encountered the reformulate() function, which is a more robust way of building formulae than the combination of paste() and as.formula() I was using before.
Those were the things I was thinking about when I was answering the question.
test.function <- function(variable, group, data) {
if(!require(dplyr)){install.packages("dplyr")}
if(!require(ggplot2)){install.packages("ggplot2")}
if(!require(ggrepel)){install.packages("ggrepel")}
library(dplyr)
library(ggplot2)
require(ggrepel)
# Compute individual group stats
data[[group]] <- as.factor(data[[group]])
dat_text <- data %>% group_by(.data[[group]]) %>%
summarise(text=shapiro.test(.data[[variable]])$p.value) %>%
mutate(text=as.character(round(text, 3)))
gform <- reformulate(".", response=group)
# Make the plot
ggplot(data, aes_string(x=variable, fill=group)) +
geom_density() +
facet_grid(gform) +
geom_text_repel(data = dat_text,
mapping = aes(x = Inf,
y = Inf,
label = text))
}
test.function("mpg", "vs", mtcars)
test.function("mpg", "cyl", mtcars)
I'm looking for a way to apply a function to either specified labels, or to all labels that are included in the plot. The goal is to have neat human readable labels that derive from the default labels, without having to specify each.
To demonstrate what I am looking for in terms of the input variable names and the output, I am including an example based on the starwars data set, that uses the versatile snakecase::to_sentence_case() function, but this could apply to any function, including ones that expand short variable names in pre-determined ways:
library(tidyverse)
library(snakecase)
starwars %>%
filter(mass < 1000) %>%
mutate(species = species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
ggplot(aes(height, mass, color=species, size=birth_year)) +
geom_point() +
labs(
x = to_sentence_case("height"),
y = to_sentence_case("mass"),
color = to_sentence_case("species"),
size = to_sentence_case("birth_year")
)
Which produces the following graph:
The graph is the desired output, but requires that each of the labels be specified by hand, increasing the possibility of error if the variables are later changed. Note that if I had not specified the labels, all the labels would have been applied automatically, but with the variable names instead of the prettier versions.
This issue seems to be somewhat related to what the labeller() function is intended for, but it seems that it only applies to facetting. Another related issue is raised in this question. However, both of these seem to apply only to values contained within the data, not to the variable names that are being used in the plot, which is what I am looking for.
The very helpful answer by #z-lin demonstrated to me a simple way to do this by simply modifying the plot object before printing.
The intended result can be achieved with the help of gg_apply_labs(), a short function that will apply an arbitrary string processing function to the $labels of a plot object. The resulting code should be a self-contained illustration of this approach:
# Packages
library(tidyverse)
library(snakecase)
# This applies fun to each label present in the plot object
#
# fun should accept and return character vectors, it can either be a simple
# prettyfying function or it can perform more complex lookup to replace
# variable names with variable labels
gg_apply_labs <- function(p, fun) {
p$labels <- lapply(p$labels, fun)
p
}
# This gives the intended result
# Note: The plot is assigned to a named variable before piping to apply_labs()
p <- starwars %>%
filter(mass < 1000) %>%
mutate(species = species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
ggplot(aes(height, mass, color=species, size=birth_year)) +
geom_point()
p %>% gg_apply_labs(to_sentence_case)
# This also gives the intended result, in a single pipeline
# Note: It is important to put in the extra parentheses!
(starwars %>%
filter(mass < 1000) %>%
mutate(species = species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
ggplot(aes(height, mass, color=species, size=birth_year)) +
geom_point()) %>%
gg_apply_labs(to_sentence_case)
# This DOES NOT give the intended result
# Note: The issue is probably order precedence
starwars %>%
filter(mass < 1000) %>%
mutate(species = species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
ggplot(aes(height, mass, color=species, size=birth_year)) +
geom_point() %>%
gg_apply_labs(to_sentence_case)
A simple solution is to pipe through rename_all (or rename_if if you want more control) before plotting:
library(tidyverse)
library(snakecase)
starwars %>%
filter(mass<1000) %>%
mutate(species=species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
rename_all(to_sentence_case) %>%
#rename_if(is.character, to_sentence_case) %>%
ggplot(aes(Height, Mass, color=Species, size=`Birth year`)) +
geom_point()
#> Warning: Removed 23 rows containing missing values (geom_point).
Created on 2019-11-25 by the reprex package (v0.3.0)
Note, though, that the variables given to aes in ggplot in this case must be modified to match the modified sentence case variable names.
You can modify a ggplot object's appearance at the point of printing / plotting it, without affecting the original plot object, using trace:
trace(what = ggplot2:::ggplot_build.ggplot,
tracer = quote(plot$labels <- lapply(plot$labels,
<whatever string function you desire>)))
This will change the appearance of all existing / new ggplot objects you wish to plot / save, until you turn off the trace via either untrace(...) or tracingState(on = FALSE).
Illustration
Create a normal plot with default labels in lower case:
library(tidyverse)
p <- starwars %>%
filter(mass < 1000) %>%
mutate(species=species %>% fct_infreq %>% fct_lump(5) %>% fct_explicit_na) %>%
ggplot(aes(height, mass, color=species, size=birth_year)) +
geom_point() +
theme_bw()
p # if we print the plot now, all labels will be lower-case
Apply a function to modify the appearance of all labels:
trace(what = ggplot2:::ggplot_build.ggplot,
tracer = quote(plot$labels <- lapply(plot$labels,
snakecase::to_sentence_case)))
p # all labels will be in sentence case
trace(what = ggplot2:::ggplot_build.ggplot,
tracer = quote(plot$labels <- lapply(plot$labels,
snakecase::to_screaming_snake_case)))
p # all labels will be in upper case
trace(what = ggplot2:::ggplot_build.ggplot,
tracer = quote(plot$labels <- lapply(plot$labels,
snakecase::to_random_case)))
p # all letters in all labels may be in upper / lower case randomly
# (exact order can change every time we print the plot again, unless we set the same
# random seed for reproducibility)
trace(what = ggplot2:::ggplot_build.ggplot,
tracer = quote(plot$labels <- lapply(plot$labels,
function(x) paste("!!!", x, "$$$"))))
p # all labels now have "!!!" in front & "$$$" behind (this is a demonstration for
# an arbitrary user-defined function, not a demonstration of good taste in labels)
Toggle between applying & not applying the function:
tracingState(on = FALSE)
p # back to sanity, temporarily
tracingState(on = TRUE)
p # plot labels are affected by the function again
untrace(ggplot2:::ggplot_build.ggplot)
p # back to sanity, permanently
I have a dataset in which I have one numeric variable and many categorical variables. I would like to make a grid of density plots, each showing the distribution of the numeric variable for different categorical variables, with the fill corresponding to subgroups of each categorical variable. For example:
library(tidyverse)
library(nycflights13)
dat <- flights %>%
select(carrier, origin, distance) %>%
mutate(origin = origin %>% as.factor,
carrier = carrier %>% as.factor)
plot_1 <- dat %>%
ggplot(aes(x = distance, fill = carrier)) +
geom_density()
plot_1
plot_2 <- dat %>%
ggplot(aes(x = distance, fill = origin)) +
geom_density()
plot_2
I would like to find a way to quickly make these two plots. Right now, the only way I know how to do this is to create each plot individually, and then use grid_arrange to put them together. However, my real dataset has something like 15 categorical variables, so this would be very time intensive!
Is there a quicker and easier way to do this? I believe that the hardest part about this is that each plot has its own legend, so I'm not sure how to get around that stumbling block.
This solutions gives all the plots in a list. Here we make a single function that accepts a variable that you want to plot, and then use lapply with a vector of all the variables you want to plot.
fill_variables <- vars(carrier, origin)
func_plot <- function(fill_variable) {
dat %>%
ggplot(aes(x = distance, fill = !!fill_variable)) +
geom_density()
}
plotlist <- lapply(fill_variables, func_plot)
If you have no idea of what those !! mean, I recommend watching this 5 minute video that introduces the key concepts of tidy evaluation. This is what you want to use when you want to create this sorts of wrapper functions to do stuff programmatically. I hope this helps!
Edit: If you want to feed an array of strings instead of a quosure, you can change !!fill_variable for !!sym(fill_variable) as follows:
fill_variables <- c('carrier', 'origin')
func_plot <- function(fill_variable) {
dat %>%
ggplot(aes(x = distance, fill = !!sym(fill_variable))) +
geom_density()
}
plotlist <- lapply(fill_variables, func_plot)
Alternative solution
As #djc wrote in the comments, I'm having trouble passing the column names into 'fill_variables'. Right now I am extracting column names using the following code...
You can separate the categorical and numerical variables like; cat_vars <- flights[, sapply(flights, is.character)] for categorical variables and cat_vars <- flights[, sapply(flights, !is.character)] for continuous variables and then pass these vectors into the wrapper function given by mgiormenti
Full code is given below;
library(tidyverse)
library(nycflights13)
cat_vars <- flights[, sapply(flights, is.character)]
cont_vars<- flights[, !sapply(flights, is.character)]
dat <- flights %>%
select(carrier, origin, distance) %>%
mutate(origin = origin %>% as.factor,
carrier = carrier %>% as.factor)
func_plot_cat <- function(cat_vars) {
dat %>%
ggplot(aes(x = distance, fill = !!cat_vars)) +
geom_density()
}
func_plot_cont <- function(cont_vars) {
dat %>%
ggplot(aes(x = distance, fill = !!cont_vars)) +
geom_point()
}
plotlist_cat_vars <- lapply(cat_vars, func_plot_cat)
plotlist_cont_vars<- lapply(cont_vars, func_plot_cont)
print(plotlist_cat_vars)
print(plotlist_cont_vars)
The actual data (and aim) I have is different but for reproducing purposes I used the Titanic dataset. My aim is create a plot of the age outliers (1 time SD) per class and sex.
Therefore the first thing I did is calculating the sd values and ranges:
library(dplyr)
library(ggplot2)
#Load titanic set
titanic <- read.csv("titanic_total.csv")
group <- group_by(titanic, Pclass, Sex)
#Create outlier ranges
summarise <- summarise(group, mean=mean(Age), sd=sd(Age))
summarise <- as.data.frame(summarise)
summarise$outlier_max <- summarise$mean + summarise$sd
summarise$outlier_min <- summarise$mean - summarise$sd
#Create a key
summarise$key <- paste0(summarise$Pclass, summarise$Sex)
#Create a key for the base set
titanic$key <- paste0(titanic$Pclass, titanic$Sex)
total_data <- left_join(titanic, summarise, by = "key")
total_data$outlier <- 0
Next, using a loop I determine whether the age is inside or outside the range
for (row in 1:nrow(total_data)){
if((total_data$Age[row]) > (total_data$outlier_max[row])){
total_data$outlier[row] <- 1
} else if ((total_data$Age[row]) < (total_data$outlier_min[row])){
total_data$outlier[row] <- 1
} else {
total_data$outlier[row] <- 0
}
}
Do some data cleaning ...
total_data$Pclass.x <- as.factor(total_data$Pclass.x)
total_data$outlier <- as.factor(total_data$outlier)
Now this code gives me the plot I am looking for.
ggplot(total_data, aes(x = Age, y = Pclass.x, colour = outlier)) + geom_point() +
facet_grid(. ~Sex.x)
However, this not really seems like the easiest way to crack this problem. Any thoughts on how I can include best practises to make this more efficients.
One way to reduce your code and make it less repetitive is to get it all into one procedure thanks to the pipe. Instead of creating a summary with the values, re-join this with the data, you could basically do this within one mutate step:
titanic %>%
mutate(Pclass = as.factor(Pclass)) %>%
group_by(Pclass, Sex) %>%
mutate(Age.mean = mean(Age),
Age.sd = sd(Age),
outlier.max = Age.mean + Age.sd,
outlier.min = Age.mean - Age.sd,
outlier = as.factor(ifelse(Age > outlier.max, 1,
ifelse(Age < outlier.min, 1, 0)))) %>%
ggplot() +
geom_point(aes(Age, Pclass, colour = outlier)) +
facet_grid(.~Sex)
Pclass is mutated to a factor in advance, as it is a grouping factor. Then, the steps are done within the original dataframe, instead of creating two new ones. No changes are made to the original dataset however! If you would want this, just reassign the results to titanic or another data frame, and execute the ggplot-part as next step. Else you would assign the result of the figure to your data.
For the identification of outliers, one way is to work with the ifelse. Alternatively, dplyr offers the nice between function, however, for this, you would need to add rowwise, i.e. after creating the min and max thresholds for outliers:
...
rowwise() %>%
mutate(outlier = as.factor(as.numeric(between(Age, outlier.min, outlier.max)))) %>% ...
Plus:
Additionally, you could even reduce your code further, depends on which variables you want to keep in which way:
titanic %>%
group_by(Pclass, Sex) %>%
mutate(outlier = as.factor(ifelse(Age > (mean(Age) + sd(Age)), 1,
ifelse(Age < (mean(Age) - sd(Age)), 1, 0)))) %>%
ggplot() +
geom_point(aes(Age, as.factor(Pclass), colour = outlier)) +
facet_grid(.~Sex)