Why does ggplot2 allow grouping by invalid aesthetic mappings? - r

Question

I'm having trouble understanding how grouping works in ggplot. Suppose I have a dataframe with date, value, categorical, and a dichotomous variable like so:
library(ggplot2)
set.seed(42)
data <- data.frame(date = sample(seq(as.Date('2015/01/01'), as.Date('2020/01/01'), by="day"), 20),
values = rnorm(20, 0, 1),
categories = sample(c("a", "b", "c"), 20, replace = TRUE))
data$pre_post <- ifelse(data$date <= '2018/01/01', "pre", "post")
If I group by the 'pre_post' variable (dichotomous) using aes(group =), I get:
ggplot(data, aes(x = date, y = values, color = categories, group = pre_post)) +
geom_line()
Using aes(by =), the results change to:
ggplot(data, aes(x = date, y = values, color = categories, by = pre_post)) +
geom_line()
In my current use case, 'by=' gives me the desired results but I don't know how to explain what exactly it is doing and why. Especially since, as #markus points out that 'by=' isn't even a valid aesthetic and that replacing it with 'foo=' would do the same thing:
ggplot(data, aes(x = date, y = values, color = categories, foo = pre_post)) +
geom_line()

Answer 1

If I understand the examples and explanations in Aesthetics: grouping of the ggplot2 documention correctly, the group aesthetic maps a different line for each subject.
group = pre_post seems to take precedence over the grouping by the color aesthetic enforcing to draw a single line connecting all "pre" data points and another line connecting all "post" data points.
In order to get six different lines for each combination of categories and pre_post you may follow Axeman's suggestion to use interaction():
ggplot(data, aes(x = date, y = values, color = categories, group = interaction(categories, pre_post))) +
geom_line()
or you may use a different aesthetic like linetype:
ggplot(data, aes(x = date, y = values, color = categories, linetype = pre_post)) +
geom_line()
Using an undefined aesthetic like by or foo causes the data to be grouped as well but without effect on the aesthetical appearance.
However, grouping a line chart may lead to a loss of information as can be seen here:
ggplot(data, aes(x = date, y = values, color = categories)) +
geom_line()
Without grouping by pre_post the blue line extends into 2018 which was not visible on the previous plots.
To avoid the loss of information I prefer to plot the data points as well as the lines in order to mark the end points of the line segments or to show a single data point:
ggplot(data, aes(x = date, y = values, color = categories, linetype = pre_post, shape = pre_post)) +
geom_line() +
geom_point() +
geom_label(aes(label = date), data = data[data$categories == "c" & data$pre_post == "post", ],
hjust = -0.1, vjust = -0.1, show.legend = FALSE)
geom_label() is optional and only used in this post to highlight the single data point.