color lines ggplot by last value r - r

Question

I need help with plotting > 741 lines in a ggplot.
The color of one specific line should not change, e.g. the color line should be assigned only by the final value of eci.
I would want to display the name (in the code example “unit”) of each line at the beginning and the end of each line
Of course over 700 lines are hard to distinguish with the bare eye but any suggestions how to make the lines more distinguishable?
df <- data.frame(unit=rep(1:741, 4),
year=rep(c(2012, 2013, 2014, 2015), each=741),
eci=round(runif(2964, 1, 741), digits = 0))
g = ggplot(data = df, aes(x=year, y=eci, group=unit)) +
geom_line(aes(colour=eci), size=0.01) +
scale_colour_gradientn(colours = terrain.colors(10)) +
geom_point(aes(colour=eci), size=0.04)
# The colour of the line should be determined by all eci for which year=2015

Answer 1

One way to achieve your desired result is creating new columns with extra information to use when plotting with ggplot2.
With dplyr, we group data by unit, and then arrange it, so we can create a column that stores the value of the last eci, and two columns with labels for the first and last year, so we can add them as text to the plot.
df_new <- df %>%
group_by(unit) %>%
arrange(unit, year, eci) %>%
mutate(last_eci = last(eci),
first_year = ifelse(year == 2012, unit, ""),
last_year = ifelse(year == 2015, unit, ""))
Then, we plot it.
ggplot(data = df_new,
aes(x = year, y = eci, group = unit, colour = last_eci)) +
geom_line(size = 0.01) +
geom_text(aes(label = first_year), nudge_x = -0.05, color = "black") +
geom_text(aes(label = last_year), nudge_x = 0.05, color = "black") +
scale_colour_gradientn(colours = terrain.colors(10)) +
geom_point(aes(colour = eci), size = 0.04)
Of course, looking at the resulting plot it's easy to see that trying to plot >700 lines of different colors and >1400 labels in a single plot is not very advisable.
I'd use relevant subsets of df, so we produce plots that helps us to better understand the data.
df_new %>%
filter(unit %in% c(1:10)) %>%
ggplot(data = .,
aes(x = year, y = eci, group = unit, colour = last_eci)) +
geom_line(size = 0.01) +
geom_text(aes(label = first_year), nudge_x = -0.05, color = "black") +
geom_text(aes(label = last_year), nudge_x = 0.05, color = "black") +
scale_colour_gradientn(colours = terrain.colors(10)) +
geom_point(aes(colour = eci), size = 0.04)

Answer 2

For better readability, I have opted for a 10-line example, using the directlabels-package.
library(ggplot2)
library(dplyr)
library(directlabels)
set.seed(95)
l <- 10
df1 <- data.frame(unit=rep(1:l, 4),
year=rep(c(2012, 2013, 2014, 2015), each=l),
eci=round(runif(4*l, 1, l), digits = 0))
df2 <- df1 %>% filter (year == 2015) %>% select(-year, end = eci)
df <- left_join(df1,df2, by = "unit")
g <-
ggplot(data = df, aes(x=year,
y=eci,
group=unit)) +
geom_line(aes(colour=end), size=0.01) +
scale_colour_gradientn(colours = terrain.colors(10)) +
geom_point(aes(colour=eci), size=0.04) +
geom_dl(aes(label = unit,color = end), method = list(dl.combine("first.points", "last.points"), cex = 0.8))
g

Answer 3

Half a year later, I think there is a much easier solution based on parcoord() applied to a wide df.
set.seed(95)
l <- 1000 # really 1000 observations per year this time
df1 <- data.frame(unit=rep(1:l, 4),
year=rep(c(2012, 2013, 2014, 2015), each=l),
eci=round(runif(4*l, 1, l), digits = 0))
df1 <- tidyr::spread(df1, year, eci) # change from long to wide
df1 <- df1 %>%
dplyr::arrange(desc(`2015`)) # Assign after which column (year) rows should be ordered
# create 10 different colrs which are repeated 100 times
my_colors=rep(terrain.colors(11)[-1], each=100)
parcoord(df1[, c(2:5)] , col= my_colors)
This is more efficient and easily scaleable.