How calculate growth rate in long format data frame? - r

Question

With data structured as follows...
df <- data.frame(Category=c(rep("A",6),rep("B",6)),
Year=rep(2010:2015,2),Value=1:12)
I'm having a tough time creating a growth rate column (by year) within category. Can anyone help with code to create something like this...
Category Year Value Growth
A 2010 1
A 2011 2 1.000
A 2012 3 0.500
A 2013 4 0.333
A 2014 5 0.250
A 2015 6 0.200
B 2010 7
B 2011 8 0.143
B 2012 9 0.125
B 2013 10 0.111
B 2014 11 0.100
B 2015 12 0.091

Answer 1

For these sorts of questions ("how do I compute XXX by category YYY")? there are always solutions based on by(), the data.table() package, and plyr. I generally prefer plyr, which is often slower, but (to me) more transparent/elegant.
df <- data.frame(Category=c(rep("A",6),rep("B",6)),
Year=rep(2010:2015,2),Value=1:12)
library(plyr)
ddply(df,"Category",transform,
Growth=c(NA,exp(diff(log(Value)))-1))
The main difference between this answer and #krlmr's is that I am using a geometric-mean trick (taking differences of logs and then exponentiating) while #krlmr computes an explicit ratio.
Mathematically, diff(log(Value)) is taking the differences of the logs, i.e. log(x[t+1])-log(x[t]) for all t. When we exponentiate that we get the ratio x[t+1]/x[t] (because exp(log(x[t+1])-log(x[t])) = exp(log(x[t+1]))/exp(log(x[t])) = x[t+1]/x[t]). The OP wanted the fractional change rather than the multiplicative growth rate (i.e. x[t+1]==x[t] corresponds to a fractional change of zero rather than a multiplicative growth rate of 1.0), so we subtract 1.
I am also using transform() for a little bit of extra "syntactic sugar", to avoid creating a new anonymous function.

Answer 2

You can simply use dplyr package:
> df %>% group_by(Category) %>% mutate(Growth = (Value - lag(Value))/lag(Value))
which will produce the following result:
# A tibble: 12 x 4
# Groups: Category [2]
Category Year Value Growth
<fct> <int> <int> <dbl>
1 A 2010 1 NA
2 A 2011 2 1
3 A 2012 3 0.5
4 A 2013 4 0.333
5 A 2014 5 0.25
6 A 2015 6 0.2
7 B 2010 7 NA
8 B 2011 8 0.143
9 B 2012 9 0.125
10 B 2013 10 0.111
11 B 2014 11 0.1
12 B 2015 12 0.0909

Answer 3

Using R base function (ave)
> dfdf$Growth <- with(df, ave(Value, Category,
FUN=function(x) c(NA, diff(x)/x[-length(x)]) ))
> df
Category Year Value Growth
1 A 2010 1 NA
2 A 2011 2 1.00000000
3 A 2012 3 0.50000000
4 A 2013 4 0.33333333
5 A 2014 5 0.25000000
6 A 2015 6 0.20000000
7 B 2010 7 NA
8 B 2011 8 0.14285714
9 B 2012 9 0.12500000
10 B 2013 10 0.11111111
11 B 2014 11 0.10000000
12 B 2015 12 0.09090909
#Ben Bolker's answer is easily adapted to ave:
transform(df, Growth=ave(Value, Category,
FUN=function(x) c(NA,exp(diff(log(x)))-1)))

Answer 4

Very easy with plyr:
library(plyr)
ddply(df, .(Category),
function (d) {
d$Growth <- c(NA, tail(d$Value, -1) / head(d$Value, -1) - 1)
d
}
)
We have two problems here:
Splitting by category
Computing the growth rate
ddply is the workhorse, the split and the function to compute the growth rate is defined by parameters to this function.

Answer 5

A more elegant variant based on Ben's idea with the new gdiff function in my R package:
df <- data.frame(Category=c(rep("A",6),rep("B",6)),
Year=rep(2010:2015,2),Value=1:12)
library(plyr)
ddply(df, "Category", transform,
Growth=c(NA, kimisc::gdiff(Value, FUN = `/`)-1))
Here, gdiff is used to compute a lagged rate (instead of a lagged difference as diff would).

Answer 6

Many years later: the tsbox package aims to work with all kind of time series objects, including data frames, and offers a standard time series toolkit. Thus, calculating growth rates is as simple as:
df <- data.frame(Category=c(rep("A",6),rep("B",6)),
Year=rep(2010:2015,2),Value=1:12)
library(tsbox)
ts_pc(df)
#> [time]: 'Year' [value]: 'Value'
#> Category Year Value
#> 1 A 2010-01-01 NA
#> 2 A 2011-01-01 100.000000
#> 3 A 2012-01-01 50.000000
#> 4 A 2013-01-01 33.333333
#> 5 A 2014-01-01 25.000000
#> 6 A 2015-01-01 20.000000
#> 7 B 2010-01-01 NA
#> 8 B 2011-01-01 14.285714
#> 9 B 2012-01-01 12.500000
#> 10 B 2013-01-01 11.111111
#> 11 B 2014-01-01 10.000000
#> 12 B 2015-01-01 9.090909

Answer 7

The package collapse available in CRAN provides an easy and fully C/C++ based solution to these kinds of problems: with the generic function fgrowth and the associated growth operator G:
df <- data.frame(Category=c(rep("A",6),rep("B",6)),
Year=rep(2010:2015,2),Value=1:12)
library(collapse)
G(df, by = ~Category, t = ~Year)
Category Year G1.Value
1 A 2010 NA
2 A 2011 100.000000
3 A 2012 50.000000
4 A 2013 33.333333
5 A 2014 25.000000
6 A 2015 20.000000
7 B 2010 NA
8 B 2011 14.285714
9 B 2012 12.500000
10 B 2013 11.111111
11 B 2014 10.000000
12 B 2015 9.090909
# fgrowth is more of a programmers function, you can do:
fgrowth(df$Value, 1, 1, df$Category, df$Year)
[1] NA 100.000000 50.000000 33.333333 25.000000 20.000000 NA 14.285714 12.500000 11.111111 10.000000 9.090909
# Which means: Calculate the growth rate of Value, using 1 lag, and iterated 1 time (you can compute arbitrary sequences of lagged / leaded and iterated growth rates with these functions), identified by Category and Year.
fgrowth / G also has methods for the plm::pseries and plm::pdata.frame classes available in the plm package.