Count number of value increases/decreases per group - r

Question

I have a df with entries in 10 columns grouped by unit and year. I want to calculate a) how often the values per column increased and b) how often the values per column decreased from one year to the other (e.g. from 2010 to 2011, 2011 to 2012 and so on) per group.
This is my df
df <- data.frame(unit=rep(1:250, 4),
year=rep(c(2012, 2013, 2014, 2015), each=250),
replicate(10,sample(0:50000,1000,rep=TRUE)))
So a solution should show information how often unit 1 in X1 had increases and decreases from one year to the other, how often unit 1 had increases/decreases in X2 and so forth
A tidyverse solution would be preferable ;)

Answer 1

One solution that produces a wide format. Each one of the Xs will get 2 new columns of counts: X_incr and X_decr:
# example data
df <- data.frame(unit=rep(1:250, 4),
year=rep(c(2012, 2013, 2014, 2015), each=250),
replicate(10,sample(0:50000,1000,rep=TRUE)))
library(dplyr)
# function to count increases and decreases
f_incr = function(x) sum(lead(x) > x, na.rm = T)
f_decr = function(x) sum(lead(x) < x, na.rm = T)
df %>%
group_by(unit) %>% # for each unit
summarise_at(vars(matches("X")), funs(incr = f_incr, # apply functions
decr = f_decr))
# # A tibble: 250 x 21
# unit X1_incr X2_incr X3_incr X4_incr X5_incr X6_incr X7_incr X8_incr X9_incr X10_incr X1_decr X2_decr
# <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int> <int>
# 1 1 1 0 2 1 1 1 1 1 2 2 2 3
# 2 2 1 2 1 2 0 1 1 3 2 2 2 1
# 3 3 3 1 1 1 2 1 1 2 2 2 0 2
# 4 4 1 1 2 1 1 1 1 1 2 1 2 2
# 5 5 3 2 2 1 2 2 1 2 2 2 0 1
# 6 6 1 2 1 2 2 2 1 2 2 1 2 1
# 7 7 1 2 1 1 2 0 2 3 1 1 2 1
# 8 8 2 1 1 2 2 1 1 2 1 1 1 2
# 9 9 1 2 3 1 2 2 1 1 2 2 2 1
#10 10 2 1 2 2 2 2 0 1 2 1 1 2
# # ... with 240 more rows, and 8 more variables: X3_decr <int>, X4_decr <int>, X5_decr <int>, X6_decr <int>,
# # X7_decr <int>, X8_decr <int>, X9_decr <int>, X10_decr <int>
Or if you prefer a format where each X has 2 rows of counts (X_incr and X_decr):
library(tidyr)
df %>%
group_by(unit) %>%
summarise_at(vars(matches("X")), funs(incr = f_incr,
decr = f_decr)) %>%
gather(type, counts, -unit)
# # A tibble: 5,000 x 3
# unit type counts
# <int> <chr> <int>
# 1 1 X1_incr 1
# 2 2 X1_incr 1
# 3 3 X1_incr 3
# 4 4 X1_incr 1
# 5 5 X1_incr 3
# 6 6 X1_incr 1
# 7 7 X1_incr 1
# 8 8 X1_incr 2
# 9 9 X1_incr 1
#10 10 X1_incr 2
# # ... with 4,990 more rows
Or this:
df %>%
gather(type,value,-unit,-year) %>% # reshape data
group_by(unit, type) %>% # for each combination
summarise(incr = f_incr(value), # get increasing counts
decr = f_decr(value)) %>% # get decreasing counts
arrange(type, unit) %>% # order (just for visualisation purposes)
ungroup() # forget the grouping
# # A tibble: 2,500 x 4
# unit type incr decr
# <int> <chr> <int> <int>
# 1 1 X1 1 2
# 2 2 X1 1 2
# 3 3 X1 3 0
# 4 4 X1 1 2
# 5 5 X1 3 0
# 6 6 X1 1 2
# 7 7 X1 1 2
# 8 8 X1 2 1
# 9 9 X1 1 2
#10 10 X1 2 1
# # ... with 2,490 more rows

Answer 2

I hope I understand the question (a) correctly. You are trying to see for each row how many times the value increases (first from x1 to x2, then from x2 to x3, and so on)
I am using apply to iterate over each row. Then overlay the second through last value over the first through second-to-last value and see if they if the first is bigger or smaller than the second. And add sum the boolean values to see how many times an increase or decrease occurs for that row. Note the switch from '>' to '<'
increases <- apply(df[,3:12], 1, function(x) {sum(x[2:length(x)] > x[1:(length(x)-1)])})
decreases <- apply(df[,3:12], 1, function(x) {sum(x[2:length(x)] < x[1:(length(x)-1)])})
For question (b) you can subtract the subset where year equals 2012 from the subset where year equals 2013, and test whether values are bigger than 0 for increases and smaller than 0 for decreases. Then use colSums to see for how many 'units' the increase or decrease is true.
Increase:
colSums((subset(df, year==2013) - subset(df, year==2012))>0)[3:12]
Decrease:
colSums((subset(df, year==2013) - subset(df, year==2012))<0)[3:12]