Creating a new table that shows the percent change between two different categories from a single column in R - r

Question

I'm trying to learn how to use some of the functions in the R "reshape2" package, specifically dcast. I'm trying to create a table that shows the aggregate sum (the sum of one category of data for all files divided by the max "RepNum" in one "Case") for two software versions and the percent change between the two.
Here's what my data set looks like (example data):
| FileName | Version | Category | Value | TestNum | RepNum | Case |
|:--------:|:-------:|:---------:|:-----:|:-------:|:------:|:-----:|
| File1 | 1.0.18 | Category1 | 32.5 | 11 | 1 | Case1 |
| File1 | 1.0.18 | Category1 | 31.5 | 11 | 2 | Case1 |
| File1 | 1.0.18 | Category2 | 32.3 | 11 | 1 | Case1 |
| File1 | 1.0.18 | Category2 | 31.4 | 11 | 2 | Case1 |
| File2 | 1.0.18 | Category1 | 34.6 | 11 | 1 | Case1 |
| File2 | 1.0.18 | Category1 | 34.7 | 11 | 2 | Case1 |
| File2 | 1.0.18 | Category2 | 34.5 | 11 | 1 | Case1 |
| File2 | 1.0.18 | Category2 | 34.6 | 11 | 2 | Case1 |
| File1 | 1.0.21 | Category1 | 31.7 | 12 | 1 | Case1 |
| File1 | 1.0.21 | Category1 | 32.0 | 12 | 2 | Case1 |
| File1 | 1.0.21 | Category2 | 31.5 | 12 | 1 | Case1 |
| File1 | 1.0.21 | Category2 | 32.4 | 12 | 2 | Case1 |
| File2 | 1.0.21 | Category1 | 31.5 | 12 | 1 | Case1 |
| File2 | 1.0.21 | Category1 | 34.6 | 12 | 2 | Case1 |
| File2 | 1.0.21 | Category2 | 31.7 | 12 | 1 | Case1 |
| File2 | 1.0.21 | Category2 | 32.4 | 12 | 2 | Case1 |
| File1 | 1.0.18 | Category1 | 32.0 | 11 | 1 | Case2 |
| File1 | 1.0.18 | Category1 | 34.6 | 11 | 2 | Case2 |
| File1 | 1.0.18 | Category2 | 34.6 | 11 | 1 | Case2 |
| File1 | 1.0.18 | Category2 | 34.7 | 11 | 2 | Case2 |
| File2 | 1.0.18 | Category1 | 32.3 | 11 | 1 | Case2 |
| File2 | 1.0.18 | Category1 | 34.7 | 11 | 2 | Case2 |
| File2 | 1.0.18 | Category2 | 31.4 | 11 | 1 | Case2 |
| File2 | 1.0.18 | Category2 | 32.3 | 11 | 2 | Case2 |
| File1 | 1.0.21 | Category1 | 32.4 | 12 | 1 | Case2 |
| File1 | 1.0.21 | Category1 | 34.7 | 12 | 2 | Case2 |
| File1 | 1.0.21 | Category2 | 31.5 | 12 | 1 | Case2 |
| File1 | 1.0.21 | Category2 | 34.6 | 12 | 2 | Case2 |
| File2 | 1.0.21 | Category1 | 31.7 | 12 | 1 | Case2 |
| File2 | 1.0.21 | Category1 | 31.4 | 12 | 2 | Case2 |
| File2 | 1.0.21 | Category2 | 34.5 | 12 | 1 | Case2 |
| File2 | 1.0.21 | Category2 | 31.5 | 12 | 2 | Case2 |
The actual data set has 6 unique files, the two most previous "TestNums & Versions", 2 unique categories, and 4 unique cases.
Using the magic of the internet, I was able to cobble together a table that looks like this for a different need (but the code should be similarish):
| FileName | Category | 1.0.1 | 1.0.2 | PercentChange |
|:--------:|:---------:|:-----:|:-----:|:-------------:|
| File1 | Category1 | 18.19 | 18.18 | -0.0045808520 |
| File1 | Category2 | 18.05 | 18.06 | -0.0005075721 |
| File2 | Category1 | 19.27 | 18.83 | -0.0224913494 |
| File2 | Category2 | 19.13 | 18.69 | -0.0231780146 |
| File3 | Category1 | 26.02 | 26.91 | 0.0342729019 |
| File3 | Category2 | 25.88 | 26.75 | 0.0335598775 |
| File4 | Category1 | 31.28 | 28.70 | -0.0823371327 |
| File4 | Category2 | 31.13 | 28.56 | -0.0826670833 |
| File5 | Category1 | 31.77 | 25.45 | -01999731215 |
| File5 | Category2 | 31.62 | 25.30 | -0.0117180458 |
| File6 | Category1 | 46.23 | 45.68 | -0.0119578545 |
| File6 | Category2 | 46.08 | 45.53 | -0.0045808520 |
This is the code that made that table:
vLatest and vPrevious are variables with the latest and second latest verion numbers
deviations<-subset(df, df$Version %in% c(vLatest, vPrevious))
deviationsCast<-dcast(df[,1:4], FileName + Category ~ Version, value.var = "Value", fun.aggregate=mean)
deviationsCast$PercentChange<-(deviationsCast[,dim(deviationsCast)[2]]-deviationsCast[,dim(deviationsCast)[2]-1])/deviationsCast[,dim(deviationsCast)[2]-1]
I'm really just hoping someone can help me understand the syntax of dcast. The initial generation of deviationsCast is where I'm most fuzzy on how everything is working together. Instead of getting this for the Files, I really want to get it so that its the sum of all files for each category for a unique "Case" and show the Percent change between them.
| Case | Measure | 1.0.18 | 1.0.21 | PercentChange |
|:------:|:----------:|:------:|:------:|:-------------:|
| Case 1 | Category 1 | 110 | 100 | 9.09% |
| Case 2 | Category 1 | 95 | 89 | 9.32% |
| Case 3 | Category 1 | 92 | 84 | 8.70% |
| Case 4 | Category 1 | 83 | 75 | 9.64% |
| Case 1 | Category 2 | 112 | 101 | 9.82% |
| Case 2 | Category 2 | 96 | 89 | 7.29% |
| Case 3 | Category 2 | 94 | 86 | 8.51% |
| Case 4 | Category 2 | 83 | 76 | 8.43% |
Note: The rounding and percent sign is a plus but a very preferred plus
The numbers do not reflect actual maths done correctly, just random numbers I put in there to show for an example. I hopefully explained the math that I'm trying to do sufficiently.
Example dataset to test with
FileName<-rep(c("File1","File2","File3","File4","File5","File6"),times=8,each=6)
Version<-rep(c("1.0.18","1.0.21"),times=4,each=36)
Category<-rep(c("Category1","Category2"),times=48,each=3)
Value<-rpois(n=288,lambda=32)
TestNum<-rep(11:12,times=4,each=36)
RepNum<-rep(1:3,times=96)
Case<-rep(c("Case1","Case2","Case3","Case4"),each=72)
df<-data.frame(FileName,Version,Category,Value,TestNum,RepNum,Case)
Its worth noting that the df here is essentially what deviations data frame is from the above code (with vLatest and vPrevious)
EDIT:
MrFlick's answer is almost perfect but when I try to implement it in my actual dataset I run into problems. The issue is due to using vLatest and vPrevious as my Versions instead of just writing the string. Here's the code that I use to get those two variables
vLatest<-unique(df[df[,"TestNum"] == max(df$TestNum), "Version"])
vPrevious<-unique(df[df[,"TestNum"] == sort(unique(df$TestNum), T)[2], "Version"])
And when I tried this:
pc <- function(a,b) (b-a)/a
summary <- df %>%
group_by(Case, Category, Version) %>%
summarize(Value=mean(Value)) %>%
spread(Version, Value) %>%
mutate(Change=scales::percent(pc(vPrevious,vLatest)))
I received this error: Error: non-numeric argument to binary operator
2nd EDIT:
I tried creating new variables that were for the two TestNum values (since they could be numeric values and wouldn't need to have factors).
maxTestNum<-max(df$TestNum)
prevTestNum<-sort(unique(df$TestNum), T)[2]
(The reason I don't use "prevTestNum<-maxTestNum-1" is because sometimes versions are omitted from the data results)
However when I put in those two variables into the code, the "Change" column is all the same value.

Answer 1

With the sample data set supplied by the OP, and from analysing the edits, I believe the following code might produce the desired result even with OP's production data set.
My understanding is that the OP has a data.frame with many test results but he wants only to show the relative change of the two most recent versions.
The OP has asked for help in using the dcast() function. This function is available from two packages, reshape2 and data.table. Here the data.table version is used for speed and concise code. In addition, functions from the forcats and formattable packages are used.
library(data.table) # CRAN version 1.10.4 used
# coerce to data.table object
DT <- data.table(df)
# reorder factor levels of Version according to TestNum
DT[, Version := forcats::fct_reorder(Version, TestNum)]
# determine the two most recent Versions
# trick: pick 1st and 2nd entry of the _reversed_ levels
vLatest <- DT[, rev(levels(Version))[1L]]
vPrevious <- DT[, rev(levels(Version))[2L]]
# filter DT, reshape from long to wide format,
# compute change for the selected columns using get(),
# use formattable package for pretty printing
summary <- dcast(
DT[Version %in% c(vLatest, vPrevious)],
Case + Category ~ Version, mean, value.var = "Value")[
, PercentChange := formattable::percent(get(vLatest) / get(vPrevious) - 1.0)]
summary
Case Category 1.0.18 1.0.21 PercentChange
1: Case1 Category1 33.00000 31.94444 -3.20%
2: Case1 Category2 31.83333 31.83333 0.00%
3: Case2 Category1 33.05556 33.61111 1.68%
4: Case2 Category2 30.77778 32.94444 7.04%
5: Case3 Category1 33.16667 31.94444 -3.69%
6: Case3 Category2 33.44444 33.72222 0.83%
7: Case4 Category1 30.83333 34.66667 12.43%
8: Case4 Category2 32.27778 33.44444 3.61%
Explanations
Sorting Version
The OP has recognized that simply sorting Version alphabetically doesn't ensure the proper order. This can be demontrated by
sort(paste0("0.0.", 0:12))
[1] "0.0.0" "0.0.1" "0.0.10" "0.0.11" "0.0.12" "0.0.2" "0.0.3" "0.0.4" "0.0.5"
[10] "0.0.6" "0.0.7" "0.0.8" "0.0.9"
where 0.0.10 comes before 0.0.2.
This is crucial as data.frame() turns character variables to factor by default.
Fortunately, TestNum is associated with Version. So, TestNum is used to reorder the factor levels of Version with help of the fct_reorder() function from the forcats package.
This also ensures that dcast() creates the new columns in the appropriate order.
Accessing columns through variables
Using vLatest / vPrevious in an expression returns the error message
Error in vLatest/vPrevious : non-numeric argument to binary operator
This is to be expected because vLatests and vPrevious contain character values "1.0.21" and "1.0.18", resp., which can't be divided. What is meant here is take the values of the columns which names are given by vLatests and vPrevious and divide. This is achieved by using get().
Formatting as percent
While scales::percent() returns a character vector, formattable::percent() does return a numeric vector with a percent representation, i.e., we're still able to do numeric calculations.
Data
As given by the OP:
FileName <- rep(c("File1", "File2", "File3", "File4", "File5", "File6"),
times = 8, each = 6)
Version <- rep(c("1.0.18", "1.0.21"), times = 4, each = 36)
Category <- rep(c("Category1", "Category2"), times = 48, each = 3)
Value <- rpois(n = 288, lambda = 32)
TestNum <- rep(11:12, times = 4, each = 36)
RepNum <- rep(1:3, times = 96)
Case <- rep(c("Case1", "Case2", "Case3", "Case4"), each = 72)
df <- data.frame(FileName, Version, Category, Value, TestNum, RepNum, Case)