Rmd re-run all blocks that read_csv - r

Question

I have an Rmd document with several code blocks that call read_csv on many different csv's. The Rmd makes various graphs and tables and I use cache=TRUE to speed up the rendering.
Another program produces the CSVs and generates different results depending on different experiments and configurations. So I want the Rmd to reload those CSVs which have changed and use the cache for those that have not.
At the moment I have a block parameter {r lastrun=100} on each code block that has a read_csv, and i search for lastrun and replace e.g. to 101 for those blocks that have read_csv i think should be reread, but I'd like this to be automated somehow.
so right now my Rmd looks like:
```{r lastrun=100}
a<-map_dfr(paste0(1:10, ".a.csv"), read_csv)
```
lots of text and including code blocks
```{r}
#whatever
print(f(a))
```
```{r lastrun=100}
b<-map_dfr(paste0(1:20, ".b.csv"), read_csv)
```
So on rendering that, if later any of *.a.csv or *.b.csv change, then i have to search/replace lastrun or i'll just see the stale cached versions of a and b. I want a and b to update when the files change (i don't need to be able to identify the exact file and only reread that one, just reloading the block would be fine.)
How can this be done?
Thanks
-Neal

Answer 1

Here's a trick that might work:
---
title: 'nothing to report'
output: html_document
---
```{r setup}
files <- c("file1", "file2")
fileinfo <- file.info(files)
```
```{r test1, file=fileinfo['file1',-6], cache = TRUE}
Sys.sleep(5)
```
```{r test2, file=fileinfo['file2',-6], cache = TRUE}
Sys.sleep(5)
```
To test:
Render this, it should take 10 seconds.
Render it again, it should be instantaneous.
Create one or both of the files file1 and file2.
Render it again, it should take 5 seconds (if one created) or 10 seconds (both).
Touch one of the files (either updating its mtime or add/replace contents, doesn't matter).
Render it again, there should be a pause.
Notes:
files do not need to exist at first; NA row contents (in fileinfo) is just as changing when files "appear"
files can be in the current directory, a subdirectory, or any absolute path, the name of the row is the name set within my files variable
with [,-6], I'm omitting atime, which indicates when something "looks at the file"; this updates with each file.info, so obviously this would defeat the purpose of the cache (invalidating on each test)
other than that, it is currently invalidating on a change in: size, isdir, mode, mtime, ctime, and exe, all fields within file.info; you can easily choose specific fields (instead of omitting one, as I did above);
one could have a chunk depend on multiple files, you either need to be creative in the file= used in order to grab multiple rows, or your setup block should know how to aggregate summaries
if this is on a network drive, sometimes things like mtime and such are less reliable; in that case, one might consider using md5sum somehow
(You will likely want the code in the {r setup} block to be hidden.)