Best practices for drake pipeline experimentation - r

Question

I'm new to drake but loving it so far. One thing I'm having trouble with is how to best go about experimenting with different pipeline configurations. That is, my plans consist purely of a chain of targets where the output from the first target is the input for the second, the second forms the input for the third, etc. My targets all have the same basic structure (dynamic targets with tibbles as individual entries) expected as input and supplied as output, and I want to experiment with different orderings, inclusion/exclusion of certain steps, etc. For example:
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
, b_transformed = target(
compute_b_transform(a_transformed)
, dynamic = map(a_transformed)
)
, c_transformed = target(
compute_c_transform(c_transformed)
, dynamic = map(c_transformed)
)
)
The way I've been using drake so far is that each target has a unique/meaningful name, so when I, for example, remove a target, I have to rename the input supplied to the subsequent target:
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
#, b_transformed = target(
# compute_b_transform(a_transformed)
# , dynamic = map(a_transformed)
#)
#note the b-transform step has been removed (commented-out), requiring inputs to c_transform to be changed from `b_transform` to `a_transform`
, c_transformed = target(
compute_c_transform(a_transformed) #had to rename things here
, dynamic = map(a_transformed) #and here
)
)
Would it be too much to hope that there's a better way of experimenting that doesn't require this manual commenting-out and renaming?

Answer 1

I worked out a method that is a bit of a hack but works for me. I simply add a skip argument to each function that triggers return of the input if TRUE:
compute_a_transform = function(x,skip=F){
if(skip){
return(x)
}
... #regular compute_a_transform stuff here
}
Then, when I want to skip a step in the processing chain, I simply set skip=TRUE without commenting-out or renaming anything
plan = drake::drake_plan(
a_transformed = target(
compute_a_transform(list_of_input_data)
, dynamic = map(list_of_input_data)
)
, b_transformed = target(
compute_b_transform(a_transformed, skip=TRUE) #skip=TRUE means the b-transform isn't actually applied
, dynamic = map(a_transformed)
)
, c_transformed = target(
compute_c_transform(c_transformed)
, dynamic = map(c_transformed)
)
)