I find the snakemake documentation quite terse. Since I've been asked this a few times I thought posting the question and my own answer.
How do you execute or integrate this R script in a snakemake rule?
my-script.R
CUTOFF <- 25
dat <- read.table('input-table.txt')
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, 'output-table.txt')
Snakefile:
rule one:
input:
txt= 'input-table.txt',
output:
out= 'output-table.txt',
params:
cutoff= 25,
# now what?
I'm going to propose three solutions, roughly in order from most canonical.
OPTION 1: Use the script directory in combination with the snakemake R object
Replace the actual filenames inside the R script with the S4 object snakemake
which holds input, output, params etc:
my-script.R
CUTOFF <- snakemake#params[['cutoff']]
dat <- read.table(snakemake#input[['txt']])
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, snakemake#output[['out']])
Similarly, wildcards values would be accessible via snakemake#wildcards[['some-wildcard']]
The rule would look like:
rule one:
input:
txt= 'input-table.txt',
output:
out= 'output-table.txt',
params:
cutoff= 25,
script:
'/path/to/my-script.R'
Note that the script filename must end in .R or .r to
instruct snakemake that this is an R script.
Pros
This is the simplest option, just replace the actual filenames and parameters with snakemake#... in the R script
Cons
You may have many short scripts that make the pipeline difficult to follow.
What does my-script.R actually do?
--printshellcmds option is not helpful here
Debugging the R script may be cluncky because of the embedded snakemake object
OPTION 2: Write a standalone R script executable via shell directive
For example using the argparse library for R:
my-script.R
#!/usr/bin/env Rscript
library(argparse)
parser <- ArgumentParser(description= 'This progrom does stuff')
parser$add_argument('--input', '-i', help= 'I am the input file')
parser$add_argument('--output', '-o', help= 'I am the output file')
parser$add_argument('--cutoff', '-c', help= 'Some filtering cutoff', type= 'double')
xargs<- parser$parse_args()
CUTOFF <- xargs$cutoff
dat <- read.table(xargs$input)
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, xargs$output)
The snakemake rule is like any other one executing shell commands:
rule one:
input:
txt= 'input-table.txt',
output:
out= 'output-table.txt',
params:
cutoff= 25,
shell:
r"""
/path/to/my-script.R --input {input.txt} --output {output.out} --cutoff {params.cutoff}
"""
Pros
You get a standalone, nicely documented script that you can use elsewhere
--printshellcmds tells you what is being executed and you can re-run it outside snakemake
Cons
Some setting up to do via argparse
Not so easy to debug by opening the R interpreter and running the individual R commands
OPTION 3 Create a temporary R script as an heredoc that you run via Rscript:
This is all self-contained in the rule:
rule one:
input:
txt= 'input-table.txt',
output:
out= 'output-table.txt',
params:
cutoff= 25,
shell:
r"""
cat <<'EOF' > {rule}.$$.tmp.R
CUTOFF <- {params.cutoff}
dat <- read.table('{input.txt}')
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, '{output.out}')
EOF
Rscript {rule}.$$.tmp.R
rm {rule}.$$.tmp.R
"""
Explanantion: The syntax cat <<'EOF' tells Bash that everything until the EOF
marker is a string to be written to file {rule}.$$.tmp.R. Before running this
shell script, snakemake will replace the placeholders {...} as usual. So file
{rule}.$$.tmp.R will be one.$$.tmp.R where bash will replace $$ with the
current process ID. The combination of {rule} and $$ reasonably ensures
that each temporary R script has a distinct filename. NB: EOF is not a special
keyword, you can use any marker string to delimit the heredoc.
The content of {rule}.$$.tmp.R will be:
CUTOFF <- 25
dat <- read.table('input-table.txt')
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, 'output-table.txt')
after execution via Rscript, {rule}.$$.tmp.R will be deleted by rm provided
that Rscripts exited clean.
Pros
Especially for short scripts, you see what the rule actually does. No
need to look into other script files.
--printshellcmds option shows the exact R code. You can copy & paste it
to the R interpreter as it is which is very useful for debugging and developing
Cons
Clunky, quite a bit of boiler plate code
If input/output/params are lists, you need to split them with e.g.
strsplit('{params}', sep= ' ') inside the R script. This is not great
especially if you have space inside the list items.
If Rscript fails the temp R script is not deleted and it litters your working
dir.
I've created a simple template for running snakemake with R - here's the Github link:
https://github.com/fritzbayer/snakemake-with-R
It shows two simple options for passing variables between snakemake and R.
#dariober has a thorough answer. I wanted to share the variation on Option 1 that I use to make interactive development/debugging of R scripts a bit easier.
Mock snakemake Object
One can include a preamble to create a mock snakemake object conditioned on whether the script is run interactively or not. This mimics the actual object that gets instantiated, enabling one to step through the code with a realistic input, but gets skipped over when Snakemake is executing the script.
scripts/my-script.R
#!/usr/bin/env Rscript
################################################################################
## Mock `snakemake` preamble
################################################################################
if (interactive()) {
library(methods)
Snakemake <- setClass(
"Snakemake",
slots=c(
input='list',
output='list',
params='list',
threads='numeric'
)
)
snakemake <- Snakemake(
input=list(txt="input-table.txt"),
output=list(out="output-table.txt"),
params=list(cutoff=25),
threads=1
)
}
################################################################################
CUTOFF <- snakemake#params$cutoff
dat <- read.table(snakemake#input$txt)
out <- dat[dat$V1 > CUTOFF, ]
write.table(out, snakemake#output$out)
Object Details
The full set of slots on the Snakemake class can be seen in the generate_preamble code from the snakemake.script.Rscript class. Since this is subject to change, one can inspect the code in the installed version using (from Python where Snakemake is installed):
from inspect import getsource
from snakemake.script import RScript
print(getsource(RScript.generate_preamble))
for my work, I run scripts on virtual machines on a computer cluster. These jobs are typically large and have a big output. What I'd like to do is to run a script via the terminal. In the end, the script creates a duplicate of itself so that it contains every line that was part of the script (minus the last if necessary). This is quite vital for replicability and debugging in my work-life because I sometimes can't see which parameters or variables a particular job included as I submit the same script repeatedly just with slightly different parameters and the folders can't be version controlled.
Imagine this file test.R:
a <- rnorm(100)
#test
# Saving history for reproducibility to folder
savehistory(file = 'test2.R')
Now, I run this via the console on my virtual node and get the following error:
[XX home]$ Rscript test.R
Error in.External2(C_savehistory, file): no history available to save
Calls: save history
Execution halted
Is there any command like save history that works inside a script that is just executed?
The desired outcome is a file called test2.R is saved that contains this:
a <- rnorm(100)
#test
# Saving history for reproducibility to folder
You could copy the file instead. Since you're using Rscript, the script name is included in commandArgs() in the form --file=test.R. A simple function like this will return the path of the executing script:
get_filename <- function() {
c_args <- commandArgs()
r_file <- c_args[grepl("\\.R$", c_args, ignore.case = TRUE)]
r_file <- gsub("--file=", "", r_file)
r_file <- normalizePath(r_file)
return(r_file)
}
Then you can copy the file as you see fit. For example, appending ".backup":
script_name <- get_filename()
backup_name <- paste0(script_name, ".backup")
file.copy(script_name, backup_name)
I have groovy script file (.groovy). I want to execute this groovy script file in R. Please note the input for groovy file has 3 different variables and I created all 3 variables in the form of list
# This installs the package on your machine to read the groovy file
install.packages("readr")
variables <- list ()
variables["first"] = "First"
variables["second"] = "Second"
variables["third" = "Third"
library(readr)
myscript <- read_file("path/to/groovycode.groovy")
result <- Execute (groovyScript=myscript, variables=variables)
result
I didn't test the code, but can be a good start if you can fix any issues when trying to run it.
I would like to access, update and run an R script from another R script. Specifically, I would like to create a loop that reads in the lines of another R script in order to update some variable names by adding increments of 1 to each of them. Once the variables have been updated, I would like to run the R script.
Currently my code for this is as follows:
for (i in 1:n) {
x <- readLines("Mscript0.R")
y <- gsub(paste0("Mtrain",i),paste0("Mtrain",i+1), x)
cat(y, file="Mscript0.R", sep="\n")
source("Mscript0.R")
}
Please note that the string "Mtrain" in the source script takes on various different forms such as:
Mtrain4 <- read.csv("Mtrain4.csv",header=T,sep=",")
s <- Mtrain4$Weight
Any Ideas?
Thanks
I am new to R and I am trying to have a script get arguments from a file. I am using the following code within my R script:
args <- commandArgs(TRUE)
covr <- args[1]
rpts <- args[2]
The arguments will come from a parameters.tsv which will have two fields, one for each argument.
I want to run the R script with the parameters given in a line from parameters.tsv until all lines have been used to run the R script.
The end result will be qsub'ing a bash script to run each line into the R script.
This is what I came up with:
#!/bin/bash
cat parameters.tsv | while read v1 v2; do RScript --slave ‘--args $v1 $v2’ myscript.R; done
It's currently terminating almost immediately after i submit it and i don't understand why.
Any help is greatly appreciated since i am very new to this and anything i read prior did not explain in enough detail to grasp.
How about something like:
var_df <- read.csv([your_file_here]) # or read table with correct specs
for (i in 1:dim(var_df)[1]) { # vectorise for speed; doing it with loops to
# make this clearer
this_var_a <- var_df[i,1]
this_var_b <- var_df[i,2]
source([Rscript file here], local=TRUE) #set local=T as otherwise the vars
# will not be visible to the script's operations
}