This question already has an answer here:
R - Run source() in background
(1 answer)
Closed 4 years ago.
Is there a way to source a different R script and continuing to execute the remainder of the current script without stoping to wait for the sourced scrip to finish?
eg.
Script 1 - run 00:00
source(Script2) - run 00:01
script 1 - end 00:05
script 2 - end 01:00
I hope this makes sense
I believe you can accomplish this with doParallel like so:
require(doParallel)
scripts <- c('script1.r', 'script2.r')
foreach(x = 1:length(scripts)) %dopar%
{
script <- scripts[x]
source(script)
}
Which would run each script simultaneously on two different workers.
Related
This question already has answers here:
How to include interactive input in script to be run from the command line
(3 answers)
Closed 5 years ago.
I cannot find a solution to this particular problem, even though more or less similar questions have been questioned before in:
Run R script from command line
http://www.cureffi.org/2014/01/15/running-r-batch-mode-linux/
Running a script from bash is easy enough, however once one needs user interaction I couldn't find a solution. Please consider the example:
userInput<-function(question) {
n = 0
while(n < 1 ){
n <- readline(question)
n <- ifelse(grepl("\\D",n),-1,as.integer(n))
if(is.na(n)){break} # breaks when hit enter
}
return(n)
}
investedLow<- userInput("Invested value in low risk since last time: ")
Now if I save this script as test.R and run it for R --no-save < teste.R the entire script is run and the time for user input does not happen.
The script works fine in Rstudio, for example.
How to wait for user input in a script to be run in the command line?
Here is a total hack, repurposing a very specific purpose-built package for your more-general question:
library(getPass)
userInput<-function(question) {
n = 0
while(n < 1 ){
n <- getPass::getPass(msg = question)
n <- ifelse(grepl("\\D",n),-1,as.integer(n))
if(is.na(n)){break} # breaks when hit enter
}
return(n)
}
investedLow <- userInput("Invested value in low risk since last time: ")
print(investedLow)
Maybe the worst part about this is that getPass hides the user input. There must be a way to modify the source code to fix that.
Update: The getPass author pointed out that the solution could be as simple as using readLines slightly differently:
cat(question)
readLines(file("stdin"), n=1)
Background/Motivation:
I am running a bioinformatics pipeline that, if executed from beginning to end linearly takes several days to finish. Fortunately, some of the tasks don't depend upon each other so they can be performed individually. For example, Task 2, 3, and 4 all depend upon the output from Task 1, but do not need information from each other. Task 5 uses the output of 2, 3, and 4 as input.
I'm trying to write a script that will open new instances of R for each of the three tasks and run them simultaneously. Once all three are complete I can continue with the remaining pipeline.
What I've done in the past, for more linear workflows, is have one "master" script that sources (source()) each task's subscript in turn.
I've scoured SO and google and haven't been able to find a solution for this particular problem. Hopefully you guys can help.
From within R, you can run system() to invoke commands within a terminal and open to open a file. For example, the following will open a new terminal instance:
system("open -a Terminal .",wait=FALSE)
Similarly, I can start a new r session by using
system("open -a r .")
What I can't figure out for the life of me is how to set the "input" argument so that it sources one of my scripts. For example, I would expect the following to open a new terminal instance, call r within the new instance, and then source the script.
system("open -a Terminal .",wait=FALSE,input=paste0("r; source(\"/path/to/script/M_01-A.R\",verbose=TRUE,max.deparse.length=Inf)"))
Answering my own question in the event someone else is interested down the road.
After a couple of days of working on this, I think the best way to carry out this workflow is to not limit myself to working just in R. Writing a bash script offers more flexibility and is probably a more direct solution. The following example was suggested to me on another website.
#!/bin/bash
# Run task 1
Rscript Task1.R
# now run the three jobs that use Task1's output
# we can fork these using '&' to run in the background in parallel
Rscript Task2.R &
Rscript Task3.R &
Rscript Task4.R &
# wait until background processes have finished
wait %1 %2 %3
Rscript Task5.R
You might be interested in the future package (I'm the author). It allows you to write your code as:
library("future")
v1 %<-% task1(args_1)
v2 %<-% task2(v1, args_2)
v3 %<-% task3(v1, args_3)
v4 %<-% task4(v1, args_4)
v5 %<-% task5(v2, v3, v4, args_5)
Each of those v %<-% expr statements creates a future based on the R expression expr (and all of it's dependencies) and assigns it to a promise v. It is only when v is used, it will block and wait for the value v to be available.
How and where these futures are resolved is decided by the user of the above code. For instance, by specifying:
library("future")
plan(multiprocess)
at the top, then the futures (= the different tasks) are resolved in parallel on your local machine. If you use,
plan(cluster, workers = c("n1", "n3", "n3", "n5"))
they're resolved on those for machine (where n3 accepts two concurrent jobs).
This works on all operating systems (including Windows).
If you have access to a HPC compute with schedulers such as Slurm, SGE, and TORQUE / PBS, you can use the future.BatchJobs package, e.g.
plan(future.BatchJobs::batchjobs_torque)
PS. One reason for creating future was to do large-scale Bioinformatics in parallel / distributed.
This question already has an answer here:
Is there anyway to add shutdown hook to R process
shutdown PC after finishing a script
(1 answer)
Closed 6 years ago.
I'd like to do some cleanup work before the R process is exited by accidently. Is there any api like java's shutdown hook in R ? Thanks
Defining a .Last function will do the trick.
.Last <- function() {
printf("\n == END of R | %s (runtime %s) ==\n", Sys.time(),format(round(difftime(Sys.time(), .r.start.time), 2)))
}
similarly .First function is called at the start of the script.
I want to run a program in batch and I want it to wait for the user input in the program.
This is what I have:
library('XLConnect')
FolderPath<-"C:/Documents/Testing/"
load(paste('C:/Documents/Testing/Program.ml',sep=''));Run()
This code is named Test.R, and it calls a compiled file that runs a function that does what I need. I need to have some inputs in this exercise, so I wanted to ask for the user to input some dates.
The program should do something like this:
What are the years for this simulation?
Then the user enter:
>2001, 2002, 2003, 2004
The program will save this vector in variable, lets call y.
Then, when I load the compiled function, I will use y.
The problem is that I run this R code in cmd batch.
Yukio, try the function readline (documentation here) as in the following example, in which a user inputs two years separated by a comma.
> years = readline('What are the years for this simulation? ')
What are the years for this simulation? 2001, 2002
> years = as.numeric(unlist(strsplit(years, ",")))
> years
[1] 2001 2002
By the way, your English is great!
As far as I understand (someone please correct me if this is wrong), but you can't send messages to the user at the command line when executing a script with R CMD BATCH (it writes all input and output to an .Rout file). But executing a script with Rscript.exe does in fact let you send stuff to stdout. However, in order to get input from a user when executing with Rscript.exe, you need to use file('stdin') as the connection to your input function, which, funny enough, won't work if you then try to run the script from the R interpreter with the source function. Here's an example that showing you how to prompt the user for a series of comma separated years, regardless of whether the script runs with Rscript.exe or the source function in interactive mode.
con <- if (interactive()) stdin() else file('stdin')
message('what are the years')
years <- scan(file=con, sep=',', nlines=1, quiet=TRUE)
print(years)
You can deal easily with user inputs in a batch file not in the R.
You create a batch file , say launcher.bat , like this for example
ECHO I am Rscript launcher
set /p years= What are the years for this simulation?
cd R_SCRIPT_PATH
Rscript youscript.R %years%
The user can write as many years he want, it will go in the variable years. Then in your script you parse this variable to set it as a valid list of years.
Say my executable is c:\my irectory\myfile.exe and my R script calls on this executeable with system(myfile.exe)
The R script gives parameters to the executable programme which uses them to do numerical calculations. From the ouput of the executable, the R script then tests whether the parameters are good ore not. If they are not good, the parameters are changed and the executable rerun with updated parameters.
Now, as this executable carries out mathematical calculations and solutions may converge only slowly I wish to be able to kill the executable once it has takes to long to carry out the calculations (say 5 seconds)
How do I do this time dependant kill?
PS:
My question is a little related to this one: (time non dependant kill)
how to run an executable file and then later kill or terminate the same process with R in Windows
You can add code to your R function which issued the executable call:
setTimeLimit(elapse=5, trans=T)
This will kill the calling function, returning control to the parent environment (which could well be a function as well). Then use the examples in the question you linked to for further work.
Alternatively, set up a loop which examines Sys.time and if the expected update to the parameter set has not taken place after 5 seconds, break the loop and issue the system kill command to terminate myfile.exe .
There might possibly be nicer ways but it is a solution.
The assumption here is, that myfile.exe successfully does its calculation within 5 seconds
try.wtl <- function(timeout = 5)
{
y <- evalWithTimeout(system(myfile.exe), timeout = timeout, onTimeout= "warning")
if(inherits(y, "try-error")) NA else y
}
case 1 (myfile.exe is closed after successfull calculation)
g <- try.wtl(5)
case 2 (myfile.exe is not closed after successfull calculation)
g <- try.wtl(0.1)
MSDOS taskkill required for case 2 to recommence from the beginnging
if (class(g) == "NULL") {system('taskkill /im "myfile.exe" /f',show.output.on.console = FALSE)}
PS: inspiration came from Time out an R command via something like try()