I am running a script that starts as follows:
#!/usr/bin/env Rscript
#./geneiase -t static -i mydata.tab
If I run the script on my data directly in the command line, it starts without errors or warnings.
But the program is very demanding computationally so I need to submit my jobs to a cluster using a job scheduler called Slurm.
When I write the exact same expression (as in the second paragraph) within the batch job file, and then I submit the job using sbatch, it is immediatelly terminated and does not return any error or output that can help me understand the problem.
I think it has to do with having Rscript in $PATH, but even though I added the directory where Rscript is located to $PATH by: PATH=$PATH:path/to/R/build/R-3.4.0/lib64/R/bin, the problem remains.
Is there a way that I can make Rscript be run in a Slurm batch job?
You'll need to keep the environment of your SLURM script as bash
#!/bin/bash
Since you can run your R script from the command line, it likely means the path to R is already included in your $PATH. On the command line, you might already do something like:
Rscript ./path-to-script/script
To run R from within your SLURM script, it's the same as running from the command line:
Rscript ./path-to-script/script
Related
I have multiple scripts naming R001.r, R002.r, and so on. I need to schedule them so that they run in a sequential manner one after the other. What would be the best approach to do this.
I think you want to wrap your r scripts in a caller sh file and then invoke it through terminal. Here is what I would do.
Open up a terminal or any text editor available and fill it up with the following commands:
Rscript R0001.r
Rscript R0002.r
Rscript R0003.r
...
Save this file into something like call_my_scripts. You can then execute it via standard unix shell commands as follows:
./call_my_scripts
This will run sequentially by definition. Make sure you give exec permissions to the file before you invoke it as follows:
chmod u+x call_my_scripts
I want to source an asynchronous background processes using System() and Rscript but it doesn't seem to run the script. The line I'm using is below:
system("Rscript -e 'source(\"/Users/Federico/Documents/R/win-library/3.4/taskscheduleR/extdata/PriceTesting.R\")'", wait=FALSE)
In the sourced script I have it write a simple csv and it does not write which leads me to believe its not running the script at all.
Am I doing something wrong?
Rscript already runs the script, so you can simply pass the path to the script as argument:
system("Rscript '/Users/colin/R/dslinr/plop.R'")
That being said, I have no pblm here running :
system("Rscript -e 'source(\"/Users/colin/R/dslinr/plop.R\")'", wait=FALSE)
Does the script works if you launch it from the shell? Are you sure your path is correct ? It seems to be mixing unix path (with /User), and win-library (windows library for R).
I changed it to this and it runs. All I can think of is that maybe Rscript isn't a recognized command.
system("C:/PROGRA~1/R/R-34~1.0/bin/Rscript.exe
C:/Users/Federico/Documents/R/win-
library/3.4/taskscheduleR/extdata/PriceTesting.R", wait=FALSE)
I'm trying to follow the conventional method to trigger R scripts through batch like
RScript Example.R
but what i look to is some way to run multiple R scripts through a batch file.
I tried to do use Start command to open multiple sessions but that doesn't work either. (RScript START ex1.R START ex2.R)
PS complete noob to batch files.
On Windows - if you want to run them in parallel make sure you add start in yoyr batch (.bat) script. Otherwise Example2.R waits for Example1.R to complete etc.
start RScript Example1.R
start RScript Example2.R
...
If you're using sh to launch your scripts, this could do it.
cd /path_to_script1/
sh script1.sh &
cd /path_to_script2/
sh script2.sh &
cd /path_to_script3/
sh scipt2.sh &
This launches parallel R sessions (one for each script) so careful with memory and CPU use. Each script file contains Rscript command.
Simply save the RScript commands in a Windows batch file (.bat) then double-click the file in directory or call it via command line. Below assumes RScript is an environment variable.
Batch file (type below in Notepad and save with extension .bat, not default .txt)
cd "C:\Path\To\Scripts"
RScript Example1.R
RScript Example2.R
RScript Example3.R
RScript Example4.R
RScript Example5.R
CMD Command line
call myRScriptBatchFile.bat
PowerShell Command line
cmd.exe /c myRScriptBatchFile.bat
My Mac OS command line application is making Unix calls such as:
system("rm -rf /Users/stu/Developer/file);
perfectly successfully.
So why is the following not changing the current directory?
system("cd /Users/me/whatever");
system("pwd"); //cd has not changed
Because
system() executes a command specified in command by calling /bin/sh -c command, and returns after the command has been completed.
So each command is executed independently, each in a new instance of the shell.
So your first call spawns a new sh (with your current working directory), changes directories, and then exits. Then the second call spawns a new sh (again in your CWD).
See the man page for system().
The better solution is to not use system. It has some inherent flaws that can leave you open to security vulnerabilities. Instead of executing system() commands, you should use the equivalent POSIX C functions. Everything that you can do from the command-line, you can do with C functions (how do you think those utilities work?)
Instead of system("rm -rf ...") use this.
Instead of system("cd ...") use chdir().
Instead of system("pwd ...") use getcwd().
There are some differences, of course, but these are the fundamental equivalents of what you're trying to do.
I am trying to schedule my R script using cron, but it is not working. It seems R can not find packages in cron. Anyone can help me? Thanks.
The following is my bash script
# source my profile
. /home/winie/.profile
# script.R will load packages
R CMD BATCH /home/script.R
Consider these tips
Use Rscript (or littler) rather than R CMD BATCH
Make sure the cron job is running as you
Make sure the script runs by itself
Test it a few times in verbose mode
My box is running the somewhat visible CRANberries via a cronjob calling an R script
(which I execute via littler but Rscript
should work just as well). For this, the entry in /etc/crontab on my Ubuntu server is
# every few hours, run cranberries
16 */3 * * * edd cd /home/edd/cranberries && ./cranberries.r
so every sixteen minutes past every third hour, a shell command is being run with my id. It changes into the working directory, and call the R script (which has executable modes etc).
Looking at this, I could actually just run the script and have setwd() command in it....