I'd like to run all the r scripts located in a directory called scripts using a bash script. How would you do it? My script so far (not working) looks as follow:
#!/usr/bin/bash
#SBATCH --job-name=name
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
module load R/4.0.2-gnu9.1
module load sqlite/3.26.0
Rscript $R_SCRIPT
Related
Here is my slurm job script. I requested 4 gpu and 1 computing node. My script is as follows:
#!/bin/bash
#SBATCH --partition=gpu
#SBATCH --gres=gpu:4
#SBATCH --ntasks-per-gpu=12
#SBATCH --mem-per-gpu=40G
#SBATCH --time=0:15:00
module use /ifs/opt_cuda/modulefiles
module load python/gcc/3.10
module load cuda11.1/toolkit cuda11.1/blas cuda11.1/fft cudnn8.0-cuda11.1 tensorrt-cuda11.1/7.2.3.4
# activate TF venv
source /ifs/groups/rweberGrp/venvs/py310-tf210/bin/activate
python -c "import torch;print(torch.cuda.device_count())"
so the torch.cuda.device_count() should give me 4 but actually the output is 0
0
I have no idea why this is happening. Anyone has any idea? Thanks
I am executing an R script on a cluster, using the following sbatch file:
#!/bin/bash
#SBATCH --job-name=ReconstructScans
#SBATCH --mem=4g
#SBATCH --account=XXXXXX
#SBATCH --array=1-1013%5
module load StdEnv/2020
module load gcc/9.3.0
module load r/4.0.2
srun Rscript ReconstructScans.R
ReconstructScans.R calls another R script, using
source("path_to_script.R", local=FALSE)
However, for some reason, the variables declared within path_to_script.R are not accessible to ReconstructScans.R. I get the object 'DataPath' not found error.
Why might this be happening? When I open an interactive R shell and source ReconstructScans.R, everything works fine. It's when ReconstructScans.R is executed through sbatch that the problem arises.
Is it a problem with how I'm executing it using:
srun Rscript ReconstructScans.R?
Thank you,
Mrinmayi
I am trying to get a very basic job array script working using Slurm job scheduler on a HPC. I am getting the error:
slurmstepd: error: execve(): Rscript: No such file or directory
This is similar to this but I am not using any export commands so this isn't the cause here. Some sources say it could be something to do with creating these scripts in Windows so the line ends will not work for Unix. Could this be the issue? If so how can I check for this?
My shell script:
#!/bin/bash
# Example of running R script with a job array
#SBATCH --nodes=1
#SBATCH --array=1-10 # how many tasks in the array
#SBATCH --ntasks-per-node=10
#SBATCH -o hello-%j-%a.txt
#SBATCH --mail-user=user#email.address
# Load software
module load R/4.0.0-foss-2020a
# Run R script with a command line argument
srun Rscript hello.R $SLURM_ARRAY_TASK_ID
The script -hello.R is:
#!/usr/bin/env
# Rscript
# accept command line arguments and save them in a list called args
args = commandArgs(trailingOnly=TRUE)
# print task number
print(paste0('Hello! I am a task number: ', args[1]))
Thanks to some offline help I have got an answer to my question. Seems I didn't need to use srun in the shell script and needed to include Rscript in the shebang line in hello.R
Shell script is:
#!/bin/bash
# Example of running R script with a job array
#SBATCH --nodes=1
#SBATCH --array=1-10 # how many tasks in the array
#SBATCH --ntasks-per-node=10
#SBATCH -o hello-%j-%a.txt
#SBATCH --mail-user=tclewis1#sheffield.ac.uk
# Load software
module load R/4.0.0-foss-2020a
# Run R script with a command line argument
Rscript hello.R $SLURM_ARRAY_TASK_ID
And hello.R is now:
#!/usr/bin/env/Rscript
# accept command line arguments and save them in a list called args
args = commandArgs(trailingOnly=TRUE)
# print task number
print(paste0('Hello! I am a task number: ', args[1]))
I try to submit a R script to SLURM in CentOS 7, like this:
#!/bin/bash
#SBATCH -J test
#SBATCH -o test.out
#SBATCH -p compute
#SBATCH --qos=normal
#SBATCH -N 1
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=1
#SBATCH --job-name=rtest
Rscript --vanilla Rhelp.R
Then system will return a jobid, but the R script does not work. I can assure this script can run in command line. Then I have found in test.out, like this:
error while loading shared libraries: libicuuc.so.50:
cannot open shared object file: No such file or directory
I am a freshman in SLURM and Linux, thx!
It looks like the libicu RPM package is not installed on the compute nodes.
Just because it may be installed on the head node doesn't mean it's installed on the compute node(s). You could fire off the same ldconfig command in a Slurm job and view the results to confirm that's the case.
With the module avail command from the head node you list all available modules and loaded modules are marked somehow depending on your OS; for me they are marked with (L). All you need to do is load those loaded modules from your file script each of which is invoked with the line
module load path_to_module. Whereas, path_to_module is as is indicated with the previous command module avail.
Or without resorting to module avail you could use module list for currently loaded modules.
I would like to run job using R.
I wrote r script as follow
#!/bin/sh
#SBATCH --time=168:00:00
#SBATCH --mem=50gb
#SBATCH --ntasks=1
#SBATCH --job-name=StrAuto
#SBATCH --error=R.%J.err
#SBATCH --output=R.%J.out
#SBATCH --mail-type=ALL
#SBATCH --mail-user=asallam#unl.edu
module load R/3.3
R CMD BATCH AF1.R
Then I worte the command lines in AF1.R file as follow
outer(names(m[,-1]), names(m[,-1]), function(x,y){colSums((m[,x]-m[,y])**2/156854,na.rm=TRUE)})
Now I would like to ask how I load my data m.txt. what is the command that I should write before outer(.......
Thanks in advance