Grep R error message in bash to halt a pipeline - r

I have a pipeline I am working on. I have a wrappper.sh that pipes together various .R scripts. However, This pipeline will run through an error message. I want to add a way to grep our the word Error if True, shut down pipeline. I know I need an if/else statment, but dont know how to grep this info out of .R script running in bash.sh. See an example error.
Current script:
#!/bin/bash
#Bash script for running GeoMx Pipeline
####
# Install required R packages for pipeline
echo "installing R packages"
Rscript installPackages.R
echo "DONE! R packages installed"
#####
# Created required folders
echo "Creating Folders"
Rscript CreateFolder.R
echo "DONE! Folders created"
####
# Copy data over
cp -u -p Path/Initial\ Dataset.xlsx /PATO_TO
####
# Run Statistical Models
echo "Running Statistical Analysis"
Rscript GLM_EdgeR.R
echo "DONE! Statistical Models completed"
Example error:
Error in glmLRT(glmfit, coef = coef, contrast = contrast) :
contrast vector of wrong length, should be equal to number of coefficients in the linear model.
Calls: glmQLFTest -> glmLRT
Execution halted
What I want:
#!/bin/bash
#Bash script for running GeoMx Pipeline
####
# Install required R packages for pipeline
echo "installing R packages"
Rscript installPackages.R
if grep error == TRUE
then
echo "Fatal Error, STOP Pipeline"
STOP
else
echo "DONE! R packages installed"
#####
# Created required folders
echo "Creating Folders"
Rscript CreateFolder.R
if grep error == TRUE
then
echo "Fatal Error, STOP Pipeline"
STOP
else
echo "DONE! Folders created"
####
# Copy data over
cp -u -p Path/Initial\ Dataset.xlsx /PATO_TO
####
# Run Statistical Models
echo "Running Statistical Analysis"
Rscript GLM_EdgeR.R
if grep error == TRUE
then
echo "Fatal Error, STOP Pipeline"
STOP
else
echo "DONE! Statistical Models completed"

You don't need to grep for errors, you can test if last status-code was non-zero:
#!/bin/bash
Rscript CreateFolder.R
exit_code=$?
if test $exit_code -ne 0
then
echo "Fatal Error, STOP Pipeline"
exit $exit_code
else
echo "DONE! Folders created"
fi
If Rscript CreateFolder.R fails, bash script will exit with the same status-code.
Though if you have more of those conditions you want to check against, it makes sense to use set -e instead.
Exit immediately if a pipeline, which may consist of a
single simple command, a list, or a
compound command returns a non-zero status.
https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html
Basically it makes your script run until something fails:
#!/bin/bash
set -e
Rscript installPackages.R
echo "DONE! R packages installed"
Rscript CreateFolder.R
echo "DONE! Folders created"
Rscript GLM_EdgeR.R
echo "DONE! Statistical Models completed"
With 2nd script, CreateFolder.R, failing, it will look like this:
~/r# ./wrappper.sh
[1] "OK"
DONE! R packages installed
Error: object 'will_fail' not found
Execution halted

Related

Exit Powershell if R errors

I have a series of R files I'm calling in powershell. If one errors, I don't want it to carry on to try running the subsequent R scripts. The problem I have is capturing the R error in powershell, and using this to terminate the script. Any ideas?
I've tried this so far to no avail, as the error isn't being caught.
Powershell script:
# loop some scripts, try running them and if they fail then stop the process
foreach ($element in #(1,2,3)) {
Write-Host $element
try{
# minimal working eg of a script that stops early
Rscript -e "stop('error')"
}
catch {
Write-Host 'Rscript failed on run' $element'. Process killed in 10 seconds'
Start-Sleep 10
Exit
}
}
As it doesn't capture the error, the output message is:
1
Error: error
Execution halted
2
Error: error
Execution halted
3
Error: error
Execution halted
Any suggestions would be great! Thank you
EDIT:
Based off of the comment, am using this now (for future reference):
foreach ($element in #(1,2,3)) {
Write-Host $element
# minimal working eg of a script that stops early
Rscript -e "stop('error')"
if ($LastExitCode -ne 0){
Write-Host 'Rscript failed on run' $element'. Process killed in 10 seconds'
Start-Sleep 10
Exit
}
}
The errors are only available in the scope of the RScript not thrown into the PowerShell environment. You might consider to redirect the display output and evaluate the text but using the $LastExitCode is probably easiest.
(I don't have RScript installed but guess you can just capture the output like: $Result = Rscript -e "stop('error')".)
The exitcode can be controlled by the Status property of the Quit command and is apparently automatically set on a stop('error'):
foreach ($element in #(1,2,3)) {
Write-Host $element
# minimal working eg of a script that stops early
Rscript -e "stop('error')"
if ($LastExitCode -ne 0){
Write-Host 'Rscript failed on run' $element'. Process killed in 10 seconds'
Start-Sleep 10
Exit
}
}

How to get an Rscript to return a status code in non-interactive bash mode

I am trying to get the status code out of an Rscript run in an non-interactive way in the form of a bash script. This step is part of larger data processing cycle that involves db2 scripts among other things.
So I have the following contents in a script sample.sh:
Rscript --verbose --no-restore --no-save /home/R/scripts/sample.r >> sample.rout
when this sample.sh is run it always returns a status code of 0, irrespective of if the sample.r script run fully or error out in an intermediate step.
I tried the following things but no luck
1 - in the sample.sh file, I added an if and else condition for a return code like the below, but it again wrote back 0 despite sample.r failing in one of the functions inside.
if Rscript --verbose --no-restore --no-save /home/R/scripts/sample.r >> sample.rout
then
echo -e "0"
else
echo -e "1"
fi
2 - I also tried a wrapper script, like in a sample.wrapper.sh file
r=0
a=$(./sample.sh)
r=$?
echo -e "\n return code of the script is: $a\n"
echo -e "\n The process completed with status: $r"
here also I did not get the expected '1' in the case of failure of the sample.r in an intermediate step on both the variables a and r. Ideally, i would like a way to capture the error (as '1') in a.
Could someone please advice how to get rscript to write '0' only in case of completion of the entire script without any errors and '1' in all other cases?
greatly appreciate the input! thank you!
I solved the problem by returning the status code in addition to echo. below is the code snipped from sample.sh script. In addition, in sample.R code i have added trycatch to catch the errors and quit(status = 1).
function fun {
if Rscript --verbose --no-restore --no-save /home/R/scripts/sample.r > sample.rout 2>&1
then
echo -e "0"
return 0
else
echo -e "1"
return 1
fi
}
fun
thanks everyone for your inputs.
The above code works for me. I modified it so that I could reuse the function and have it exit when there's an error
Rscript_with_status () {
rscript=$1
if Rscript --vanilla $rscript
then
return 0
else
exit 1
fi
}
run r scripts by:
Rscript_with_status /path/to/script/sample.r
Your remote script needs to provide a proper exit status.
You can make a 1st test by providing i.e. "exit 1" at the end of the remote script and see that it will make a difference.
remote.sh:
#!/bin/sh
exit 1
From local machine:
ssh -l username remoteip /home/username/remote.sh
echo $?
1
But the remote script should also provide to you the exit status of the last executed command. Experiment further by modifying your remote script:
#!/bin/sh
#exit 1
/bin/false
The exit status of the remote command will now also be 1.

Running programmatically created Rscript call at command line

I have the following script:
rstest
text=$1
cmd="Rscript -e \"a='$1'; print(a)\""
echo $cmd
$cmd
This is the output I get when I run it:
balter#spectre3:~$ bash rstest hello
Rscript -e "a='hello'; print(a)"
Error: unexpected end of input
Execution halted
However, if I run the echoed command directly, it runs fine:
balter#spectre3:~$ Rscript -e "a='hello'; print(a)"
[1] "hello"
I would like to understand why this is. I've tried various combinations of quoting the bash variables and adding eval. But that doesn't seem to be the issue.
EDIT
I tried the answer given below, but get a different result!
balter#spectre3:~$ cat rstest
text=$1
cmd="Rscript -e \"a=$1; print(a)\""
echo $cmd
eval $cmd
balter#spectre3:~$ bash rstest
Rscript -e "a=; print(a)"
Error in cat("pointing to conda env:", env_name, "and lib location", lib, :
argument "env_name" is missing, with no default
Calls: startCondaEnv -> cat
Execution halted
Below script worked for me.
text=$1
cmd="Rscript -e \"a='$1'; print(a)\""
echo $cmd
eval $cmd
Removing eval gave the same error you posted.
Rscript -e "a='Hello'; print(a)"
Error: unexpected end of input
Execution halted

Run shell script from within an R script on a windows machine

I have a R-script within which i call a shell script using system command and use the paste command to pass arguments to the shell script ( on unix machine) and i now would like execute the same R-script on a windows machine and am struggling to get it working
Here are the steps i followed
R code
source('C:\\Users\\xxxx\\Documents\\R\\R-3.5.2\\ms\\ms\\MS_Config.R')
if(is.null(git_version) | git_version == "" | length(git_version) == 0){
print('ERROR: EXECUTION STOPPED !!!')
print('PLEASE SPECIFY GITHUB TAG_ID')
stop()
}
print("test4444")
print(enable_data_pull)
print (getwd())
system(paste('C:\\Users\\xxxx\\Documents\\R\\R- 3.5.2\\ms\\ms\\MS_ALLM_Parallel_Runner.sh -c ', num_cores,
'-s ', snapshot_dt,
'-p ' , local_storage_path,
'-t ', tag,
'-g ', git_version,
'-y ', enable_data_pull
))
print ("after shell script execution")
I tried the following, but did not succeed
Installed cygwin and called the rscript from the cygwin terminal(PATH variable is updated to include R and its binaries)
rscript "C:\Users\xxxx\Documents\R\R-3.5.2\ms\ms\MS_Model_Kickoff.R"
Below is the error message that i see after the r-script attempts to run the shell script
'CreateProcess' failed to run 'C:\Users\xxxx\DOCUME~1\R\R-35~1.2\ms\ms\CONRM_~1.SH -c 25 -s 201811 -p C:\Users\xxxx\Documents\Test -t Analytical -g verModelRefit2.2.2 -y N'
what does the above error mean and how do i fix this and execute the shell script within from the R-script on windows machine?r

AIX Interactive Shell

I was hoping I could distinguish between when a script is run interactively versus by 'at' or 'cron'. If the user is running a script on the command line I want to put output to their screen, but if it's running via 'at' or 'cron' then the output would go to a log file.
I searched online and saw many suggestions (although not AIX specific) on using the "$-". That sounded promising, but when I tried it, it wasn't as useful.
If I type 'echo "$-"' at the prompt I get "ims" back. If I create a script with the echo command, it returns "h". If I submit the script via "at" I get "h", and if I have cron run it I get "h".
I also looked at using TERM and PS1, but again they don't allow me to distinguish between a script run by either 'cron' or 'at' versus a script invoked at the command line.
Is there something else I could try on AIX?
Thanks.
If I run this script
Glenn. Here's a script I'm running. I get the desired result using "tty -s" but not with "$-". Am I doing something wrong? You see the "$-" results always says I'm not in an interactive shell.
#!/bin/ksh93
echo "tty -s"
if tty -s
then
echo " - I'm an interactive shell"
else
echo " - I'm not interactive"
fi
echo "\$-"
case $- in
*i*) echo " - I'm an interactive shell";;
*) echo " - I'm not interactive";;
esac
I get these three results sets when the script is run via 1) the command line, 2) "at", and 3) cron.
command line result
tty -s
- I'm an interactive shell
$-
- I'm not interactive
at result
tty -s
- I'm not interactive
$-
- I'm not interactive
Cron result
tty -s
- I'm not interactive
$-
- I'm not interactive
By running at the command line I mean I'm running "script.ksh >> script.log". If I run "echo $-" from the command line it returns 'ims' but when $- is reference within a script it always returns 'h'.
If the user is running a script on the command line I want to put output to their screen, but if it's running via 'at' or 'cron' then the output would go to a log file.
That's what standard output and redirection is good for. Also you can use command tty -s to determine if your standard input is a terminal or not. (If you wish to check your standard output instead: tty -s <&1)
Example (~projects/tmp/tty_test.sh):
#!/bin/sh
exec >>~projects/tmp/tty_test.log 2>&1
tty
if tty -s; then echo terminal
else echo otherwise
fi
You want to check if $- contains the letter i. If it does, you have an interactive shell:
case $- in
*i*) echo "I'm an interactive shell";;
*) echo "I'm not interactive";;
esac
Testing
$ ksh
$ case $- in
> *i*) echo "I'm an interactive shell";;
> *) echo "I'm not interactive";;
> esac
I'm an interactive shell
$ ksh <<'END'
> case $- in
> *i*) echo "I'm an interactive shell";;
> *) echo "I'm not interactive";;
> esac
> END
I'm not interactive
Given the comments below, you probably want to use this builtin test:
if [ -t 0 ]; then
: # script running interactively
else
# running non-interactively (cron or at)
# redirect output to a log file
exec 1>/path/to/logfile 2>&1
fi
# and now, your script can just output to stdout/stderr
# and the output will go to the screen or to the logfile
echo "hello"
date
print -u2 "bye"

Resources