executing file contents as commands under R from Linux terminal - r

I've written a text file containing script for R. I've gotten it to run under Windows from a .bat file running a .txt file under R with CMD BATCH.
I'm trying to replicate that (minus the clickability) in the Terminal
I've changed the permissions for program execution, I've set the file to have the shebang, and have tried rewriting for it a few different programmes such as
#!/usr/bin/R
library(rvest)
library(plyr)
which returns an error "Syntax error near unexpected symbol 'rvest'
and
#!/home/robert/Téléchargements/R-3.2.3/src/unix/Rscript.c
library(rvest)
library(plyr)
which also returns an error "Syntax error near unexpected symbol 'rvest'
Separately, on both of these I changed the file extension from nothing to .R
In one case it gave the same error, in the other it started a session of R but didn't execute the commands.
I realise it's a messy question, but I'm having difficulty getting these ducks in a row.

Ultimately, this was what worked:
R < /home/robert/R/scraper1.R --no-save
But here's the rest of my answer in case that doesn't work for someone else:
I'm not sure if you've seen any of these, but here's some stuff to try:
Dupe?
The top answer from a very similar post: I'm going to assume you googled your question before posting it, so I'm sure you've already seen this, but it's not referenced in your question, so here it is [Source]:
Content of script.r:
#!/usr/bin/Rscript
cat("Hello")
Invocation from command line:
./script.r
?Rscript
?Rscript allows you to run R scripts in a Unix-esque system [Source]:
## example #! script for a Unix-alike
#! /path/to/Rscript --vanilla --default-packages=utils
args <- commandArgs(TRUE)
res <- try(install.packages(args))
if(inherits(res, "try-error")) q(status=1) else q()
Batch
Here's something from an old R mail pipe [Source]:
Place the line: R --vanilla < foo.txt foo.results into a file named foo.batch. No other text should be in the file.
Make this file executable via chmod 755 foo.batch
At the command line try at -f foo.batch now or perhaps, batch -f foo.batch.
If this does not work, ask your system administrator how to set up a batch process.
The advantage of the batch process is 1. you need not be logged in, 2.
your job will take a lower priority than interactive jobs.
R -e
Loading two libraries and running an R command [Source]
R -e 'library("rmarkdown");library("knitr");rmarkdown::render("NormalDevconJuly.Rmd")'
R -e 'library("markdown");rpubsUpload("normalDev","NormalDevconJuly.html")'
Other
R < scriptname.R --no-save [Source]
$ source("scriptname.R") [Source]

Related

How to write console output to a text file with Rscript, like you can with R CMD BATCH

In the past I have used R CMD BATCH to execute R code from the command line on a Linux server. The syntax I used was
R CMD BATCH --no-save --no-restore rcode.r output.txt
The above code writes console output to output.txt which can be monitored as the script is running. Is this also possible with Rscript? I would prefer to use Rscript since I have heard that R CMD BATCH is deprecated.
To clarify my original question, R CMD BATCH writes all console output, including messages, warnings, and print() statements, to output.txt. In contrast Rscript rcode.r > output.txt writes only the print()ed output to the text file and everything else to the terminal. How can I replicate the behavior of R CMD BATCH with Rscript?
I discovered after some digging that, at least on the Linux system I'm using, Rscript is just a convenience function. If you call
Rscript --verbose foobar.r
you will see the underlying call is:
running
'/usr/lib/R/bin/R --no-echo --no-restore --file=foobar.r'
This means that --no-echo is baked into Rscript.
Therefore the solution is to run
/usr/lib/R/bin/R --no-restore --file=foobar.r > output.txt
where the --no-echo is removed, and the output is redirected to a text file as suggested by #MrFlick. The commands will be echoed in addition to the output.
You can create a new alias for Rscript if you want --no-echo to be removed by default. For example, in my .bashrc file I have the following:
function Rscript2() { R --no-restore --file="$1"; }
export -f Rscript2
Now, in my Slurm batch job scripts, I can run Rscript2 file.R to get the desired behavior: all R console output is included in the slurm-*.out files.
Just redirect the output to a file like you would with any other command line output
Rscript rcode.r > output.txt

Rscript not working with packaged R for AWS Lambda

I'm trying to run an R script on the command line of an AWS EC2 instance using packaged R binaries and libraries (without installation) -- the point is to test the script for deployment to AWS Lambda. I followed these instructions. The instructions are for packaging up all the R binaries and libraries in a zip file and moving everything to a Amazon EC2 instance for testing. I unzipped everything on the new machine, ran 'sudo yum update' on the machine, and set R's environment variables to point to the proper location:
export R_HOME=$HOME
export LD_LIBRARY_PATH=$HOME/lib
NOTE: $HOME is equal to /home/ec2-user.
I created this hello_world.R file to test:
#!/home/ec2-user/bin/Rscript
print ("Hello World!")
But when I ran this:
ec2-user$ Rscript hello_world.R
I got the following error:
Rscript execution error: No such file or directory
So I checked the path, but everything checks out:
ec2-user$ whereis Rscript
Rscript: /home/ec2-user/bin/Rscript
ec2-user$ whereis R
R: /home/ec2-user/bin/R /home/ec2-user/R
But when I tried to evaluate an expression using Rscript at the command line, I got this:
ec2-user$ Rscript -e "" --verbose
running
'/usr/lib64/R/bin/R --slave --no-restore -e '
Rscript execution error: No such file or directory
It seems Rscript is still looking for R in the default location '/usr/lib64/R/bin/R' even though my R_HOME variable is set to '/home/ec2-user':
ec2-user$ echo $R_HOME
/home/ec2-user
I've found sprinkles of support, but I can't find anything that addresses my specific issue. Some people have suggested reinstalling R, but my understanding is, for the purposes of Lambda, everything needs to be self-contained so I installed R on a separate EC2 instance, then packaged it up. I should mention that everything runs fine on the machine where R was installed with the package manager.
SOLUTION: Posted my solution in the answers.
It thinkt it is staring at you right there:
ec2-user$ whereis R
R: /home/ec2-user/bin/R /home/ec2-user/R
is where you put R -- however it was built for / expects this:
ec2-user$ Rscript -e "" --verbose
running
'/usr/lib64/R/bin/R --slave --no-restore -e '
These paths are not the same. The real error may be your assumption that you could just relocate the built and configured R installation to a different directory. You can't.
You could build R for the new (known) path and install that. On a system where the configured-for and installed-at path are the same, all is good:
$ Rscript -e "q()" --verbose
running
'/usr/lib/R/bin/R --slave --no-restore -e q()'
$
This blog post walks through a similar problem and offers a potential solution. I also had to implement part of the solution from this post.
I changed the very first line of R's source code from this:
#!/bin/sh
# Shell wrapper for R executable.
R_HOME_DIR=${R_ROOT_DIR}/lib64${R_ROOT_DIR}
To this:
R_HOME_DIR=${RHOME}/lib64${R_ROOT_DIR}
I'll explain why below.
NOTE -- The rest of the code is:
if test "${R_HOME_DIR}" = "${R_ROOT_DIR}/lib64${R_ROOT_DIR}"; then
case "linux-gnu" in
linux*)
run_arch=`uname -m`
case "$run_arch" in
x86_64|mips64|ppc64|powerpc64|sparc64|s390x)
libnn=lib64
libnn_fallback=lib
;;
*)
libnn=lib
libnn_fallback=lib64
;;
esac
if [ -x "${R_ROOT_DIR}/${libnn}${R_ROOT_DIR}/bin/exec${R_ROOT_DIR}" ]; then
R_HOME_DIR="${R_ROOT_DIR}/${libnn}${R_ROOT_DIR}"
elif [ -x "${R_ROOT_DIR}/${libnn_fallback}${R_ROOT_DIR}/bin/exec${R_ROOT_DIR}" ]; then
R_HOME_DIR="${R_ROOT_DIR}/${libnn_fallback}${R_ROOT_DIR}"
## else -- leave alone (might be a sub-arch)
fi
;;
esac
fi
if test -n "${R_HOME}" && \
test "${R_HOME}" != "${R_HOME_DIR}"; then
echo "WARNING: ignoring environment value of R_HOME"
fi
R_HOME="${R_HOME_DIR}"
export R_HOME
You can see at the bottom, the code sets R_HOME equal to R_HOME_DIR, which it originally assigned based on R_ROOT_DIR.
No matter what you set the R_HOME_DIR or R_HOME variable to, R resets everything using the R_ROOT_DIR variable.
With the change, I can set all my environment variables:
export RHOME=$PWD/R #/home/ec2-user/R
export R_HOME=$PWD/R #/home/ec2-user/R
export R_ROOT_DIR=/R #/R
I set RHOME to my working directory where the R package sits. RHOME basically acts as a prefix, in my case, it's /home/ec2-user/.
Also, Rscript appends /R/bin to whatever RHOME is, so now I can properly run...
Rscript hello_world.R
...on the command line. Rscript knows where to find R, which knows where to find all it's stuff.
I feel like packaging up R to run in a portable self-contained folder, without using Docker or something, should be easier than this, so if anyone has a better way of doing this, I'd really appreciate it.
Another more quickly method:
create same folder /usr/lib/R/bin/
then put R into this folder.

Rscript in silent mode

I am using Rscript to run an R script but I get a lot of output on my screen. Can I run Rscript in silent mode (meaning without any screen output)?
Several options come to mind:
within R: use sink() to divert output to a file, see help(sink)
on the shell: Rscript myscript.R 2>&1 >/dev/null
edit the code :)
on Linux, use our littler frontend as it runs in --slave mode by default :)
Options 3 is the most involved but possibly best. You could use a logging scheme where you print / display in "debug" or "verbose" but not otherwise. I often do that, based on a command-line toggle given to the script.
You can redirect the output with
Rscript myscript.R >& >/dev/null (linux)
or
Rscript myscript.R >$null (windows)
or use R directly:
R --quiet --vanilla < myscript.R
or
R CMD BATCH myscript.R
(That last version writes the output to a file myscript.Rout)
One more option: if you want to separate the output and the error message into different files, which makes it easier to identify the problems, you can use the command on the shell:
Rscript myscript.R >a.Rout 2>a.Rerr
This will write the program output to a.Rout and the error messages to a.Rerr. Note that the files of a.Rout and a.Rerr should be removed beforehand, to avoid an error.

/usr/bin/env: RScript: No such file or directory | After recent R-3.0.1. installation.

I am a bit lost when dealing with installing and using R. I installed R 3.0.1 from source and did the ./configure, make, make check, and make install as suggested. However I tried running R but it said that R wasn't in the /usr/bin folder. So I then copied the entire R-3.0.1/bin directory into my /usr/bin directory using cp. Now I'm getting a few errors regarding /usr/bin/env when trying to use RScript on a hello_world.R script I wrote from the O'Reilly R In a Nutshell book I store in a file hello_world.R the contents are below:
#! /usr/bin/env RScript
print("Hello World!");
Simple enough, but when I try to load it I get the following error:
$ ./hello_world.R
/usr/bin/env: RScript: No such file or directory
I'm not sure if this is a PATH problem or something, but when I search in my /usr/bin directory I do see the RScript file in there along with (R, BATCH, and the others associated with R programming language). Any help is greatly appreciated. Cheers.
You may be using an invalid command line option for Rscript in your shebang line.
For instance ...
#!/usr/bin/env RScript --vanilla
remove "--vanilla" (or other offending option) and rerun your script
#!/usr/bin/env RScript
I know you didn't put this in your example, but the solution may help others searching for the same issue.
Again, the good solution to this problem is very simple and clearly explained in the man page of env. The script should use the env command to invoke Rscript and not Rscript directly:
#!/usr/bin/env Rscript
some R code now...
But a script like this will read the user's .Rprofile among other things. When we want to have a vanilla R session (in order to start with a clean and controlled R), we must pass the option --vanilla. If you try something like
#!/usr/bin/env Rscript --vanilla
some R code now...
env will take the string Rscript --vanilla a the command to execute and will inevitably return the error message
/usr/bin/env: ‘Rscript --vanilla’: No such file or directory
In env's man page, there is an option called -S for splitting the strings. Its role is exactly to solve the problem above and use the first string Rscript as the command name, and the following strings (like --vanilla) as options to pass to Rscript.
The solution is therefore:
#!/usr/bin/env -S Rscript --vanilla
some R code now...
Put in the shebang line of your script #!/usr/bin/Rscript and it should work.
As a side remark if you want to keep up-to-date with the R versions from CRAN and not relying on the native R of your Linux distro (Ubuntu) then add the following line in your apt sources:
deb http://my_favorite_cran_mirror/bin/linux/ubuntu raring/
After that you can always use the apt system to install R which -I would agree with Jake above- it should be the preferable way to install R.
*Change the my_favorite_cran_mirror with a valid CRAN mirror that is close to you.
#! /usr/bin/env RScript
print("Hello World!");
Simple enough, but when I try to load it I get the following error:
$ ./hello_world.R
/usr/bin/env: RScript: No such file or directory
Here u make mistake is that instead of RScript write Rscript.
The syntax will be
#! /usr/bin/env Rscript
print("Hello World!");
Then run it it will work (y) all the best.
$./hello_world.R
I arrived at this question trying to understand this error message on a cluster computer where I did not have control over the R installation.
In general, when I converted Rscript in my makefile to /usr/bin/Rscript the error message no longer occurred.

Problems executing script from command line in R. Error message: cannot find path specified

I have been trying to execute a simple test.R
setwd("C:\Users\jdd\Documents")
test <- 2*6598
filename = "test.csv"
write.csv(test,file=filename)
Via the following command line command in Windows:
"C:\Program Files\R\R-2.15.2\bin\R.exe" CMD BATCH --vanilla --slave "C:\Users\jdd\Documents\test.R"
When I execute this I get the following error:
The system cannot find the path specified.
I have been trying to work out a solution on the basis of the provided error message, but failed so far. Wondering if somebody can help me so I can execute the script directly from the command line. Thanks
Thanks #sebastian-c! I tried to use RScript, which I investigated before. However, the problem was a different one. Appears that in my installation there is a R.exe and Rscript.exe file in .\bin, but also one in .\bin\x64. The first one is not working properly, but the second one is. The comment made by #Roland is very important as well, since once working I got this error message!
The following command did the job:
"C:\Program Files\R\R-2.15.2\bin\x64\Rscript.exe" "C:\Users\jdd\Documents\test.R"
and the corrected text.R is:
setwd("C:\\Users\\jdd\\Documents")
test <- 2*6598
filename = "test.csv"
write.csv(test,file=filename)
As mentioned here, it might has something to do with 64bit version of R.
The problem is that Rscript.exe itself is attempting to access a missing file on the system. The obvious fix is explicitly add 'x64' to the path of the other Rscript.exe that was installed:
"C:\Program Files\R\R-2.15.2\bin\x64\Rscript.exe" --version
R scripting front-end version 3.0.2 (2013-09-25)

Resources