Remote install of multiple R packages (with, without depends) from Ubuntu CLI - r

I have to perform a remote R installation on an Ubuntu 16.10 system. As part of this, I have to install specific packages on that host. I want to install these packages Rcmdr,list,ggplot2,afex,lsmeans. Since I am doing this remotely, I cannot use
sudo -i R
to first enter the R CLI and then install with install.packages(). Instead I must somehow install the packages from the Ubuntu CLI.
I found these links:
multiple R package installation with
install.packages()
R CMD INSTALL -l usage syntax to install multiple packages in
section
6.3
Use of repos parameter inside
install.packages()
However, some packages have dependencies:
The list package depends on utils and sandwich.
The Rcmdr package depends on grDevices, utils, splines, RcmdrMisc, car.
The ggplot2 package also has dependencies.
I would like to install only the packages Rcmdr,list,ggplot2 with all their dependencies. Normally, I would do it this way:
install.packages(c('Rcmdr','list','ggplot2'), dependencies=TRUE)
QUESTIONS
How do I specify the dependencies option in R CMD for one package
only? Is this the way to install them
R CMD INSTALL -l Rcmdr dependencies=TRUE, list dependencies=TRUE, \
ggplot2 dependencies=TRUE, afex, lsmeans
or this incorrect?
Also, how to I specify the repos parameter inside R CMD INSTALL -l?
EDIT
As per the first comment below, sudo is not needed above.i.e. sudo -i R can be replaced by R.

Regarding your questions:
Question 1
This may not be the best approach. Consider instead Rscript -e 'install.packages(...)' which is what R CMD INSTALL ... calls anyway. You have better control over options here. And read on...
Question 2
On all Ubuntu machines at work and home I do this via /etc/R/Rprofile.site via something like
## Example of Rprofile.site
local({
r <- getOption("repos")
r["CRAN"] <- "https://cloud.r-project.org"
r["ghrr"] <- "https://ghrr.github.io/drat"
options(repos = r)
})
where we usually add a third and network-local repo. You may just want CRAN here -- it uses the 'always-close to you' CDN administered by RStudio for the R Project and R Consortium. The ghrr drat is a helper repo I set up.
Question 3
sudo is not needed per something I add to the official Debian/Ubuntu package for R -- but you need to be a member of the group that owns /usr/local/lib/R/site-library.
Now, if I may, two more suggestions:
Littler
The r executable is available to you via sudo apt-get install r-cran-littler. I use it on the command-line; and you probably want to look into the basic install.r script and the extended install2.r. I tend to create a softlink from /usr/local/bin to the package directory for these and other (such as update.r). I have been running many (Ubuntu and Debian) machines like that for many years.
Michael Rutter repos, and Docker
We actually have about 3000 CRAN packages as binaries for Ubuntu so you could just do sudo apt-get install ... and all dependendies would get resolved. Look eg in this script of mine (which uses them on Travis) or some of the Docker files I maintain such as this one.

Related

R won't update the data.table package for me [duplicate]

A friend sent me along this great tutorial on webscraping The New York Times with R. I would really love to try it. However, the first step is to install a package called [RJSONIO][2] from source.
I know R reasonably well, but I have no idea how to install a package from source.
I'm running macOS (OS X).
If you have the file locally, then use install.packages() and set the repos=NULL:
install.packages(path_to_file, repos = NULL, type="source")
Where path_to_file would represent the full path and file name:
On Windows it will look something like this: "C:\\RJSONIO_0.2-3.tar.gz".
On UNIX it will look like this: "/home/blah/RJSONIO_0.2-3.tar.gz".
Download the source package, open Terminal.app, navigate to the directory where you currently have the file, and then execute:
R CMD INSTALL RJSONIO_0.2-3.tar.gz
Do note that this will only succeed when either: a) the package does not need compilation or b) the needed system tools for compilation are present. See: R for Mac OS X
You can install directly from the repository (note the type="source"):
install.packages("RJSONIO", repos = "http://www.omegahat.org/R", type="source")
A supplementarily handy (but trivial) tip for installing older version of packages from source.
First, if you call "install.packages", it always installs the latest package from repo. If you want to install the older version of packages, say for compatibility, you can call install.packages("url_to_source", repo=NULL, type="source"). For example:
install.packages("http://cran.r-project.org/src/contrib/Archive/RNetLogo/RNetLogo_0.9-6.tar.gz", repo=NULL, type="source")
Without manually downloading packages to the local disk and switching to the command line or installing from local disk, I found it is very convenient and simplify the call (one-step).
Plus: you can use this trick with devtools library's dev_mode, in order to manage different versions of packages:
Reference: doc devtools
From CRAN, you can install directly from a GitHub repository address. So if you want the package at https://github.com/twitter/AnomalyDetection, using
library(devtools)
install_github("twitter/AnomalyDetection")
does the trick.
In addition, you can build the binary package using the --binary option.
R CMD build --binary RJSONIO_0.2-3.tar.gz
If you have source code you wrote yourself, downloaded (cloned) from GitHub, or otherwise copied or moved to your computer from some other source, a nice simple way to install the package/library is:
In R
It's as simple as:
# install.packages("devtools")
devtools::install('path/to/package')
From terminal
From here, you can clone a GitHub repo and install it with:
git clone https://github.com/user/repo.git
R -e "install.packages('devtools');devtools::install('path/to/package')"
Or if you already have devtools installed, you can skip that first bit and just clone the repo and run:
R -e "devtools::install('path/to/package')"
Note that if you're on ubuntu, install these system libraries before installing devtools (or devtools won't install properly).
apt-get update
apt-get install build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev -y

Installing of SparkR

I have the last version of R - 3.2.1. Now I want to install SparkR on R. After I execute:
> install.packages("SparkR")
I got back:
Installing package into ‘/home/user/R/x86_64-pc-linux-gnu-library/3.2’
(as ‘lib’ is unspecified)
Warning in install.packages :
package ‘SparkR’ is not available (for R version 3.2.1)
I have also installed Spark on my machine
Spark 1.4.0
How I can solve this problem?
You can install directly from a GitHub repository:
if (!require('devtools')) install.packages('devtools')
devtools::install_github('apache/spark#v2.x.x', subdir='R/pkg')
You should choose tag (v2.x.x above) corresponding to the version of Spark you use. You can find a full list of tags on the project page or directly from R using GitHub API:
jsonlite::fromJSON("https://api.github.com/repos/apache/spark/tags")$name
If you've downloaded binary package from a downloads page R library is in a R/lib/SparkR subdirectory. It can be used to install SparkR directly. For example:
$ export SPARK_HOME=/path/to/spark/directory
$ cd $SPARK_HOME/R/pkg/
$ R -e "devtools::install('.')"
You can also add R lib to .libPaths (taken from here):
Sys.setenv(SPARK_HOME='/path/to/spark/directory')
.libPaths(c(file.path(Sys.getenv('SPARK_HOME'), 'R', 'lib'), .libPaths()))
Finally, you can use sparkR shell without any additional steps:
$ /path/to/spark/directory/bin/sparkR
Edit
According to Spark 2.1.0 Release Notes should be available on CRAN in the future:
Standalone installable package built with the Apache Spark release. We will be submitting this to CRAN soon.
You can follow SPARK-15799 to check the progress.
Edit 2
While SPARK-15799 has been merged, satisfying CRAN requirements proved to be challenging (see for example discussions about 2.2.2, 2.3.1, 2.4.0), and the packages has been subsequently removed (see for example SparkR was removed from CRAN on 2018-05-01, CRAN SparkR package removed?). As the result methods listed in the original post are still the most reliable solutions.
Edit 3
OK, SparkR is back up on CRAN again, v2.4.1. install.packages('SparkR') should work again (it may take a couple of days for the mirrors to reflect this)
SparkR requires not just an R package but an entire Spark backend to be pulled in. When you want to upgrade SparkR, you are upgrading Spark, not just the R package. If you want to go with SparkR then this blogpost might help you out: https://blog.rstudio.org/2015/07/14/spark-1-4-for-rstudio/.
It should be said though: nowadays you may want to refer to the sparklyr package as it makes all of this a whole lot easier.
install.packages("devtools")
devtools::install_github("rstudio/sparklyr")
library(sparklyr)
spark_install(version = "1.6.2")
spark_install(version = "2.0.0")
It also offers more functionality than SparkR as well as a very nice interface to dplyr.
I also faced similar issue while trying to play with SparkR in EMR with Spark 2.0.0. I'll post the steps here that I followed to install rstudio server, SparkR, sparklyr, and finally connecting to a spark session in a EMR cluster:
Install rstudio server:
After the EMR cluster is up and running, ssh to the master node with user 'hadoop#' and download rstudio server
wget https://download2.rstudio.org/rstudio-server-rhel-0.99.903-x86_64.rpm
then install using yum install
sudo yum install --nogpgcheck rstudio-server-rhel-0.99.903-x86_64.rpm
finally add a user to access rstudio web console as:
sudo su
sudo useradd username
sudo echo username:password | chpasswd
To acess rstudio Web console you need to create a SSH tunnel from your machine to the EMR master node like below:
ssh -NL 8787:ec2-emr-master-node-ip.compute-1.amazonaws.com:8787 hadoop#ec2-emr-master-node-ip.compute-1.amazonaws.com&
Now open any browser and type localhost:8787 to go the rstudio Web console and use the username:password combo to login.
To install the required R packages you need to install libcurl into the master node first like below:
sudo yum update
sudo yum -y install libcurl-devel
Resolve permission issues with:
sudo -u hdfs hadoop fs -mkdir /user/
sudo -u hdfs hadoop fs -chown /user/
Check Spark version in EMR and set SPARK_HOME:
spark-submit --version
export SPARK_HOME='/usr/lib/spark/'
Now in the rstudio console install SparkR like below:
install.packages('devtools')
devtools::install_github('apache/spark#v2.0.0', subdir='R/pkg')
install.packages('sparklyr')
library(SparkR)
library(sparklyr)
Sys.setenv(SPARK_HOME='/usr/lib/spark')
sc <- spark_connect(master = "yarn-client")
Now versions 2.1.2 and 2.3.0 of SparkR are now available in the repository of CRAN, you can install version 2.3.0 as follows:
install.packages("https://cran.r-project.org/src/contrib/Archive/SparkR/SparkR_2.3.0.tar.gz", repos = NULL, type="source")
Note: You must first download and install the corresponding version of Apache Spark from download, so that the package works correctly.

How to install "arm" package in R/Rstudio for downloading it directly and not from the Packages ->Install interface? [duplicate]

A friend sent me along this great tutorial on webscraping The New York Times with R. I would really love to try it. However, the first step is to install a package called [RJSONIO][2] from source.
I know R reasonably well, but I have no idea how to install a package from source.
I'm running macOS (OS X).
If you have the file locally, then use install.packages() and set the repos=NULL:
install.packages(path_to_file, repos = NULL, type="source")
Where path_to_file would represent the full path and file name:
On Windows it will look something like this: "C:\\RJSONIO_0.2-3.tar.gz".
On UNIX it will look like this: "/home/blah/RJSONIO_0.2-3.tar.gz".
Download the source package, open Terminal.app, navigate to the directory where you currently have the file, and then execute:
R CMD INSTALL RJSONIO_0.2-3.tar.gz
Do note that this will only succeed when either: a) the package does not need compilation or b) the needed system tools for compilation are present. See: R for Mac OS X
You can install directly from the repository (note the type="source"):
install.packages("RJSONIO", repos = "http://www.omegahat.org/R", type="source")
A supplementarily handy (but trivial) tip for installing older version of packages from source.
First, if you call "install.packages", it always installs the latest package from repo. If you want to install the older version of packages, say for compatibility, you can call install.packages("url_to_source", repo=NULL, type="source"). For example:
install.packages("http://cran.r-project.org/src/contrib/Archive/RNetLogo/RNetLogo_0.9-6.tar.gz", repo=NULL, type="source")
Without manually downloading packages to the local disk and switching to the command line or installing from local disk, I found it is very convenient and simplify the call (one-step).
Plus: you can use this trick with devtools library's dev_mode, in order to manage different versions of packages:
Reference: doc devtools
From CRAN, you can install directly from a GitHub repository address. So if you want the package at https://github.com/twitter/AnomalyDetection, using
library(devtools)
install_github("twitter/AnomalyDetection")
does the trick.
In addition, you can build the binary package using the --binary option.
R CMD build --binary RJSONIO_0.2-3.tar.gz
If you have source code you wrote yourself, downloaded (cloned) from GitHub, or otherwise copied or moved to your computer from some other source, a nice simple way to install the package/library is:
In R
It's as simple as:
# install.packages("devtools")
devtools::install('path/to/package')
From terminal
From here, you can clone a GitHub repo and install it with:
git clone https://github.com/user/repo.git
R -e "install.packages('devtools');devtools::install('path/to/package')"
Or if you already have devtools installed, you can skip that first bit and just clone the repo and run:
R -e "devtools::install('path/to/package')"
Note that if you're on ubuntu, install these system libraries before installing devtools (or devtools won't install properly).
apt-get update
apt-get install build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev -y

How do I install non-CRAN R packages onto AWS?

I have recently, set up an AWS accountand am in the process of uploading R packages I require
The process runs smoothly with CRAN packages using the command e.g.
sudo su ­ ­c "R ­e \"install.packages('ggplot2', repos='http://cran.rstudio.com/')\""
For non-CRAN packages, I got some help from this source
https://github.com/hadley/devtools/issues/414 and tried
Rscript -e 'library(devtools); library(methods); install_github(commandArgs(TRUE))' "ramnathv/rCharts"
The zipped package (and others on github ) download but then I get the error
'lib="/usr/local/lib/R/site-library"' is not writable .
although that is where the R packages were written to
Has anyone successfully achieved this process and could give me the solution
Tx
It looks like you just need sudo to install to that directory. You could try
sudo Rscript -e 'library(devtools); library(methods); install_github(commandArgs(TRUE))' "ramnathv/rCharts"
or
git clone https://github.com/ramnathv/rCharts.git
sudo R CMD INSTALL rCharts

How do I install an R package from source?

A friend sent me along this great tutorial on webscraping The New York Times with R. I would really love to try it. However, the first step is to install a package called [RJSONIO][2] from source.
I know R reasonably well, but I have no idea how to install a package from source.
I'm running macOS (OS X).
If you have the file locally, then use install.packages() and set the repos=NULL:
install.packages(path_to_file, repos = NULL, type="source")
Where path_to_file would represent the full path and file name:
On Windows it will look something like this: "C:\\RJSONIO_0.2-3.tar.gz".
On UNIX it will look like this: "/home/blah/RJSONIO_0.2-3.tar.gz".
Download the source package, open Terminal.app, navigate to the directory where you currently have the file, and then execute:
R CMD INSTALL RJSONIO_0.2-3.tar.gz
Do note that this will only succeed when either: a) the package does not need compilation or b) the needed system tools for compilation are present. See: R for Mac OS X
You can install directly from the repository (note the type="source"):
install.packages("RJSONIO", repos = "http://www.omegahat.org/R", type="source")
A supplementarily handy (but trivial) tip for installing older version of packages from source.
First, if you call "install.packages", it always installs the latest package from repo. If you want to install the older version of packages, say for compatibility, you can call install.packages("url_to_source", repo=NULL, type="source"). For example:
install.packages("http://cran.r-project.org/src/contrib/Archive/RNetLogo/RNetLogo_0.9-6.tar.gz", repo=NULL, type="source")
Without manually downloading packages to the local disk and switching to the command line or installing from local disk, I found it is very convenient and simplify the call (one-step).
Plus: you can use this trick with devtools library's dev_mode, in order to manage different versions of packages:
Reference: doc devtools
From CRAN, you can install directly from a GitHub repository address. So if you want the package at https://github.com/twitter/AnomalyDetection, using
library(devtools)
install_github("twitter/AnomalyDetection")
does the trick.
In addition, you can build the binary package using the --binary option.
R CMD build --binary RJSONIO_0.2-3.tar.gz
If you have source code you wrote yourself, downloaded (cloned) from GitHub, or otherwise copied or moved to your computer from some other source, a nice simple way to install the package/library is:
In R
It's as simple as:
# install.packages("devtools")
devtools::install('path/to/package')
From terminal
From here, you can clone a GitHub repo and install it with:
git clone https://github.com/user/repo.git
R -e "install.packages('devtools');devtools::install('path/to/package')"
Or if you already have devtools installed, you can skip that first bit and just clone the repo and run:
R -e "devtools::install('path/to/package')"
Note that if you're on ubuntu, install these system libraries before installing devtools (or devtools won't install properly).
apt-get update
apt-get install build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev -y

Resources