For the latest versions of SparkR (1.6 and 2.0) do I need R to be installed ? If I need R to run sparkR is it required to install R on all data nodes in a Yarn cluster with CDH and spark?
Yes, you need:
R installation.
all packages and native dependencies
installed on each worker node.
Related
I try to install azuremlsdk for R on rocker/tidyverse:latest image:
dir.create("library")
install.packages("azuremlsdk", repos="https://cloud.r-project.org", lib="library")
azuremlsdk::install_azureml()
However, I get the following error on the last line:
Error in loadNamespace(name) : there is no package called ‘azuremlsdk’
Calls: :: ... loadNamespace -> withRestarts -> withOneRestart ->
doWithOneRestart
Why does it happen and how to fix it?
P.S. I must install everything inside library because otherwise the folder is not writable.
If you can suggest a docker image that includes R and AzureML SDK for R, then I'd appreciate as well.
Step-by-step instructions for installing the Azure ML SDK for R:
Step1: Install Conda
If you do not have Conda already installed on your machine, you will first need to install it, since the Azure ML R SDK uses reticulate to bind to the Python SDK. We recommend installing Miniconda, which is a smaller, lightweight version of Anaconda. Choose the 64-bit binary for Python 3.5 or later.
Step2: Install the azuremlsdk R package
You will need remotes to install azuremlsdk from the GitHub repo.
install.packages('remotes')
Then, you can use the install_github function to install the package.
remotes::install_cran('azuremlsdk', repos = 'https://cloud.r-project.org/')
If you are using R installed from CRAN, which comes with 32-bit and 64-bit binaries, you may need to specify the parameter INSTALL_opts=c("--no-multiarch") to only build for the current 64-bit architecture.
remotes::install_cran('azuremlsdk', repos = 'https://cloud.r-project.org/', INSTALL_opts=c("--no-multiarch"))
For more details, refer "Install the Azure ML SDK for R" and "Azure ML SDK for R".
Hope this helps. Do let us know if you any further queries.
I have access to a headless, air-gapped server, where I have Anaconda installed.
The default R on server is located at /usr/local/bin/R with version 3.1.2.
On one of my environments, I wish to install R, along with certain packages. I downloaded R version 3.5.1 source files from Anaconda repository and ran the command
conda install Downloads/r-3.5.1-r351_0.tar.bz2
which executed successfully
Downloading and Extracting Packages
################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
However, when I check if R is installed inside the environment, I still get the previous location and version. How can I access the newly installed R ?
As suggested in comments section, I installed the dependencies separately (r-base >=3.5.1,<3.5.2.0a0, r-recommended 3.5.1.*) and the problem was solved.
On my Win 7 Ent x64 I installed RStudio R-3.4.2.
I installed few packages that I need (RPostgreSQL, sqldf, etc..)
But when I'm executing code, these libraries give me errors:
library(RPostgreSQL)
Error: package ‘RPostgreSQL’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version
In addition: Warning message:
package ‘RPostgreSQL’ was built under R version 3.5.0
All of the packages are downloaded from CRAN. RPostgrSQL is 0.6-2 version.
I'm pretty sure that solution is quite simple, but I searched a lot and still cannot find the solution.
How did you install the package RPostgreSQL? If you used
install.packages("RPostgreSQL") on your Windows it installed using binaries, which may have been built using a different R version. In your case, 3.5.0 vs 3.4.2.
Try using install.packages("RPostgreSQL", type="source"). This way your machine will compile the package from source using your version of R, and it should work.
If the package needs C/C++/Fortran compilation, you will als need RTools installed on your machine.
You can try to find all packages installed with the old version R like this:
grep 'Built: R 3.4' /usr/local/lib/R/site-library/*/DESCRIPTION > temp
and then you can uninstall the packages with remove.packages()
I need to install R packages in several nodes (10+) in AWS.
I wont be able to open R shell in each and do install.packages("foo")
This will be done using a configuration management tool like Puppet and it'll be easier if i can do an apt-get installation of R packages automatically.
I found a list of R debian packages here:
http://cran.cnr.berkeley.edu/bin/linux/ubuntu/lucid/
But it does not contain all the packages that i need.
Is there a way to convert any R package and it's internal dependencies to a Debian package similar to the approach used in creating r-cran-*.deb?
Have you looked at http://debian-r.debian.net/ ?
All CRAN (and many other) packages already packaged
You can install packages without starting the R console. You can download the tar.gz packages from the cran website. For example here is the tar.gz for the randomForest package: http://cran.r-project.org/src/contrib/randomForest_4.6-7.tar.gz
R CMD INSTALL ${package}.tar.gz
The cran2deb project claims to do exactly this, turning an R package into a Debian package and noting the correct dependencies.
I haven't used it myself yet.
I'd like to install the latest version of R in RHEL 5 as I tried to install some library (plyr) but it complained a dependency could not be used with my installed version of R: package ‘Rcpp’ is not available (for R version 2.15.2)
2.15.2 is the latest precompiled R binary available for RHEL5 from epel. I guess they stopped updating it. Is compiling from source to get R 3.x my only solution here or there is some binary installer somewhere else? I much rather use something like yum since it take care of all the R dependency packages as well.