Error installing RODBC or ODBC on a Sagemaker Jupyter NoteBook Instance - r

I have been trying to establish a connection to Teradata from a Sagemaker Jupyter Notebook instance. I was trying to do it the way I would through R Studio. But when ever I try to install the package in the instance I get an non-zero exit status error.
I have tried installing the following ways:
remotes::install_github()
or
devtools::install_github()
and also:
install.packages('odbc', repo="https://cran.rstudio.com/")
I tryed the same with RODBC, and I get the same warnings or errors.
Any ideas on how I can get around this problem?
Thanks in advance.
UPDATE:
If I run this line of code:
devtools::install_github("r-dbi/odbc")`
I get the following error (This is just an extract):
Error: Failed to install 'odbc' from GitHub:
System command error, exit status: 1, stdout + stderr (last 10 lines):
E> ** testing if installed package can be loaded from temporary location
E> Error: package or namespace load failed for ‘odbc’ in dyn.load(file, DLLpath = DLLpath, ...):
E> unable to load shared object '/tmp/RtmpBuMhbW/Rinst32846cdd20a9/00LOCK-odbc/00new/odbc/libs/odbc.so':
E> libodbc.so.2: cannot open shared object file: No such file or directory
E> Error: loading failed
E> Execution halted
E> ERROR: loading failed
E> * removing ‘/tmp/RtmpBuMhbW/Rinst32846cdd20a9/odbc’
E> -----------------------------------
E> ERROR: package installation failed
Traceback:

Alright, after hours of reading AWS/Sagemaker/Conda/R documentation I arrived at the conclusion that, I don't know how to explain the phenomena.
Nonetheless, I have found a workaround!
So we know how we can install packages through the Jupyter terminal or directly from a cell in the .ipynb file. Nonetheless, if you try to run those commands inside the R kernel just like you would on the terminal or in a Python kernel you would run into an error.
Also, for some people installing it in the terminal, or even on a Python kernel might just make conda install the package on a different environment. Yes, turns out there are a couple of environments on your Jupyter instance.
So, to get to the point, just like I said, you would use a conda install command, not a pip, sudo or yum command.
Make your life easier. Here is the code that you would need to run on your R Kernel cell, to install a package directly to the correct environment if, install.packages() and install_github() didn't work:
system(command = 'conda install -c r packageName --yes')
Example:
system(command = 'conda install -c r tidyverse --yes')
And that's it. After that you can call library() and resume your R workflow as you do.
Hope this helps guys!

Related

RGDAL installation error: libnetcdf.so.6: cannot open shared object file: No such file or directory

I run RStudio Server on ubuntu 14.04. When trying to install the rgdal package I get the following error message:
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/admin2/R/x86_64-pc-linux-gnu- library/2.14/rgdal/libs/rgdal.so':
libnetcdf.so.6: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
I upgraded RStudio Server, R-Core and all Ubuntu packages to their latest versions but no success. I also tried apt-get install lib32stdc++6 as suggested here, but also no changes. I think it is a problem specific to GDAL and the indexing of shared libraries in unix. Can anyone help please?
Ok I got it done removing Gdal completly, compiling it from source again and setting library paths correctly as described here. Later adding /usr/local/lib/ to /etc/ld.so.conf and run sudo ldconfig as described here.

How to install R-packages not in the conda repositories?

I am using Anaconda to manage my R-installation. It works great for packages available in the R-channels provided by Anaconda, but I am having troubles installing packages not contained in the Anaconda repos.
I have tried a few different approaches, all listed below together with their error output.
1. install.packages('rafalib')
Suggested to work here conda - How to install R packages that are not available in "R-essentials"?. My .libPaths() points to '/home/user/anaconda2/lib/R/library'.
Out:
--- Please select a CRAN mirror for use in this session ---
Error in download.file(url, destfile = f, quiet = TRUE) :
unsupported URL scheme
Error: .onLoad failed in loadNamespace() for 'tcltk', details:
call: fun(libname, pkgname)
error: Can't find a usable init.tcl in the following directories:
/opt/anaconda1anaconda2anaconda3/lib/tcl8.5 ./lib/tcl8.5 ./lib/tcl8.5 ./library ./library ./tcl8.5.18/library ./tcl8.5.18/library
This probably means that Tcl wasn't installed properly.
I installed tcl from the conda channel r-old, but install.packages() still threw the same error message.
2. options(menu.graphics=FALSE) and then install.packages('rafalib')
I got a list of mirrors and chose one.
Out:
Selection: 15
trying URL 'http://cran.utstat.utoronto.ca/src/contrib/rafalib_1.0.0.tar.gz'
Content type 'application/x-gzip' length 11798 bytes (11 KB)
==================================================
downloaded 11 KB
sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
The downloaded source packages are in
‘/tmp/Rtmphwpta0/downloaded_packages’
Warning message:
In install.packages("rafalib") :
installation of package ‘rafalib’ had non-zero exit status
Both 2 and 3 are from Disable/suppress tcltk popup for CRAN mirror selection in R
3. Setting the mirror in ~/.Rprofile
Before trying install.packages(), I added the following to my ~/.Rprofile.
## Default repo
local({r <- getOption("repos");
r["CRAN"] <- "http://cran.us.r-project.org";
options(repos=r)})
Out:
trying URL 'http://cran.us.r-project.org/src/contrib/rafalib_1.0.0.tar.gz'
Content type 'application/x-gzip' length 11798 bytes (11 KB)
==================================================
downloaded 11 KB
sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
The downloaded source packages are in
‘/tmp/RtmppIz9rT/downloaded_packages’
Warning message:
In install.packages("rafalib") :
installation of package ‘rafalib’ had non-zero exit status
4. Setting the download method to 'curl' or 'wget'.
While keeping the new ~/.Rprofile configuration. I guess this wasn't necessary since the package seems to be downloading fine now, but I tested it just in case.
Out:
sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
Warning in download.packages(pkgs, destdir = tmpd, available = available, :
download of package ‘rafalib’ failed
Warning message:
In download.file(url, destfile, method, mode = "wb", ...) :
download had nonzero exit status
5. Manual download of rafalib
install.packages('../Downloads/rafalib_1.0.0.tar.gz', repos=NULL, type='source')
Out:
sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook
Warning message:
In install.packages("../Downloads/rafalib_1.0.0.tar.gz", repos = NULL, :
installation of package ‘../Downloads/rafalib_1.0.0.tar.gz’ had non-zero exit status
6. Building a conda package from rafalib
I opened a separate issue for this Errors building R-packages for conda. In short, it complains about missing dependencies that I already have installed. Update I got a round the dependency problem and I am now stuck at the same rl_signal_event_hook-error as for my other approaches.
7. sudo ln /usr/lib/libncursesw.so.6 /usr/lib/libncursesw.so.5
As per https://github.com/conda/conda/issues/1679, but it didn't fix the issue for me.
So it seems like I can now download the package fine, but installing it fails. I have seen the error message sh: symbol lookup error: sh: undefined symbol: rl_signal_event_hook previously when using R with irkernel in the Jupyter Notebook, but it has never obstructed my work. I have never seen anything relating to that error message when running python through anaconda.
I'm out of ideas. Does anyone know how I can install R-packages not provided by anaconda, such as rafalib or swirl?
I am on Linux (Antergos, an Arch derivative) with kernel 4.4.5-1-ARCH.
UPDATE 2016/04/15
There is some related discussion in this thread. I have tried to get around this error by installing different versions of ncurses, including this patched version, and I have tried to link the readline libraries, as suggested here, but I keep running into the same error. I'm quite lost at this point and any help to solve this would be greatly appreciated.
Detailed post on managing packages that are and are not in Anaconda R: http://ihrke.github.io/conda.html
Essentially is using commands:
conda skeleton cran <package_name>
conda build <package_name>
If the package has dependencies that are also not in Anaconda:
conda skeleton cran <dependency1>
conda skeleton cran <dependency2>
conda build <package_name>
Essentially I would agree with this post in saying that I don't understand how install.packages() works with Anaconda. What I seem to see is that Anaconda creates a R environment where all the packages installed from install.packages() are kept.
Whenever I am working in Jupyter with R, I use this environment and am able to access all the packages that I have installed with install.packages()
In the end, I got around the rl_event_hookproblems by following the approach recommended here and symlinking anaconda's libreadline to the system one:
mv ~/anaconda3/lib/libreadline.s.6.2 ~/anaconda3/lib/libreadline.s.6.2.bak
ln -s /usr/lib/libreadline.so.6.3 ~/anaconda3/lib/libreadline.s.6.2
I am still having troubles installing some dependency heavy R-packages due to failure to load shared objects when using install.packages() from withing R. However, simpler packages work fine and I can get most of the dependency heavy packages from anacondas R-repositories.

Error while installing "nloptr_1.0.4.tar.gz" from local

When i tried to install nloptr_1.0.4.tar.gz from local, It needed another unix library nlopt-2.4.2.tar.gz which I got it resolved from the following thread Error while installing a tar.gz package in R
But, even after that the package is not getting installed
It throws the following error,
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/cloudera/R/x86_64-redhat-linux-gnu-library/3.2/nloptr/libs/nloptr.so':
/home/cloudera/R/x86_64-redhat-linux-gnu-library/3.2/nloptr/libs/nloptr.so: undefined symbol: nlopt_set_maxtime
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/cloudera/R/x86_64-redhat-linux-gnu-library/3.2/nloptr’
For the above error, I got a lead from the following link:https://cran.r-project.org/web/packages/ROracle/INSTALL
and http://ab-initio.mit.edu/wiki/index.php/NLopt_Installation
Based on the above links, I had to set LD_LIBRARY_PATH variable to /usr/local/lib and /usr/local/include
Even after setting the above Environment variable, R CMD INSTALL nloptr_1.0.4.tar.gz is not getting installed and throwing the same error.
The fix is easy. On a Debian/Ubuntu system do
sudo apt-get install libnlopt-dev
as you always need the development package to compile a given library. It will then be found, and nloptr will install without a hitch.
Similarly on a RH/FC/CentOS with rpms.
Now, when I helped Jelmer rewrite the configuration for nloptr, we did make sure it worked both ways: with an install nlopt library, and without. You seem to stuck in the middle with one that is installed, but different ("missing symbol"). If you cannot uninstall that 'wrong' libnlopt, I would recommend altering the nloptr sources to not check for it but rather always build itself against a copy of nlopt 2.4.2 which it will download and compile statically.

RPostgreSQL installation Error - RPostgreSQL.so: undefined symbol: PQpass

I am trying to install R-Package RPostgreSQL, but getting the following error,
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/lib64/R/library/RPostgreSQL/libs/RPostgreSQL.so':
/usr/lib64/R/library/RPostgreSQL/libs/RPostgreSQL.so: undefined symbol: PQpass
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/usr/lib64/R/library/RPostgreSQL’
I have libpq and postgresql-dev installed. All the library files are in the directory
/usr/lib64/pgsql/ and all header files are in the directory
/usr/include/pgsql/
Postgre Version - 9.3.4
R Version - 3.0.2
Operating System - CentOS-6.4
I am missing some small thing, but unable to find out why this is happening.
What am I doing wrong? How to correct this?
Looks like the pgsql libraries are no longer installed in their previous locations. I linked both:
ln -s /usr/pgsql-9.3/lib /usr/lib/pgsql
ln -s /usr/pgsql-9.3/include /usr/include/pgsql
This worked for me :)
The package RPostgreSQL checks for PostgreSQL libraries only in the following directory paths,
/usr/lib
/usr/lib/pgsql
/usr/lib/postgresql
/usr/local/lib
/usr/local/lib/pgsql
/usr/local/lib/postgresql
/usr/local/pgsql/lib
/usr/local/postgresql/lib
/opt/lib
/opt/lib/pgsql
/opt/lib/postgresql
/opt/local/lib
/opt/local/lib/postgresql
/opt/local/lib/postgresql84
/sw/opt/postgresql-8.4/lib
/Library/PostgresPlus/8.4SS/lib
/sw/lib
It does not check for the directories either in LD_LIBRARY_PATH or in /etc/ld.so.conf.
So RPostgreSQL installation will be successful only if the PostgreSQL libraries exist in any of the above directories.
When I copied the libraries from /usr/lib64/pgsql to /usr/lib/pgsql and tried installing the package. It worked. :)

Problems installing rjags and JAGS on Mac OsX 10.5.8

I am having issues getting rjags the R package for R to call JAGS (Just Another Gibbs Sampler http://www-ice.iarc.fr/~martyn/software/jags/ ).
I am running Mac Os X. 10.5.8 Leopard and using R64 2.12.1. I compiled JAGS from the command line. Then I installed rjags in the R command line.
install.packages(rjags)
When I try to call rjags within R, I get the following error.
> library(rjags)
Error : .onLoad failed in loadNamespace() for 'rjags', details:
call: dyn.load(file, DLLpath = DLLpath, ...)
error: unable to load shared object '/Library/Frameworks/R.framework/Versions/2.12/Resources /library/rjags/libs/x86_64/rjags.so':
dlopen(/Library/Frameworks/R.framework/Versions/2.12/Resources/library/rjags/libs/x86_64/rjags.so, 10): Library not loaded: /usr/local/lib/libjags.2.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/2.12/Resources/library/rjags/libs/x86_64/rjags.so
Reason: no suitable image found. Did find:
/usr/local/lib/libjags.2.dylib: mach-o, but wrong architecture
/usr/local/lib/libjags.2.dylib: mach-o, but wrong architecture
Error: package/namespace load failed for 'rjags'
I am guessing there is some issue with rjags referencing an unknown file in the library, but I am not sure. I am a fairly novice at the command line and messing around with directories, so any help would be much appreciated.
I found (after downloading and installing the dmg from SourceForge) that the following commands worked for me:
wget http://cran.r-project.org/src/contrib/rjags_2.2.0-4.tar.gz
wget http://cran.r-project.org/src/contrib/R2jags_0.02-15.tar.gz
sudo R --with-jags-modules=/usr/local/lib/JAGS/modules-2.2.0 CMD INSTALL rjags_2.2.0-4.tar.gz
sudo R --with-jags-modules=/usr/local/lib/JAGS/modules-2.2.0 CMD INSTALL R2jags_0.02-15.tar.gz
sudo R --with-jags-modules=/usr/local/lib/JAGS/modules-2.2.0 --arch=i386 CMD INSTALL rjags_2.2.0-4.tar.gz
sudo R --with-jags-modules=/usr/local/lib/JAGS/modules-2.2.0 --arch=x86_64 CMD INSTALL rjags_2.2.0-4.tar.gz
They certainly aren't all necessary, but they were sufficient!
I had the same problem while running Rstudio 0.97.237 and R 2.15.2. The issue had something to do with JAGS.
The fix: Update to JAGS 3-3 by downloading and installing it from here.
I encountered the error: unable to load shared object '/Library/Frameworks/R.framework/Resources/modules//R_X11.so': on my Mac and DuckDuckGo led me to this post. I was using R from the command line, so I can't speak to how it works with RStudio although I'm pretty sure it's the fix for that too. I updated JAGS from the SourceForge link provided in another answer. I can at least verify that it worked for me, though the developer was unsigned (I needed to ignore a MacOS warning during the installation procedure).

Resources