I am no longer able to compile an R package from source, and this has never been a problem before today. This is what happens:
R CMD build BAMMtools
* checking for file ‘BAMMtools/DESCRIPTION’ ... OK
* preparing ‘BAMMtools’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* looking to see if a ‘data/datalist’ file should be added
* building ‘BAMMtools_2.1.0.tar.gz’
R CMD INSTALL BAMMtools
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘BAMMtools’ ...
** libs
make: Nothing to be done for 'all'.
installing to /usr/local/lib/R/site-library/BAMMtools/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/local/lib/R/site-library/BAMMtools/libs/BAMMtools.so':
/usr/local/lib/R/site-library/BAMMtools/libs/BAMMtools.so: invalid ELF header
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/usr/local/lib/R/site-library/BAMMtools’
* restoring previous ‘/usr/local/lib/R/site-library/BAMMtools’
This does not seem to be a problem with this R package in particular, as I am able to install the package from within R, as:
> install.packages('BAMMtools_2.1.0.tar.gz', repos=NULL)
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘BAMMtools’ ...
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c cohort_matrix.c -o cohort_matrix.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c dtrates.c -o dtrates.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c jenksBrks.c -o jenksBrks.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c treetraverse.c -o treetraverse.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o BAMMtools.so cohort_matrix.o dtrates.o jenksBrks.o treetraverse.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/BAMMtools/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (BAMMtools)
I am running R v3.2.2, under Ubuntu v15.10 wily. I can't think of anything I've done within the last 24 hours that would have this effect, except updating my system with sudo apt-get update/upgrade.
Although this is not too much of a problem for installing the package, I can't run the R CMD check commands either, and this is a problem for development.
Any suggestions or advice would be greatly appreciated! Thanks!
As was found in the comments to the question, thanks to Jack Wasey, the problem was simply that I was running R CMD INSTALL on the name of the source directory, rather than on the built .tar.gz file.
Related
I apologize in advance if this question has been asked before but I haven't been able to find a suitable answer after days of searching.
I have a package that has a couple of small functions written in C++ using Rcpp (package can be found here). When trying to install this package on Linux systems (both local install and via Github Actions - Ubuntu 18.04, 20.04) the compiling procedure was successful but R cannot load the package because an .so file was not found. Mac and Windows systems work fine and there were no R CMD CHECK issues when tested on these systems.
I was wondering if there is any way to solve this issue? I was thinking of adding something in .Makevars but I don't know what to add.
Thank you so much in advance.
─ installing *source* package ‘CBEA’ ...
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c gmeans.cpp -o gmeans.o
g++ -std=gnu++11 -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o CBEA.so RcppExports.o gmeans.o -L/usr/local/lib/R/lib -lR
installing to /tmp/RtmpwFufM0/Rinst8982598ffd0c/00LOCK-CBEA/00new/CBEA/libs
** R
** data
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘CBEA’ in library.dynam(lib, package, package.lib):
shared object ‘cbea.so’ not found
Error: loading failed
Execution halted
ERROR: loading failed
─ removing ‘/tmp/RtmpwFufM0/Rinst8982598ffd0c/CBEA’
Your (generated) NAMESPACE file has two entries referring to the package:
useDynLib(CBEA, .registration = TRUE)
useDynLib(cbea, .registration = TRUE)
Remove the second one (i.e. remove whereever you tell roxygen2 to add it, and you will be fine. The rror message you see and quote indicates a case issue as on Linux, cbea.so is different from CBEA.so (whereas macOS and Windows let you get away with it).
Upon executing 'Rscript -e "install.packages('mclust')"` with R 4.0.1 on Ubuntu 20.04, the process hangs during package preparation. Any idea how to fix or debug this issue?
ENV: R v4.0.1, GCC v9, Ubuntu v20.04
CMD: Rscript -e "install.packages('mclust')"
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/mclust_5.4.6.tar.gz'
Content type 'application/x-gzip' length 2877519 bytes (2.7 MB)
==================================================
downloaded 2.7 MB
* installing *source* package ‘mclust’ ...
** package ‘mclust’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gfortran -fno-optimize-sibling-calls -fpic -g -O2 -fdebug-prefix-map=/build/r-base-dEscXG/r-base-4.0.1=. -fstack-protector-strong -c dmvnorm.f -o dmvnorm.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -fpic -g -O2 -fdebug-prefix-map=/build/r-base-dEscXG/r-base-4.0.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c init.c -o init.o
gfortran -fno-optimize-sibling-calls -fpic -g -O2 -fdebug-prefix-map=/build/r-base-dEscXG/r-base-4.0.1=. -fstack-protector-strong -c mclust.f -o mclust.o
gfortran -fno-optimize-sibling-calls -fpic -g -O2 -fdebug-prefix-map=/build/r-base-dEscXG/r-base-4.0.1=. -fstack-protector-strong -c mclustaddson.f -o mclustaddson.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o mclust.so dmvnorm.o init.o mclust.o mclustaddson.o -llapack -lblas -lgfortran -lm -lquadmath -lgfortran -lm -lquadmath -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/00LOCK-mclust/00new/mclust/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
Based on the response on R-sig-debian for a similar issue, I figured this issue is due to a bug in the pthread-based versions of Blas and Lapack libraries.
Until the bug is fixed, the workaround is to use libopenblas-openmp-blas and libopenblas-openmp-lapack libraries.
There are two ways to achieve use these libraries.
Uninstall the pthread-based version of the libraries and install the openmp-based version of the libraries.
Switch to openmp-based version of the libraries via update-alternatives. This can be done via sudo update-alternatives --config <name> command with being liblapack.so.3-x86_64-linux-gnu and libblas.so.3-x86_64-linux-gnu.
I need to access an Azure SQL Database from an R notebook in Databricks. To do this I aimed to use the odbc package, which installed fine on my local instance of R.
I have tried to install the package to the cluster using Databricks' interface, which always fails. I have also tried the following code within a notebook:
install.packages("odbc")
which results in:
Installing package into ‘/databricks/spark/R/lib’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/odbc_1.1.6.tar.gz'
Content type 'application/x-gzip' length 288033 bytes (281 KB)
==================================================
downloaded 281 KB
* installing *source* package ‘odbc’ ...
** package ‘odbc’ successfully unpacked and MD5 sums checked
PKG_CFLAGS=
PKG_LIBS=-lodbc
<stdin>:1:17: fatal error: sql.h: No such file or directory
compilation terminated.
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because odbc was not found. Try installing:
* deb: unixodbc-dev (Debian, Ubuntu, etc)
* rpm: unixODBC-devel (Fedora, CentOS, RHEL)
* csw: unixodbc_dev (Solaris)
* brew: unixodbc (Mac OSX)
To use a custom odbc set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘odbc’
* removing ‘/databricks/spark/R/lib/odbc’
The downloaded source packages are in
‘/tmp/RtmpqHp2QM/downloaded_packages’
I have also tried installing from github:
library(devtools)
devtools::install_github("r-dbi/odbc")
Which gives a different error:
Downloading GitHub repo r-dbi/odbc#master
Installing 3 packages: assertthat, BH, Rcpp
Installing packages into ‘/databricks/spark/R/lib’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/assertthat_0.2.1.tar.gz'
Content type 'application/x-gzip' length 12742 bytes (12 KB)
==================================================
downloaded 12 KB
trying URL 'https://cloud.r-project.org/src/contrib/BH_1.69.0-1.tar.gz'
Content type 'application/x-gzip' length 12378154 bytes (11.8 MB)
==================================================
downloaded 11.8 MB
trying URL 'https://cloud.r-project.org/src/contrib/Rcpp_1.0.1.tar.gz'
Content type 'application/x-gzip' length 3661123 bytes (3.5 MB)
==================================================
downloaded 3.5 MB
* installing *source* package ‘assertthat’ ...
** package ‘assertthat’ successfully unpacked and MD5 sums checked
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (assertthat)
* installing *source* package ‘BH’ ...
** package ‘BH’ successfully unpacked and MD5 sums checked
** inst
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (BH)
* installing *source* package ‘Rcpp’ ...
** package ‘Rcpp’ successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Date.cpp -o Date.o
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Module.cpp -o Module.o
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Rcpp_init.cpp -o Rcpp_init.o
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c api.cpp -o api.o
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c attributes.cpp -o attributes.o
g++ -I/usr/share/R/include -DNDEBUG -I../inst/include/ -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c barrier.cpp -o barrier.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o Rcpp.so Date.o Module.o Rcpp_init.o api.o attributes.o barrier.o -L/usr/lib/R/lib -lR
installing to /databricks/spark/R/lib/Rcpp/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (Rcpp)
The downloaded source packages are in
‘/tmp/RtmpqHp2QM/downloaded_packages’
Error in processx::run(bin, args = real_cmdargs, stdout_line_callback = real_callback(stdout), :
System command error
In addition: Warning messages:
1: In install.packages("odbc") :
installation of package ‘odbc’ had non-zero exit status
2: In install.packages("odbc") :
installation of package ‘odbc’ had non-zero exit status
Any idea why this package will not install on Databricks when it works fine locally and when all other packages I have tried install on Databricks work fine using the same syntax?
Best option to access a SQL Database is using the preinstalled JDBC connectivity (see the Documentation).
If you want to use ODBC this requires (as mentioned in one of the comments) unix odbc. Good practice to install multiple packages is using init-scripts. Following python code is for creating an init-script for pyodbc installation.
script = """
sudo apt-get -q -y install unixodbc unixodbc-dev
sudo apt-get -q -y install python3-dev
sudo pip install pyodbc
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
sudo curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get -q -y install msodbcsql
"""
dbutils.fs.put("/databricks/init/pyodbc/pyodbc.sh", script, True)
Hope this helps.
While installing the package RcppArmadillo, I am getting below error.
installing to /usr/lib64/R/library/RcppArmadillo/libs
** R
** inst
** preparing package for lazy loading
Error in get(name, envir = asNamespace(pkg), inherits = FALSE) :
object 'SHLIB.maker' not found
Error : unable to load R code in package ‘RcppArmadillo’
ERROR: lazy loading failed for package ‘RcppArmadillo’
* removing ‘/usr/lib64/R/library/RcppArmadillo’
Warning message:
In install.packages("RcppArmadillo_0.3.920.1.tar.gz", repos = NULL, :
installation of package ‘RcppArmadillo_0.3.920.1.tar.gz’ had non-zero exit status
You need to add R's repository to your system:
Use your favorite text editor (I'm using gedit as an example) to open /etc/apt/sources.list:
sudo gedit /etc/apt/sources.list
Add this line to the file (if this is slow, use another mirror. You may also want to change precise into the codename for your Ubuntu version --- e.g., trusty for 12.04):
deb http://cran.rstudio.com/bin/linux/ubuntu precise/
This works for me install.packages('RcppArmadillo')
> install.packages('RcppArmadillo')
Installing package into ‘/home/priyankara/R/i686-pc-linux-gnu-library/3.2’
(as ‘lib’ is unspecified)
trying URL 'http://mirror.bjtu.edu.cn/cran/src/contrib/RcppArmadillo_0.5.300.4.tar.gz'
Content type 'application/octet-stream' length 967193 bytes (944 KB)
==================================================
downloaded 944 KB
* installing *source* package ‘RcppArmadillo’ ...
** package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked
* checking LAPACK_LIBS: divide-and-conquer complex SVD available via system LAPACK
** libs
g++ -I/usr/share/R/include -DNDEBUG -I"/home/priyankara/R/i686-pc-linux-gnu-library/3.2/Rcpp/include" -I../inst/include -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c RcppArmadillo.cpp -o RcppArmadillo.o
g++ -I/usr/share/R/include -DNDEBUG -I"/home/priyankara/R/i686-pc-linux-gnu-library/3.2/Rcpp/include" -I../inst/include -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o
g++ -I/usr/share/R/include -DNDEBUG -I"/home/priyankara/R/i686-pc-linux-gnu-library/3.2/Rcpp/include" -I../inst/include -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c fastLm.cpp -o fastLm.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o RcppArmadillo.so RcppArmadillo.o RcppExports.o fastLm.o -llapack -lblas -lgfortran -lm -lquadmath -L/usr/lib/R/lib -lR
installing to /home/priyankara/R/i686-pc-linux-gnu-library/3.2/RcppArmadillo/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (RcppArmadillo)
I am getting error while installing topicmodels package in R.
on running install.packages("topicmodels",dependencies=TRUE) following are the last few lines I am getting. Please help. My R version is 3.1.3.
g++ -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c utilities.cpp -o utilities.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c utils.c -o utils.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o topicmodels.so cokus.o common.o ctm.o dataset.o gsl-wrappers.o inference.o lda-alpha.o lda-inference.o lda-model.o model.o rGibbslda.o rctm.o rlda.o utilities.o utils.o -lgsl -lgslcblas -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/topicmodels/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/local/lib/R/site-library/topicmodels/libs/topicmodels.so':
libgsl.so.0: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/usr/local/lib/R/site-library/topicmodels’
Warning message:
In install.packages("http://cran.r-project.org/src/contrib/topicmodels_0.2-1.tar.gz", :
installation of package ‘/tmp/RtmpbX6uuH/downloaded_packages/topicmodels_0.2-1.tar.gz’ had non-zero exit status
I have searched for this error at many places but haven't found solution. Please help
You don't have GSL installed:
sudo apt-get install gsl-bin libgsl0-dev