Shared .so file not found when loading R package with Rcpp component Linux systems - r

I apologize in advance if this question has been asked before but I haven't been able to find a suitable answer after days of searching.
I have a package that has a couple of small functions written in C++ using Rcpp (package can be found here). When trying to install this package on Linux systems (both local install and via Github Actions - Ubuntu 18.04, 20.04) the compiling procedure was successful but R cannot load the package because an .so file was not found. Mac and Windows systems work fine and there were no R CMD CHECK issues when tested on these systems.
I was wondering if there is any way to solve this issue? I was thinking of adding something in .Makevars but I don't know what to add.
Thank you so much in advance.
─ installing *source* package ‘CBEA’ ...
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c gmeans.cpp -o gmeans.o
g++ -std=gnu++11 -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o CBEA.so RcppExports.o gmeans.o -L/usr/local/lib/R/lib -lR
installing to /tmp/RtmpwFufM0/Rinst8982598ffd0c/00LOCK-CBEA/00new/CBEA/libs
** R
** data
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘CBEA’ in library.dynam(lib, package, package.lib):
shared object ‘cbea.so’ not found
Error: loading failed
Execution halted
ERROR: loading failed
─ removing ‘/tmp/RtmpwFufM0/Rinst8982598ffd0c/CBEA’

Your (generated) NAMESPACE file has two entries referring to the package:
useDynLib(CBEA, .registration = TRUE)
useDynLib(cbea, .registration = TRUE)
Remove the second one (i.e. remove whereever you tell roxygen2 to add it, and you will be fine. The rror message you see and quote indicates a case issue as on Linux, cbea.so is different from CBEA.so (whereas macOS and Windows let you get away with it).

Related

R package RcppArmadillo "has a configure script" and "probably needs manual configuration"

I just updated my R packages packages in RStudio. For the RcppArmadillo package (version 0.11.4.3.1), RStudio warned me that no binary package is available for my system (x86_64, Windows 11) and asked if I wanted to install the source package instead. Since I have RTools installed for just this purpose, I said yes.
The installation succeeded, but a warning was issued that the package "has a configure script" and "probably needs manual configuration".
* installing *source* package 'RcppArmadillo' ...
** package 'RcppArmadillo' successfully unpacked and MD5 sums checked
** using staged installation
**********************************************
WARNING: this package has a configure script
It probably needs manual configuration
**********************************************
** libs
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c RcppArmadillo.cpp -o RcppArmadillo.o
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c fastLm.cpp -o fastLm.o
g++ -shared -s -static-libgcc -o RcppArmadillo.dll tmp.def RcppArmadillo.o RcppExports.o fastLm.o -fopenmp -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lRlapack -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lRblas -lgfortran -lm -lquadmath -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lR
installing to C:/.../R/win-library/4.2/00LOCK-RcppArmadillo/00new/RcppArmadillo/libs/x64
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppArmadillo)
My questions are:
Should I be concerned?
How can I tell that the package is working correctly?
If manual configuration is needed, how do I do it?
An aside for the second of those: I'm not using this package directly, and presume that it was installed as a dependency of another package that I used in the past. Since I do not remember which package that may have been (or when I last used it), I can't just dig into my older code, find a script that uses this package, and see if it still works.
Is there such a thing in the R world as automated tests to ensure that a newly-compiled package actually works before it gets installed? I'm a little surprised that this is apparently not done by default.

ROracle on R Version 4.0.0

When trying to use ROracle I get the following error message
> library("ROracle")
Error: package or namespace load failed for ‘ROracle’:
package ‘ROracle’ was installed before R 4.0.0: please re-install it
I've installed/downloaded DBI, RTools, oracle isntant client (basic + sdk) and tried following these instructions but it did not work: ROracle install
I'm able to download ROracle just fine (see below) so I'm stumped on what is missing. Do I need to downgrade a version?
> Roracle_path <- "filepath.../ROracle_1.3-2.zip"
> install.packages(Roracle_path, repos = NULL)
Installing package into ‘C:/Users/<my username>/Documents/R/win-library/4.0’
(as ‘lib’ is unspecified)
package ‘ROracle’ successfully unpacked and MD5 sums checked
ROracle_1.3-2.tar.gz which is built for R 3.6.x works also with R 4.x if you install the package from SOURCES.
On Linux systems at first you have to install the Oracle Instant Client basic and SDK:
/home/jovyan/R
unzip instantclient-basic-linux.x64-21.4.0.0.0dbru.zip -d /opt/oracle/
unzip instantclient-sdk-linux.x64-21.4.0.0.0dbru.zip -d /opt/oracle
The linux package libaio1 have to be installed on the system:
apt install -y libaio1
Next, the environment variables must be set:
export PATH="$PATH:/opt/oracle/instantclient_21_4"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/oracle/instantclient_21_4"
Finally the package can be installed from Sources (use ROracle_1.3-2.tar.gz (sources) but not ROracle_1.3-2_R_x86_64-linux-gnu.tar.gz (binary) ):
R CMD INSTALL --configure-args='--with-oci-lib=/opt/oracle/instantclient_21_4/ --with-oci-inc=/opt/oracle/instantclient_21_4/sdk/include' ROracle_1.3-2.tar.gz
During the installation process you will see that the package will be installed from SOURCES:
* installing to library ‘/opt/conda/lib/R/library’
* installing *source* package ‘ROracle’ ...
** using staged installation
configure: creating ./config.status
config.status: creating src/Makevars
** libs
x86_64-conda-linux-gnu-cc -I"/opt/conda/lib/R/include" -DNDEBUG -I/opt/oracle/instantclient_21_4/sdk/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/conda/include -I/opt/conda/include -Wl,-rpath-link,/opt/conda/lib -fpic -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/r-base-split_1639563404388/work=/usr/local/src/conda/r-base-4.1.2 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix -c rodbi.c -o rodbi.o
x86_64-conda-linux-gnu-cc -I"/opt/conda/lib/R/include" -DNDEBUG -I/opt/oracle/instantclient_21_4/sdk/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /opt/conda/include -I/opt/conda/include -Wl,-rpath-link,/opt/conda/lib -fpic -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /opt/conda/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/r-base-split_1639563404388/work=/usr/local/src/conda/r-base-4.1.2 -fdebug-prefix-map=/opt/conda=/usr/local/src/conda-prefix -c rooci.c -o rooci.o
x86_64-conda-linux-gnu-cc -shared -L/opt/conda/lib/R/lib -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,-rpath,/opt/conda/lib -Wl,-rpath-link,/opt/conda/lib -L/opt/conda/lib -o ROracle.so rodbi.o rooci.o -L/opt/oracle/instantclient_21_4/ -lclntsh -Wl,-rpath,/opt/oracle/instantclient_21_4/ -L/opt/conda/lib/R/lib -lR
installing to /opt/conda/lib/R/library/00LOCK-ROracle/00new/ROracle/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (ROracle)
#
Check the line in the output to see it will be installed from Sources:
* installing *source* package ‘ROracle’ ...
I downgraded to R 3.6.3 and used RTools 35 and now it works.

Error during installation of readxl in RStudio

I am using R version 3.4.4 (2018-03-15) and RStudio Version 1.1.456 and I am getting this error when I am trying to install the readxl extension:
* installing *source* package ‘utf8’ ...
** package ‘utf8’ successfully unpacked and MD5 sums checked
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -Iutf8lite/src -fpic -g -O2 -fdebug-prefix-map=/build/r-base-AitvI6/r-base-3.4.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c as_utf8.c -o as_utf8.o
as_utf8.c:17:10: fatal error: assert.h: No such file or directory
#include <assert.h>
^~~~~~~~~~
compilation terminated.
This happened with me because of outdated R version.
Kindly update your R and try again. For instructions regarding updating R, follow this.
Make sure you update R to 3.5.2.

R :: override C compiler using Makevars

Our R installation define in:
R$HOME/etc/Makeconf that CC = gcc -std=gnu99
I have one specific package (mix of C++ and C code) that need to be compiled using
CC = gcc
without -std=gnu99
As far as I understood I have 3 ways of doing that:
1) system wide, edit R$HOME/etc/Makeconf
2) per user basis, play with ~/.R/Makevars
3) per package basis, set PACKAGE/src/Makevars
Even if 1 and 2 is not what I want I tested the 3 options, using 1 and 2
R CMD INSTALL -l pack.tgz is OK "gcc -std=gnu99" is corectly replaced by "gcc"
But when using PACKAGE/src/Makevars approach it fails
I must admit that I'm lost at this point, where should I look ?
edit.
this is not really a duplicate with Building R Packages using Alternate GCC
for sure I have read the previous one.is the one that pointed me to Makevars
my key concern is that PACKAGE/src/Makevars is not taken in account for CC=alternate compiler while other one are working prefectly.
I had a similar problem in fortran. Anyway I made a model of your package and I found an half solution. It seems that not all the variables in PACKAGE/src/Makevars are considered and used. In order to make it work I used this Makevars file:
MY_PKG_LIBS =
MY_PKG_CCLAGS = -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g
all: $(SHLIB)
hello.o: hello.c
gcc $(MY_PKG_CCLAGS) -c hello.c -o hello.o $(MY_PKG_LIBS)
PKG_LIB = -std=gnu++11
Obviously the hello.c file should be replaced by your_file_name.c. If you can not change the CC and you adopt my workaround, the real problem become that when the shared file .so is created, the compiler flags should be overwritten using PKG_CFLAGS or PKG_CPPFLAGS as stated in Writing R Extensions (again in the Makevars file). In my personal situation (Ubuntu 15.04, R 3.1.2), I tried those and other vars following the guidelines in /etc/R/Makeconf file:
ALL_CFLAGS = $(R_XTRA_CFLAGS) $(PKG_CFLAGS) $(CPICFLAGS) $(SHLIB_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(R_XTRA_CPPFLAGS) $(PKG_CPPFLAGS) $(CPPFLAGS) $(CLINK_CPPFLAGS)
The only thing that worked in adding a flag to the -shared final compiling of the package was adding the library linker flag (As I did originally with my fortran code) PKG_LIB = -std=gnu++11 in the PACKAGE/src/Makevars.
My final result in installing the package is:
Installing package into ‘/home/home/R/x86_64-pc-linux-gnu-library/3.1’
(as ‘lib’ is unspecified)
* installing *source* package ‘question1’ ...
** libs
gcc -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c hello.c -o hello.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o question1.so hello.o -std=gnu++11 -L/usr/lib/R/lib -lR
installing to /home/dgarolini/R/x86_64-pc-linux-gnu-library/3.1/question1/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (question1)

compiling R package on ubuntu, invalid ELF header

I am no longer able to compile an R package from source, and this has never been a problem before today. This is what happens:
R CMD build BAMMtools
* checking for file ‘BAMMtools/DESCRIPTION’ ... OK
* preparing ‘BAMMtools’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* looking to see if a ‘data/datalist’ file should be added
* building ‘BAMMtools_2.1.0.tar.gz’
R CMD INSTALL BAMMtools
* installing to library ‘/usr/local/lib/R/site-library’
* installing *source* package ‘BAMMtools’ ...
** libs
make: Nothing to be done for 'all'.
installing to /usr/local/lib/R/site-library/BAMMtools/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/usr/local/lib/R/site-library/BAMMtools/libs/BAMMtools.so':
/usr/local/lib/R/site-library/BAMMtools/libs/BAMMtools.so: invalid ELF header
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/usr/local/lib/R/site-library/BAMMtools’
* restoring previous ‘/usr/local/lib/R/site-library/BAMMtools’
This does not seem to be a problem with this R package in particular, as I am able to install the package from within R, as:
> install.packages('BAMMtools_2.1.0.tar.gz', repos=NULL)
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘BAMMtools’ ...
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c cohort_matrix.c -o cohort_matrix.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c dtrates.c -o dtrates.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c jenksBrks.c -o jenksBrks.o
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c treetraverse.c -o treetraverse.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o BAMMtools.so cohort_matrix.o dtrates.o jenksBrks.o treetraverse.o -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/BAMMtools/libs
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (BAMMtools)
I am running R v3.2.2, under Ubuntu v15.10 wily. I can't think of anything I've done within the last 24 hours that would have this effect, except updating my system with sudo apt-get update/upgrade.
Although this is not too much of a problem for installing the package, I can't run the R CMD check commands either, and this is a problem for development.
Any suggestions or advice would be greatly appreciated! Thanks!
As was found in the comments to the question, thanks to Jack Wasey, the problem was simply that I was running R CMD INSTALL on the name of the source directory, rather than on the built .tar.gz file.

Resources