R :: override C compiler using Makevars - r

Our R installation define in:
R$HOME/etc/Makeconf that CC = gcc -std=gnu99
I have one specific package (mix of C++ and C code) that need to be compiled using
CC = gcc
without -std=gnu99
As far as I understood I have 3 ways of doing that:
1) system wide, edit R$HOME/etc/Makeconf
2) per user basis, play with ~/.R/Makevars
3) per package basis, set PACKAGE/src/Makevars
Even if 1 and 2 is not what I want I tested the 3 options, using 1 and 2
R CMD INSTALL -l pack.tgz is OK "gcc -std=gnu99" is corectly replaced by "gcc"
But when using PACKAGE/src/Makevars approach it fails
I must admit that I'm lost at this point, where should I look ?
edit.
this is not really a duplicate with Building R Packages using Alternate GCC
for sure I have read the previous one.is the one that pointed me to Makevars
my key concern is that PACKAGE/src/Makevars is not taken in account for CC=alternate compiler while other one are working prefectly.

I had a similar problem in fortran. Anyway I made a model of your package and I found an half solution. It seems that not all the variables in PACKAGE/src/Makevars are considered and used. In order to make it work I used this Makevars file:
MY_PKG_LIBS =
MY_PKG_CCLAGS = -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g
all: $(SHLIB)
hello.o: hello.c
gcc $(MY_PKG_CCLAGS) -c hello.c -o hello.o $(MY_PKG_LIBS)
PKG_LIB = -std=gnu++11
Obviously the hello.c file should be replaced by your_file_name.c. If you can not change the CC and you adopt my workaround, the real problem become that when the shared file .so is created, the compiler flags should be overwritten using PKG_CFLAGS or PKG_CPPFLAGS as stated in Writing R Extensions (again in the Makevars file). In my personal situation (Ubuntu 15.04, R 3.1.2), I tried those and other vars following the guidelines in /etc/R/Makeconf file:
ALL_CFLAGS = $(R_XTRA_CFLAGS) $(PKG_CFLAGS) $(CPICFLAGS) $(SHLIB_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(R_XTRA_CPPFLAGS) $(PKG_CPPFLAGS) $(CPPFLAGS) $(CLINK_CPPFLAGS)
The only thing that worked in adding a flag to the -shared final compiling of the package was adding the library linker flag (As I did originally with my fortran code) PKG_LIB = -std=gnu++11 in the PACKAGE/src/Makevars.
My final result in installing the package is:
Installing package into ‘/home/home/R/x86_64-pc-linux-gnu-library/3.1’
(as ‘lib’ is unspecified)
* installing *source* package ‘question1’ ...
** libs
gcc -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c hello.c -o hello.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o question1.so hello.o -std=gnu++11 -L/usr/lib/R/lib -lR
installing to /home/dgarolini/R/x86_64-pc-linux-gnu-library/3.1/question1/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (question1)

Related

R package RcppArmadillo "has a configure script" and "probably needs manual configuration"

I just updated my R packages packages in RStudio. For the RcppArmadillo package (version 0.11.4.3.1), RStudio warned me that no binary package is available for my system (x86_64, Windows 11) and asked if I wanted to install the source package instead. Since I have RTools installed for just this purpose, I said yes.
The installation succeeded, but a warning was issued that the package "has a configure script" and "probably needs manual configuration".
* installing *source* package 'RcppArmadillo' ...
** package 'RcppArmadillo' successfully unpacked and MD5 sums checked
** using staged installation
**********************************************
WARNING: this package has a configure script
It probably needs manual configuration
**********************************************
** libs
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c RcppArmadillo.cpp -o RcppArmadillo.o
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"C:/PROGRA~1/R/R-42~1.2/include" -DNDEBUG -I'C:/.../R/win-library/4.2/Rcpp/include' -I"C:/rtools42/x86_64-w64-mingw32.static.posix/include" -I../inst/include -I. -fopenmp -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c fastLm.cpp -o fastLm.o
g++ -shared -s -static-libgcc -o RcppArmadillo.dll tmp.def RcppArmadillo.o RcppExports.o fastLm.o -fopenmp -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lRlapack -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lRblas -lgfortran -lm -lquadmath -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/rtools42/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-42~1.2/bin/x64 -lR
installing to C:/.../R/win-library/4.2/00LOCK-RcppArmadillo/00new/RcppArmadillo/libs/x64
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (RcppArmadillo)
My questions are:
Should I be concerned?
How can I tell that the package is working correctly?
If manual configuration is needed, how do I do it?
An aside for the second of those: I'm not using this package directly, and presume that it was installed as a dependency of another package that I used in the past. Since I do not remember which package that may have been (or when I last used it), I can't just dig into my older code, find a script that uses this package, and see if it still works.
Is there such a thing in the R world as automated tests to ensure that a newly-compiled package actually works before it gets installed? I'm a little surprised that this is apparently not done by default.

Shared .so file not found when loading R package with Rcpp component Linux systems

I apologize in advance if this question has been asked before but I haven't been able to find a suitable answer after days of searching.
I have a package that has a couple of small functions written in C++ using Rcpp (package can be found here). When trying to install this package on Linux systems (both local install and via Github Actions - Ubuntu 18.04, 20.04) the compiling procedure was successful but R cannot load the package because an .so file was not found. Mac and Windows systems work fine and there were no R CMD CHECK issues when tested on these systems.
I was wondering if there is any way to solve this issue? I was thinking of adding something in .Makevars but I don't know what to add.
Thank you so much in advance.
─ installing *source* package ‘CBEA’ ...
** using staged installation
** libs
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/local/lib/R/include" -DNDEBUG -I'/__w/_temp/Library/Rcpp/include' -I/usr/local/include -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -Wall -pedantic -c gmeans.cpp -o gmeans.o
g++ -std=gnu++11 -shared -L/usr/local/lib/R/lib -L/usr/local/lib -o CBEA.so RcppExports.o gmeans.o -L/usr/local/lib/R/lib -lR
installing to /tmp/RtmpwFufM0/Rinst8982598ffd0c/00LOCK-CBEA/00new/CBEA/libs
** R
** data
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘CBEA’ in library.dynam(lib, package, package.lib):
shared object ‘cbea.so’ not found
Error: loading failed
Execution halted
ERROR: loading failed
─ removing ‘/tmp/RtmpwFufM0/Rinst8982598ffd0c/CBEA’
Your (generated) NAMESPACE file has two entries referring to the package:
useDynLib(CBEA, .registration = TRUE)
useDynLib(cbea, .registration = TRUE)
Remove the second one (i.e. remove whereever you tell roxygen2 to add it, and you will be fine. The rror message you see and quote indicates a case issue as on Linux, cbea.so is different from CBEA.so (whereas macOS and Windows let you get away with it).

R cannot load libraries calling lapack functions

I'm trying to install some of the popular R packages but somehow my R does not see the lapack installation. Or rather the compiler doesn't see it, I guess, because when it checks LAPACK_LIBS it says it found it. I checked sessionInfo() and La_version() and they both point to where lapack is installed. How do I even go about solving it?
> install.packages("RcppArmadillo")
Installing package into ‘/homeUPDATES
/marcin/R/x86_64-pc-linux-gnu-library/4.1’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/RcppArmadillo_0.10.5.0.0.tar.gz'
Content type 'application/x-gzip' length 1669395 bytes (1.6 MB)
==================================================
downloaded 1.6 MB
* installing *source* package ‘RcppArmadillo’ ...
** package ‘RcppArmadillo’ successfully unpacked and MD5 sums checked
** using staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++14 accepts -g... yes
checking how to run the C++ preprocessor... g++ -std=gnu++14 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -std=gnu++14 accepts -g... (cached) yes
checking whether we have a suitable tempdir... /tmp
checking whether R CMD SHLIB can already compile programs using OpenMP... yes
checking LAPACK_LIBS... system LAPACK found
configure: creating ./config.status
config.status: creating inst/include/RcppArmadilloConfigGenerated.h
config.status: creating src/Makevars
** libs
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I'/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I../inst/include -fpic -g -O2 -fdebug-prefix-map=/build/r-base-tbZjLv/r-base-4.1.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppArmadillo.cpp -o RcppArmadillo.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I'/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I../inst/include -fpic -g -O2 -fdebug-prefix-map=/build/r-base-tbZjLv/r-base-4.1.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -I"/usr/share/R/include" -DNDEBUG -I'/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/Rcpp/include' -I../inst/include -fpic -g -O2 -fdebug-prefix-map=/build/r-base-tbZjLv/r-base-4.1.0=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c fastLm.cpp -o fastLm.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -L/usr/local/opt/gsl/lib -lgsl -llapack -lgslcblas -o RcppArmadillo.so RcppArmadillo.o RcppExports.o fastLm.o -L/usr/local/opt/gettext/lib -L/usr/lib/R/lib -lR
installing to /home/marcin/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-RcppArmadillo/00new/RcppArmadillo/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘RcppArmadillo’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-RcppArmadillo/00new/RcppArmadillo/libs/RcppArmadillo.so':
/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/00LOCK-RcppArmadillo/00new/RcppArmadillo/libs/RcppArmadillo.so: undefined symbol: dgesvx_
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/marcin/R/x86_64-pc-linux-gnu-library/4.1/RcppArmadillo’
Warning in install.packages("RcppArmadillo") :
installation of package ‘RcppArmadillo’ had non-zero exit status
The downloaded source packages are in
‘/tmp/Rtmpw0yc9E/downloaded_packages’
>
UPDATE 1:
I made sure that the blas and lapack .so files have 755 permissions but it didn't help.
UPDATE 2:
The same problem occurs when I try to build from source a package that needs lapack. For example, I tried installing the GpGp package with the following Makevars
PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(SHLIB_OPENMP_CXXFLAGS)
PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS)
and it gives the same problem.
Try sudo apt-get update in terminal. Does it say something is locked? It looks to me like something else is using the depenent files and thus the process is locked. If this is the case you can either wait for the process to be done or reboot.
You can also try to install forcing no lock:
install.packages("RcppArmadillo",INSTALL_opts = '--no-lock')
You also might also try installing via remotes and the github page which will get you the latest dev version rather than the cran version:
library(remotes)
remotes::install_github("RcppCore/RcppArmadillo")
Please install openblas-dev libraries:
sudo apt -y install libopenblas-dev
The symbol dgesvx_ is not found. These are the extern C defines, that wrap to the fortran drivers. Then grepping for the symbol you'll find it here:
~> grep dgesvx_ /usr/lib/x86_64-linux-gnu/liblapack.so.3
grep: /usr/lib/x86_64-linux-gnu/liblapack.so.3: binary file matches
So eventually I figured out that the problem was my ~/.R/Makevars file. Dirk's comment under my original question made me wonder where do these extra commands come from and I found them in Makevars. I don't remember when and why I added something to it but when I reverted back to just an empty Makevars file everything worked! Thanks Dirk!

Issues compiling Rpackage: error in asNamespace(ns) using Rcpp

Working on a small Rcpp package to use Boost and some of its geometry functions in C.
Finished writing the functions and everything was working well. Tested that everything was working properly (Clean and Rebuild and testing the functions) one last time before pushing up to GitHub. Once I double-checked the directory was all cleanly stored in GitHub I removed the directory from my local machine. It should also be noted that I have Roxygen2 running on this and managing the NAMESPACE file.
Upon cloning the directory back and Clean and Rebuild I get the following error:
==> Rcpp::compileAttributes()
* Updated R/RcppExports.R
==> R CMD INSTALL --preclean --no-multiarch --with-keep.source MinimumRcpp
* installing to library ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library’
* installing *source* package ‘MinimumRcpp’ ...
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include -fPIC -Wall -g -O2 -c RcppExports.cpp -o RcppExports.o
** libs
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include -fPIC -Wall -g -O2 -c findParetoSet.cpp -o findParetoSet.o
clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include -fPIC -Wall -g -O2 -c kintersection.cpp -o kintersection.o
clang++ -std=gnu++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o MinimumRcpp.so RcppExports.o findParetoSet.o kintersection.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.5/Resources/library/MinimumRcpp/libs
** R
** byte-compile and prepare package for lazy loading
Error in asNamespace(ns) : not a namespace
Error : unable to load R code in package ‘MinimumRcpp’
ERROR: lazy loading failed for package ‘MinimumRcpp’
* removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/MinimumRcpp’
Exited with status 1.
Of course, nothing changed in the code. However, I do have the following .gitignore files:
# /.gitignore
inst/lib
.Rproj.user ## Could this be an issue?
.Rproj.user/ ## Could this be an issue?
.Rproj.user/** ## Could this be an issue?
src/*.o
src/*.so
src/*.dll
src/symbols.rds
.Rhistory
.RData
.DS_Store
## QtCreator
Rcpp.pro
Rcpp.pro.user
*.autosave
.#*
*.tar.gz
vignettes/*_cache
## GNU global
GPATH
GRTAGS
GTAGS
##
local/
## docker helpers
docker/*sh
docker/*/*.sh
# /.Rbuildignore
debian
inst/doc/html
inst/doc/latex
inst/doc/man
inst/doc/jss.bst
vignettes/jss.bst
^.*\.Rproj$
^\.Rproj\.user$
vignettes/notyet
doxyfile
\.travis\.yml
\.dir-locals\.el
\.clang_format
vignettes/getCurrentVersionsOfCitedPackages.R
^Contributing.md$
^\.codecov\.yml$
inst/include/Rcpp.h.new
inst/include/Rcpp.h.old
LICENSE
.*\.tar\.gz$
\.editorconfig
docker
^GPATH
^GRTAGS
^GTAGS
^local
vignettes/Makefile
vignettes/rmd
\.github
Is it possible that some package data that was necessary for building the package was left behind and is causing the current issues?
Any assistance would be appreciated. Additionally, any advice on how to debug this would help as the current output messages are quite nebulous and are leaving me directionless.
Here is the GitHub repo for reproducibility: https://github.com/eduardoRubioG/MinimumRcpp
Rookie mistake, but a serious one:
never ever keep script files in your R/ directory
everything (and we mean everything) in the directory gets sourced (provided it looks like R code, _i.e. end in .R)
you left a script with a library(MinimumRcpp) call in there so now your package byte-code compilation wants to source itself --> not a good plan
in short, keep such scripts but put them in e.g. local/ and exclude local/ via .Rbuildignore.
Plus an important style lesson
do not leave rm(list = ls(all = TRUE)) in your code
So if you do mv R/script.R R/script.R.txt and rebuild, all is good.
(I get a half-dozen warnings because n is not a size_t but compared to one. You may want to cast it earlier.)

Compiling an adjusted package in Windows

I am trying to create a package from an existed package called "rDEA". Basically, I have adjusted and added some functions to the source .R files and the DESCRIPTION file to include Depends:. However, when I checked the package I got the following 00install.out:
* installing *source* package 'rDEA' ...
** libs
*** arch - i386
gcc -m32 -I"C:/PROGRA~1/R/R-32~1.2/include" -DNDEBUG -g -D_R_=1 -DUSE_R=1 -I/include -DCHECK_GLPK_ARGS -I"d:/RCompile/r-compiling/local/local320/include" -O3 -Wall -std=gnu99 -mtune=core2 -c multi_glp_solve.c -o multi_glp_solve.o
gcc -m32 -I"C:/PROGRA~1/R/R-32~1.2/include" -DNDEBUG -g -D_R_=1 -DUSE_R=1 -I/include -DCHECK_GLPK_ARGS -I"d:/RCompile/r-compiling/local/local320/include" -O3 -Wall -std=gnu99 -mtune=core2 -c rDEA_initialize.c -o rDEA_initialize.o
gcc -m32 -shared -s -static-libgcc -o rDEA.dll tmp.def multi_glp_solve.o rDEA_initialize.o -L/lib -lglpk -lgmp -Ld:/RCompile/r-compiling/local/local320/lib/i386 -Ld:/RCompile/r-compiling/local/local320/lib -LC:/PROGRA~1/R/R-32~1.2/bin/i386 -lR
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -lglpk
c:/rtools/gcc-4.6.3/bin/../lib/gcc/i686-w64-mingw32/4.6.3/../../../../i686-w64-mingw32/bin/ld.exe: cannot find -lgmp
collect2: ld returned 1 exit status
no DLL was created
ERROR: compilation failed for package 'rDEA'
* removing 'C:/Users/Dell/Documents/rDEA.Rcheck/rDEA'
Here, it seems that the files lglpk and lgmp are not found when compiling. They are created from Makevars.win in the src as follows:`
#-*- Makefile -*-
#
PKG_CPPFLAGS=-g -D_R_=1 -DUSE_R=1 -I${GLPK_HOME}/include -DCHECK_GLPK_ARGS
PKG_LIBS=-L${GLPK_HOME}/lib -lglpk -lgmp
My questions are: How to solve this issue? Do I need to change directory? Or do I need to use GSL (GNU Scientific Library) or other compilers? and how?
Please excuse my ignorance as I am no a computer scientist nor had good programming background.
Your help would be greatly appreciated. Many thanks!!

Resources