I am preparing a package (which makes use of Rcpp and RcppArmadillo) for submission to CRAN. I am using Travis CI and I have included the following to test my package more thoroughly:
os:
- linux
- osx
compiler:
- clang
- gcc
This, however, yields the (familiar, it seems) error when using clang on OS X
* installing *source* package ‘my_pkg’ ...
** libs
clang++ -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I"/Users/travis/R/Library/Rcpp/include" -I"/Users/travis/R/Library/RcppArmadillo/include" -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c RcppExports.cpp -o RcppExports.o
clang: error: unsupported option '-fopenmp'
make: *** [RcppExports.o] Error 1
ERROR: compilation failed for package ‘pkg’
* removing ‘/Users/travis/build/my_user/pkg/pkg.Rcheck/pkg’
By appropriately adjusting my travis.yml file to adhere to one of the workarounds suggested here, it seems like I can avoid the issue.
My question is: what should I do if I want to put this on CRAN? Jim's suggestions seems to me only local, do I have to do something in my package to automatically avoid the compile issue for the user in order to be accepted by CRAN?
Edit: To be a little bit more specific, could I just put CXX=g++ in Makevars, or is that frowned upon (and would that be sufficient)?
So ... one step at a time, please. Which RcppArmadillo version are you pulling in here? Which clang version are you pulling in? Where is that clang version from? Does it have OpenMP support?
Starting with the previous release of RcppArmadillo, we ensured we simply turn OpenMP off on macOS as the build support is too fragile, see this PR on GitHub. So you may have to remove unconditional use of -fopenmp in your files, or simple not use that "cell" of your Matrix (ie clang && macOS).
Otherwise, the answer by Jim Hester is spot-on. Things are just more work on macOS.
Related
I've recently installed R version 4.0.5 (2021-03-31) onto Ubuntu 20.04.2 LTS. R is working as expected.
However, while some packages have installed without issue (e.g., R.matlab), there are several packages that have not been able to install. As as example, running install.packages("data.table") throws the following error:
* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
gcc -std=gnu99 9.3.0
zlib 1.2.11 is available ok
R CMD SHLIB supports OpenMP without any extra hint
** libs
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -fopenmp -fpic -g -O2 -fdebug-prefix-map=/build/r-base-tRgc13/r-base-4.0.5=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c assign.c -o assign.o
during GIMPLE pass: ccp
assign.c: In function ‘memrecycle’:
assign.c:1205:1: internal compiler error: Segmentation fault
1205 | }
| ^
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-9/README.Bugs> for instructions.
make: *** [/usr/lib/R/etc/Makeconf:172: assign.o] Error 1
ERROR: compilation failed for package ‘data.table’
I get similar errors (i.e., "internal compiler error") with other package installation attempts, but with slightly different output. For example, install.packages("xfun") throws the following error, truncated for brevity:
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -fpic -g -O2 -fdebug-prefix-map=/build/r-base-tRgc13/r-base-4.0.5=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c base64.c -o base64.o
during GIMPLE pass: ccp
base64.c: In function ‘base64_decode_impl’:
base64.c:237:1: internal compiler error: Segmentation fault
237 | }
| ^
I have tried reinstalling gcc-9, which has not helped. I had found that the PATH in Sys.getenv("PATH") was pointing to a miniconda3 install on my system, so updated that to /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/rstudio/bin/postback, which also has not helped.
My goal is to install packages among those above without throwing these compiler errors. Any help would be greatly appreciated.
In case others also run into this issue in the future, I am posting the solution that was suggested to me by https://answers.launchpad.net/ubuntu. Here is the link to the question I posted: https://answers.launchpad.net/ubuntu/+question/696623.
The issue turned out to be that R was using gcc-9 rather than gcc-10 to compile packages. The older version of gcc was throwing an error. Here are the steps I took to solve the problem:
Install gcc-10, which was not available on my system: sudo apt install gcc-10.
Edit the CC= pointer in the /usr/lib/R/etc/Makeconf file to gcc-10: open Terminal, type sudo nano /usr/lib/R/etc/Makeconf and replace the current CC= to CC=gcc-10. Save the file.
Restart R and run the install.packages() command for those packages that were not compiling correctly.
EDIT: Please see the comments below for a discussion. The above steps resolve the issue, but are not recommended. The issue was related to R not using the system package manager to install packages when the call install.packages() was used in R.
Installing the package bspm solved the issue for me. This package and its utility is discussed here for those who are curious.
To use install.packages() within R, bspm can be used in two ways:
bspm::enable() within R and then install.packages()
As written in its documentation: To enable bspm system-wide by default, include the following: suppressMessages(bspm::enable()) into the Rprofile.site file.
Thanks very much to Dirk for his guidance.
Since I installed Catalina, any attempt to compile an R package fails. My system is set up to use home-brew. The failure messages look like this:
/usr/local/clang8/bin/clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I/usr/local/clang8/include -I/usr/local/Cellar/gcc/8.3.0_2/include -fopenmp -fPIC -Wall -g -O2 -c R_register_native.c -o R_register_native.o
In file included from R_register_native.c:1:
/Library/Frameworks/R.framework/Resources/include/R.h:55:11: fatal error: 'stdlib.h' file not found
# include <stdlib.h> /* Not used by R itself, but widely assumed in packages */
After googling I found this link in the home-brew site on GitHub. I can't follow much of the discussion but it seems that Apple removed the usr/include directory and that the home-brew folks are looking for a solution. I don't immediately have a need to compile these packages so my plan is to wait until home-brew finds a solution. But if there is an easy one now, please share.
After some additional searching I found this link with a bunch of discussion about this problem, a lot of which I didn't understand. But this bit of code seemed promising.
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
I ran it, rebooted my mac, opened RStudio and got the list of new package updates. Ran the update, answering yes to compile. Lots of messages and some warnings but at the end of it all, there were no errors. But all the messages start with something like
/usr/local/clang8/bin/clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I/usr/local/clang8/include -I/usr/local/Cellar/gcc/8.3.0_2/include -fopenmp -fPIC -Wall -g -O2 -c R_register_native.c -o R_register_native.o
When I cd into /usr/local/clang8/bin and get the version of clang it is Apple clang version 11.0.0. Also clang symlinks to clang-8 and clang++ symlinks to clang
The contents of /usr/local/Cellar/gcc is a directory called 9.2.0_1. When I go to /usr/bin and run gcc --version I get messages about this being Apple clang version 11.0.0. It seems like R is compiling using the Apple clang version. Is that a problem? Is there some way to move everything to using the brew version of clang? FYI, here's my current path - /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Library/TeX/texbin:/opt/X11/bin:/usr/local/CrossPack-AVR/bin:/usr/local/git/bin
I am trying to submit a package to CRAN that fails during the pretest
process on Debian.
I use some C++ code that interfaces to the nlopt optimization
library with Rcpp/RcppArmadillo (using <nlopt.hpp>). Thus, my package requires a system version of nlopt that can be installed via the deb package libnlopt-dev (I added this to the SystemRequirements field).
I use a configure script and pkg-config to retrieve the (hopefully)
appropriate compiler flags which I then send to src/Makevars
NLOPT_LIBS=`pkg-config --libs ${PKG_CONFIG_NAME}`
NLOPT_FLAG=`pkg-config --cflags ${PKG_CONFIG_NAME}`
This approach works on my system (Ubuntu 18.04, R 3.5.2) and on Travis
(linux and mac OS), but fails on the Debian CRAN server :
https://win-builder.r-project.org/incoming_pretest/PLNmodels_0.7_20190119_161032/Debian/00install.out
I added some verbosity to check that the flags were correctly exported
on CRAN servers, and they seems alright to me: at least, a version of
nlopt is found on the system (see below, NLOPT_LIBS=-lnlopt -lm).
However, the <nlopt.hpp> is not found latter on...
Any help would be greatly appreciated. Thank for taking the time to read this.
* installing *source* package ‘PLNmodels’ ...
Using NLOPT_LIBS=-lnlopt -lm
Using NLOPT_FLAG=
** libs
g++-8 -std=gnu++11 -I"/home/hornik/tmp/R/include" -DNDEBUG -fopenmp
-I"/home/hornik/lib/R/Library/3.6/x86_64-linux-gnu/Rcpp/include"
-I"/home/hornik/lib/R/Library/3.6/x86_64-linux-gnu/RcppArmadillo/include"
-I/usr/local/include -fopenmp -fpic -g -O2 -Wall -pedantic
-mtune=native -c RcppExports.cpp -o RcppExports.o
g++-8 -std=gnu++11 -I"/home/hornik/tmp/R/include" -DNDEBUG -fopenmp
-I"/home/hornik/lib/R/Library/3.6/x86_64-linux-gnu/Rcpp/include"
-I"/home/hornik/lib/R/Library/3.6/x86_64-linux-gnu/RcppArmadillo/include"
-I/usr/local/include -fopenmp -fpic -g -O2 -Wall -pedantic
-mtune=native -c call_nlopt_PLN_VE.cpp -o call_nlopt_PLN_VE.o
In file included from call_nlopt_PLN_VE.cpp:6:
utils.h:5:10: fatal error: nlopt.hpp: No such file or directory
#include <nlopt.hpp>
^~~~~~~~~~~
compilation terminated.
make: *** [/home/hornik/tmp/R/etc/Makeconf:173: call_nlopt_PLN_VE.o] Error 1
ERROR: compilation failed for package ‘PLNmodels’
* removing ‘/srv/hornik/tmp/CRAN/PLNmodels.Rcheck/PLNmodels’
The C++ interface for nlopt has migrated to a separate package in Debian, c.f. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=855600. So it looks like that particular CRAN machine uses either Debian Testing or Stable + Backports and has only libnlopt-dev but not libnlopt-cxx-dev installed. You will have to contact the CRAN maintainers to install this dependency.
BTW, ideally such errors should be cought in the configure script, i.e. you should try to compile a simple program that requires nlopt.hpp.
You asked the same question here on r-package-devel and I had no good answer for you.
In essence, you have fine variation within one required upstream package. That is very delicate. I cannot think of anything but what #Ralf suggested: test for it.
Or maybe you just fail in communicating the requirements of your package to CRAN. But that would be an even simpler question, and when the wrong version is encountered, you are bound to fail installation.
So overall, my recommendation is the same as previously: work with Jelmer, and wait for him to release an updated nloptr release. Until then, all solutions are bound to be fudges as you cannot control what version of nloptr your package may encounter.
I created RcppArmadillo package on windows (Win 7, 64 bit) using R 3.1.2 and the latest versions of Rcpp 0.11.3 and RcppArmadillo 0.4.600.0. Package compiles, installs, and runs fine. Afterwards i wrote some code to use BigMatrix from bigmemory package with my RcppArmadillo package. As it turns out bigmemory is not available for R 3.1.2 on windows. To use bigmemory i downgraded R to R 2.15.3 and got Rcpp 0.10.4 and RcppArmadillo 0.4.000.2 for it from CRAN. Now i can use bigmemory but i cannot get my RcppArmadillo package to compile. Basically, it barfs a long list of error messages starting with
C:/Program Files/R/R-2.15.3/library/RcppArmadillo/include/RcppArmadilloForward.h:99:33: error: 'ConstReferenceInputParameter' is not a template
The errors look like a version incompatibility issue RcppArmadillo "ReferenceInputParameter is not a template" for which Dirk recommended installing Rcpp 0.10.5 from source. So i found tarball of Rcpp 0.10.5 and installed it manually. Problem is that i still get a ton of different error messages
g++ -m32 -I"C:/PROGRA~1/R/R-215~1.3/include" -DNDEBUG -I"C:/Program Files/R/R-2.15.3/library/Rcpp/include" -I"C:/Program Files/R/R-2.15.3/library/RcppArmadillo/include" -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O2 -Wall -mtune=core2 -c RcppExports.cpp -o RcppExports.o
g++ -m32 -I"C:/PROGRA~1/R/R-215~1.3/include" -DNDEBUG -I"C:/Program Files/R/R-2.15.3/library/Rcpp/include" -I"C:/Program Files/R/R-2.15.3/library/RcppArmadillo/include" -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O2 -Wall -mtune=core2 -c rcpparma_hello_world.cpp -o rcpparma_hello_world.o
g++ -m32 -shared -s -static-libgcc -o rcppArma.dll tmp.def RcppExports.o rcpparma_hello_world.o -LC:/PROGRA~1/R/R-215~1.3/bin/i386 -lRlapack -LC:/PROGRA~1/R/R-215~1.3/bin/i386 -lRblas -lgfortran -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/PROGRA~1/R/R-215~1.3/bin/i386 -lR
RcppExports.o:RcppExports.cpp:(.text+0x82): undefined reference to `Rcpp::internal::enterRNGScope()'
RcppExports.o:RcppExports.cpp:(.text+0xa7): undefined reference to `SEXPREC* Rcpp::internal::r_true_cast<14>(SEXPREC*)'
RcppExports.o:RcppExports.cpp:(.text+0xc9): undefined reference to `Rcpp::RObject::RObject(SEXPREC*)'
RcppExports.o:RcppExports.cpp:(.text+0xf5): undefined reference to `Rcpp::traits::storage_type<14>::type* Rcpp::internal::r_vector_start<14>(SEXPREC*)'
RcppExports.o:RcppExports.cpp:(.text+0x199): undefined reference to `Rcpp::traits::storage_type<14>::type* Rcpp::internal::r_vector_start<14>(SEXPREC*)'
I suspect it is still version incompatibility issue, but i have no idea how to match Rcpp version and RcppArmadillo version. Is there a compatibility table, could someone give a suggestion?
All CRAN packages have an Archive/ subdirectory. By picking the date of the last available bigmemory package you can find that corresponding package for Rcpp and RcppArmadillo: the most recent release prior to that particular bigmemory release. You can then download and install these (provided you are set up to install from source, which I presume you are, given the context).
There is some ongoing work by some folks giving 'snap-shotting' ability but I am not aware of a high-level helper function you could use right now.
Edit: What you should probably do here, though, is to take a recent bigmemory source package and install from source.
The checkpoint package can create a local R library that contains packages as of a given date.
There is a tutorial [here].
I am writing an R package that has Fortran source code. In short, my problem is when I try to install the package I get the error "gfortran -m32: not found" however, I am able compile my code using gfortran -m32 when I'm not installing the package.
Detailed Version:
I have installed the newest version of R(3.1.1) along with the newest version of Rtools(31) and renamed my path variables accordingly (C:\Rtools\bin;C:\Rtools\gcc-4.6.3\bin;C:\Program Files\R\R-3.1.1\bin\x64;C:\msys;C:\Program Files (x86)\SSH Secure Shell;C:\Program Files\MiKTeX 2.9\miktex\bin).
For testing purposes, I downloaded the PropClust package from CRAN which has Fortran source code.
When I type "R CMD INSTALL PropClust_1.4.tar.gz" after building it, I get the following error:
* installing *source* package 'PropClust' ...
** libs
*** arch - i386
C:/PROGRA~1/R/R-31~1.1/etc/i386/Makeconf:202: warning: overriding recipe for target `.m.o'
C:/PROGRA~1/R/R-31~1.1/etc/i386/Makeconf:195: warning: ignoring old recipe for target `.m.o'
gfortran -m32 -O3 -mtune=core2 -c PropClustParallelTrials.f90 -o PropClustParallelTrials.o
gcc -m32 -I"C:/PROGRA~1/R/R-31~1.1/include" -DNDEBUG -I"d:/RCompile/CRANpkg/extralibs64/local/include" -O3 -Wall -std=gnu99 -mtune=core2 -c minWhichMin.c -o minWhichMin.o
gfortran -m32 -shared -s -static-libgcc -o PropClust.dll tmp.def PropClustParallelTrials.o minWhichMin.o -Ld:/RCompile/CRANpkg/extralibs64/local/lib/i386 -Ld:/RCompile/CRANpkg/extralibs64/local/lib -LC:/PROGRA~1/R/R-31~1.1/bin/i386 -lR
gfortran -m32: not found
C:/PROGRA~1/R/R-31~1.1/etc/i386/Makeconf:202: warning: overriding recipe for target `.m.o'
C:/PROGRA~1/R/R-31~1.1/etc/i386/Makeconf:195: warning: ignoring old recipe for target `.m.o'
cygwin warning:
MS-DOS style path detected: C:/PROGRA~1/R/R-31~1.1/bin/i386/Rterm.exe
Preferred POSIX equivalent is: /cygdrive/c/PROGRA~1/R/R-31~1.1/bin/i386/Rterm.exe
CYGWIN environment variable option "nodosfilewarning" turns off this warning.
Consult the user's guide for more details about POSIX paths:
http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
no DLL was created
ERROR: compilation failed for package 'PropClust'
* removing 'C:/Users/kamicz/Documents/IsotopeR/PropClust.Rcheck/PropClust'
Although it says "gfortran -m32: not found" I have compiled my code in the same window with the same options(gfortran -m32 -O3 -mtne=core2 -c a.f90 -o a.out) with no problems. Has anyone had this problem before? I am running Windows 7 64-bit if that matters.
I had a similar problem : R was invoking gfortran-4.8 - an unrecognized command on my system - instead of gfortran. This problem was fixed upon creating the file ~/.R/Makevars within my home directory. Here are the contents of Makevars which points to the appropriate gfortran installation:
F77 = gfortran
FC = gfortran
FLIBS = -L/usr/local/gfortran/lib