Certain R packages, such as mgcv, fail to compile from source with clang 10+ (under macOS 10.14 and R version 3.6+). The error reported during compilation is
ld: unknown option: -platform_version
How do I resolve this error and compile these packages?
The problem with ld is the same as in Clang 10 fails to link C++ application with CMake on macOS 10.12. However, the suggestion to add the flag -DCMAKE_CXX_FLAGS="-mlinker-version=305" is not applicable to the R package compilation process. For R, you need to add -mlinker-version=305 to LDFLAGS to your Makevars file, typically located in $HOME/.R/.
My Makevars is based on this GitHub gist. I changed LDFLAGS from this:
LDFLAGS+=-L$(HO)/llvm/lib -Wl,-rpath,$(HO)/llvm/lib
to this:
LDFLAGS+=-L$(HO)/llvm/lib -Wl,-rpath,$(HO)/llvm/lib -mlinker-version=305
That resolved the ld error when compiling mgcv from source.
For the igraph package, adding the mlinker flag to LDFLAGS was not enough; it had to be added to the C++ flags as well. In the gist Makevars above, this is done by adding -mlinker-version=305 to STD_FLAGS, which then adds the flag to CXX**FLAGS for all C++ versions.
UPDATE, June 24 2020: unfortunately, some packages (rJava in my case) fail to use the STD_FLAGS. My workaround was to put the mlinker flag in the C compiler invocation:
CC=$(CCACHE) $(HO)/llvm/bin/clang -mlinker-version=305
Related
I'd like to keep my OS, R, and R packages up to date. I unknowingly treaded into deep water by upgrading to OSX 10.15.6 and upgrading to R 4.0.2. Currently, Rcpp is failing to compile c++ code, which I believe is causing the CRAN installation for certain packages to fail (e.g. glmmTMB), and is also causing installations from source to fail. I'll describe what I've done and hopefully someone can elucidate a solution.
In a rough order, here's what I've done:
Installed OSX 10.15.6
I don't remember the date but it was fairly recent. I don't know how closely this is related to the problem, I was having issues installing other R packages (e.g. rstan) that needed Rcpp prior to this.
Ran into an issue trying to run some old glmmTMB models
The exact error was identical to this issue. I followed the various solutions on that thread to no avail. Perhaps the most frustrating was when attempting to install from source, which then failed to compile some c++.
This wasn't the first time I had seen similar errors. Something similar happened when I tried installing rstan, so I asked for some help on their repo under a similar issue. Nothing helped.
This was on R 3.6, so I thought maybe it'd be worth updating R and glmmTMB. Before doing this I wanted to install Xcode, as opposed to just the Command Line Developer Tools, in hopes that the installation would ensure I'd have everything clang- and c++ compiler-related squared away.
Installed Xcode to help set a baseline c++ environment
Relatively painless, am able to build and compile fairly simple c++ projects.
Installed R 4.0.2
From the r-project site, I installed R-4.0.2.pkg without issues and with the default install settings. According to the R project Tools site, I was reassured in installing Xcode and continued on to install the GNU Fortran 8.2 installer.
Followed #coatless's guide to installing Rcpp
Discovered this guide and followed each step, except for those related to xcode-select.
Testing Rcpp and RcppArmadillo from the guide
One of the last steps is to test the installation with a simple helloworld.cpp test. This fails with this error (abridged for clarity):
> Rcpp::sourceCpp("~/Documents/BergenLab/nlp_cancer_metaphor/helloworld.cpp")
In file included from helloworld.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadillo.h:31:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include/RcppArmadilloForward.h:26:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/RcppCommon.h:29:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/r/headers.h:67:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include/Rcpp/platform/compiler.h:100:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: error: no member named 'signbit' in the global namespace
using ::signbit;
~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [helloworld.o] Error 1
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppArmadillo/include" -I"/Users/alex/Documents/BergenLab/nlp_cancer_metaphor" -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c helloworld.cpp -o helloworld.o
Error in Rcpp::sourceCpp("~/Documents/BergenLab/nlp_cancer_metaphor/helloworld.cpp") :
Error 1 occurred building shared library.
Tried to debug Rcpp to no avail
Found some somewhat related reports, such as Packages cannot build from source due to math.h not found, Rcpp doesn't under macOS, math.h not found, Source Cpp file failed due to math.h not found, and Cannot compile R packages with c++ code after updating to macOS Catalina.
None of the solutions in these answers have helped at all. The only solution which gave me feedback was using installer to install macOS_SDK_headers_for_macOS_10.14.pkg which subsequently gave me installer: Error - the package path specified was invalid (not sure how what the implications of this are).
Installing glmmTMB from source fails
One of the solutions from an above glmmTMB issue recommends installing from source. Here is the error on my attempt:
* installing *source* package ‘glmmTMB’ ...
** package ‘glmmTMB’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/TMB/include' -I'/Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include' -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c glmmTMB.cpp -o glmmTMB.o
In file included from glmmTMB.cpp:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/TMB/include/TMB.hpp:53:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Dense:1:
In file included from /Library/Frameworks/R.framework/Versions/4.0/Resources/library/RcppEigen/include/Eigen/Core:96:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/complex:245:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cmath:317:9: error: no member named 'signbit' in the global namespace
using ::signbit;
~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
13 warnings and 20 errors generated.
make: *** [glmmTMB.o] Error 1
ERROR: compilation failed for package ‘glmmTMB’
* removing ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/glmmTMB’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library/glmmTMB’
Warning in install.packages :
installation of package ‘glmmTMB’ had non-zero exit status
Overall, I'm still unable to run my glmmTMB models (and I presume unable to build rstan as well). Something with my c++ compiler must be off but I'm a c++ beginner and am unsure how to interpret the errors I'm getting. Something's wrong with the header files, but it seems perhaps like it's not due to math.h being missing (like most other posts). It should go without saying but any help would be greatly appreciated!
Edit 1
I've tried to narrow down the error and thus followed each step from #coatless's guide, exactly as listed there and according to the R installation documentation. My exact steps:
Removed gfortran 8.2. Removed ~/.Renviron and ~/.R/Makevars. Unlinked both of these files from Rstudio with unlink().
Ran xcode-select --install, which returned with "use "Software Update" to install updates." Accordingly, softwareupdate --list estimated that nothing needed updating. Regardless, I uninstalled Command Line Tools with sudo rm -rf /Library/Developer/CommandLineTools and installed the Command Line Tools with xcode-select —install. Up to this point everything is working as expected. Small c++ programs are being compiled without issue by both gcc and clang++.
According to the R manual, from https://mac.r-project.org/libs-4/, I installed pcre2-10.34-darwin.17-x86_64.tar.gz, xz-5.2.4-darwin.17-x86_64.tar.gz , and readline-5.2.14-darwin.17-x86_64.tar.gz to /usr/local. No big issues here, although I'm not really sure how to verify that these work as expected.
Restarted an R session in RStudio.
Installed Rcpp and RcppArmadillo with install.packages(c(‘Rcpp’, ‘RcppArmadillo’))
Created a ~/helloworld.cpp with the code from Slide 15 mentioned in the comments.
Running Rcpp::sourceCpp("~/helloworld.cpp") fails with the exact error as mentioned in the body of the post. Similarly evalCpp() fails with a similar error.
After a bit of digging, the issue turned out to be neither glmmTMB, Rcpp, or really anything high-level. From what I understand: When R uses Rcpp to compile c++ code, a certain set of arguments are passed to the system c++ compiler. In my case this compiler was clang++ since I had just installed Xcode. The c++ compiler relies on a set of c++ header files to be included in exactly the right order. Seemed like the order of mine were incorrect, thus causing a slew of errors regarding "missing template definitions."
I looked into changing the order of the includes but couldn't find any straightforward solution. I knew that Rcpp was calling a very specific array of arguments to clang++, so I looked into how that might elucidate an answer. I looked into the manuals for clang++ to figure out what -isysroot and -I meant and felt like they might play a part in reconfiguring how the system header files were being compiled. From one of the threads below I discovered the $(R RHOME)/etc/Makeconf Makefile and proceeded to change the directory of the OSX SDK from what I believe was the Command Line Tools path to the Xcode path. This seemed to solve the issue.
Some threads I used to find this solution: Cannot compile R packages with c++ code after updating to macOS Catalina and Catalina C++: Using <cmath> headers yield error: no member named 'signbit' in the global namespace.
What worked for me:
Note the current header file search
From clang++ -Wp,-v -E -:
#include <...> search starts here:
/usr/local/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
Note the OSX SDK path
From xcrun --show-sdk-path:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
Modify the R configuration
With vim $(R RHOME)/etc/Makeconf, changed the CPPFLAGS flag to:
CPPFLAGS = -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/usr/local/include
Notice how this SDK directory is what was found in step 2. This apparently cleans up the location and order of the SDK header file includes.
Reinstall glmmTMB from source
I was still seeing a FreeADFunObject error when trying to fit a glmmTMB model, so I went back to this GH thread and followed the first suggestion: to reinstall from source. This wasn't working before since I needed a properly set up c++ configuration, which I fixed in the last two steps.
Hence, install.packages("glmmTMB", type="source") installed without issue and I can now build and fit glmmTMB models.
For posterity's sake:
> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.6
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
In any R package I try to install, I get the following error message:
ld: library not found for -lintl
collect2: error: ld returned 1 exit status
make: *** [utf8.so] Error 1
I'm not sure how to make this a reprex, but I am running:
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin19.5.0 (64-bit)
Running under: macOS Catalina 10.15.5
What does this error message (ld: library not found for -lintl) mean and how can I fix it to be able to install R packages (the packages I have tried are texreg and lme4)? Thank you.
Edit: I apologize, I thought I had included this with my post. I am not missing gettext (as per Link error installing Rcpp "library not found for -lintl"), and I followed the instructions to export the LPDFLAGS and CPPFLAGS so that they are linked.
As best I can tell, the problem is the ~/.R/Makevars file, which currently looks like this, where I've commented out things I've added to it based on googling.
CC=/usr/local/Cellar/gcc/9.3.0_1/bin/gcc-9
CXX=/usr/local/Cellar/gcc/9.3.0_1/bin/g++-9
CXX11=/usr/local/Cellar/gcc/9.3.0_1/bin/g++-9
CXX14=/usr/local/Cellar/gcc/9.3.0_1/bin/g++-9
cxx17=/usr/local/cellar/gcc/9.3.0_1/bin/g++-9
cxx1X=/usr/local/cellar/gcc/9.3.0_1/bin/g++-9
LDFLAGS=-L/usr/local/Cellar/gcc/9.3.0_1/lib
#FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin18/8.2.0
#CC=/usr/local/clang4/bin/clang
#CXX=/usr/local/clang4/bin/clang++
#CXX1X=/usr/local/clang4/bin/clang++
#CXX98=/usr/local/clang4/bin/clang++
#CXX11=/usr/local/clang4/bin/clang++
#CXX14=/usr/local/clang4/bin/clang++
#CXX17=/usr/local/clang4/bin/clang++
#LDFLAGS=-L/usr/local/clang4/lib
Apparently, I needed to remove all the lines from ~/.R/Makevars and I was able to install lme4. I did get some warnings, but library(lme4) works.
If you're using a mac with an Apple Silicon chip (like the M1), then you can try adding this to ~/.R/Makevars:
CFLAGS=-I/opt/homebrew/include
CPPFLAGS=-I/opt/homebrew/include
CXXFLAGS=-I/opt/homebrew/include
CXX11FLAGS=-I/opt/homebrew/include
LDFLAGS=-L/opt/homebrew/lib
This allows R to find the needed libraries. You will need to first ensure you have the needed compilers and dependencies installed. You can usually do this with brew.
I am working on an R package containing Fortran source files. The structure of the Fortran code is rather complex, with many dependencies, therefore I have a Makefile in the src folder for the compilation of the shared library.
So far I have been compiling this package on my machine running Ubuntu 14.04, without any problems. I am now trying to compile it on Windows using Rtools, and I am running into a problem when linking the objects to produce the shared library. More precisely, the linker does not find the functions REXIT, RWARN and RCHKUSR, as I get the following error messages:
undefined reference to `rchkusr_'
undefined reference to `rexit_'
undefined reference to `rwarn_'
These functions are supposed to be in the shared library R.dll (libR.so under Linux). I checked this library with Dependencies Walker and I could find these functions. I have tried to link with -I C:/Program\ Files/R/R-3.1.1/bin/i386/R.dll to make explicit the reference to the shared library. I have tried to recompile with -fno-underscoring to make sure the underscores were not the problem. Nothing helped.
Any ideas where the problem could come from? Any suggestions would be more than welcome!
Under Linux I did not have to do anything special, the linker found the functions without specifying anything in the Makefile.
I am using R version 3.1.1 and Rtools version 3.1.
Many thanks for your help.
After fiddling with the Makefile, I finally found the solution to my problem: The flags I was trying to pass to the linker were wrong. Only the path of the shared library R.dll should be specified, using -L, and the name of the library should be specified using the flag -lR at the end of the command. In short, I specified
gfortran -shared *.o -o myLib.dll -L C:/Program\ Files/R/R-3.1.1/bin/i386 -lR
to create myLib.dll and it worked! I guess I got confused because under Linux there is no need to specify anything, the linker finds the shared library on its own.
Probably not the best and most portable solution I can get (I now have two Makefiles to deal with, Makefile for Linux and Makefile.win for Windows...), but good enough for now.
I'm trying in vain to compile the armadillo linear algebra library for windows. Using the armadillo-4.200.0 source, I have Rtools-3.1 installed and in the path, msys from MinGW installed.
Because my ultimate goal is to use Rcpp and RcppArmadillo, my thought is that I need to use the same compiler for making armadillo as will be used to compile my Rcpp/RcppArmadillo files. Unfortunately, when trying to compile armadillo:
$ ./configure
[...snip...]
-- The CXX compiler identification is unknown
-- Check for working CXX compiler: cl
CMake Warning at CMakeLists.txt:3 (PROJECT):
To use the NMake generator, cmake must be run from a shell that can use the
compiler cl from the command line. This environment does not contain
INCLUDE, LIB, or LIBPATH, and these must be set for the cl compiler to
work.
CMake Error: your CXX compiler: "cl" was not found. Please set CMAKE_CXX_COMPILER to a valid compiler path or name.
[...snip...]
(I tried setting CMAKE_CXX_COMPILER to my Rtools gcc.exe with no luck.)
I believe that it is looking for the visual C compiler, though the docs imply that it can be done solely with the mingw or cygwin compiler environments.
Either:
Is there a problem with compiling the armadillo library and subsequent Rcpp code with different versions of the compiler? (Rtools-3.1 has gcc version 4.6.3, MinGW has gcc version 4.8.1.)
Is there a clean method for compiling armadillo with just the Rtools collection?
(Win7 x64, R-3.1.0, cygwin gcc 4.8.2, rtools 3.1 with gcc 4.6.3.)
RcppArmadillo ships its own copy of Armadillo to avoid exactly this problem. As RcppArmadillpo is used from R, it can rely on R (and R's configuration) to get LAPACK, BLAS, etc pp. We don't need to run configure to use Armadillo from R, and so we don't do.
RcppArmadillo installs the usual R CMD INSTALL ... way; this is tested before every release and has worked reliably.
As you say "your ultimate goal is to use Rcpp and RcppArmadillo", you are in fact done at the R CMD INSTALL .... You can replicate the step from source, and test the package -- all that should "just work".
The other thing to keep in mind is what the "Writing R Extensions" and "R Installation and Administration" manuals have to say about your compiler. As far is R is concerned, your g++ 4.8.* does not exist. Only the Rtools version matters, or you get into non-standard land real quick.
I made a simple fortran routine
subroutine add(x,y)
real(8) :: x,y
y = x + 3
end subroutine
saved as test.f90.
I compile with
gfortran -shared test.f90 -o test.so
In R (in the same directory), I use
dyn.load('test.so')
but it gives me this error:
Error in dyn.load("test.so") :
unable to load shared object '/Users/Steven/Documents/PhD/npsR/test.so':
dlopen(/Users/Steven/Documents/PhD/npsR/test.so, 6): Symbol not found: ___addtf3
Referenced from: /usr/local/gfortran/lib/libquadmath.0.dylib
Expected in: /Library/Frameworks/R.framework/Resources/lib/libgcc_s.1.dylib
in /usr/local/gfortran/lib/libquadmath.0.dylib
Does anyone know why? I'm using mac osx Lion, with R v2.15.0 and gfortran 4.6.2.
Thank you!
Unless you really understand what you're doing, you should use gfortran that comes with R tools. The missing symbol in libgcc isn't surprising since you're trying to run code compiled with a 4.6 gfortran in a 4.2 runtime environment.
You can only use the gfortran version that was used to build R.
Since you are on Lion and presumably have Xcode installed you can get the appropriate version of gfortran from here: http://r.research.att.com/tools/
Go to section: "Apple Xcode gcc-42 add-ons" and choose the appropriate version.
You'll have to get rid of your gfortran 4.6.2 completely or make it inaccessible by changing PATH if possible (which I doubt unless it is in /opt/...).
And do use R CMD SHLIB since that will pass the correct options to compiler and linker.