I have created a docker image based on Centos7 and installed R with the Dockerfile below:
FROM centos:7
ENV TZ=Etc/UTC
# OS Dependencies
RUN yum install -y \
epel-release \
centos-release-scl-rh \
openblas-Rblas \
devtoolset-8-toolchain \
tre-devel \
wget \
libcurl-devel \
&& yum group install -y "Development Tools"
# Install R
RUN yum install -y \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/libRmath-devel-4.0.2-1.sdl7.x86_64.rpm \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/libRmath-4.0.2-1.sdl7.x86_64.rpm \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/R-java-devel-4.0.2-1.sdl7.x86_64.rpm \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/R-devel-4.0.2-1.sdl7.x86_64.rpm \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/R-core-devel-4.0.2-1.sdl7.x86_64.rpm \
http://springdale.princeton.edu/data/springdale/7/x86_64/os/Computational/R-core-4.0.2-1.sdl7.x86_64.rpm
I am trying to install the R package rlang but am getting an error related to "C99" mode that I cannot resolve.
> install.packages("rlang", repos = "https://cran.rstudio.org")
Installing package into '/usr/lib64/R/library'
(as 'lib' is unspecified)
trying URL 'https://cran.rstudio.org/src/contrib/rlang_0.4.10.tar.gz'
Content type 'application/x-gzip' length 915685 bytes (894 KB)
==================================================
downloaded 894 KB
* installing *source* package 'rlang' ...
** package 'rlang' successfully unpacked and MD5 sums checked
** using staged installation
** libs
gcc -m64 -I"/usr/include/R" -DNDEBUG -I./lib/ -I/usr/local/include -fvisibility=hidden -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c capture.c -o capture.o
gcc -m64 -I"/usr/include/R" -DNDEBUG -I./lib/ -I/usr/local/include -fvisibility=hidden -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c export.c -o export.o
In file included from export.c:1:0:
export/exported.c: In function 'rlang_env_bind_list':
export/exported.c:93:3: error: 'for' loop initial declarations are only allowed in C99 mode
for (r_ssize i = 0; i < n; ++i) {
^
export/exported.c:93:3: note: use option -std=c99 or -std=gnu99 to compile your code
In file included from export.c:1:0:
export/exported.c: In function 'rlang_is_string':
export/exported.c:572:3: error: 'for' loop initial declarations are only allowed in C99 mode
for (r_ssize i = 0; i < n; ++i) {
^
make: *** [export.o] Error 1
ERROR: compilation failed for package 'rlang'
* removing '/usr/lib64/R/library/rlang'
The downloaded source packages are in
'/tmp/RtmpEmr8lk/downloaded_packages'
Updating HTML index of packages in '.Library'
Warning messages:
1: In install.packages("rlang", repos = "https://cran.rstudio.org") :
installation of package 'rlang' had non-zero exit status
2: In file.create(f.tg) :
cannot create file '/usr/share/doc/R-4.0.2/html/packages.html', reason 'No such file or directory'
3: In make.packages.html(.Library) : cannot update HTML package index
The error is the same one as discussed here, but I was looking for a solution that I could implement during image build (i.e. an environment var or change to an R config file) that would be persistent for those using the image and installing R packages in the future, allowing them to install with a simple call to install.packages() rather than having to use withr::with_makevars() for every package install.
You can set R's config file under $R_HOME/etc/Makeconf (or add a Makevars in the same directory). By default, R_HOME should be in /usr/lib64/R, although you can get it in R by running > R.home().
Set CC = gcc -std=c11 (or whatever standard you like), you may also want to set CXX standard similarly.
Technically, you can also change what the gcc binary points to. I tried this after setting the CFLAGS environment variable failed. For example, you can rename the gcc binary (usually under /usr/bin) to something like gcc-v4.8.5. Then create a file called gcc (with proper exec permissions) in the same folder with the following shell script
#!/usr/bin/env bash
gcc-v4.8.5 -std=c99 "$#"
This will work by invoking gcc with that extra parameter but it's not recommended since it will probably break after updates to gcc. That said, I am not sure what happens to Makevars/Makeconf when R will be updated.
This shouldn't be a problem if the user runs gcc separately, it just sets the default standard that will be ignored if the user specifies the flag as well (since it will appear later in the command).
Related
I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!
I need to install data.table 1.12.0 (specifically) on macOS 11.1.
I am getting an error:
clang: error: unsupported option '-fopenmp'
make: *** [assign.o] Error 1
I have followed the directions in https://github.com/Rdatatable/data.table/wiki/Installation#openmp-enabled-compiler-for-mac but still unable to get it to work.
R 3.6.1.
I also tried a R CMD install and got the same error:
R CMD install data.table_1.12.0.tar.gz
* installing to library ‘/Users/XXX/Library/R/3.6/library’
* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c assign.c -o assign.o
clang: error: unsupported option '-fopenmp'
make: *** [assign.o] Error 1
ERROR: compilation failed for package ‘data.table’
* removing ‘/Users/XXX/Library/R/3.6/library/data.table’
Update: Seems to be an issue install latest data.table from CRAN for me as well, so its not restricted to 1.12.0 apparently.
NB. This solution works with Intel processors (not Apple M1 silicon)
These are the steps I used to install data.table from source with multithreading/openMP enabled (originally described in https://stackoverflow.com/a/65334247/12957340):
Reinstall xcode command line tools (even if it says "up to date")
sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install
Install gcc & llvm via Homebrew (instructions for installing Homebrew) or, if you already have gcc/llvm installed via Homebrew, skip to the next step
# This can take several hours
brew install gcc
brew install llvm
Once you have gcc & llvm installed via Homebrew:
brew cleanup
brew update
brew upgrade
brew reinstall gcc
brew reinstall llvm
Link some headers into /usr/local/include
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
# You can safely ignore warnings like this:
#ln: /usr/local/include//tcl.h: File exists
#ln: /usr/local/include//tclDecls.h: File exists
#ln: /usr/local/include//tclPlatDecls.h: File exists
#ln: /usr/local/include//tclTomMath.h: File exists
#ln: /usr/local/include//tclTomMathDecls.h: File exists
#ln: /usr/local/include//tk.h: File exists
#ln: /usr/local/include//tkDecls.h: File exists
#ln: /usr/local/include//tkPlatDecls.h: File exists
Install the gmp library:
brew install gmp
Create a new ~/.R/Makevars file (if you already have a ~/.R/Makevars file, save it in a different directory (away from ~/.R/)) and include only these lines in the file:
LOC=/usr/local/gfortran
CC=$(LOC)/bin/gcc -fopenmp
CXX=$(LOC)/bin/g++ -fopenmp
CXX11 = $(LOC)/bin/g++ -fopenmp
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L$(LOC)/lib -Wl,-rpath,$(LOC)/lib,-L/usr/local/lib
CPPFLAGS=-I$(LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/include
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CXX1X=/usr/local/gfortran/bin/g++
CXX98=/usr/local/gfortran/bin/g++
CXX11=/usr/local/gfortran/bin/g++
CXX14=/usr/local/gfortran/bin/g++
CXX17=/usr/local/gfortran/bin/g++
Compile data.table from source in R/Rstudio
install.packages("data.table", type = "source")
# To check whether it installed correctly, run:
library(data.table)
If your package fails to compile, a couple of SO users have had to install a fresh gfortran (re: https://stackoverflow.com/a/65334247/12957340), which you can download from https://github.com/fxcoudert/gfortran-for-macOS/releases
here has been my approach
basically you have to download clang and llvmorg
curl -SL https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_10.0.0 && \
echo 'export PATH=/clang_10.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_10.0.0/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
and make sure they are listed in you PATH and LD_LIBRARY_PATH
then get the datatable and build from the source
git clone https://github.com/h2oai/datatable.git && \
cd datatable && \
make build && \
python -m pip install ../datatable/
make sure you are using the right Python
I am trying to install Geneland (from https://i-pri.org/special/Biostatistics/Software/Geneland/#Installation) on R version 3.5.1 on my Mac (Mojave 10.14.6).
I first add gcc issue, but followed this (https://nistara.net/post/compile-issues-r/) which did the trick and the installation could go further (gcc 9.2.0 installed via Brew).
However I am now stuck here:
> install.packages("~/Desktop/Geneland_4.0.8.tar.gz", repos = NULL, type = "source")
* installing *source* package ‘Geneland’ ...
** libs
gfortran -fPIC -g -O2 -c alarousset.f -o alarousset.o
gfortran -fPIC -g -O2 -c ggfstat.f -o ggfstat.o
gfortran -fPIC -g -O2 -c hz.f -o hz.o
gfortran -fPIC -g -O2 -c main.f -o main.o
gcc-9 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I/usr/local/include -fPIC -mtune=native -g -O2 -Wall -pedantic -Wconversion -c wrapper.c -o wrapper.o
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:61,
from /Library/Frameworks/R.framework/Resources/include/R.h:55,
from wrapper.c:1:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h:257:22: error: missing binary operator before token "("
257 | #if __has_builtin(__is_target_arch)
| ^
make: *** [wrapper.o] Error 1
ERROR: compilation failed for package ‘Geneland’
* removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Geneland’
Warning in install.packages :
installation of package ‘/Users/simon/Desktop/Geneland_4.0.8.tar.gz’ had non-zero exit status
Would you have any idea how to solve this?
Thanks a lot in advance!
EDIT:
I was able to install Geneland by keeping only this line in the ~/.R/Makevars file:
FLIBS=-L/usr/local/Cellar/gcc/9.2.0/lib/gcc/9
I hope this will help others!
I was able to successfully install on Windows (R v3.5). However, I'm having similar issues installing on Mac:
install.packages("~/Desktop/Geneland_4.0.8.tar.gz", repos = NULL, type = "source")
* installing *source* package ‘Geneland’ ...
** using staged installation
** libs
gfortran -fPIC -Wall -g -O2 -c alarousset.f -o alarousset.o
gfortran-4.2: error trying to exec '/usr/local/bin/i686-apple-darwin8-gfortran-4.2': execvp: Bad CPU type in executable
make: *** [alarousset.o] Error 255
ERROR: compilation failed for package ‘Geneland’
* removing ‘/Library/Frameworks/R.framework/Versions/3.6/Resources/library/Geneland’
Warning in install.packages :
installation of package ‘/Users/jas/Desktop/Geneland_4.0.8.tar.gz’ had non-zero exit status
Perhaps the creators of this package never tested on Mac or it was not designed to run on Mac (hopefully I'm wrong).
Also regarding gcc, I believe I have the latest version as per:
Jass-MacBook-Air:~ $ brew info gcc
gcc: stable 9.2.0 (bottled), HEAD
GNU compiler collection
https://gcc.gnu.org/
/usr/local/Cellar/gcc/9.2.0 (1,462 files, 291.4MB)
Poured from bottle on 2019-09-17 at 11:08:00
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/gcc.rb
==> Dependencies
Required: gmp ✔, isl ✔, libmpc ✔, mpfr ✔
==> Options
--HEAD
Install HEAD version
==> Analytics
install: 83,384 (30 days), 202,842 (90 days), 915,671 (365 days)
install_on_request: 37,664 (30 days), 91,399 (90 days), 435,332 (365 days)
build_error: 0 (30 days)
I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!
Our R installation define in:
R$HOME/etc/Makeconf that CC = gcc -std=gnu99
I have one specific package (mix of C++ and C code) that need to be compiled using
CC = gcc
without -std=gnu99
As far as I understood I have 3 ways of doing that:
1) system wide, edit R$HOME/etc/Makeconf
2) per user basis, play with ~/.R/Makevars
3) per package basis, set PACKAGE/src/Makevars
Even if 1 and 2 is not what I want I tested the 3 options, using 1 and 2
R CMD INSTALL -l pack.tgz is OK "gcc -std=gnu99" is corectly replaced by "gcc"
But when using PACKAGE/src/Makevars approach it fails
I must admit that I'm lost at this point, where should I look ?
edit.
this is not really a duplicate with Building R Packages using Alternate GCC
for sure I have read the previous one.is the one that pointed me to Makevars
my key concern is that PACKAGE/src/Makevars is not taken in account for CC=alternate compiler while other one are working prefectly.
I had a similar problem in fortran. Anyway I made a model of your package and I found an half solution. It seems that not all the variables in PACKAGE/src/Makevars are considered and used. In order to make it work I used this Makevars file:
MY_PKG_LIBS =
MY_PKG_CCLAGS = -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g
all: $(SHLIB)
hello.o: hello.c
gcc $(MY_PKG_CCLAGS) -c hello.c -o hello.o $(MY_PKG_LIBS)
PKG_LIB = -std=gnu++11
Obviously the hello.c file should be replaced by your_file_name.c. If you can not change the CC and you adopt my workaround, the real problem become that when the shared file .so is created, the compiler flags should be overwritten using PKG_CFLAGS or PKG_CPPFLAGS as stated in Writing R Extensions (again in the Makevars file). In my personal situation (Ubuntu 15.04, R 3.1.2), I tried those and other vars following the guidelines in /etc/R/Makeconf file:
ALL_CFLAGS = $(R_XTRA_CFLAGS) $(PKG_CFLAGS) $(CPICFLAGS) $(SHLIB_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(R_XTRA_CPPFLAGS) $(PKG_CPPFLAGS) $(CPPFLAGS) $(CLINK_CPPFLAGS)
The only thing that worked in adding a flag to the -shared final compiling of the package was adding the library linker flag (As I did originally with my fortran code) PKG_LIB = -std=gnu++11 in the PACKAGE/src/Makevars.
My final result in installing the package is:
Installing package into ‘/home/home/R/x86_64-pc-linux-gnu-library/3.1’
(as ‘lib’ is unspecified)
* installing *source* package ‘question1’ ...
** libs
gcc -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c hello.c -o hello.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o question1.so hello.o -std=gnu++11 -L/usr/lib/R/lib -lR
installing to /home/dgarolini/R/x86_64-pc-linux-gnu-library/3.1/question1/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (question1)