I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!
Related
I want to compile an R package with specific linker flags. In my case, I want to use -rpath to "hard code" some library paths for the package so I don't have to have them always in my LD_LIBRARY_PATH.
If I simply try to do so using R's environment variables, it seems to have no effect:
Sys.setenv(LDFLAGS="-Wl,-rpath,/stornext/System/data/tools/proj/proj-4.9.3/lib -Wl,-rpath,/stornext/System/data/apps/geos/geos-3.7.0/lib")
install.packages("sf")
The final gcc invocation is:
/stornext/System/data/apps/gcc/gcc-11.1.0/bin/g++ -std=gnu++11 -shared -L/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib -L/usr/local/lib64 -o sf.so RcppExports.o bbox.o gdal.o gdal_geom.o gdal_read.o gdal_utils.o gdal_write.o geos.o hex.o mdim.o ops.o polygonize.o proj.o proj_info.o raster2sf.o sfc-sfg.o signed_area.o stars.o wkb.o zm_range.o -L/stornext/System/data/tools/proj/proj-4.9.3/lib -lproj -L/stornext/System/data/tools/gdal/gdal-2.4.4/lib -lgdal -L/stornext/System/data/apps/geos/geos-3.7.0/lib -lgeos_c -L/stornext/System/data/apps/R/R-4.2.1/lib64/R/lib -lR
And when I check the final .so file with readelf -d, I can see that the RPATH headers are missing.
If I try to add them via configure, that doesn't work either:
install.packages("sf", configure.vars="LDFLAGS=-Wl,-rpath,/stornext/System/data/tools/proj/proj-4.9.3/lib -Wl,-rpath,/stornext/System/data/apps/geos/geos-3.7.0/lib -Wl,-rpath,/stornext/System/data/tools/gdal/gdal-2.4.4/lib")
This gives me:
sh: -Wl,-rpath,/stornext/System/data/apps/geos/geos-3.7.0/lib: No such file or directory
In this case, the flags are being passed through numerous stages in order to finally get to the makefile: they go from R to R CMD INSTALL in the shell, then to the configure script, so escaping quotes and spaces seems incredibly difficult here.
Can anyone offer help as to how to provide multiple linker flags when compiling an R package?
I can’t figure out how to give my R package’s shared library’s debug symbols source line information. What am I missing?
I create the following src/Makevars file:
PKG_CXXFLAGS=-O0 -ggdb
PKG_LIBS=-O0 -ggdb
I compile the package using R CMD INSTALL --no-multiarch --with-keep.source:
* installing to library ‘~/.local/lib/R/3.6’
* installing *source* package ‘reticulate’ ...
** using staged installation
g++ -std=gnu++11 -I"/usr/include/R/" -DNDEBUG -I"$HOME/.local/lib/R/3.6/Rcpp/include" -D_FORTIFY_SOURCE=2 -O0 -ggdb -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c RcppExports.cpp -o RcppExports.o
** libs
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o -O0 -ggdb -L/usr/lib64/R/lib -lR
installing to ~/.local/lib/R/3.6/00LOCK-reticulate/00new/reticulate/libs
I debug like this:
R -d gdb --slave -e 'reticulate::py_eval("print")()'
GNU gdb (GDB) 8.3
[...]
(No debugging symbols found in /usr/lib64/R/bin/exec/R)
(gdb) break py_get_formals
Function "py_get_formals" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (py_get_formals) pending.
(gdb) run
Starting program: /usr/lib/R/bin/exec/R --slave -e reticulate::py_eval\(\"print\"\)\(\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[...]
Thread 1 "R" hit Breakpoint 1, 0x00007fffeb6b79a0 in py_get_formals(PyObjectRef, bool) () from /home/angerer/.local/lib/R/3.6/reticulate/libs/reticulate.so
(gdb) step
Single stepping until exit from function _Z14py_get_formals11PyObjectRefb,
which has no line number information.
[...]
Why does my function not have line numbers even though I specified -ggdb in both compilation? I see that only RcppExports.cpp is mentioned in the command line, is that the problem? If so, how can I change this?
Changing the Makevars doesn’t prompt recompilation.
I needed to rm -f src/*.o src/*.so before the object files get recompiled.
This is specifically for Windows. The simplest way to do it is to set the R_MAKEVARS_USER environment to point to the Makevars.win file. That seems to work. However, debug break points have stopped working!!!!
I am running MRO 3.5.0 on Win10, and trying to install a package image.CannyImage from bnosac/image on the github. But it keeps reporting fatal errors as following.
* installing *source* package 'image.CannyEdges' ...
** libs
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -mtune=core2 -c RcppExports.cpp -o RcppExports.o
c:/Rtools/mingw_64/bin/gcc -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -std=gnu99 -mtune=core2 -c adsf.c -o adsf.o
c:/Rtools/mingw_64/bin/g++ -m64 -I"C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/include" -DNDEBUG -I"C:/Users/eric1/Documents/R/win-library/3.5/Rcpp/include" -I"C:/swarm/workspace/External-R-3.5.0/vendor/extsoft/include" -O2 -Wall -mtune=core2 -c rcpp_canny.cpp -o rcpp_canny.o
In file included from rcpp_canny.cpp:13:0:
canny.h:8:19: fatal error: fftw3.h: No such file or directory
#include <fftw3.h>
^
compilation terminated.
make: *** [C:/PROGRA~1/MICROS~1/ROPEN~1/R-35~1.0/etc/x64/Makeconf:215: rcpp_canny.o] Error 1
ERROR: compilation failed for package 'image.CannyEdges'
* removing 'C:/Users/eric1/Documents/R/win-library/3.5/image.CannyEdges'
In R CMD INSTALL
Installation failed: Command failed (1)
^
compilation terminated.
Google says I can solve the issue by sudo apt-get remove libfftw3-dev. Unfortunately, I am on Windows, and some advised, apply lib /machine:i386 /def:libfftw3-3.def on cmd mode. Ouch, there is no such command or file called lib.exe on Win10.
Please advise, how I can solve this issue on my Win10+MRO system. Thanks.
Even though this question is already a bit old:
By now (Feb 2020) there are precompiled packages available that work also under Windows:
Canny Edges - Package
and more general:
List of all available packages
I have tried installing them and (at least for me) it worked.
I am trying to install R’s plyr package. Here is the error message:
* installing *source* package ‘plyr’ ...
** package ‘plyr’ successfully unpacked and MD5 sums checked
** libs
clang++ -I/opt/R-3.4.1/include -DNDEBUG -I"/home/isomorphismes/R/i686-pc-linux-gnu-library/3.4/Rcpp/include" -I/usr/local/include -fpic -I/opt/boost_1_61_0/boost -c RcppExports.cpp -o RcppExports.o
clang -I/opt/R-3.4.1/include -DNDEBUG -I"/home/cd/R/i686-pc-linux-gnu-library/3.4/Rcpp/include" -I/usr/local/include -fpic -g -O2 -flto -c loop_apply.c -o loop_apply.o
clang++ -I/opt/R-3.4.1/include -DNDEBUG -I"/home/isomorphismes/R/i686-pc-linux-gnu-library/3.4/Rcpp/include" -I/usr/local/include -fpic -I/opt/boost_1_61_0/boost -c split-numeric.cpp -o split-numeric.o
clang++ -shared -L/usr/local/lib -o plyr.so RcppExports.o loop_apply.o split-numeric.o
loop_apply.o: file not recognized: File format not recognized
clang: error: linker command failed with exit code 1 (use -v to see invocation)
/opt/R-3.4.1/share/make/shlib.mk:6: recipe for target 'plyr.so' failed
make: *** [plyr.so] Error 1
ERROR: compilation failed for package ‘plyr’
* removing ‘/home/cd/R/i686-pc-linux-gnu-library/3.4/plyr’
The *.o files are in /opt/plyr/src, from github.com/hadley/plyr. They look like this on my system:
i#scheherezade:/opt/plyr/src$ file *o
loop_apply.o: LLVM IR bitcode
RcppExports.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped
split-numeric.o: ELF 32-bit LSB relocatable, Intel 80386, version 1 (SYSV), not stripped
In case you didn't know, -flto specifies link time optimization, and has been added by R. How did you manage to end up with such a mis-configured R install?
Adding -flto to the link command may work? Or remove it from the loop_apply compilation line. If either of those works, you need to fix your R install.
Compiling with -flto using clang requires (on Ubuntu) installing the llvm-dev package. Otherwise, the linker is unable to handle -flto object files.
apt-get install clang-10 llvm-10-dev
Now the linking should succeed.
Our R installation define in:
R$HOME/etc/Makeconf that CC = gcc -std=gnu99
I have one specific package (mix of C++ and C code) that need to be compiled using
CC = gcc
without -std=gnu99
As far as I understood I have 3 ways of doing that:
1) system wide, edit R$HOME/etc/Makeconf
2) per user basis, play with ~/.R/Makevars
3) per package basis, set PACKAGE/src/Makevars
Even if 1 and 2 is not what I want I tested the 3 options, using 1 and 2
R CMD INSTALL -l pack.tgz is OK "gcc -std=gnu99" is corectly replaced by "gcc"
But when using PACKAGE/src/Makevars approach it fails
I must admit that I'm lost at this point, where should I look ?
edit.
this is not really a duplicate with Building R Packages using Alternate GCC
for sure I have read the previous one.is the one that pointed me to Makevars
my key concern is that PACKAGE/src/Makevars is not taken in account for CC=alternate compiler while other one are working prefectly.
I had a similar problem in fortran. Anyway I made a model of your package and I found an half solution. It seems that not all the variables in PACKAGE/src/Makevars are considered and used. In order to make it work I used this Makevars file:
MY_PKG_LIBS =
MY_PKG_CCLAGS = -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g
all: $(SHLIB)
hello.o: hello.c
gcc $(MY_PKG_CCLAGS) -c hello.c -o hello.o $(MY_PKG_LIBS)
PKG_LIB = -std=gnu++11
Obviously the hello.c file should be replaced by your_file_name.c. If you can not change the CC and you adopt my workaround, the real problem become that when the shared file .so is created, the compiler flags should be overwritten using PKG_CFLAGS or PKG_CPPFLAGS as stated in Writing R Extensions (again in the Makevars file). In my personal situation (Ubuntu 15.04, R 3.1.2), I tried those and other vars following the guidelines in /etc/R/Makeconf file:
ALL_CFLAGS = $(R_XTRA_CFLAGS) $(PKG_CFLAGS) $(CPICFLAGS) $(SHLIB_CFLAGS) $(CFLAGS)
ALL_CPPFLAGS = $(R_XTRA_CPPFLAGS) $(PKG_CPPFLAGS) $(CPPFLAGS) $(CLINK_CPPFLAGS)
The only thing that worked in adding a flag to the -shared final compiling of the package was adding the library linker flag (As I did originally with my fortran code) PKG_LIB = -std=gnu++11 in the PACKAGE/src/Makevars.
My final result in installing the package is:
Installing package into ‘/home/home/R/x86_64-pc-linux-gnu-library/3.1’
(as ‘lib’ is unspecified)
* installing *source* package ‘question1’ ...
** libs
gcc -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c hello.c -o hello.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o question1.so hello.o -std=gnu++11 -L/usr/lib/R/lib -lR
installing to /home/dgarolini/R/x86_64-pc-linux-gnu-library/3.1/question1/libs
** R
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (question1)