`ld` error when installing hdp package in R - r

I am try to install the hdp package by Nicola Robers using the following command in R:
devtools::install_github("nicolaroberts/hdp", build_vignettes = TRUE)
This used to get compiled and installed fine when I had the R version 3.6.2 and gcc version 9.2.1 on an (Arch) Linux machine. After updating the system to R 4.0.0 and gcc 10.1.0, the installation fails with the following message:
* installing *source* package ‘hdp’ ...
** using staged installation
** libs
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-base.c -o R-base.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-conparam.c -o R-conparam.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-dp.c -o R-dp.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-hdp.c -o R-hdp.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-hdpMultinomial_iterate.c -o R-hdpMultinomial_iterate.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-multinomial.c -o R-multinomial.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c R-utils.c -o R-utils.o
gcc -I"/usr/include/R/" -DNDEBUG -D_FORTIFY_SOURCE=2 -fpic -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -c randutils.c -o randutils.o
gcc -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o hdp.so R-base.o R-conparam.o R-dp.o R-hdp.o R-hdpMultinomial_iterate.o R-multinomial.o R-utils.o randutils.o -L/usr/lib64/R/lib -lR
/usr/bin/ld: R-conparam.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: R-dp.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: R-hdp.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: R-hdpMultinomial_iterate.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: R-multinomial.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: R-utils.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
/usr/bin/ld: randutils.o:(.bss+0x0): multiple definition of `DEBUG'; R-base.o:(.bss+0x0): first defined here
collect2: error: ld returned 1 exit status
make: *** [/usr/share/R//make/shlib.mk:6: hdp.so] Error 1
ERROR: compilation failed for package ‘hdp’
The individual files are compiling fine, but the linking seems to fail. Any idea on how to go about debugging what is missing?

In case someone else runs into this issue: I manged to compile it by commenting all the references to the debug printing code. I commented all lines with rdebug* and DEBUG. These functions are mainly defined in R-utils.h. It still would be nice to find a better solution. There seems to be a circular definition of the DEBUG?

I had the same issue and ended up wrangling with the C code for several hours. My knowledge in C/C++ is very limited so perhaps there are better solutions but these changes worked for me.
Defining NODEBUG in R-utils.h after #ifndef NODEBUG and adding an extern to the declaration of DEGUB per below.
#ifndef NODEBUG
#define NODEBUG
extern int DEBUG;
Added int DEBUG to R-utils.c after #include "R-utils.h".

Related

devtools::build issue with conda

this is my first Stack Overflow question so please do not mind the terminology.
I try to build a package using devtools::build using a conda environment from my WSL:
devtools::build('pkg')
However, i get the following error
Error: Could not find tools necessary to compile a package
Call `pkgbuild::check_build_tools(debug = TRUE)` to diagnose the problem.
When i diagnose the problem i get the following
> pkgbuild::check_build_tools(debug = TRUE)
Trying to compile a simple C file
Running /home/name/miniconda3/envs/pipeline_env/lib/R/bin/R CMD SHLIB foo.c
x86_64-conda-linux-gnu-cc -I"/home/name/miniconda3/envs/pipeline_env/lib/R/include" -DNDEBUG -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/name/miniconda3/envs/pipeline_env/include -I/home/name/miniconda3/envs/pipeline_env/include -Wl,-rpath-link,/home/name/miniconda3/envs/pipeline_env/lib -fpic -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/name/miniconda3/envs/pipeline_env/include -fdebug-prefix-map=/home/conda/feedstock_root/build_artifacts/r-base-split_1648745733215/work=/usr/local/src/conda/r-base-4.1.3 -fdebug-prefix-map=/home/name/miniconda3/envs/pipeline_env=/usr/local/src/conda-prefix -c foo.c -o foo.o
/bin/sh: 1: x86_64-conda-linux-gnu-cc: not found
make: *** [/home/name/miniconda3/envs/pipeline_env/lib/R/etc/Makeconf:170: foo.o] Error 127

OpenMP support in R and RcppArmadillo

I struggle to enable OpenMP support for R in Ubuntu 16.04. When installing RcppArmadillo, OpenMP is 'found', but I get the following message when installing packages (e.g. 'forecast') that depend on RcppArmadillo:
g++ -I/usr/share/R/include -DNDEBUG -I"/usr/local/lib/R/site-
library/Rcpp/include" -I"/usr/local/lib/R/site-
library/RcppArmadillo/include" -fpic -g -O2 -fstack-protector-strong -
Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c
calcBATS.cpp -o calcBATS.o
In file included from /usr/local/lib/R/site-
library/RcppArmadillo/include/armadillo:53:0,
from /usr/local/lib/R/site-
library/RcppArmadillo/include/RcppArmadilloForward.h:46,
from /usr/local/lib/R/site-library/RcppArmadillo/include/RcppArmadillo.h:31,
from calcBATS.h:36,
from calcBATS.cpp:1:
/usr/local/lib/R/site-
library/RcppArmadillo/include/armadillo_bits/compiler_setup.hpp:474:96:
note: #pragma message: WARNING: use of OpenMP disabled; this compiler
doesn't support OpenMP 3.0+
#pragma message ("WARNING: use of OpenMP disabled; this compiler doesn't
support OpenMP 3.0+")
My ~/.R/Makevars are the following:
VER=-5.4
CC=ccache gcc-$(VER)
CXX=ccache g++-$(VER)
FC=ccache gfortran
F77=ccache gfortran
CXX11 =ccache g++-$(VER)
CXX14=ccache g++-$(VER)
CXX_STD = CXX11
CFLAGS= -O3 -Wall -pipe -pedantic -std=gnu11 -fopenmp
FCFLAGS=-O3 -Wall -pipe -pedantic -std=gnu11 -fopenmp
CXXFLAGS = -O3 -Wall -pipe -pedantic -std=gnu11 -fopenmp
CXX11FLAGS = -O3 -Wall -pipe -pedantic -std=gnu11 -fopenmp
CXX14FLAGS = -O3 -Wall -pipe -pedantic -std=gnu14 -fopenmp
PKG_CFLAGS= -O3 -Wall -pipe -pedantic -std=gnu99 -fopenmp
PKG_CXXFLAGS= -fopenmp
PKG_LIBS = -fopenmp -lgomp
CFLAGS= -O3 -Wall -pipe -pedantic -std=gnu99 -fopenmp
FCFLAGS=-O3 -g0 -Wall -pipe -fopenmp
I have played around with these settings, but nothing seems to help. Any wise thoughts would be appreciated. Many thanks.
Update:
I think some packages are using openMP now (e.g. xgboost). However, I still get the error message with some packages (e.g. Amelia):
* installing *source* package ‘Amelia’ ...
** package ‘Amelia’ successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/share/R/include -DNDEBUG -I"/usr/local/lib/R/site-
library/Rcpp/include" -I"/usr/local/lib/R/site-
library/RcppArmadillo/include" -fpic -g -O2 -fstack-protector-strong -
Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c
em.cpp -o em.o
In file included from /usr/local/lib/R/site-
library/RcppArmadillo/include/armadillo:53:0,
from /usr/local/lib/R/site-
library/RcppArmadillo/include/RcppArmadilloForward.h:46,
from /usr/local/lib/R/site-
library/RcppArmadillo/include/RcppArmadillo.h:31,
from em.h:3,
from em.cpp:2:
/usr/local/lib/R/site-
library/RcppArmadillo/include/armadillo_bits/compiler_setup.hpp:474:96:
note: #pragma message: WARNING: use of OpenMP disabled; this compiler
doesn't support OpenMP 3.0+
#pragma message ("WARNING: use of OpenMP disabled; this compiler doesn't
support OpenMP 3.0+")
^
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o
Amelia.so em.o -llapack -lblas -lgfortran -lm -lquadmath -L/usr/lib/R/lib -
lR
installing to /usr/local/lib/R/site-library/Amelia/libs
I can see that Amelia does not finish with '-fopenmp' unlike xgboost, but I don't know what else to add to my ~/.R/Makevars. Installation of xgboost ends in the following (note the -fopenmp):
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-
z,relro -o xgboost.so ./xgboost_R.o ./xgboost_custom.o ./xgboost_assert.o
./amalgamation/xgboost-all0.o ./amalgamation/dmlc-minimum0.o
./rabit/src/engine_empty.o ./rabit/src/c_api.o -fopenmp -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/xgboost/libs
The comment by #hejseb is correct. (Rcpp)Armadillo now wants to use OpenMP when it can---on new enough compilers.
But client packages need to turn this on, meaning they have to change their file src/Makevars from having just
PKG_LIBS = $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
to
PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS)
PKG_LIBS = $(SHLIB_OPENMP_CFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS)
to enable OpenMP in the client package on both compilation and linking.
And e.g. the forecast package does not yet do that.
You can also set "global" #define in your ~/.R/Makevars to skip OpenMP if you find this too noisy. But you'd leave some performance on the table, which is not a good default. Which is why (Rcpp)Armadillo makes the noisy comment.
Edit: There is one minor nuisance in that you need to set the flags for every C++ dialect so make sure you set CFLAGS, CXXFLAGS, CXX11FLAGS and CXX14FLAGS. You should see the -fopenmp on the command-line when R CMD INSTALL ... does its work.

RcppEigen.h: No such file or directory during installation of ... RcppEigen on Centos 7

I'm not new to R but can't figure out what went wrong. I'm just trying to install RcppEigen package using install.packages('RcppEigen') and receive the above error.
The command below (issued by the installer) fails:
g++ -m64 -I/usr/include/R -DNDEBUG -I"/usr/lib64/R/library/Rcpp/include" -I/usr/local/include -std=c++11 -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c RcppEigen.cpp -o RcppEigen.o
I have installed both R-Rcpp and R-Rcpp-devel version 0.12.12 from EPEL repository as well as eigen3-devel v 3.2.5 (not sure whether it is required but anyway ...)
I cloned RcppEigen from Github and tried to build in RStudio - the same error.
Makevars has PKG_CXXFLAGS = -I../inst/include but the compiler is invoked as below:
g++ -m64 -I/usr/include/R -DNDEBUG -I"/home/zer0hedge/R/x86_64-redhat-linux-gnu-library/3.4/Rcpp/include" -I/usr/local/include -std=c++11 -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -c RcppEigen.cpp -o RcppEigen.o
For some reason, I had PKG_CXXFLAGS defined in $HOME/.R/Makevars. It erroneously overrided PKG_CXXFLAGS in Makevars in src directory of the package and prevented compilation of C++ files

Installation of rgdal fails in Gentoo with inverser.c include directive

I am trying to install rgdal (a dependency of gstat) in R on a Calculate Linux (effectively a generic Gentoo) clean installation. I have sci-libs/gdal installed. Here are the last few lines of output:
x86_64-pc-linux-gnu-g++ -I/usr/lib64/R/include -DNDEBUG -I/usr/include/gdal -I"/home/wjc721/R/x86_64-pc-linux-gnu-library/3.2/sp/include" -fpic -O2 -march=x86-64 -pipe -c OGR_write.cpp -o OGR_write.o
x86_64-pc-linux-gnu-g++ -I/usr/lib64/R/include -DNDEBUG -I/usr/include/gdal -I"/home/wjc721/R/x86_64-pc-linux-gnu-library/3.2/sp/include" -fpic -O2 -march=x86-64 -pipe -c gdal-bindings.cpp -o gdal-bindings.o
x86_64-pc-linux-gnu-gcc -std=gnu99 -I/usr/lib64/R/include -DNDEBUG -I/usr/include/gdal -I"/home/wjc721/R/x86_64-pc-linux-gnu-library/3.2/sp/include" -fpic -O2 -march=x86-64 -pipe -c init.c -o init.o
x86_64-pc-linux-gnu-gcc -std=gnu99 -I/usr/lib64/R/include -DNDEBUG -I/usr/include/gdal -I"/home/wjc721/R/x86_64-pc-linux-gnu-library/3.2/sp/include" -fpic -O2 -march=x86-64 -pipe -c inverser.c -o inverser.o
inverser.c:3:22: fatal error: projects.h: No such file or directory
#include <projects.h>
^
compilation terminated.
Existing answers on Stackoverflow are for Linux distributions other than Gentoo. They suggest installation of packages (in Debian) such as libgdal1h, libgdal1-dev, libproj-dev and gdal-bin, none of which exist in Gentoo.
Any help would be very much appreciated! It was working fine on the previous version of Calculate :(
Thanks,
Bill
Edit: I upgraded R from v3.2.2 to 3.3.2 and gdal from 2.0.2-r3 to 2.0.3. This did not help, the error is the same.
I encountered the same problem. Upgrading to proj-4.9.2 did the trick.

GCC in error when compiling R package on Cloudera CDH4

I am trying to install RHadoop on Cloudera's CDH4 Hadoop VM. RHadoop is a framework for using Hadoop via an R programming interface. One of the prerequisites to installing the RHadoop R package "rmr" is another R package called Rcpp, which is for calling C++ code from R. When I attempt to install the package, I get the following error. I contacted both the Rcpp and RHadoop communities and both say its a GCC problem. When I try to update or reinstall GCC, I get
"Package gcc-4.4.6-4.e16.x86_64 already installed and latest version, nothing to do." Any suggestions?
Here is the error (from the R command line).
> install.packages("Rcpp")
Installing package(s) into ‘/usr/lib64/R/library’
(as ‘lib’ is unspecified)
--- Please select a CRAN mirror for use in this session ---
Loading Tcl/Tk interface ... done
trying URL 'http://cran.csdb.cn/src/contrib/Rcpp_0.9.15.tar.gz'
Content type 'application/x-gzip' length 2119769 bytes (2.0 Mb)
opened URL
==============================
====================
downloaded 2.0 Mb
* installing *source* package ‘Rcpp’ ...
** package ‘Rcpp’ successfully unpacked and MD5 sums checked
** libs
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Date.cpp -o Date.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c DateVector.cpp -o DateVector.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Datetime.cpp -o Datetime.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c DatetimeVector.cpp -o DatetimeVector.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Dimension.cpp -o Dimension.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c DottedPair.cpp -o DottedPair.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Environment.cpp -o Environment.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Evaluator.cpp -o Evaluator.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Formula.cpp -o Formula.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Function.cpp -o Function.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Language.cpp -o Language.o
g++ -m64 -I/usr/include/R -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c Module.cpp -o Module.o
{standard input}: Assembler messages:
{standard input}:10671: Warning: end of file not at end of a line; newline inserted
{standard input}:11544: Error: bad register name `%'
{standard input}:11544: Error: open CFI at the end of file; missing .cfi_endproc directive
g++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
make: *** [Module.o] Error 1
ERROR: compilation failed for package ‘Rcpp’
* removing ‘/usr/lib64/R/library/Rcpp’
The downloaded source packages are in
‘/tmp/RtmpGdXO9h/downloaded_packages’
Updating HTML index of packages in '.Library'
Making packages.html ... done
Warning message:
In install.packages("Rcpp") :
installation of package ‘Rcpp’ had non-zero exit status
As one of the Rcpp authors, I can assure that the error is probably at your end and with either gcc (unlikely) or CDH. Rcpp gets built and rebuild a bazillion times both on CRAN (and yes there are pages summarizing the builds on CRAN and also on Debian. The package does not normally kill the compiler.
Could you be running out of ram on the box?

Resources