I'm trying to replicate CRAN sanitizer failures for my R package that depends on Rcpp on the rocker/r-devel-ubsan-clang docker image.
When I try to install Rcpp I get an error about no type named 'R_ContinueUnwind'. Is this something I'm doing wrong, or is it an incompatibility with Rcpp and Rdevel?
If the latter, is there a way to get a similar docker image that uses address sanitizers for a non-development version of R?
Here's the command I've been trying to run and output:
docker run --cap-add SYS_PTRACE --rm -it rocker/r-devel-ubsan-clang /bin/bash
root#e9f11aceb087:/# Rscriptdevel -e "install.packages('Rcpp')"
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'http://cran.rstudio.com/src/contrib/Rcpp_0.12.18.tar.gz'
Content type 'application/x-gzip' length 3809164 bytes (3.6 MB)
==================================================
downloaded 3.6 MB
* installing *source* package ‘Rcpp’ ...
** package ‘Rcpp’ successfully unpacked and MD5 sums checked
** libs
clang++-4.0 -stdlib=libc++ -fsanitize=address,undefined -fno-sanitize=float-divide-by-zero -fno-omit-frame-pointer -fsanitize-address-use-after-scope -I/usr/local/lib/R/include -DNDEBUG -I../inst/include/ -I/usr/local/include -fpic -g -O3 -Wall -pedantic -mtune=native -c Date.cpp -o Date.o
In file included from Date.cpp:31:
In file included from ../inst/include/Rcpp.h:27:
In file included from ../inst/include/RcppCommon.h:128:
../inst/include/Rcpp/exceptions.h:150:7: error: no type named 'R_ContinueUnwind' in the global namespace
::R_ContinueUnwind(token);
~~^
1 error generated.
/usr/local/lib/R/etc/Makeconf:166: recipe for target 'Date.o' failed
make: *** [Date.o] Error 1
ERROR: compilation failed for package ‘Rcpp’
* removing ‘/usr/local/lib/R/site-library/Rcpp’
The downloaded source packages are in
‘/tmp/RtmpqVuvlD/downloaded_packages’
Warning message:
In install.packages("Rcpp") :
installation of package ‘Rcpp’ had non-zero exit status
Edit
Here's the result of docker image list and from within the container Rdevel --version
docker image list rocker/r-devel-ubsan-clang
REPOSITORY TAG IMAGE ID CREATED SIZE
rocker/r-devel-ubsan-clang latest c769161f77fb 10 months ago 4.47GB
And Rdevel --version
Rdevel --version
R Under development (unstable) (2017-09-16 r73288) -- "Unsuffered Consequences"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.
I got that image by using docker pull, I'm now trying to build it from scratch, following the lead that the image is 10 months old.
Maintainer of that repo here. We could probably discuss that over at the repo in an issue ticket too. I have not tried compiling R-devel with clang myself in some time (as I build r-devel at home via gcc), so not sure why the symbol is missing.
The whole concept was picked up by Winston who created a whole battery of debug Docker containers in this repo.
Otherwise, the (excellent) R hub builder also has sanitizer builds. One remaining problem is that CRAN is not too forthcoming with changes to their setup (besides publishing their short descriptions) so it is never clear if the setup is in fact identical to theirs.
Edit: Actually grep'ing for your symbol shows that it is contingent on R version 3.5.0 or later:
#if (defined(R_VERSION) && R_VERSION >= R_Version(3, 5, 0))
::R_ContinueUnwind(token);
#endif
Can you double check your container setup if that is given? We may have a mismatch here then between R version and R-devel version -- should be easy to fix.
Related
I'm on Centos 8, dictated by corporate IT. I have EPEL 8 added to the repositories, and I've installed R. I'm trying to install the popular CRAN package, dplyr, so I can use its functions in R. I've used it a pile of times on different environments, including Centos 7.
> install.packages("dplyr")
Installing package into ?/home/user/R/x86_64-redhat-linux-gnu-library/3.6?
(as ?lib? is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/dplyr_0.8.5.tar.gz'
Then it goes and downloads the package and goes through a lot of compile steps. Skipping to the last part of the compiling,
g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o dplyr.so RcppExports.o address.o api.o arrange.o between.o bind.o distinct.o encoding.o filter.o group_indices.o hybrid.o init.o join.o join_exports.o mutate.o rlang-export.o rlang.o select.o set.o summarise.o test.o utils-bindings.o utils.o window.o -L/usr/lib64/R/lib -lR
installing to /home/user/R/x86_64-redhat-linux-gnu-library/3.6/00LOCK-dplyr/00new/dplyr/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** byte-compile and prepare package for lazy loading
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/home/user/R/x86_64-redhat-linux-gnu-library/3.6/Rcpp/libs/Rcpp.so':
/lib64/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /home/user/R/x86_64-redhat-linux-gnu-library/3.6/Rcpp/libs/Rcpp.so)
So, I'm clearly unable to find GLIBCXX_3.4.26. I'm unclear on a few things, including which of my packages requires it (does dplyr require it, or is it innocently calling a library which requires it). I know that I'm not gong to get GLIBCXX_3.4.26 in Centos 8. But now I need to understand if I need to find an installer for R that needs an earlier version (which I might find in EPEL 7?) or if I need to find an earlier version of dplyr (which I might find in archives of CRAN?)
My R version that yum list installed reports is 3.6.3-1.el8. My stdc++ is 8.3.1-4.5.el8.
I am trying to build and check a package with some Fortran code in it under Windows 10
with R 3.4.0 updating some code previously in R 2.8.
I am doing this in three steps in the command window:
>R CMD build src
[...]
* checking DESCRIPTION meta-information ... OK
So I assume everything is ok with the DESCRIPTION file
then:
>R CMD check --no-examples --no-tests src
I get among other stuff:
* checking for file 'src/DESCRIPTION' ... OK
A bit further it gets wrong at
* checking whether package 'pckgname' can be installed
Looking at the log inside the file /src.Rcheck/00install.out, I read:
Error in .get_package_metadata(dir, FALSE) :
Files 'DESCRIPTION' and 'DESCRIPTION.in' are missing.
ERROR: installing Rd objects failed for package 'samara'
I have copied and pasted the DESCRIPTION file pretty much everywhere it made sense, including the top level directory and the \src directory or in those directory alone in some other trials, with always the same result.
This is strange because sometimes the DESCRIPTION file is recognized, sometimes it is not
There is only one similar post treating this subject:
R package building error
the author seems to have solved his problem but he doesn't know how.
Does anybody know what happened?
Following some requests, here is the content of the files 00check.log and 00install.out:
00check.log
C:\Users\username\Documents\PCKGNAME\SRC_5.0.2E\pckgname>R CMD check --no- examples --no-tests src
* using log directory 'C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck'
* using R version 3.4.0 (2017-04-21)
* using platform: x86_64-w64-mingw32 (64-bit)
* using session charset: ISO8859-1
* using options '--no-examples --no-tests'
* checking for file 'src/DESCRIPTION' ... OK
* checking extension type ... Package
* this is package 'pckgname' version '5.0.2.5'
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... NOTE
Found the following apparent object files/libraries:
src-x64/FortPrg1.o src-x64/FortPrg2.o src-x64/FortPrg3.o
Object files/libraries should not be included in a source package.
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... NOTE
Found the following hidden files and directories:
R/.Rhistory
These were most likely included in error. See section 'Package
structure' in the 'Writing R Extensions' manual.
* checking for portable file names ... OK
* checking whether package 'pckgname' can be installed ... ERROR
Installation failed.
See 'C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck/00install.out' for details.
* DONE
Status: 1 ERROR, 2 NOTEs
In Prompt, there is this information in addition to what appeared before:
* checking whether package 'pckgname' can be installed ...
Warning: l'exécution de la commande '"C:/PROGRA~1/R/R-34~1.0/bin/x64/Rcmd.exe"
INSTALL -l "C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck" --no-html
"C:\Users\username\DOCUME~1\PCKGNAME\SRC_30~1.7E\pckgname\src"' renvoie un statut 1
Now in the file 00install.out:
* installing *source* package 'pckgname' ...
** libs
*** arch - i386
c:/Rtools/mingw_32/bin/gfortran -O3 -mtune=core2 -c FortPrg1.f90 -o FortPrg1.o
c:/Rtools/mingw_32/bin/gfortran -O3 -mtune=core2 -c FortPrg2.f90 -o FortPrg2.o
c:/Rtools/mingw_32/bin/gfortran -O3 -mtune=core2 -c FortPrg3.f90 -o FortPrg3.o
c:/Rtools/mingw_32/bin/gfortran -shared -s -static-libgcc -o pckgname.dll tmp.def FortPrg1.o FortPrg2.o FortPrg3.o -Ld:/Compiler/gcc-4.9.3/local330/lib/i386 -Ld:/Compiler/gcc-4.9.3/local330/lib -LC:/PROGRA~1/R/R-34~1.0/bin/i386 -lR
installing to C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck/pckgname/libs/i386
*** arch - x64
c:/Rtools/mingw_64/bin/gfortran -O2 -mtune=core2 -c FortPrg1.f90 -o FortPrg1.o
c:/Rtools/mingw_64/bin/gfortran -O2 -mtune=core2 -c FortPrg2.f90 -o FortPrg2.o
c:/Rtools/mingw_64/bin/gfortran -O2 -mtune=core2 -c FortPrg3.f90 -o FortPrg3.o
c:/Rtools/mingw_64/bin/gfortran -shared -s -static-libgcc -o pckgname.dll tmp.def FortPrg1.o FortPrg2.o FortPrg3.o -Ld:/Compiler/gcc-4.9.3/local330/lib/x64 -Ld:/Compiler/gcc-4.9.3/local330/lib -LC:/PROGRA~1/R/R-34~1.0/bin/x64 -lR
installing to C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck/pckgname/libs/x64
** R
** demo
** preparing package for lazy loading
Warning: package 'survival' was built under R version 3.4.4
Warning: package 'boot' was built under R version 3.4.4
Warning: package 'xlsx' was built under R version 3.4.4
Warning: package 'ggplot2' was built under R version 3.4.4
Warning: package 'GGally' was built under R version 3.4.4
[1] "Chargement de gdata ..."
[1] "Chargement de survival ..."
[1] "Chargement de boot ..."
[1] "Chargement de tcltk ..."
[1] "Chargement de xlsx ..."
---------------------------------------------
Package PCKGNAME genere le ven. sept. 28 17:14:21 2018
---------------------------------------------
Librairie Pckgname V5.0.2E Chargee
** help
No man pages found in package 'pckgname'
Error in .get_package_metadata(dir, FALSE) :
Files 'DESCRIPTION' and 'DESCRIPTION.in' are missing.
ERROR: installing Rd objects failed for package 'pckgname'
* removing 'C:/Users/username/Documents/PCKGNAME/SRC_5.0.2E/pckgname/src.Rcheck/pckgname'
Usually you would store a package named samara in a directory of the same name. You appear to have it in src. That should work, but I wouldn't be surprised if something in the check code implicitly assumes you were following the usual convention.
The other thing that is usually done is that you create a source tarball of the package and check that. That is, you run R CMD check on the output of R CMD build, not on the input. Again, what you are doing should mostly work (though not perfectly in this case; tarball contents typically are not identical to source directories), but perhaps these two nonstandard ways of working are causing your problems.
With the help of user2554330, I solved the problem.
I did the following things:
-I reorganized into pckgname/R and pckgname/src
(instead of pckgname/src/R and pckgname/src/src previously )
-I used R Studio. First I created a new project (create a new package) with RStudio and devtools: File>Create new project>R Package using devtools. And I put my files there
-I built and checked exclusively with the Rstudio interface
I'm running Mac OS X 10.7.5 (Lion), which means that the latest version of R I can run is 3.2.1. Since binaries of packages are no longer being made for this version, it means that I need to install the latest package versions from source. I can do this with
install.packages(pkgs, type = "source")
I installed GNU Fortran, although it's not clear to me whether I need it or not in order to compile R packages.
When I try to compile packages from source in R, however, I get error messages indicating that I don't have (or that R cannot find) the necessary compiling tools. Cf. e.g.:
* installing *source* package ‘scales’ ...
** package ‘scales’ successfully unpacked and MD5 sums checked
** libs
sh: make: command not found
ERROR: compilation failed for package ‘scales’
and
* installing *source* package ‘stringi’ ...
** package ‘stringi’ successfully unpacked and MD5 sums checked
checking for local ICUDT_DIR... icu55/data
checking for R_HOME... /Library/Frameworks/R.framework/Resources
checking for R... /Library/Frameworks/R.framework/Resources/bin/R
checking for R >= 3.1.0 for C++11 use... yes
checking for R < 3.4.0 for CXX1X flag use... yes
checking for cat... /bin/cat
/Library/Frameworks/R.framework/Resources/bin/config: line 150: make: command not found
[...]
checking for gcc... no
checking for cc... no
checking for cl.exe... no
/Library/Frameworks/R.framework/Resources/bin/config: line 153: make: command not found
/Library/Frameworks/R.framework/Resources/bin/config: line 269: make: command not found
configure: error: in `/private/var/folders/2r/6c5vhb4s3zx3zs4d1dbf9p940000gn/T/Rtmpob4pMm/R.INSTALLd4676ad8595/stringi':
configure: error: no acceptable C compiler found in $PATH
I assume what's needed is to install the necessary compiling tools for R packages. What do I need to do?
(If I need to provide additional information, just ask for it in a comment, and I will provide it)
It might be tricky. First of all, double check whether you can download XCode from AppStore or not. If you can, you will be able to install command line tools as well. Either via Preferences -> Downloads (as far as I recall), or they will be installed during first startup.
Alternatively, create free account here:
http://developer.apple.com
Then, head to:
https://developer.apple.com/download/more/
and find XCode. It might be you will need to install some really old version.
As #hrbrmstr mentioned, it might be that installing virtual machine (e.g. inside VirtualBox) with Linux will be less painful.
Note that you will probably encounter more and more issues while installing packages. There is no easy way if you are forced to stick with old OS X :(
Update:
As #Sverre pointed out, it would be better to have a complete list of links here:
XCode - (Lion version) download from Apple (not to confuse with AppStore)
Command Line Tools - (Lion version) you can install them separately, without installing XCode
I'm trying to install the CausalImpact package and failing due to a dependency on bsts.
I've tried to install bsts without success, and looking at the results I see what I think is a failure on a dependency for Boom:
install.packages("bsts", lib="C:/R/win-library/3.3")
Package which is only available in source form, and may need compilation of C/C++/Fortran: ‘bsts’
Do you want to attempt to install these from sources?
y/n: y
installing the source package ‘bsts’
trying URL 'https://cran.rstudio.com/src/contrib/bsts_0.6.3.tar.gz'
Content type 'application/x-gzip' length 148815 bytes (145 KB)
downloaded 145 KB
* installing *source* package 'bsts' ...
** package 'bsts' successfully unpacked and MD5 sums checked
** libs
*** arch - i386
c:/Rtools/mingw_32/bin/g++ -std=c++0x -I"C:/PROGRA~1/R/R-33~1.0/include" -DNDEBUG -I`C:/PROGRA~1/R/R-33~1.0/bin/Rscript -e "cat(system.file(package='Boom'))"`/include -DNO_BOOST_THREADS -DNO_BOOST_FILESYSTEM -DADD_ -DR_NO_REMAP -DRLANGUAGE -I"C:/Users/Firstname Lastname/Documents/R/win-library/3.3/Boom/include" -I"C:/Users/Firstname Lastname/Documents/R/win-library/3.3/BH/include" -I"d:/Compiler/gcc-4.9.3/local330/include" -O2 -Wall -mtune=core2 -c aggregate_time_series.cc -o aggregate_time_series.o
g++.exe: error: Lastname/Documents/R/win-library/3.3/Boom/include: No such file or directory
What I assume is that g++ is getting upset because R has put the library in a subfolder of C:Users/Firstname Lastname, and it's not liking the space in my name.
I tried setting the R_LIBS_USER environment variable to C:/R/win-library/3.3 to eliminate this problem with a space in the path, but with no success - even after a full restart of the computer, the error repeats - it's still going to the C:/Users/Firstname Lastname/Documents/R/win-library/3.3 directory.
I'm using RStudio 0.99.896 with R 3.3.0 on Windows 10, if that makes any difference
As suggested in this related question if you don't necessarily need the latest version of bsts you can still install version 0.6.2 on Windows, and that will let you install CausalImpact:
install.packages("https://cran.r-project.org/bin/windows/contrib/3.2/bsts_0.6.2.zip",
repos = NULL, type = "local")
library(devtools)
devtools::install_github("google/CausalImpact")
works on Win 10, R 3.3.0
I have been installing R packages from tar.gz type files on Edinburgh university's compute and data facility which is a high performance computing cluster which is accessed via SSH. Once logged in, I can install the files as the example below:
install.packages("/exports/work/eng_AntonyHydroData/Pkges/DEoptim_2.2-3.tar.gz",rpeo=NULL,target="source")
There are approximately 40 packages and 38 of them install no problem, but two fail: RGEOS (rgeos_0.3-14.tar.gz) and RGDAL (rgdal_1.0-7.tar.gz).
The reason for failure is given below:
Installing package into '/exports/work/eng_AntonyHydroData/library'
(as 'lib' is unspecified)
inferring 'repos = NULL' from 'pkgs'
* installing *source* package 'rgdal' ...
** package 'rgdal' successfully unpacked and MD5 sums checked
configure: CC: gcc -std=gnu99
configure: CXX: g++
configure: rgdal: 1.0-5
checking for /usr/bin/svnversion... yes
configure: svn revision: 559
checking for gdal-config... no
no
configure: error: gdal-config not found or not executable.
ERROR: configuration failed for package 'rgdal'
Note that RGEOS gives the same reason i.e. gdal-config is missing.
I have looked around the net and have found various answers to this problem that mainly suggests to install libgeos-dev or something similar using the command 'sudo' or 'apt-get' but neither of these commands are recognised by the computing cluster system.
Any help in getting RGEOS and RGDAL to work would be very much appreciated.
Antony
I managed to solve this by first installing the geos-3.5.0.tar.bz2 package and issuing the following linux commands:
tar xjvf geos-3.5.0.tar.bz2
cd geos-3.5.0
./configure --help
mkdir /exports/work/....
./configure --help,
./configure --prefix=eng_A... --enable-python
make
make check
make install
and similar for rgdal
I'm not sure how your HPC system is setup but as the previous comment implied, you need "geos" installed/loaded. The system I work on uses the "module" command to load programs, so I use "module available" to find where "geos" is on the system so you can load it.
Here's an example from another system: http://docs.hpc.shef.ac.uk/en/latest/iceberg/software/apps/r.html