I was using Windows as OS and RStudio for windows, switched to Ubuntu today and installed R and RStudio again. When I try to install some packages from CRAN (only tidyverse !) using install.packages(), I see something something on console I have never seen on Windows, it looks like this;
* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
gcc -std=gnu99 9.3.0
zlib 1.2.11 is available ok
OpenMP supported
** libs
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -fopenmp -fpic -g -O2 -fdebug-prefix-map=/build/r-base-5iUtQS/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c assign.c -o assign.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -fopenmp -fpic -g -O2 -fdebug-prefix-map=/build/r-base-5iUtQS/r-base-4.0.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c between.c -o between.o
...
then it continues like this and installing process is taking much longer than the time I encountered on Windows. (tidyverse which includes about seven packages took 30 minutes)
Is this normal ? if it is not, how can I solve it ?
A summary of the comments that helped resolve the issue.
In general, the default installation method in linux is type="source", which means that any package with non-R source code (e.g., C++, Fortran) needs to be compiled. Further, CRAN and all of its repository mirrors only provide binary packages windows and macos.
It appears that there are two ways to install binary packages on linux vice the default:
#duckmayr's suggestion to read https://cran.r-project.org/bin/linux/ubuntu/README.html (similar pages exist for other linux distros) identifies how to set up the CRAN2deb4ubuntu PPA, so that one can the the OS-level software management apt (and related tools). These ubuntu/debian packages are maintained well and reasonably up-to-date, though not nearly as frequently updated as a straight-shot to the CRAN servers.
But instead of the long-running R command install.packages("tidyverse"), you can simply run this for a binary installation, same effect:
$ sudo apt-get install r-cran-tidyverse
Unfortunately, the PPA does not include every single contributed package supplied to CRAN (as that would take significant effort on the PPA maintainers' side, trying to keep up with CRAN's daily onslaught of new packages and package-updates ... the testing alone sounds prohibitive). (The number suggested is 4000+ packages in the PPA, out of CRAN's 16,278 packages (as of 30 seconds ago).
A recent addition to the repository scene is RStudio's Public Package Manager. In a recent blog post, RStudio announced it to have (at least) three fundamental features:
Access to pre-compiled packages on Linux via install.packages ...
Historical checkpoints for CRAN enabling reproducible work ...
Expanded Windows support for older versions of R ...
(That's just a small snapshot of the blog post, I suggest you read the original for more details and context.)
A quick search (of packages for Ubuntu 20.04 Focal) reveals: 15,217 binary and 16,216 source packages.
Borrowing from RStudio's "Setup" page, for Ubuntu 20.04 Focal you can set this as your repository with
options(repos = c(PkgMgr="https://packagemanager.rstudio.com/all/__linux__/focal/latest"))
(I named it "PkgMgr", that's arbitrary. You may want/need more repos, over to you, see ?options and ?setRepository. This is a sample only, provided for convenience; please go to RStudio's documentation for how to set up your R for your installation.)
Related
I've inherited some F77 code from the 1980s that I'd like to eventually use in an R package that is on CRAN. Basically I'll be substituting a slow function written in straight R with a .Call to a F77 subroutine that runs 100x faster. I've done similar heavy lifting with C in the past. However, when I compile the fortran subroutine via R CMD SHLIB I get a warning about one of the arrays being "larger than limit set by '-fmax-stack-var-size='". I can fix this by compiling explicitly with gfortran and setting -fmax-stack-var-size appropriately. However, how can I set the flag when running R CMD SHLIB? In a Makevars file? Eventually, I'll include this in a new release and submit to CRAN and I want to head off any issues. Many thanks.
Yes, in ~/.R/Makevars for your compilations, in src/Makevars for all builds of the package. CRAN has views on which options are portable so you may not be able to ship with the option in src/Makevars.
Here is what I have:
edd#rob:~$ grep ^F .R/Makevars
FLAGS=-Wall -O3 -g -pipe $(PEDANTIC) $(XTRAFLAGS)
FFLAGS=-O3 -g0 -Wall -pipe
FCFLAGS=-O3 -g0 -Wall -pipe
FC=$(CCACHE) gfortran
F77=$(CCACHE) gfortran
F95=$(CCACHE) gfortran
edd#rob:~$
where some of the other values are previoysly set as you can surmise. Note that there are several Fortran compiler variables, details as usual in Writing R Extensions.
I have been having some difficulties with my R install. I realised that I was unable to install packages and thought that it was due to having been tinkering with the packages I had installed recently. As I am a novice I thought that a clean install might fix it so I uninstalled R and Rstudio (via homebrew).
On reinstall I'm faced with the same issue. This is part of the console output from
install.packages("tidyverse")
/usr/local/opt/llvm/bin/clang -fopenmp -I"/usr/local/Cellar/r/3.5.1/lib/R/include" -DNDEBUG -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include -fPIC -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c assumptions.c -o assumptions.o
In file included from assumptions.c:21:
/usr/local/Cellar/r/3.5.1/lib/R/include/Rinternals.h:39:11: fatal error: 'stdio.h' file not found
# include <stdio.h>
^~~~~~~~~
1 error generated.
Searching this site and others suggested that this is to do with lacking the Xcode command line tools. I have checked that these are installed.
Searching for stdio.h in terminal shows numerous versions in the various subfolders (tvos, iOS etc.)
other missing files include: stdlib.h, assert.h and unistd.h
I can provide the full output if that would help
Further searches have identified that in Xcode 10 (as is bundled with macOS Mojave), the location of the standard path for stdio.h and other utilities has moved.
"Legacy software" that looks for the macOS headers in the base system under /usr/include will not find the required files.
To patch this, Xcode includes a package to create links for such software to find the files.
For Xcode 10 the package file is located at:
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg
This information was sourced from the apple Developer forums
After installing this package and restarting the computer I have successfully installed the tidyverse.
I did run into another issue where my C compiler was not able to create executables but this was solved in this thread.
RcppArmadillo 0.8.400.0.0 works just fine on my computer, both from sourceCpp and cxxfunction. Now I want to upgrade to 0.8.500.0, which requires compilation from source. Then I got the error
ccache /usr/local/clang6/bin/clang++ -std=gnu++11 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I"/Library/Frameworks/R.framework/Versions/3.5/Resources/library/Rcpp/include" -I/usr/local/include -I/Applications/CPLEX_Studio128/cplex/include -I/Applications/CPLEX_Studio128/concert/include -DIL_STD -fPIC -Wall -g -O2 -c RcppArmadillo.cpp -o RcppArmadillo.o
RcppArmadillo.cpp:22:10: fatal error: 'RcppArmadillo.h' file not found
#include <RcppArmadillo.h>
^~~~~~~~~~~~~~~~~
1 error generated.
I got this error whether I ran install.packages("RcppArmadillo") in RStudio or in R console outside RStudio. My compiler is clang4, and I have no problem installing RcppParallel or compiling my own Rcpp code that uses RcppArmadillo and RcppParallel. R version: 3.5.0, MacOS High Sierra.
Edit: I looked at https://github.com/RcppCore/RcppArmadillo/issues/143, which is very relevant. I did install clang6 and gfortran 6.1, both downloaded from the CRAN website. I also changed the .R/Makevars file to reflect the changes to clang6. I have also installed ccache as suggested, to make compilation faster. Here's my Makevars:
F77 = /usr/local/gfortran/bin/gfortran
FC = $F77
FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin16/6.3.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm
CC=ccache /usr/local/clang6/bin/clang
SHLIB_CXXLD=ccache /usr/local/clang6/bin/clang++
CXX= ccache /usr/local/clang6/bin/clang++ -Wall
CXX1X= ccache /usr/local/clang6/bin/clang++
CXX98= ccache /usr/local/clang6/bin/clang++
CXX11= ccache /usr/local/clang6/bin/clang++
CXX14= ccache /usr/local/clang6/bin/clang++
CXX17= ccache /usr/local/clang6/bin/clang++
LDFLAGS=-L/usr/local/clang6/lib
From the outputs in the R console when I was installing the package (added to the error message above), I can tell that clang6 was used for compilation. I also found the downloaded RcppArmadillo tarball, unpacked it, and ran ./configure from the command line. Nothing seemed wrong there; no config.log file was generated. This doesn't seem to be a configuration problem. Then I still got the exact same error when trying to install the package in R, after configuration, in libs. I also didn't get any message about -fopenmp. It seems that my problem is a bit different from that in the GitHub ticket.
R 3.5.x requires clang6 and adding clang6 to your system PATH variable.
Steps:
x Download and install the clang-6.0.0.pkg official package from CRAN.
x Delete your ~/.R/Makevars as it is likely set to use clang4. To do this, open Terminal and type:
sudo rm ~/.R/Makevars
x Then add to your ~/.bash_profile the following:
if [ -d "/usr/local/clang6/bin" ] ; then
export PATH="/usr/local/clang6/bin:$PATH"
fi
Note: If you are using a shell other than bash you will need to add the above into the appropriate file, e.g. for zsh add to ~/.zshrc, as this was not implemented in the official installer.
I should have a few cycles to finish writing a new unofficial Rtools installer a bit later. See: https://github.com/coatless/r-macos-rtools/issues/7
Note: Alternatively, you can remove the paths individual with
sudo vi ~/.R/Makevars
Pressing I to enter insert mode, deleting lines with clang4 in the path, and then pressing ESC + :wq to write changes to file.
Never mind, this problem went away with version 0.8.600. Later, this problem appeared again when I upgraded to RcppArmadillo 0.9.100.5.0. Somehow I only got this error when building from source; I downloaded the binary manually from the CRAN website, and used R CMD INSTALL to install the prebuilt version, and it worked; somehow R doesn't seem to know that there is a binary available for the newest version.
I did install clang6 and modified .R/Makevars to use clang6, but the same problem persisted; probably something else caused the problem.
I have now been searching the web and tried different methods to get a R library (ctsmr) installed from the source, but without luck. I know it works on Linux and Windows, but have not succeeded on my Mac (OSX v10.12.6) with R v3.3.3.
When I try to install it like this:
install.packages("ctsmr", repo="http://ctsm.info/repo/dev", type="source")
I get the some warnings during the installation process, but the installation finishes.
* installing *source* package ‘ctsmr’ ...
** libs
gfortran-4.8 -fopenmp -fPIC -Wall -g -O2 -c fdf.f90 -o fdf.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘16.7.0
fdf.f90:19.30:
INTEGER I,ISTOP
1
Warning: Unused variable 'istop' declared at (1)
gfortran-4.8 -fPIC -g -O2 -c matutil.f -o matutil.o
gfortran-4.8: warning: couldn’t understand kern.osversion ‘16.7.0
gfortran-4.8 -fPIC -g -O2 -c opkda1.f -o opkda1.o
...
When I run an example in RStudio (the one that is called "Building 1" on the packages webpage, http://ctsm.info) I get an error message for one of the function from the package. (This error message actually does not show up at the moment, as RStudio just crash when I run the line):
fit <- model$estimate(data=X)
Error in dyn.load(x = m$libfile) :
unable to load shared object '/var/folders/7g/81dqdb9s4qq2r79ncggcv2l40000gn/T//RtmpNoNnQl/model5a613c359b3f.so':
dlopen(/var/folders/7g/81dqdb9s4qq2r79ncggcv2l40000gn/T//RtmpNoNnQl/model5a613c359b3f.so, 6): Symbol not found: _GOMP_parallel_end
Referenced from: /var/folders/7g/81dqdb9s4qq2r79ncggcv2l40000gn/T//RtmpNoNnQl/model5a613c359b3f.so
Expected in: dynamic lookup
It looks a little different if I run the same script in the R application:
fit <- model$estimate(data=X)
*** caught illegal operation ***
address 0x7fffa39b0e97, cause 'illegal opcode'
Trackback:
1: dyn.load(x = m$libfile)
2: CallFunction("ctsmdriver", .self, rho, verbose = verbose)
3: model$estimate(data = X)
So, I am wondering if the package is not installed correctly or if their is a bug in the package (maybe because of the double slashes in the path in the error message).
If I manually go look for the shared object, model5a613c359b3f.so, it actually exist on my computer.
Regarding the gfortran compiler I get the following form the therminal:
~ $ gfortran --version
GNU Fortran (GCC) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
~ $ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
I would appreciate if someone could tell me if it is a installation error or a bug in the package. And secondly, come up with possible solutions.
I am trying to install the R package nloptr on a CentOS Linux machine that doesn't have internet connection as follows:
install.packages("/home/ravi/nloptr_1.0.4.tar.gz", repos = NULL, type="source")
This command in turn looks for the following file online
http://ab-initio.mit.edu/nlopt/nlopt-2.4.2.tar.gz
However, this fails since there is no internet connection to the machine.
I tried the suggestion from the following stackoverflow post:
trouble with Installing nloptr by locally on Ubuntu
I changed the URL in configure and configure.ac files as follows:
NLOPT_URL="file:///home//ravi//${NLOPT_TGZ}"
However, I get the following error when I try to install the package again:
> install.packages("/home/ravi/nloptr_1.0.4.tar.gz", repos = NULL, type="source")
* installing *source* package 'nloptr' ...
files 'configure', 'configure.ac' have the wrong MD5 checksums
ERROR: 'configure' exists but is not executable -- see the 'R Installation and Administration Manual'
* removing '/opt/vertica/R/library/nloptr'
Warning message:
In install.packages("/home/ravi/nloptr_1.0.4.tar.gz", :
installation of package '/home/ravi/nloptr_1.0.4.tar.gz' had non-zero exit status
Can someone guide me on how to install this R package locally?
Update 1
Based on the suggestion from Dirk on installing nlopt first, I followed the instructions given in the following page:
http://ab-initio.mit.edu/wiki/index.php/NLopt_Installation
I installed nlopt as follows :
./configure --enable-shared
make
make install
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib
When I tried to re-install nloptr in R, it doesn't look for the nlopt link anymore but throws the following error:
Error in dyn.load(file, DLLpath = DLLpath, ...) :
unable to load shared object '/opt/vertica/R/library/nloptr/libs/nloptr.so':
/opt/vertica/R/library/nloptr/libs/nloptr.so: undefined symbol: nlopt_set_maxtime
Error: loading failed
Execution halted
ERROR: loading failed
* removing '/opt/vertica/R/library/nloptr'
Warning message:
In install.packages("/home/ravi/nloptr_1.0.4.tar.gz", :
installation of package '/home/ravi/nloptr_1.0.4.tar.gz' had non-zero exit status
Update 2
As suggested by Dirk, I looked into the ldconfig command and used the following reference:
http://codeyarns.com/2014/01/14/how-to-add-library-directory-to-ldconfig-cache/
I edited the /etc/ld.so.conf file, added the directory /usr/local/lib which contains the shared library and ran the ldconfig command. This added the relevant shared library as shown below:
libnlopt.so.0 (libc6,x86-64) => /usr/local/lib/libnlopt.so.0
libnlopt.so (libc6,x86-64) => /usr/local/lib/libnlopt.so
However, when I tried reinstalling the nloptr package, I still get the same shared object error.
Could someone guide me on the shared library error?
When you say [t]his command in turn looks for the following file online you only get half the story. Together with Jelmer, the maintainer of the actual nloptr package, I modified the package to do the following:
look for an install libnlopt library, and, if found, use it
if not found fall back to the old behaviour and download the library
So you could simply install nlopt via
sudo apt-get install libnlopt-dev
(or the equivalent sudo dpkg -i /media/.... pointing to the file from a
USB drive etc pp) and then reinstall the nloptr package. It will just work. On my machine:
edd#max:~$ install.r nloptr ## install.r is in littler
trying URL 'http://cran.rstudio.com/src/contrib/nloptr_1.0.4.tar.gz'
Content type 'application/x-gzip' length 353942 bytes (345 KB)
==================================================
downloaded 345 KB
* installing *source* package ‘nloptr’ ...
** package ‘nloptr’ successfully unpacked and MD5 sums checked
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking how to run the C++ preprocessor... g++ -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking for pkg-config... yes
configure: Now testing for NLopt header file.
[...]
checking for nlopt.h... yes
configure: Suitable NLopt library found.
configure: creating ./config.status
config.status: creating src/Makevars
** libs
g++ -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -O3 -Wall -pipe -Wno-unused -pedantic -c dummy.cpp -o dummy.o
gcc -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -O3 -Wall -pipe -pedantic -std=gnu99 -c nloptr.c -o nloptr.o
g++ -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o nloptr.so dummy.o nloptr.o -lnlopt -lm -L/usr/lib/R/lib -lR
installing to /usr/local/lib/R/site-library/nloptr/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (nloptr)
The downloaded source packages are in
‘/tmp/downloaded_packages’
edd#max:~$
Note how it compiled only two files from the actual R packages having found the nlopt installation.
I had exactly the same problem, on a production machine without direct Internet access, with Red Hat Enterprise Linux Server release 6.6 (Santiago). The fixups proposed above did not work: when installing the nlopt library on the system, the subsequent attempt to install nloptr fail with the "nlopt_set_maxtime" error. I tried to fiddle with compilation options for nlopt, to no avail. I even reproduced the problem on a Ubuntu machine with Internet connection: I uninstalled nloptr, installed nlopt on the system and then the nloptr installation failed.
At last I solved the issue taking the following steps:
Un-install nlopt from the operating system (make sure that it is thoroughly removed).
Manually download nlopt-2.4.2.tar.gz and copy it to /tmp/
Manually download nloptr_1.0.4.tar.gz to a work directory and unpack it; this creates the directory "nloptr/".
Edit "nloptr/configure", comment out the row
$("${R_HOME}/bin/Rscript" --vanilla -e
"download.file(url='${NLOPT_URL}', destfile='${NLOPT_TGZ}')")
and insert a new row:
$("${R_HOME}/bin/Rscript" --vanilla -e
"file.copy(from='/tmp/nlopt-2.4.2.tar.gz', to='${NLOPT_TGZ}')")
Install with "R CMD INSTALL nloptr".
I had a similar problem on Ubuntu. In addition to installing nlopt-dev as Dirk answered, I also had to install pkg-config:
sudo apt-get install pkg-config
Hope this helps.
If you are on CentOS (and have internet access, but the nlopt server is not reachable), you can install nlopt first using
yum install nlopt nlopt-devel
Then you can install nloptr normally.
I had the same issue setting up nloptr on my RHEL 7.2 instance. For those using RHEL you can do the following:
Download & install the following RPMS in the same order:
NLopt-2.4.2-2.el7.x86_64.rpm- https://pkgs.org/centos-7/epel-x86_64/NLopt-2.4.2-2.el7.x86_64.rpm.html
NLopt-devel-2.4.2-2.el7.x86_64.rpm - https://pkgs.org/centos-7/epel-x86_64/NLopt-devel-2.4.2-2.el7.x86_64.rpm.htm
Please note that these RPMs may need additional dependencies to be installed based on your system configuration.
Post this you can install nloptr directly using the install.packages utility or download source from CRAN and install from source as suggested in earlier responses.
Hope this helps!
Kiran
if you are using AWS Linux the below should help
yum --enablerepo=epel install NLopt-devel -y
I just solved this on my server with following two steps:
Install libnlopt-dev and nlopt with
sudo apt-get install libnlopt-dev
pip install nlopt
Use sudo R to launch R and type:
withr::with_makevars(c(PKG_LIBS='-lnlopt'),install.packages('nloptr'),assignment='+=')
I think I have the same problem as you and I am using CentOS 6.5
Remove nlopt and install nloptr with install.packages() in R, a nlopt will be installed automatically.
I removed installed nlopt with the following command in the unzipped nlopt folder.
make uninstall
If you had tried to use epel repository in CentOS and had installed nlopt with yum, try:
yum remove nlopt
After removing existed nlopt, I am able to install nloptr with
install.packages("nloptr")