Cannot install rpy2 in python 2.7 - r

I am attempting to install RPy2 in the Python environment (2.7.x) that comes bundled in Mac OS X Sierra (10.12.6), and as such, I am attempting to install the latest non-2.9.x version from Terminal using pip via the command:
pip install 'rpy2<2.9.0' --user
The --user flag is there because I have installed Homebrew on a user which is normally not admin and for which I turned on admin rights briefly for the sole purpose of installing Homebrew.
Every time I run this command, I get the following error:
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/1x/d_trx3556sl61by_tp4093j80000gq/T/pip-build-uV52fY/rpy2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/1x/d_trx3556sl61by_tp4093j80000gq/T/pip-LoW5FN-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /private/var/folders/1x/d_trx3556sl61by_tp4093j80000gq/T/pip-build-uV52fY/rpy2/
Looking back into the output, it shows the following at time of failure:
creating build/temp.macosx-10.12-intel-2.7/rpy/rinterface
/usr/local/Cellar/gcc/7.2.0/bin/gcc-7 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I./rpy/rinterface -I/Library/Frameworks/R.framework/Resources/include -c ./rpy/rinterface/r_utils.c -o build/temp.macosx-10.12-intel-2.7/./rpy/rinterface/r_utils.o
unable to execute '/usr/local/Cellar/gcc/7.2.0/bin/gcc-7': No such file or directory
error: command '/usr/local/Cellar/gcc/7.2.0/bin/gcc-7' failed with exit status 1
I am unable to find anyone who is getting a similar error, and I cannot figure out what is going on.
Does anyone know why this is failing?

I would discourage you from installing Python packages in your system's Python (unless you are happy with reinstalling the system if "something suddenly stopped working").
I also encourage you to use the latest rpy2 (the 2.9.x series are arguably much better than the 2.8.x series), and move to Python 3.
Finally, a binary wheel for rpy2-2.9.3 on OS X is now available on pypi (making the installation no longer require a development tools).

Related

How to fix /bin/sh: gcc-11.3.0_1: command not found

When trying to install RTidyHTML remotely from Github on RStudio (2022.02.2 Build 485) with R version 4.2.0, I'm told /bin/sh: gcc-11.3.0_1: command not found. This is after installing gcc through homebrew and creating an ~/.R/Makevars with the following details:
VER=-11.3.0_1
CC=gcc$(VER)
CXX=g++$(VER)
CFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
CXXFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
Running brew list --versions gcc, return gcc 11.3.0_1,
Running gcc --version returns:
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.27.3)
Target: arm64-apple-darwin21.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
My PATH is
/opt/homebrew/opt/openssl#3/bin:/Library/Frameworks/GDAL.framework/Programs:/Library/Frameworks/Python.framework/Versions/3.9/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
which gcc returns gcc is /usr/bin/gcc.
I don't know how to make RStudio recognize the correct path and thus how I'm able to compile packages. I'd appreciate troubleshooting advice.
Running macOS Monterey (V12.2) on an M1 Mac.
Xposted on https://community.rstudio.com/t/how-to-troubleshoot-c-compiler-problems-on-mac/137675 but no response.

r compile packages fails with Mac catalina

Since I installed Catalina, any attempt to compile an R package fails. My system is set up to use home-brew. The failure messages look like this:
/usr/local/clang8/bin/clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I/usr/local/clang8/include -I/usr/local/Cellar/gcc/8.3.0_2/include -fopenmp -fPIC -Wall -g -O2 -c R_register_native.c -o R_register_native.o
In file included from R_register_native.c:1:
/Library/Frameworks/R.framework/Resources/include/R.h:55:11: fatal error: 'stdlib.h' file not found
# include <stdlib.h> /* Not used by R itself, but widely assumed in packages */
After googling I found this link in the home-brew site on GitHub. I can't follow much of the discussion but it seems that Apple removed the usr/include directory and that the home-brew folks are looking for a solution. I don't immediately have a need to compile these packages so my plan is to wait until home-brew finds a solution. But if there is an easy one now, please share.
After some additional searching I found this link with a bunch of discussion about this problem, a lot of which I didn't understand. But this bit of code seemed promising.
sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/
I ran it, rebooted my mac, opened RStudio and got the list of new package updates. Ran the update, answering yes to compile. Lots of messages and some warnings but at the end of it all, there were no errors. But all the messages start with something like
/usr/local/clang8/bin/clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I/usr/local/clang8/include -I/usr/local/Cellar/gcc/8.3.0_2/include -fopenmp -fPIC -Wall -g -O2 -c R_register_native.c -o R_register_native.o
When I cd into /usr/local/clang8/bin and get the version of clang it is Apple clang version 11.0.0. Also clang symlinks to clang-8 and clang++ symlinks to clang
The contents of /usr/local/Cellar/gcc is a directory called 9.2.0_1. When I go to /usr/bin and run gcc --version I get messages about this being Apple clang version 11.0.0. It seems like R is compiling using the Apple clang version. Is that a problem? Is there some way to move everything to using the brew version of clang? FYI, here's my current path - /usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/Library/Apple/bin:/Library/TeX/texbin:/opt/X11/bin:/usr/local/CrossPack-AVR/bin:/usr/local/git/bin

R: can't update packages clang can't find head files

I have several R packages failed to be updated/complied from the source. Followings are snippets of error messages;
clang++ -std=gnu++11 -I"/usr/local/Cellar/r/3.5.2_2/lib/R/include" -DNDEBUG -I"/Volumes/Documents/Users/akihoji/Library/R/3.x/library/Rcpp/include" -I"/Volumes/Documents/Users/akihoji/Library/R/3.x/library/RcppArmadillo/include" -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/include -fPIC -g -O2 -c RcppExports.cpp -o RcppExports.o
In file included from RcppExports.cpp:4:
In file included from /Volumes/Documents/Users/akihoji/Library/R/3.x/library/RcppArmadillo/include/RcppArmadillo.h:31:
In file included from /Volumes/Documents/Users/akihoji/Library/R/3.x/library/RcppArmadillo/include/RcppArmadilloForward.h:26:
In file included from /Volumes/Documents/Users/akihoji/Library/R/3.x/library/Rcpp/include/RcppCommon.h:29:
In file included from /Volumes/Documents/Users/akihoji/Library/R/3.x/library/Rcpp/include/Rcpp/r/headers.h:59:
In file included from /Volumes/Documents/Users/akihoji/Library/R/3.x/library/Rcpp/include/Rcpp/platform/compiler.h:100:
In file included from /usr/local/Cellar/llvm/7.0.1/include/c++/v1/cmath:305:
/usr/local/Cellar/llvm/7.0.1/include/c++/v1/math.h:301:15: **fatal error: 'math.h' file not found**
Another one;
> ** libs clang -I"/usr/local/Cellar/r/3.5.2_2/lib/R/include" -DNDEBUG -I"/Volumes/Documents/Users/akihoji/Library/R/3.x/library/Matrix/include"
> -I/usr/local/opt/gettext/include -I/usr/local/opt/readline/include -I/usr/local/include -fPIC -g -O2 -c irlb.c -o irlb.o irlb.c:19:10: fatal error: 'stdlib.h' file not found
> #include <stdlib.h>
Error messages point to several missing headers but they are all there in my
/usr/local/Cellar/llvm/7.0.1/include/c++/v1
I'd appreciate any help solving this issue.
sw_vers
ProductName: Mac OS X
ProductVersion: 10.14.3
BuildVersion: 18D4
clang -v
clang version 7.0.1 (tags/RELEASE_701/final)
Target: x86_64-apple-darwin18.2.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
gcc --version
gcc (Homebrew GCC 8.2.0) 8.2.0
setting value
version R version 3.5.2 (2018-12-20)
os macOS Mojave 10.14.3
system x86_64, darwin17.7.0
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz America/New_York
date 2019-02-06
Update;
I just remembered that I had the exact same issue when I installed OSX Mojave in my other Mac. It turns out that /usr/include is absent even when you install command tools, particularly for the latest version by Xcode-select install.
My installed Xcode is
pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 10.1.0.0.1.1539992718
volume: /
location: /
install-time: 1549499389
groups: com.apple.FindSystemFiles.pkg-group
This CLT pkg will install /include in
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
/Library/Developer/CommandLineTools/usr/include
So all you have to do is either copy or link one of these include folders to /usr but in order to do this, you have to take disable System Integrity Protection by provoking crstutil in the recovery mode.
It seems like you can install packages from an R console if you cannot install them inside RStudio.
Source:
https://github.com/catboost/catboost/issues/137

R compilation failed for package fst

I'm trying to install fst R package from CRAN but I get error:
gcc -I"/path/apps/R/3.5.1-20180807-test/x86_64-linux-2.6-rhel6/lib64/R/include" -DNDEBUG
-fopenmp -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common
-Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress
-I"/path/apps/R/3.5.1-20180807-test/x86_64-linux-2.6-rhel6/r_libs_site/Rcpp/include"
-I /path/apps/R/static-zlib-gcc6.3/x86_64-linux-2.6-rhel6/include -fpic
-I /path/apps/R/static-zlib-gcc6.3/x86_64-linux-2.6-rhel6/include
-c fstcore/ZSTD/compress/huf_compress.c -o fstcore/ZSTD/compress/huf_compress.o
gcc -I"/path/apps/R/3.5.1-20180807-test/x86_64-linux-2.6-rhel6/lib64/R/include" -DNDEBUG
-fopenmp -I. -Ifstcore -Ifstcore_v1 -Ifstcore/LZ4 -Ifstcore/ZSTD -Ifstcore/ZSTD/common
-Ifstcore/ZSTD/decompress -Ifstcore/ZSTD/compress
-I"/path/apps/R/3.5.1-20180807-test/x86_64-linux-2.6-rhel6/r_libs_site/Rcpp/include"
-I /path/apps/R/static-zlib-gcc6.3/x86_64-linux-2.6-rhel6/include -fpic
-I /path/apps/R/static-zlib-gcc6.3/x86_64-linux-2.6-rhel6/include
-c fstcore/ZSTD/decompress/zstd_decompress.c -o fstcore/ZSTD/decompress/zstd_decompress.o
/tmp/cc0LoSFX.s: Assembler messages:
/tmp/cc0LoSFX.s:7353: Error: no such instruction: `shlx %rax,%rdx,%rax'
/tmp/cc0LoSFX.s:8204: Error: no such instruction: `shlx %rax,%rdx,%rax'
/tmp/cc0LoSFX.s:8608: Error: no such instruction: `shlx %rax,%rdx,%rax'
make: *** [fstcore/ZSTD/decompress/zstd_decompress.o] Error 1
ERROR: compilation failed for package ‘fst’
I there any required linux library that can be compiled from source? What else I can do?
EDIT:
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/path/apps/gcc/6.3.0/x86_64-linux-2.6-rhel6/bin/../libexec/gcc/x86_64-pc-linux-gnu/6.3.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ./configure --prefix=/path/apps/gcc/6.3.0/x86_64-linux-2.6-rhel6 --disable-multilib
Thread model: posix
gcc version 6.3.0 (GCC)
Putting this here for posterity in case others eventually land on this question when facing similar issues.
It turns out that as far as we can tell, the problem was incompatible versions of the assembler (/usr/bin/as) and gcc (our non-system 6.3.0 installation).
So if you landed here at the end of your rope with no idea why you're seeing these errors, looking into getting a compatible version of the assembler (in linux world this is part of the binutils module/project). Hope that helps.

Mac OS X R error "ld: warning: directory not found for option"

I am trying to install an R package from source, but getting an error:
* installing *source* package ‘mclust’ ...
** package ‘mclust’ successfully unpacked and MD5 sums checked
** libs
gfortran-4.8 -fPIC -g -O2 -c mclust.f -o mclust.o
gfortran-4.8 -fPIC -g -O2 -c mclustaddson.f -o mclustaddson.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o mclust.so mclust.o mclustaddson.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 -lgfortran -lquadmath -lm -L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2 -lgfortran -lquadmath -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
ld: warning: directory not found for option '-L/usr/local/lib/gcc/x86_64-apple-darwin13.0.0/4.8.2'
ld: library not found for -lquadmath
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [mclust.so] Error 1
ERROR: compilation failed for package ‘mclust’
* removing ‘/Library/Frameworks/R.framework/Versions/3.3/Resources/library/mclust’
Warning in install.packages :
installation of package ‘mclust’ had non-zero exit status
I don't have /usr/local/lib/gcc/x86_64-apple-darwin13.0.0, so it makes sense that it can't be found. I have /usr/local/lib/gcc/i686-apple-darwin11 and /usr/local/lib/gcc/4.8 (symlink to Homebrew installation). Where is it getting x86_64-apple-darwin13.0.0 from?
There are a lot of references to a similar error online. However, all of them are related to compiling in Xcode and resolved by updating project settings, which is not applicable here.
You need to modify the ~/.R/Makevars file. For a greater overview of this see: https://cran.r-project.org/doc/manuals/r-release/R-admin.html#OS-X-packages
Alternatively, this has been answered before in a bit more depth by #kevin-ushey in Rcpp warning: "directory not found for option '-L/usr/local/Cellar/gfortran/4.8.2/gfortran'".
What is happening is your code is not being run under gcc instead it is being forwarded to clang
You will need to change your compile statements in ~/.R/Makevars/ to gcc using:
VER=-5.3.0
CC=gcc$(VER)
CXX=g++$(VER)
CFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
CXXFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
FLIBS=-L/usr/local/Cellar/gcc/5.3.0/lib/gcc/5
This assumes you have already installed gcc via homebrew under:
brew install gcc
(gfortran ships with gcc in brew now)
Incorporating previous solutions with additional help from the comments, the following solution has worked for me on Mac OS X High Sierra.
Create/edit ~/.R/Makevars with the following contents:
VER=-8
CC=gcc$(VER)
CXX=g++$(VER)
CXX11=g++$(VER)
CXX14=g++$(VER)
CXX17=g++$(VER)
CFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
CXXFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
FLIBS=-L/usr/local/Cellar/gcc/8.2.0/lib/gcc/8
Note, I am using homebrew and have gcc version 8.2.0 installed.
From http://thecoatlessprofessor.com/programming/rcpp-rcpparmadillo-and-os-x-mavericks-lgfortran-and-lquadmath-error/ you can fix this by downloading the optional gfortran libraries from http://r.research.att.com/libs/ and extracting them. To do this on the command line
curl -O http://r.research.att.com/libs/gfortran-4.8.2-darwin13.tar.bz2
sudo tar fvxz gfortran-4.8.2-darwin13.tar.bz2 -C /
I had this issue for a long time, working on a Mac. Following the other answers on this page and other questions, I did the following and it finally worked!
Step 1 - brew install gcc (it installed gcc-11 for me. If this is different for you in the future, replace gcc-11 below with the corresponding version. You can find the version by checking in the folder /usr/local/bin and seeing the latest gcc version file there)
Step 2 - xcode-select --install (this installs Xcode CLI)
Step 3 - ln -sf /usr/local/bin/gcc-11 /usr/local/bin/gcc (this makes a new gcc symlink under /usr/local/bin/. Source)
Step 4 - I did not have a ~/.R/Makevars file. Heck, I didn't even have the ~./R folder. Made the folder and the file. Here is what I wrote in the Makevars file (remember to change the GCC version in the first and last line according to the version you have. Also remember to not have the entire version number that's in the last line, in the first line. Just the overall version number - 11 in my case) -
VER=-11
CC=gcc$(VER)
CXX=g++$(VER)
CXX11=g++$(VER)
CXX14=g++$(VER)
CXX17=g++$(VER)
CFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
CXXFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
FLIBS=-L/usr/local/Cellar/gcc/11.2.0_3/lib/gcc/11
Now restart, and voila! It works.
I'm working on MacOS Mojave (10.14.5) and on R 4.0.0. The problem here is that "CRAN R 4.0.0 builds and higher no longer use any custom compilers" (see here), so the Makevars solution does not appear to work anymore.
The solution for me was to download and install the GNU Fortran compiler from the official R-Project website. Note that you will also need Xcode and Xcode command-line tools.
After running the installer with default settings, compilation of gfortran code worked without problems.
I received the same error on MAC. All I needed to do was to install gfortran-6.1.pkg from https://cran.r-project.org/bin/macosx/tools/. Make sure that the package is installed under /usr/local/gfortran so it can be found by R.
Alternatively you can install it via homebrew typing brew cask install gfortran in the terminal (it may ask for your password).
In my case I combined this answer with this one, to produce the following code in the ~./R/.Makevars-file.
touch ~./R/.Makevars
(because it didn't exist there)
open -a BBEdit ~./R/.Makevars
(I use BBEdit as text editor)
Added the following lines to the Makevars-file:
VER=-11
CC=gcc$(VER)
CXX=g++$(VER)
CFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
CXXFLAGS=-mtune=native -g -O2 -Wall -pedantic -Wconversion
# FLIBS=-L/usr/local/Cellar/gcc/11.1.0_1/lib/gcc/11
# FLIBS = -L`gfortran -print-file-name=libgfortran.dylib | xargs dirname`
FLIBS=`gfortran -print-search-dirs | grep ^libraries: | sed 's|libraries: =||' | sed 's|:| -L|g' | sed 's|^|-L|'`
These two lines were suggested by #KevinUshy.
# FLIBS=-L/usr/local/Cellar/gcc/11.1.0_1/lib/gcc/11
# FLIBS = -L`gfortran -print-file-name=libgfortran.dylib | xargs dirname`
I commented these out, as I figure that the final line would probably work best.
Two notes.
I have brew installed and I used brew to install gcc using the command brew install gcc.
I figure out the version-naming using brew info gcc, which gave me 11.1.0_1, but it is linked as gcc-11, so VER=-11 in the .Makevars-file.
Hope this helps others.
I am having Mac OS Catalina and in my case installation of Homebrew, the newest gcc and Gfortran 8.2.0 solved the issue.
The solution was to re-install problematic packages with Homebrew.
$ brew uninstall --ignore-dependencies --force openssl
$ brew install openssl
$ brew uninstall --ignore-dependencies --force readline
$ brew install readline
Digging this topic, if you tried these and much others approaches related to this error and got no success, I strongly recommend you to check if your OSx - just in case - deleted the .zshrc file.
If that's the case, you should create again this file in the root dir. type in your terminal:
touch .zshrc
followed by
openssl_prefix=$(brew --prefix openssl) printf "CFLAGS=-I$openssl_prefix/include \nLDFLAGS=-L$openssl_prefix/lib"
example of output:
CFLAGS=-I/usr/local/opt/openssl#3/include LDFLAGS=-L/usr/local/opt/openssl#3/lib
add them to the zshrcfile content, run
source .zshrc
to refresh the file and that`s all!
Had this problem using R on macos 12.4. It was caused by old and stale entries in my ~/.R/Makevars file. Once I commented everything out everything compiled just fine. I have XCode and brew install gcc active.
For later versions of R (R 4.0 and higher) and later operating systems, the recommended fix for this problem seems to be to install gfortran via rtools (see here, already mentioned in the answer of #lks_swrx).
When following these instructions, I ran into an additional problem, since my computer uses Apple Silicon chips (not Intel anymore), for which I wanted to add a solution here:
Download the appropriate gfortran tarball from rtools
Run tar fxz gfortran-12.0.1-20220312-is-darwin20-arm64.tar.xz -C /
Add it to your path by adding this line to your .zshrc file:
nano .zshrc (or whatever you use as text editor)
export PATH=$PATH:/opt/R/arm64/gfortran/bin
So far, so good; but the installation of DESeq2, which brought me here, did still not work, because R could not find the library. This is the fix:
Create a symlink to the gfortran library in /usr/local, so that R can find it:
ln -s /opt/R/arm64/gfortran /usr/local/gfortran (might require sudo)

Resources