I am trying to install some packages to work with spatial data such as sf, stars, rgdal, on a Mac Book Air M1 with Ventura.
To do so, I need to have gdal installed to my compute, which I have. My problem is that when I try to install a library (see example with sf here below), the gdal-config file is not found.
But I have it on my computer, so my question is :
Is there a way to indicate the path to R to find that file ?
I tried to to uninstall and reinstall gdal with homebrew, I have proj that is also a dependency rightly installed.
I need to install from sf from sources as it is needed to install the stars package.
(base)#MacBook-Air ~ % echo $PATH
/opt/homebrew/bin:/opt/anaconda3/bin:/opt/anaconda3/condabin:/Library/Frameworks/Python.framework/Versions/3.9/bin:/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
(base)#MacBook-Air ~ % whereis gdal-config
gdal-config: /opt/homebrew/bin/gdal-config /opt/homebrew/share/man/man1/gdal-config.1
(base)#MacBook-Air ~ % gdal-config
Usage: gdal-config [OPTIONS]
Options:
[--prefix[=DIR]]
[--libs]
[--dep-libs]
[--cflags]
[--datadir]
[--version]
[--ogr-enabled]
[--gnm-enabled]
[--formats]
> sessionInfo()
R version 4.0.3 (2020-10-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS 13.1
Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.0.3 tools_4.0.3
(base)#MacBook-Air ~ % proj
DeprecationWarning: PROJ_LIB environment variable is deprecated, and will be removed in a future release. You are encouraged to set PROJ_DATA instead.
Rel. 9.1.1, December 1st, 2022
usage: proj [-bdeEfiIlmorsStTvVwW [args]] [+opt[=arg] ...] [file ...]
P.S: I know there are a lots of related topics (Error: gdal-config not found while installing R dependent packages whereas gdal is installed), but all of them are on Linux, and the solutions are not working for me on Mac machine.
I solved it by reinstalling the latest version of R and R Studio.
Related
While working in a conda environment, I have been observing an unexpected behavior with R>=4.1.3. Namely, in terminal I can launch R and load GDAL-based libraries
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)
... STANDARD R MESSAGE ...
> library(sf)
Linking to GEOS 3.11.1, GDAL 3.6.2, PROJ 9.1.0; sf_use_s2() is TRUE
but in RStudio I get the error below
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-conda-linux-gnu (64-bit)
... STANDARD R MESSAGE ...
> library(sf)
Error: package or namespace load failed for ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/ivan/miniconda3/envs/r41_dev/lib/R/library/sf/libs/sf.so':
/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /home/ivan/miniconda3/envs/r41_dev/lib/R/library/sf/libs/../../../../libgdal.so.32)
Even though the behavior in terminal suggested that the package libstdc++ was installed, I have reinstalled it from the Gnome repo. The package is definitely installed on my machine.
apt list --installed | grep libstdc++
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
libstdc++-8-dev/focal,now 8.4.0-3ubuntu2 amd64 [installed,automatic]
libstdc++-9-dev/focal-updates,focal-security,focal-security,now 9.4.0-1ubuntu1~20.04.1 amd64 [installed,automatic]
libstdc++6/focal,now 11.1.0-1ubuntu1~20.04 amd64 [installed]
libstdc++6/focal,now 11.1.0-1ubuntu1~20.04 i386 [installed,automatic]
The Rstudio version is
RStudio 2022.07.2+576 "Spotted Wakerobin" Release (e7373ef832b49b2a9b88162cfe7eac5f22c40b34, 2022-09-06) for Ubuntu Bionic
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) QtWebEngine/5.12.8 Chrome/69.0.3497.128 Safari/537.36
And here the session info
> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: Ubuntu 20.04.5 LTS
Matrix products: default
BLAS/LAPACK: /home/ivan/miniconda3/envs/r41_dev/lib/libopenblasp-r0.3.21.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=de_DE.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=de_DE.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.1.3 magrittr_2.0.3 class_7.3-21 DBI_1.1.3 tools_4.1.3 units_0.8-1
[7] proxy_0.4-27 Rcpp_1.0.10 KernSmooth_2.23-20 grid_4.1.3 e1071_1.7-13 classInt_0.4-8
I have been using conda environments with RStudio for a while and I have come across multiple challenges but I don't seem capable to come around this one.
Can anyone help me understand how come RStudio cannot access this package while R can access it from terminal?
As as side note, the problem does not occur with R=4.0.5 and, to the best of my knowledge, did not occur with R=4.1 either. I certainly have update apt multiple times, but I cannot recall any major update to my system.
EDIT TO ADDRESS THE POINTS RAISED BY #merv IN COMMENTS
I have created several environments with different R versions in the attempt to frame and solve this problem, without success.
I've now followed these steps:
check conda version with conda -V, returns conda 22.11.1
create a fresh environment with conda create -n r41_test -c conda-forge r-base=4.1.3 r-sf -y
activate the environment with conda activate r41_test
1 save the conda list output to file and share it through this link
launch R from terminal
load the sf library with library(sf), the library loads without issues
close R with q() and type n to avoid workspace image to be saved
launch RStudio by typing rstudio (from the activated environment)
load library sf with library(sf) from the R console in RStudio, the library fails loading with the same error reported above.
While running this test, I have noticed something I had not noticed before: when the conda environment at step 2, the following message was printed, but the environment was successfully create. I could also activate it, launch R from terminal, and load the library despite the SafetyError.
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: |
SafetyError: The package for r-base located at /home/ivan/miniconda3/pkgs/r-base-4.1.3-h2f963a2_5
appears to be corrupted. The path 'lib/R/doc/html/packages.html'
has an incorrect size.
reported size: 3061 bytes
actual size: 55120 bytes
done
Previous my post, I have tried this procedure also on another laptop running Ubuntu 20.04. I barely use that machine so it may serve as an 'independent control'. I am not sure whether the SafetyError was thrown also there, but RStudio failed to load the package also on that machine.
I'm trying to open a .nc file with terra, but I'm getting the following error:
'' not recognised as a supported file format. (GDAL error 4)
Reproducible example here (16mb nc file):
if (!file.exists("HadISST_ice.nc")) {
download.file("https://www.metoffice.gov.uk/hadobs/hadisst/data/HadISST_ice.nc.gz","HadISST_ice.nc.gz")
R.utils:::gunzip("HadISST_ice.nc.gz")
}
library(terra)
hadISST <- rast('HadISST_ice.nc')
The file opens with raster::brick, but not with terra:rast, which if I understand correctly from #robert-hijmnan answer is because raster uses ncdf4 while terra uses GDAL.
In GDAL the .nc file opens with no issues via terminal:
% gdalinfo HadISST_ice.nc
Driver: netCDF/Network Common Data Format
Files: HadISST_ice.nc
Size is 512, 512
But I can't read the file with terra without throwing a GDAL error 4. Session info below:
print(sessionInfo())
R version 4.2.0 (2022-04-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.4
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] terra_1.5-34
loaded via a namespace (and not attached):
[1] compiler_4.2.0 tools_4.2.0 Rcpp_1.0.8.3 codetools_0.2-18
and GDAL:
% gdal-config --version
GDAL 3.5.0, released 2022/05/10
edit: the problem seems to be related to be a MacOS specific issue with terra not being installed with netCDF drivers:
R terra gdal version incorrect, cannot read .nc (GDAL error 4)
installing the development version of terra solves the problem:
install.packages('terra', repos='https://rspatial.r-universe.dev’)
TL;DR
Unable to install any package from GitHub, System Error 267
I've newly setup R (v4.1), Rstudio and just installed devtools.
I also additionally installed Rtools40 and added it to my path (global environment)
Rtools seems to work properly as validated with: Sys.which("make") & maketools::rtools_find() (output below)
Problem
I am unable to install any package from GitHub, e.g. rstudio/gt. The error is nearly identical for any GitHub Package, allways stating system error 267
devtools::install_github("rstudio/gt")
Downloading GitHub repo rstudio/gt#HEAD
Error: Failed to install 'gt' from GitHub:
create process 'C:/PROGRA~1/R/R-41~1.0/bin/x64/Rcmd.exe' (system error 267, Der Verzeichnisname ist ungültig.
) #win/processx.c:1040 (processx_exec
Update
As suggested I reinstalled R 4.1.0 to the most simple folde possible C:/R/
The error ist still the same despite now lacking tildes ~
devtools::install_github("rstudio/gt")
Downloading GitHub repo rstudio/gt#HEAD
Error: Failed to install 'gt' from GitHub:
create process 'C:/R/bin/x64/Rcmd.exe' (system error 267, Der Verzeichnisname ist ungültig.
) #win/processx.c:1040 (processx_exec)
Update 2:
Changed the .libPaths to a folder without any special characters
.libPaths( c( "D:/tmp" , .libPaths() ) )
.libPaths()
[1] "D:/tmp" "C:/Users/Björn/Documents/R/win-library/4.1"
[3] "C:/R/library"
Error is still the same
Update 3:
I updated some packages, and checked the version of packageVersion('processx')= 3.5.2
install.packages("testthat")
install.packages("pkgload")
install.packages("devtools")
install.packages("remote")
Content of Sys.getenv
Sys.getenv('Path')
[1] "C:\\rtools40\\usr\\bin;C:\\R\\bin\\x64;C:\\rtools40\\usr\\bin;C:\\rtools40\\mingw64\\bin;
Diagnostics
Session Info
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19042)
Matrix products: default
locale:
[1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252 LC_MONETARY=German_Germany.1252
[4] LC_NUMERIC=C LC_TIME=German_Germany.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
Rtools successful installed / setup
Sys.which("make")
make
"C:\\rtools40\\usr\\bin\\make.exe"
maketools::rtools_find()
$version
[1] ‘4.0’
$compiler
[1] "gcc 8.3.0"
$api
[1] ‘8’
$PATH
[1] "C:\\rtools40\\usr\\bin"
$BINPREF
[1] "C:/rtools40/mingw$(WIN)/bin/"
$available
[1] TRUE
$compatible
[1] TRUE
The standalone mode of the remotes package solved the issue for me,
as suggested by the maintainer of processx (Gábor Csárdi) here
devtools::install_github() only calls remotes::install_github().
However, for the remotes, there is the option to be exectued in standalone mode
Source: Cran
Standalone mode
remotes will use the curl, git2r and pkgbuild packages if they are
installed to provide faster implementations for some aspects of the
install process. However if you are using remotes to install or update
these packages (or their reverse dependencies) using them during
installation may fail (particularly on Windows).
If you set the environment variable R_REMOTES_STANDALONE="true" (e.g.
in R Sys.setenv(R_REMOTES_STANDALONE="true")) you can force remotes to
operate in standalone mode and use only its internal R
implementations. This will allow successful installation of these
packages
With the following lines of code, gt was finally successfull installed from github.
Sys.setenv(R_REMOTES_STANDALONE="true")
remotes::install_github("rstudio/gt")
Thanks all the commentators for your help!
Update October / 2021
To avoid having to do these steps (Set in standanlone mode, and install with remotes) over and over again everytime you want to install a new package from github another convenient workaround is to just rollback to the previous version of processx as adviced by #rempsy in the github issue:
install.packages("pacman")
pacman::p_del(processx)
# Installing previous verison 3.5.1
install.packages("https://cran.r-project.org/src/contrib/Archive/processx/processx_3.5.1.tar.gz", repos=NULL, type="source")
After the rollback of processx to version 3.5.1, devtools::install_github() works as expected, e.g.
devtools::install_github("rstudio/gt")
I have an issue with the library path of R. Maybe the answer is out there already, but I was not able to find it. I am using R together with RStudio Version 1.1.456. All my additional libraries used to be in a directory dir1 <- \\SHARE\TOOLS_OLD$\R-3.5.1 which I append with .libPaths(dir1). In this setup, everything works as expected.
The problem occured after I copied all the libraries in directory \\SHARE\TOOLS_OLD$\R-3.5.1 to a new location \\NAS\TOOLSNEW$\R-3.5.1. I did not update the R version and I want to keep all the package versions the same such that I have exactly the same setup as before. That is why I did not reinstall the packages in the new location.
When I open a new R session and run the three lines of code
dir2 <- \\NAS\TOOLSNEW$\R-3.5.1
.libPaths(dir2)
.libPaths()
I get the correct output of .libPaths()
[1] "\\\\NAS/TOOLSNEW$/R-3.5.1" "C:/Apps/R-3.5.1/library"
but R also throws an error:
Error: invalid version specification ‘NA’
In addition: Warning message:
In utils:::packageDescription(packageName, fields = "Version") :
no package 'knitr' was found
Apparently some packages remember the original location where they were installed. Is there a way to copy packages to a new location without reinstalling them? If that is not possible, is there an easy (i.e. non-manual) way to reinstall my package tree without upgrading any of the packages (including the dependencies) i.e. defining the version of each package?
Thanks for your help!
PS: Here is my session info, just in case.
R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
Matrix products: default
locale:
[1] LC_COLLATE=German_Switzerland.1252 LC_CTYPE=German_Switzerland.1252
LC_MONETARY=German_Switzerland.1252 LC_NUMERIC=C
[5] LC_TIME=German_Switzerland.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.5.1 tools_3.5.1 yaml_2.2.0
The problem can have many causes, e.g. there might be some dependencies which are lost by copying the package folder or that the registry entries don't match anymore, but I'm guessing here. When you copy packages from older versions you have to run update.packages (but this might update any package to the currently available version, which you don't want). Check out the R-FAQ and this Stackoverflow question.
You can install packages of a certain version with the devtools package.
install.packages("devtools")
require(devtools)
install_version(somePackage, version = neededVersion)
How I did it was I created a data.frame of all the required packages with the corresponding version number, like
> requiredPkgs
Package Version
condformat "condformat" "0.9.0"
DT "DT" "0.17"
formattable "formattable" "0.2.1"
ggplot2 "ggplot2" "3.3.3"
ggthemes "ggthemes" "4.2.4"
htmlTable "htmlTable" "2.1.0"
and then installed them in a loop
for(iPkg in requiredPkgs) {
install_version(iPkg[1], version = iPkg[2]
}
I am trying to install stringi package after a recent update of R to newest version. However, I met this error:
* installing *source* package ‘stringi’ ...
** package ‘stringi’ successfully unpacked and MD5 sums checked
checking for local ICUDT_DIR... icu55/data
checking for R_HOME... /usr/local/Cellar/r/3.4.3/lib/R
checking for R... /usr/local/Cellar/r/3.4.3/lib/R/bin/R
checking for R >= 3.1.0 for C++11 use... yes
checking for R < 3.4.0 for CXX1X flag use... no
checking for cat... /bin/cat
checking for gcc... /usr/local/opt/llvm/bin/clang -fopenmp
checking whether the C compiler works... no
configure: error: in `/private/var/folders/y5/m7pd62wn3939vyqxygrd_ff80000gn/T/RtmpNBCRW2/R.INSTALL8ec750ee123/stringi':
configure: error: C compiler cannot create executables
See `config.log' for more details
ERROR: configuration failed for package ‘stringi’
* removing ‘/usr/local/lib/R/3.4/site-library/stringi’
Warning in install.packages :
installation of package ‘stringi’ had non-zero exit status
Following some posts, I tried:
install.packages('stringi', configure.args='--disable-cxx11')
or:
install.packages('stringi', type='mac.binary')
How could I correctly install stringi, did I miss something?
Additional info: I updated R with Homebrew. I successfully installed data.table before trying to install stringi, therefore it seems to be caused by package-specific problem.
Here is my session info:
R version 3.4.3 (2017-11-30)
Platform: x86_64-apple-darwin17.2.0 (64-bit)
Running under: macOS High Sierra 10.13.2
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.3 tools_3.4.3
I got the same problem when updating the stringi package. As was stated here https://github.com/gagolews/stringi/issues/291, the issue wasn't with stringi but instead in ~/.R/Makevars. Installation was successful upon deleting this file using rm ~/.R/Makevars in your command line.
I have no idea about what caused the error. But finally I was able to install stringi through following steps:
remove all versions of R installed by homebrew with: brew uninstall --force r
download gfortran(not sure whether this is necessary, just following the installation guide in manual) and R 3.4.3 for Mac OS X from CRAN (I chose .pkg, while .dmg files are also available) and install them.
open studio and run install.packages('stringi'), which seems installed the binary package successfully.
Upgrade you compiler, here is an example works on openSUSE:
install newer version of compiler:
zypper in gcc7 gcc7-c++ gcc7-c++ gcc7-fortran
Edit your compiler config files, open /usr/lib64/R/etc/Makeconf, and add following content:
CC = gcc-7
CXX = g++-7
CXX98 = g++-7
CXX11 = g++-7
CXX14 = g++-7
CXX17 = g++-7
FC = gfortran-7
F77 = gfortran-7
Theoretically, you do not need to restart your R, just install the package you need directly.
Happy coding ;)
If you are on MacOS your xcode license might be expired. Try running:
sudo xcodebuild -license
from the command line and agreeing with the license.