renv fails on install.packages() and restore() - r

In both new (dummy) packages and mature packages with renv already initialized, renv fails on install.packages() and restore().
Note: to avoid tripping the spam filter, I've replaced all instances of 'https://' with 'web.'
Failure on install.packages(), with, e.g., dplyr:
> install.packages("dplyr")
Retrieving 'web.cloud.r-project.org/bin/macosx/contrib/4.2/dplyr_1.1.0.tgz' ...
OK [downloaded 1.5 Mb in 0.4 secs]
Retrieving 'web.cloud.r-project.org/bin/macosx/contrib/4.2/cli_3.6.0.tgz' ...
OK [file is up to date]
[...]
Retrieving 'web.cloud.r-project.org/bin/macosx/contrib/4.2/withr_2.5.0.tgz' ...
OK [file is up to date]
Installing cli [3.6.0] ...
FAILED
Error in if (eval(cond, envir = environment(dot))) return(eval(expr, envir = environment(dot))) :
the condition has length > 1
In addition: Warning message:
In system2(R(), args, stdout = TRUE, stderr = TRUE) :
running command ''/Library/Frameworks/R.framework/Resources/bin/R' CMD config CC 2>&1' had status 71
Failure on renv::restore():
> renv::restore()
The following package(s) will be updated:
# CRAN ===============================
- KernSmooth [2.23-20 -> 2.23-18]
- MASS [7.3-58.2 -> 7.3-53.1]
[...]
- zip [* -> 2.2.2]
# GitHub =============================
- staged.dependencies [* -> remoteproject/staged.dependencies#HEAD]
Do you want to proceed? [y/N]: y
* Querying repositories for available binary packages ... Done!
* Querying repositories for available source packages ... Done!
Retrieving 'web.cran.microsoft.com/snapshot/2021-03-31/src/contrib/boot_1.3-27.tar.gz' ...
OK [file is up to date]
Retrieving 'web.cran.microsoft.com/snapshot/2021-03-31/src/contrib/class_7.3-18.tar.gz' ...
OK [file is up to date]
[...]
Retrieving 'web.cran.microsoft.com/snapshot/2021-03-31/src/contrib/visNetwork_2.0.9.tar.gz' ...
OK [file is up to date]
Installing boot [1.3-27] ...
FAILED
Error in if (eval(cond, envir = environment(dot))) return(eval(expr, envir = environment(dot))) :
the condition has length > 1
In addition: Warning messages:
1: could not retrieve available packages for url 'web.cran.microsoft.com/snapshot/2021-03-31/bin/macosx/contrib/4.2'
2: In system2(R(), args, stdout = TRUE, stderr = TRUE) :
running command ''/Library/Frameworks/R.framework/Resources/bin/R' CMD config CC 2>&1' had status 71
Session info:
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16
Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
locale:
[1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8
attached base packages:
[1] stats graphics grDevices datasets utils methods base
loaded via a namespace (and not attached):
[1] MASS_7.3-58.2 compiler_4.2.1 tools_4.2.1 renv_0.16.0
Judging from answers here and in other forums, the following might be useful information:
Output from getOption("repos"):
> getOption("repos")
CRAN MRAN
"web.cloud.r-project.org" "web.cran.microsoft.com/snapshot/2021-03-31"
Output from installing directly from utils package:
> utils::install.packages("dplyr")
Installing package into ‘/Users/dlei/Library/Caches/org.R-project.R/R/renv/library/project-01109165/R-4.2/x86_64-apple-darwin17.0’
(as ‘lib’ is unspecified)
also installing the dependencies ‘fansi’, ‘utf8’, ‘pkgconfig’, ‘withr’, ‘cli’, ‘generics’, ‘glue’, ‘lifecycle’, ‘magrittr’, ‘pillar’, ‘R6’, ‘rlang’, ‘tibble’, ‘tidyselect’, ‘vctrs’
Warning: unable to access index for repository web.cran.microsoft.com/snapshot/2021-03-31/bin/macosx/contrib/4.2:
cannot open URL 'web.cran.microsoft.com/snapshot/2021-03-31/bin/macosx/contrib/4.2/PACKAGES'
trying URL 'web.cloud.r-project.org/bin/macosx/contrib/4.2/fansi_1.0.4.tgz'
Content type 'application/x-gzip' length 364195 bytes (355 KB)
==================================================
downloaded 355 KB
trying URL 'web.cloud.r-project.org/bin/macosx/contrib/4.2/utf8_1.2.3.tgz'
Content type 'application/x-gzip' length 196823 bytes (192 KB)
==================================================
downloaded 192 KB
[...]
The downloaded binary packages are in
/var/folders/41/y38m8sw12871hpn_5nl9y88m0000gn/T//RtmpIwcHeu/downloaded_packages
Output from renv::diagnostics():
> renv::diagnostics()
Diagnostics Report [renv 0.16.0]
================================
# Session Info =======================
R version 4.2.1 (2022-06-23)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Big Sur ... 10.16
Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib
locale:
[1] en_CA.UTF-8/en_CA.UTF-8/en_CA.UTF-8/C/en_CA.UTF-8/en_CA.UTF-8
attached base packages:
[1] stats graphics grDevices datasets utils methods base
loaded via a namespace (and not attached):
[1] MASS_7.3-58.2 compiler_4.2.1 tools_4.2.1 renv_0.16.0
# Project ============================
Project path: "~/Desktop/Git_Repos/project"
# Status =============================
The following package(s) do not appear to be used in this project:
_
BH [1.75.0-0]
KernSmooth [2.23-18]
[...]
zip [2.2.2]
Use `renv::snapshot()` to remove them from the lockfile.
The following package(s) are recorded in the lockfile, but not installed:
_
DT [0.17]
covr [3.5.1]
[...]
staged.dependencies [remoteproject/staged.dependencies#HEAD]
usethis [2.1.6]
Use `renv::restore()` to install these packages.
The following package(s) are out of sync:
Package Lockfile Version Library Version
R6 2.5.0 2.5.1
cli 3.4.1 3.6.0
[...]
vctrs 0.5.0 0.5.2
Use `renv::snapshot()` to save the state of your library to the lockfile.
Use `renv::restore()` to restore your library from the lockfile.
The following package(s) are used in the project, but are not installed:
project.test
projectdev
Consider installing these packages, and then using `renv::snapshot()`
to record these packages in the lockfile.
# Packages ===========================
Library Source Lockfile Source Path Dependency
BH <NA> <NA> 1.75.0-0 CRAN <NA> <NA>
DT <NA> <NA> 0.17 CRAN <NA> direct
[...]
zip <NA> <NA> 2.2.2 CRAN <NA> <NA>
[1]: /Users/dlei/Library/Caches/org.R-project.R/R/renv/library/project-01109165/R-4.2/x86_64-apple-darwin17.0
[2]: /Users/dlei/Desktop/Git_Repos/project/renv/sandbox/R-4.2/x86_64-apple-darwin17.0/84ba8b13
# ABI ================================
* No ABI conflicts were detected in the set of installed packages.
# User Profile =======================
[no user profile detected]
# Settings ===========================
List of 10
$ bioconductor.version : chr(0)
$ external.libraries : chr(0)
$ ignored.packages : chr(0)
$ package.dependency.fields: chr [1:3] "Imports" "Depends" "LinkingTo"
$ r.version : chr(0)
$ snapshot.type : chr "implicit"
$ use.cache : logi TRUE
$ vcs.ignore.cellar : logi TRUE
$ vcs.ignore.library : logi TRUE
$ vcs.ignore.local : logi TRUE
# Options ============================
List of 9
$ defaultPackages : chr [1:6] "datasets" "utils" "grDevices" "graphics" ...
$ download.file.method : chr "libcurl"
$ download.file.extra : NULL
$ install.packages.compile.from.source: chr "interactive"
$ pkgType : chr "both"
$ repos : Named chr [1:2] "web.cloud.r-project.org" "web.cran.microsoft.com/snapshot/2021-03-31"
..- attr(*, "names")= chr [1:2] "CRAN" "MRAN"
$ renv.consent : logi TRUE
$ renv.project.path : chr "/Users/dlei/Desktop/Git_Repos/project"
$ renv.verbose : logi TRUE
# Environment Variables ==============
HOME = /Users/dlei
LANG = en_CA.UTF-8
MAKE = make
R_LIBS = <NA>
R_LIBS_SITE = /Library/Frameworks/R.framework/Resources/site-library
R_LIBS_USER = /Users/dlei/Library/Caches/org.R-project.R/R/renv/library/project-01109165/R-4.2/x86_64-apple-darwin17.0:/Users/dlei/Desktop/Git_Repos/project/renv/sandbox/R-4.2/x86_64-apple-darwin17.0/84ba8b13
RENV_DEFAULT_R_ENVIRON = <NA>
RENV_DEFAULT_R_ENVIRON_USER = <NA>
RENV_DEFAULT_R_LIBS = <NA>
RENV_DEFAULT_R_LIBS_SITE = /Library/Frameworks/R.framework/Resources/site-library
RENV_DEFAULT_R_LIBS_USER = /Users/dlei/Library/R/x86_64/4.2/library
RENV_DEFAULT_R_PROFILE = <NA>
RENV_DEFAULT_R_PROFILE_USER = <NA>
RENV_PROJECT = /Users/dlei/Desktop/Git_Repos/project
# PATH ===============================
- /usr/local/bin
- /usr/bin
- /bin
- /usr/sbin
- /sbin
- /Library/TeX/texbin
-
- /Applications/RStudio.app/Contents/MacOS/postback
# Cache ==============================
There are a total of 0 package(s) installed in the renv cache.
Cache path: "~/Library/Caches/org.R-project.R/R/renv/cache/v5/R-4.2/x86_64-apple-darwin17.0"
Output of getOption("download.file.method"):
> getOption("download.file.method")
[1] "libcurl"
Output of renv:::renv_download_method():
> renv:::renv_download_method()
[1] "curl"
As per recommendation in the RStudio community forum, I've tried setting renv's download method from curl to libcurl, but get a similar error:
> install.packages("dplyr")
Retrieving 'web.cloud.r-project.org/bin/macosx/contrib/4.2/dplyr_1.1.0.tgz' ...
OK [downloaded 1.5 Mb in 0.3 secs]
Installing dplyr [1.1.0] ...
FAILED
Error in if (eval(cond, envir = environment(dot))) return(eval(expr, envir = environment(dot))) :
the condition has length > 1
In addition: Warning messages:
1: could not retrieve available packages for url 'web.cran.microsoft.com/snapshot/2021-03-31/bin/macosx/contrib/4.2'
2: In system2(R(), args, stdout = TRUE, stderr = TRUE) :
running command ''/Library/Frameworks/R.framework/Resources/bin/R' CMD config CC 2>&1' had status 71

Given this output in the error message:
Error in if (eval(cond, envir = environment(dot))) return(eval(expr, envir = environment(dot))) :
the condition has length > 1
In addition: Warning message:
In system2(R(), args, stdout = TRUE, stderr = TRUE) :
running command ''/Library/Frameworks/R.framework/Resources/bin/R' CMD config CC 2>&1' had status 71
Did you recently update macOS on your machine? This looks like an issue in renv where attempts to query the current compiler could fail.
If so, I believe you can work around this by running:
xcode-select --install
from a terminal.

The links which were failing were cran snapshots with no corresponding url. Explicitly pointing to the CRAN repo by running restore(repos="https://cloud.r-project.org") solved this issue.
This issue was solved thanks to #nirgrahamuk in the community.rstudio forum (for those interested in the original answer, see here).

Related

Rscript called with crontab not finding local packages

I have the following R script ~/test.R :
print(.libPaths())
print(system(command = "whoami",ignore.stderr = TRUE))
library(lubridate)
ymd("2022-09-15")
If I run this script from the terminal with /opt/R/3.6.2/lib64/R/bin/Rscript test.R > test2.log I get the following output:
[1] "/home/domain/username/R/library/3.6.2"
[2] "/applis/R/site-library/x86_64-pc-linux-gnu/3.6.2"
[3] "/opt/R/3.6.2/lib64/R/library"
username#domain
[1] 0
[1] "2022-09-15"
So it's working as intended and I have 3 paths for packages. Now let's run this script with cron :
* * * * * /opt/R/3.6.2/lib64/R/bin/Rscript $HOME/test.R > $HOME/test.log 2>&1
I get this for test.log:
[1] "/opt/R/3.6.2/lib64/R/library"
username#domain
[1] 0
Error in library(lubridate) :
aucun package nommé ‘lubridate’ n'est trouvé
Exécution arrêtée
So I only have one path for libraries, consequently lubridate is not found, because it's installed in /home/domain/username/R/library/3.6.2. I cannot install packages within /opt/R/3.6.2/lib64/R/library, so I'm looking for a way to add libpaths to crontab.

unzip() is overwritting file, even when using overwrite = FALSE

saveRDS(1, tmp1 <- "test1.rds")
tmp3 <- tempfile(fileext = ".zip")
zip(tmp3, tmp1)
unlink(tmp1)
file.exists(tmp1) # FALSE
unzip(tmp3)
file.exists(tmp1) # TRUE
readRDS(tmp1) # 1
saveRDS(2, tmp1)
readRDS(tmp1) # 2
unzip(tmp3, overwrite = FALSE)
# Warning message:
# In unzip(tmp3, overwrite = FALSE) : not overwriting file './test1.rds
readRDS(tmp1) # 1
unlink(tmp1)
I was expecting the last readRDS(tmp1) to return 2, right?
Any thought?
PS: I'm on Linux CentOS 7, using R version 3.5.2.
I have confirmed that you are indeed facing a bug in R 3.5.2. I have checked on Centos 7.2 only
R 3.6.0
$ /usr/bin/R -f test2.r
R version 3.6.0 (2019-04-26) -- "Planting of a Tree"
Copyright (C) 2019 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
[Previously saved workspace restored]
During startup - Warning message:
Setting LC_CTYPE failed, using "C"
> saveRDS(1, tmp1 <- "test1.rds")
> tmp3 <- tempfile(fileext = ".zip")
> zip(tmp3, tmp1)
adding: test1.rds (deflated 2%)
> unlink(tmp1)
> file.exists(tmp1) # FALSE
[1] FALSE
>
> unzip(tmp3)
> file.exists(tmp1) # TRUE
[1] TRUE
> readRDS(tmp1) # 1
[1] 1
> saveRDS(2, tmp1)
> readRDS(tmp1) # 2
[1] 2
> unzip(tmp3, overwrite = FALSE)
Warning message:
In unzip(tmp3, overwrite = FALSE) : not overwriting file './test1.rds
> readRDS(tmp1) # 1
[1] 2
>
> unlink(tmp1)
>
R 3.5.2
$ R -f test2.r
R version 3.5.2 (2018-12-20) -- "Eggshell Igloo"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
[Previously saved workspace restored]
During startup - Warning message:
Setting LC_CTYPE failed, using "C"
> saveRDS(1, tmp1 <- "test1.rds")
> tmp3 <- tempfile(fileext = ".zip")
> zip(tmp3, tmp1)
adding: test1.rds (deflated 5%)
> unlink(tmp1)
> file.exists(tmp1) # FALSE
[1] FALSE
>
> unzip(tmp3)
> file.exists(tmp1) # TRUE
[1] TRUE
> readRDS(tmp1) # 1
[1] 1
> saveRDS(2, tmp1)
> readRDS(tmp1) # 2
[1] 2
> unzip(tmp3, overwrite = FALSE)
Warning message:
In unzip(tmp3, overwrite = FALSE) : not overwriting file './test1.rds
> readRDS(tmp1) # 1
[1] 1
>
> unlink(tmp1)
>

Error using roll_lm in a debian machine

The following piece of simple code works perfectly on my local windows machine
require(roll)
x = matrix(rnorm(100),100,1)
y = matrix(rnorm(100),100,1)
roll_lm(x,y,10)
However, on a debian distant machine, it crashes with this error message:
caught illegal operation
address 0x7f867a59ee04,
cause 'illegal operand'
Traceback:
1: .Call("roll_roll_lm", PACKAGE = "roll", x, y, as.integer(width), as.numeric(weights), as.logical(center_x), as.logical(center_y), as.logical(scale_x), as.logical(scale_y), as.integer(min_obs), as.logical(complete_obs), as.logical(na_restore), as.character(match.arg(parallel_for)))
2: roll_lm(x, y, 10)
Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace*
Option 1 : abort (with core dump, if enabled) gives:
Illegal instruction
I am clueless on how to interpret this message.
Any help? Thanks.
Some info :
R.version _
platform x86_64-pc-linux-gnu
arch x86_64
os linux-gnu
system x86_64, linux-gnu
status
major 3
minor 2.5
year 2016
month 04
day 14
svn rev 70478
language R
version.string R version 3.2.5 (2016-04-14)
nickname Very, Very Secure Dishes
The system:
Linux machineName 3.2.0-4-amd64 #1 SMP Debian 3.2.63-2+deb7u1 x86_64 GNU/Linux
Works for me:
R> require(roll)
R> x = matrix(rnorm(100),100,1)
R> y = matrix(rnorm(100),100,1)
R> str(roll_lm(x,y,10))
List of 2
$ coefficients: num [1:100, 1:2] NA NA NA NA NA ...
..- attr(*, "dimnames")=List of 2
.. ..$ : NULL
.. ..$ : chr [1:2] "(Intercept)" "x1"
$ r.squared : num [1:100, 1] NA NA NA NA NA ...
..- attr(*, "dimnames")=List of 2
.. ..$ : NULL
.. ..$ : chr "R-squared"
R>
I suggest you rebuild reinstall package roll.
Sometimes this happens when one component (Rcpp, RcppParallel, ...) gets updated.

Determining which R architectures are installed

How does one determine which architectures are supported by an installation of R? On a standard windows install, one may look for the existence of R_HOME/bin/*/R.exe where * is the architecture (typically i386 or x64). On a standard mac install from CRAN, there are no subdirectories.
I can query R for the default architecture using something like:
$ R --silent -e "sessionInfo()[[1]][[2]]"
> sessionInfo()[[1]][[2]]
[1] "x86_64"
but how do I know on mac/linux whether any sub-architectures are installed, and if so what they are?
R.version, R.Version(), R.version.string, and version provide detailed information about the version of R running.
Update, based on a better understanding of the question. This isn't a complete solution, but it seems you can get fairly close via a combination of the following commands:
# get all the installed architectures
arch <- basename(list.dirs(R.home('bin'), recursive=FALSE))
# handle different operating systems
if(.Platform$OS.type == "unix") {
arch <- gsub("exec","",arch)
if(arch == "")
arch <- R.version$arch
} else { # Windows
# any special handling
}
Note that this won't work if you've built R from source and installed the different architectures in various different places. See 2.6 Sub-architectures of the R Installation and Administration manual for more details.
Using Sys.info() you have a lot of information on your system.
May be it can help here
Sys.info()["machine"]
machine
"x86_64"
EDIT
One workaround to have all architecture possible is to download log files from the Rstudio mirror, it's not complete but it's good estimate of what you need.
start <- as.Date('2012-10-01')
today <- as.Date('2013-07-01')
all_days <- seq(start, today, by = 'day')
year <- as.POSIXlt(all_days)$year + 1900
urls <- paste0('http://cran-logs.rstudio.com/', year, '/', all_days, '.csv.gz')
files <- file.path("/tmp", basename(urls))
list_data <- lapply(files, read.csv, stringsAsFactors = FALSE)
data <- do.call(rbind, list_data)
str(data)
## 'data.frame': 10694506 obs. of 10 variables:
## $ date : chr "2012-10-01" "2012-10-01" "2012-10-01" "2012-10-01" ...
## $ time : chr "00:30:13" "00:30:15" "02:30:16" "02:30:16" ...
## $ size : int 35165 212967 167199 21164 11046 42294 435407 326143 119459 868695 ...
## $ r_version: chr "2.15.1" "2.15.1" "2.15.1" "2.15.1" ...
## $ r_arch : chr "i686" "i686" "x86_64" "x86_64" ...
## $ r_os : chr "linux-gnu" "linux-gnu" "linux-gnu" "linux-gnu" ...
## $ package : chr "quadprog" "lavaan" "formatR" "stringr" ...
## $ version : chr "1.5-4" "0.5-9" "0.6" "0.6.1" ...
## $ country : chr "AU" "AU" "US" "US" ...
## $ ip_id : int 1 1 2 2 2 2 2 1 1 3 ...
unique(data[["r_arch"]])
## [1] "i686" "x86_64" NA "i386" "i486"
## [6] "i586" "armv7l" "amd64" "000000" "powerpc64"
## [11] "armv6l" "sparc" "powerpc" "arm" "armv5tel"

Listing R Package Dependencies Without Installing Packages

Is there a simple way to get a list of R package dependencies (all recursive dependencies) for a given package, without installing the package and it's dependencies? Something similar to a fake install in portupgrade or apt.
You can use the result of the available.packages function. For example, to see what ggplot2 depends on :
pack <- available.packages()
pack["ggplot2","Depends"]
Which gives :
[1] "R (>= 2.14), stats, methods"
Note that depending on what you want to achieve, you may need to check the Imports field, too.
I am surprised no one mentioned tools::package_dependencies() , which is the simplest solution, and has a recursive argument (which the accepted solution does not offer).
Simple example looking at the recursive dependencies for the first 200 packages on CRAN:
library(tidyverse)
avail_pks <- available.packages()
deps <- tools::package_dependencies(packages = avail_pks[1:200, "Package"],
recursive = TRUE)
tibble(Package=names(deps),
data=map(deps, as_tibble)) %>%
unnest(data)
#> # A tibble: 7,125 x 2
#> Package value
#> <chr> <chr>
#> 1 A3 xtable
#> 2 A3 pbapply
#> 3 A3 parallel
#> 4 A3 stats
#> 5 A3 utils
#> 6 aaSEA DT
#> 7 aaSEA networkD3
#> 8 aaSEA shiny
#> 9 aaSEA shinydashboard
#> 10 aaSEA magrittr
#> # … with 7,115 more rows
Created on 2020-12-04 by the reprex package (v0.3.0)
Another neat and simple solution is the internal function recursivePackageDependencies from the library packrat. However, the package must be installed in some library on your machine. The advantage is that it works with selfmade non-CRAN packages as well. Example:
packrat:::recursivePackageDependencies("ggplot2",lib.loc = .libPaths()[1])
giving:
[1] "R6" "RColorBrewer" "Rcpp" "colorspace" "dichromat" "digest" "gtable"
[8] "labeling" "lazyeval" "magrittr" "munsell" "plyr" "reshape2" "rlang"
[15] "scales" "stringi" "stringr" "tibble" "viridisLite"
I do not have R installed and I needed to find out which R Packages were dependencies upon a list of R Packages being requested for usage at my company.
I wrote a bash script that iterates over a list of R Packages in a file and will recursively discover dependencies.
The script uses a file named rinput_orig.txt as input (example below). The script will create a file named rinput.txt as it does its work.
The script will create the following files:
rdepsfound.txt - Lists dependencies found including the R Package that is dependent upon it (example below).
routput.txt - Lists all R Packages (from original list and list of dependencies) along with the license and CRAN URL (example below).
r404.txt - List of R Packages where a 404 was received when trying to curl. This is handy if your original list has any typos.
Bash script:
#!/bin/bash
# CLEANUP
rm routput.txt
rm rdepsfound.txt
rm r404.txt
# COPY ORIGINAL INPUT TO WORKING INPUT
cp rinput_orig.txt rinput.txt
IFS=","
while read PACKAGE; do
echo Processing $PACKAGE...
PACKAGEURL="http://cran.r-project.org/web/packages/${PACKAGE}/index.html"
if [ `curl -o /dev/null --silent --head --write-out '%{http_code}\n' ${PACKAGEURL}` != 404 ]; then
# GET LICENSE INFO OF PACKAGE
LICENSEINFO=$(curl ${PACKAGEURL} 2>/dev/null | grep -A1 "License:" | grep -v "License:" | gawk 'match($0, /<a href=".*">(.*)<\/a>/, a) {print a[0]}' | sed "s/|/,/g" | sed "s/+/,/g")
for x in ${LICENSEINFO[*]}
do
# SAVE LICENSE
LICENSE=$(echo ${x} | gawk 'match($0, /<a href=".*">(.*)<\/a>/, a) {print a[1]}')
break
done
# WRITE PACKAGE AND LICENSE TO OUTPUT FILE
echo $PACKAGE $LICENSE $PACKAGEURL >> routput.txt
# GET DEPENDENCIES OF PACKAGE
DEPS=$(curl ${PACKAGEURL} 2>/dev/null | grep -A1 "Depends:" | grep -v "Depends:" | gawk 'match($0, /<a href=".*">(.*)<\/a>/, a) {print a[0]}')
for x in ${DEPS[*]}
do
FOUNDDEP=$(echo "${x}" | gawk 'match($0, /<a href=".*">(.*)<\/a>/, a) {print a[1]}' | sed "s/<\/span>//g")
if [ "$FOUNDDEP" != "" ]; then
echo Found dependency $FOUNDDEP for $PACKAGE...
grep $FOUNDDEP rinput.txt > /dev/null
if [ "$?" = "0" ]; then
echo $FOUNDDEP already exists in package list...
else
echo Adding $FOUNDDEP to package list...
# SAVE FOUND DEPENDENCY BACK TO INPUT LIST
echo $FOUNDDEP >> rinput.txt
# SAVE FOUND DEPENDENCY TO DEPENDENCY LIST FOR EASY VIEWING OF ALL FOUND DEPENDENCIES
echo $FOUNDDEP is a dependency of $PACKAGE >> rdepsfound.txt
fi
fi
done
else
echo Skipping $PACKAGE because 404 was received...
echo $PACKAGE $PACKAGEURL >> r404.txt
fi
done < rinput.txt
echo -e "\nRESULT:"
sort -u routput.txt
Example rinput_orig.txt:
shiny
rmarkdown
xtable
RODBC
RJDBC
XLConnect
openxlsx
xlsx
Rcpp
Example console output when running script:
Processing shiny...
Processing rmarkdown...
Processing xtable...
Processing RODBC...
Processing RJDBC...
Found dependency DBI for RJDBC...
Adding DBI to package list...
Found dependency rJava for RJDBC...
Adding rJava to package list...
Processing XLConnect...
Found dependency XLConnectJars for XLConnect...
Adding XLConnectJars to package list...
Processing openxlsx...
Processing xlsx...
Found dependency rJava for xlsx...
rJava already exists in package list...
Found dependency xlsxjars for xlsx...
Adding xlsxjars to package list...
Processing Rcpp...
Processing DBI...
Processing rJava...
Processing XLConnectJars...
Processing xlsxjars...
Found dependency rJava for xlsxjars...
rJava already exists in package list...
Example rdepsfound.txt:
DBI is a dependency of RJDBC
rJava is a dependency of RJDBC
XLConnectJars is a dependency of XLConnect
xlsxjars is a dependency of xlsx
Example routput.txt:
shiny GPL-3 http://cran.r-project.org/web/packages/shiny/index.html
rmarkdown GPL-3 http://cran.r-project.org/web/packages/rmarkdown/index.html
xtable GPL-2 http://cran.r-project.org/web/packages/xtable/index.html
RODBC GPL-2 http://cran.r-project.org/web/packages/RODBC/index.html
RJDBC GPL-2 http://cran.r-project.org/web/packages/RJDBC/index.html
XLConnect GPL-3 http://cran.r-project.org/web/packages/XLConnect/index.html
openxlsx GPL-3 http://cran.r-project.org/web/packages/openxlsx/index.html
xlsx GPL-3 http://cran.r-project.org/web/packages/xlsx/index.html
Rcpp GPL-2 http://cran.r-project.org/web/packages/Rcpp/index.html
DBI LGPL-2 http://cran.r-project.org/web/packages/DBI/index.html
rJava GPL-2 http://cran.r-project.org/web/packages/rJava/index.html
XLConnectJars GPL-3 http://cran.r-project.org/web/packages/XLConnectJars/index.html
xlsxjars GPL-3 http://cran.r-project.org/web/packages/xlsxjars/index.html
I hope this helps someone!
I tested my own solution (local installed packages checked) against packrat and tools ones.
You could find out clear differences between methods.
tools::package_dependencies looks to give too much for older R versions (till 4.1.0 and recursive = TRUE) and is not efficient solution.
R 4.1.0 NEWS
"Function tools::package_dependencies() (in package tools) can now use different dependency types for direct and recursive dependencies."
packrat:::recursivePackageDependencies is using available.packages so it is based on newest remote packages, not local ones.
My function by default is skipping base packages, change the base arg if you want to attach them too.
Tested under R 4.1.0:
get_deps <- function(package, fields = c("Depends", "Imports", "LinkingTo"), base = FALSE, lib.loc = NULL) {
stopifnot((length(package) == 1) && is.character(package))
stopifnot(all(fields %in% c("Depends", "Imports", "Suggests", "LinkingTo")))
stopifnot(is.logical(base))
stopifnot(package %in% rownames(utils::installed.packages(lib.loc = lib.loc)))
paks_global <- NULL
deps <- function(pak, fileds) {
pks <- packageDescription(pak)
res <- NULL
for (f in fileds) {
ff <- pks[[f]]
if (!is.null(ff)) {
res <- c(
res,
vapply(
strsplit(trimws(strsplit(ff, ",")[[1]]), "[ \n\\(]"),
function(x) x[1],
character(1)
)
)
}
}
if (is.null(res)) {
return(NULL)
}
for (r in res) {
if (r != "R" && !r %in% paks_global) {
paks_global <<- c(r, paks_global)
deps(r, fields)
}
}
}
deps(package, fields)
setdiff(unique(paks_global), c(
package,
"R",
if (!base) {
c(
"stats",
"graphics",
"grDevices",
"utils",
"datasets",
"methods",
"base",
"tools"
)
} else {
NULL
}
))
}
own = get_deps("shiny", fields = c("Depends", "Imports"))
packrat = packrat:::recursivePackageDependencies("shiny", lib.loc = .libPaths(), fields = c("Depends", "Imports"))
tools = tools::package_dependencies("shiny", which = c("Depends", "Imports"), recursive = TRUE)[[1]]
setdiff(own, packrat)
#> character(0)
setdiff(packrat, own)
#> character(0)
setdiff(own, tools)
#> character(0)
setdiff(tools, own)
#> [1] "methods" "utils" "grDevices" "tools" "stats" "graphics"
setdiff(packrat, tools)
#> character(0)
setdiff(tools, packrat)
#> [1] "methods" "utils" "grDevices" "tools" "stats" "graphics"
own
#> [1] "lifecycle" "ellipsis" "cachem" "jquerylib" "rappdirs"
#> [6] "fs" "sass" "bslib" "glue" "commonmark"
#> [11] "withr" "fastmap" "crayon" "sourcetools" "base64enc"
#> [16] "htmltools" "digest" "xtable" "jsonlite" "mime"
#> [21] "magrittr" "rlang" "later" "promises" "R6"
#> [26] "Rcpp" "httpuv"
packrat
#> [1] "R6" "Rcpp" "base64enc" "bslib" "cachem"
#> [6] "commonmark" "crayon" "digest" "ellipsis" "fastmap"
#> [11] "fs" "glue" "htmltools" "httpuv" "jquerylib"
#> [16] "jsonlite" "later" "lifecycle" "magrittr" "mime"
#> [21] "promises" "rappdirs" "rlang" "sass" "sourcetools"
#> [26] "withr" "xtable"
tools
#> [1] "methods" "utils" "grDevices" "httpuv" "mime"
#> [6] "jsonlite" "xtable" "digest" "htmltools" "R6"
#> [11] "sourcetools" "later" "promises" "tools" "crayon"
#> [16] "rlang" "fastmap" "withr" "commonmark" "glue"
#> [21] "bslib" "cachem" "ellipsis" "lifecycle" "sass"
#> [26] "jquerylib" "magrittr" "base64enc" "Rcpp" "stats"
#> [31] "graphics" "fs" "rappdirs"
microbenchmark::microbenchmark(get_deps("shiny", fields = c("Depends", "Imports")),
packrat:::recursivePackageDependencies("shiny", lib.loc = .libPaths(), fields = c("Depends", "Imports")),
tools = tools::package_dependencies("shiny", which = c("Depends", "Imports"), recursive = TRUE)[[1]],
times = 5
)
#> Warning in microbenchmark::microbenchmark(get_deps("shiny", fields =
#> c("Depends", : less accurate nanosecond times to avoid potential integer
#> overflows
#> Unit: milliseconds
#> expr
#> get_deps("shiny", fields = c("Depends", "Imports"))
#> packrat:::recursivePackageDependencies("shiny", lib.loc = .libPaths(), fields = c("Depends", "Imports"))
#> tools
#> min lq mean median uq max neval
#> 5.316552 5.607365 6.054568 5.674359 6.633308 7.041258 5
#> 18.767340 19.387588 21.739127 21.581457 23.526169 25.433079 5
#> 411.589734 449.179354 458.526354 465.431262 468.440211 497.991207 5
Created on 2021-06-25 by the reprex package (v0.3.0)
Proof that sth was wrong with tools solution under older R versions. Tested under R 3.6.3.
paks <- tools::package_dependencies("shiny", which = c("Depends", "Imports"), recursive = TRUE)[[1]]
"lifecycle" %in% paks
#> [1] TRUE
any(c(paks, "shiny") %in% tools::dependsOnPkgs("lifecycle"))
#> [1] FALSE
Created on 2021-06-25 by the reprex package (v0.3.0)
Try this: tools::package_dependencies(recursive = TRUE)$package_name
As an example- here are the dependencies for dplyr:
tools::package_dependencies(recursive = TRUE)$dplyr
[1] "ellipsis" "generics" "glue" "lifecycle" "magrittr" "methods"
[7] "R6" "rlang" "tibble" "tidyselect" "utils" "vctrs"
[13] "cli" "crayon" "fansi" "pillar" "pkgconfig" "purrr"
[19] "digest" "assertthat" "grDevices" "utf8" "tools"

Resources