Trouble with packrat corrupting R functioning - r

I installed the package packrat at some point, used it perhaps once and moved on with my life.
However, despite not having loaded it in months, it remains a nuisance to my regular R usage.
Seemingly at random, my R session within RStudio will fail with errors at certain operations, especially package installation. Here's the most recent error message (after running parallel::makeCluster(parallel::detectCores()):
Error in file(filename, "r", encoding = encoding) : cannot open the
connection
Calls: source -> file
In addition: Warning message:
In
file(filename, "r", encoding = encoding) : cannot open file
'packrat/init.R': No such file or directory
Execution halted
I checked all of the folders on .libPaths() and I don't even have packrat installed anymore. Why on earth is R still trying to carry out packrat operations? And how can I stop this?
My duct-tape solution so far is to simply close and reopen RStudio, which works like a charm for package installation issues.
However, I cannot seem to get around this for makeCluster(detectCores()) within just one .R script I've got. It works perfectly fine in another script for another project.
Background:
sessionInfo()
# R version 3.2.2 (2015-08-14)
# Platform: x86_64-pc-linux-gnu (64-bit)
# Running under: Ubuntu 14.04.2 LTS
# locale:
# [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8
# [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
# [7] LC_PAPER=en_US.UTF-8 LC_NAME=en_US.UTF-8 LC_ADDRESS=en_US.UTF-8
# [10] LC_TELEPHONE=en_US.UTF-8 LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=en_US.UTF-8
# attached base packages:
# [1] parallel stats graphics grDevices utils datasets methods base
# other attached packages:
# [1] doParallel_1.0.8 iterators_1.0.7 foreach_1.4.2 geosphere_1.4-3 xlsx_0.5.7 xlsxjars_0.6.1
# [7] rJava_0.9-6 xtable_1.7-4 sandwich_2.3-3 texreg_1.35 maptools_0.8-36 sp_1.1-1
# [13] ggmap_2.5.2 ggplot2_1.0.1 data.table_1.9.5
# loaded via a namespace (and not attached):
# [1] Rcpp_0.11.6 plyr_1.8.3 tools_3.2.2 digest_0.6.8 gtable_0.1.2
# [6] lattice_0.20-33 png_0.1-7 mapproj_1.2-4 proto_0.3-10 stringr_1.0.0
# [11] RgoogleMaps_1.2.0.7 maps_2.3-11 grid_3.2.2 jpeg_0.1-8 foreign_0.8-66
# [16] RJSONIO_1.3-0 reshape2_1.4.1 magrittr_1.5 codetools_0.2-11 scales_0.2.5
# [21] MASS_7.3-43 colorspace_1.2-6 stringi_0.5-9003 munsell_0.4.2 chron_2.3-47
# [26] rjson_0.2.15 zoo_1.7-12
Update 1:
Installing packrat had no effect. Running packrat::init() resulted in an error before finishing; nothing changed.
Update 2:
I've isolated the problem by identifying that it's the working directory that's causing the issues. What in the working directory I'm using might be causing the problems? Some residual file from having run packrat previously in this directory?

Through further trial and error given the prods of #BondedDust I finally appear to have solved the issue. Having previously tried to use packrat in the particular working directory in which I was working appears to have left some vestiges despite later uninstalling packrat.
In particular, packrat edits your local .Rprofile (original credit due to #zerweck and #snaut), which is source()d on R startup in that directory.
If you use the .Rprofile to store some local configurations, you should edit the file and remove the packrat lines (or any you don't recognize); otherwise, you should just delete that file to restore your project to working as expected.

Check your HOME directory for an unintentional .Rprofile.
Packrat may have put this here if you tried to packrat::init() in HOME.
install.package() with packrat looks for .Rprofile when run. The behavior I've observed has it prioritizing the HOME .Rprofile over the getwd() one, causing the
cannot open file 'packrat/init.R': No such file or directory

Related

Package ‘XXX’ was installed before R 4.0.0: please re-install it

I am using R 4.0.2.
I manually installed 2 packages from cfcdae and Stats5303lib from here. I followed the instruction here to download Rtools and use it to install packages above.
Problem is
writeLines('PATH="${RTOOLS40_HOME}\\usr\\bin;${PATH}"', con = "~/.Renviron") # runs fine
Sys.which("make") #works fine
make
"C:\\rtools40\\usr\\bin\\make.exe"
install.packages("Stat5303libs_0.7-5.zip",repos=NULL,type="source") # these all run fine
install.packages("cfcdae_0.8-4.zip",repos=NULL,type="source") # these all run fine
However, when I ran into problem below when I tried to run the libraries.
Error: package or namespace load failed for ‘cfcdae’:
package ‘cfcdae’ was installed before R 4.0.0: please re-install it
I tried below but still in vain.
update.packages(ask=FALSE, checkBuilt=TRUE)
Why is this happening? Is it because the package is too old?
Update:
As requested, i have changed my .libPaths() as below and updated the SessionInfo() as well.
> .libPaths()
[1] "C:/Users/UserME/Documents/R/win-library/4.0"
[2] "C:/Program Files/R/R-4.0.2/library"
R version 4.0.2 (2020-06-22)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)
Matrix products: default
locale:
[1] LC_COLLATE=English_Hong Kong SAR.1252 LC_CTYPE=English_Hong Kong SAR.1252
[3] LC_MONETARY=English_Hong Kong SAR.1252 LC_NUMERIC=C
[5] LC_TIME=English_Hong Kong SAR.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] Rcpp_1.0.4.6 rstudioapi_0.11 magrittr_1.5
[4] splines_4.0.2 MASS_7.3-51.6 tidyselect_1.1.0
[7] munsell_0.5.0 statmod_1.4.34 lattice_0.20-41
[10] colorspace_1.4-1 R6_2.4.1 rlang_0.4.6
[13] minqa_1.2.4 dplyr_1.0.0 tools_4.0.2
[16] grid_4.0.2 nlme_3.1-148 gtable_0.3.0
[19] ellipsis_0.3.1 lme4_1.1-23 tibble_3.0.1
[22] lifecycle_0.2.0 numDeriv_2016.8-1.1 crayon_1.3.4
[25] Matrix_1.2-18 nloptr_1.2.2.2 purrr_0.3.4
[28] ggplot2_3.3.2 vctrs_0.3.1 glue_1.4.1
[31] compiler_4.0.2 pillar_1.4.6 generics_0.0.2
[34] scales_1.1.1 boot_1.3-25 lmerTest_3.1-2
[37] pkgconfig_2.0.3
All packages need to be reinstalled under the new version (4.0). I had to first remove and then reinstall all the packages.
The following worked for me:
# check your package library path
.libPaths()
# grab old packages names
old_packages <- installed.packages(lib.loc = "/Library/Frameworks/R.framework/Versions/3.6/Resources/library")
old_packages <- as.data.frame(old_packages)
list.of.packages <- unlist(old_packages$Package)
# remove old packages
remove.packages( installed.packages( priority = "NA" )[,1] )
# reinstall all packages
new.packages <- list.of.packages[!(list.of.packages %in% installed.packages()[,"Package"])]
if(length(new.packages)) install.packages(new.packages)
lapply(list.of.packages,function(x){library(x,character.only=TRUE)})
This issue is likely to happen when R is reading from an old directory, e.g. pre R 4.0.x .
Here is a few possible ways to fix this:
Check your .libPaths() - R could be reading packages from a R 3.x.x library, which would produce the error you are getting. Alternatively, try creating a new library directory (this is very likely to fix the issue).
update.packages(ask=FALSE, checkBuilt=TRUE) (which you have already tried)
Remove previous installations of R, and their associated directories.
A zip file is a pre-built binary package for use on Windows, not a source package. Installing it with install.packages(*, type="source") won't make a difference. You'll need to contact the person/people who wrote these packages to build them again for R 4.0, or provide you with the real source -- this will usually be a .tar.gz file.
If you're having trouble with this issue while using renv, you may need to look at your renv cache. I had to delete mine to force my projects to use newly-installed versions of packages instead of the old ones.
In a similar situation, I renamed 'site-library' to 'old-site-library', made a new empty 'site-library' folder and proceeded from there successfully.
In case anyone runs into this error message when running revdepcheck::revdep_check(), what worked for me was deleting the /revdep/ folder in my R package's directory.
This code worked for me
tmp = as.data.frame(installed.packages())
max_version = max(as.numeric(substr(tmp$Built, 1,1)))
tmp = tmp[as.numeric(substr(tmp$Built, 1,1)) < max_version,]
lapply(tmp$Package, remove.packages)
lapply(tmp$Package, function(x) install.packages(x, dependencies = TRUE))

R package not available in batch mode

I have been running a bash script (under Ubuntu) that calls and runs some R scripts to plot some maps with ggplot2. As it is a production system I don't update any new R or package releases. But recently I tried to install plotly with an error message output.
Now, when trying to run the script (calling a php webpage trhough a browser; that was running fine for a lot of time) an error with crayon package halts and exits the script without plotting the maps.
'/usr/lib/R/bin/R --slave --no-restore --no-save --no-restore --file=./RAMS-mapa-onades-zones-manual.R'
Loading required package: sp
Loading required package: methods
Checking rgeos availability: TRUE
Error : .onLoad failed in loadNamespace() for 'pillar', details:
call: loadNamespace(name)
error: there is no package called 'crayon'
Error: package or namespace load failed for 'ggplot2'
Execution halted
But if I run from a terminal or with RStudio server crayon package is loaded, ggplot2 is also properly loaded, and the maps are produced.
crayon package (located at /home/meteo/R/i686-pc-linux-gnu-library/3.1) called as
library("crayon", lib.loc= c("/home/meteo/R/i686-pc-linux-gnu-library/3.1", "/usr/local/lib/R/site-library", "/usr/lib/R/library"))
R session info on Rstudio Server:
> sessionInfo()
R version 3.2.2 (2015-08-14)
Platform: i686-pc-linux-gnu (32-bit)
Running under: Ubuntu precise (12.04.5 LTS)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=C LC_COLLATE=C LC_MONETARY=C LC_MESSAGES=C
[7] LC_PAPER=C LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=C LC_IDENTIFICATION=C
attached base packages:
[1] grid stats graphics grDevices utils datasets methods base
other attached packages:
[1] crayon_1.3.4 rgdal_0.9-1 rgeos_0.3-8 raster_2.3-24 gpclib_1.5-5 mapproj_1.2-2 maps_2.3-9
[8] stringr_0.6.2 plyr_1.8.1 ggplot2_3.2.0 maptools_0.8-34 sp_1.2-3
loaded via a namespace (and not attached):
[1] Rcpp_1.0.1 magrittr_1.5 tidyselect_0.2.5 munsell_0.5.0 colorspace_1.2-4 lattice_0.20-33 R6_2.4.0
[8] rlang_0.3.4 dplyr_0.8.1 tools_3.2.2 gtable_0.1.2 withr_2.1.2 digest_0.6.8 lazyeval_0.2.2
[15] assertthat_0.2.1 tibble_2.1.3 purrr_0.3.2 glue_1.3.1 labeling_0.3 pillar_1.4.1 scales_1.0.0
[22] foreign_0.8-66 pkgconfig_2.0.2
Maybe there is a version conflict? How do I check?
what version of R is in /usr/lib/R/bin/R? Basically it appears you are in a slight "dependency hell" I would run /usr/lib/R/bin/R and check to see if crayon is installed there. #Alexis is correct though, it is not wise to mix versions like that. Let R handle library location unless you are trying to separate packages. Check the libraries for 3.2.2 and see if crayon is installed. you may have switched your R version since plotly depends on R >=3.2.0 https://cran.r-project.org/web/packages/plotly/
I think the easy fix would be to correct your .libpaths() i.e not call the 3.1 library and install the correct version of the packages you need.
OR
switch from R 3.2 back to R 3.1

R: Many problems with updates or installation of packages after updating R or RStudio

I have been having a rash of problems when I do R or RStudio updates. I get a lot of these problems over the next few sessions after an update.
Is the problem apparent on this information? If not, how can I diagnose it?
Problems take the following forms:
Packages that are present in my R library give errors as missing.
Most often these are tidyverse packages.
Certain dependencies of tidyverse are not installed, especially stringi. Similar problems occur less often with Hmisc.
Packages are correctly downloaded, but I get an error stating that the older version could not be removed. These errors vanish if
I remove the packages indicated by hand and install them
individually.
I get similar problems if I try to update tidyverse with tidyverse_update, e.g.:
tidyverse::tidyverse_update(recursive = TRUE)
Error in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]):
there is no package called ‘readr’
This although the readr package is in fact present.
I just now got a new error message on trying to install readr
individually:
Warning in install.packages:
lzma decoding result 10
Warning in install.packages:
unable to access index for repository
https://cran.case.edu/bin/windows/contrib/3.5:
Line starting 'Pac ...' is malformed!
And on trying again:
Error in install.packages : invalid version specification ‘NA’
I usually start a session with update.packages(ask=FALSE). I have RStudio set up to always run as Administrator. If asked, I generally install from binaries rather than source.
I install and load groups of packages using the following functions:
# Install packages (if not already installed)
in_packs <- function(packs){
new_packs <- packs[!(packs %in% installed.packages()[,"Package"])]
if(length(new_packs) < 1) print("Already installed.") else {
install.packages(new_packs, dependencies = TRUE, # move to fairR_fns
repos=c("https://cran.case.edu/",
"https://cran.rstudio.com"))
# Load packages (if not already installed)
load_pks <- function(pks){
invisible(sapply(X = pks[seq_along(pks)], FUN = library, character.only = TRUE))
}
Here is my session information.
R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] Rcpp_1.0.0 lubridate_1.7.4 lattice_0.20-38 tidyr_0.8.2
[5] assertthat_0.2.0 digest_0.6.18 R6_2.3.0 plyr_1.8.4
[9] backports_1.1.3 httr_1.4.0 ggplot2_3.1.0 pillar_1.3.1
[13] rlang_0.3.0.1 lazyeval_0.2.1 stringr_1.3.1 munsell_0.5.0
[17] broom_0.5.1 compiler_3.5.1 modelr_0.1.2 pkgconfig_2.0.2
[21] tidyselect_0.2.5 tibble_1.4.2 crayon_1.3.4 dplyr_0.7.8
[25] grid_3.5.1 nlme_3.1-137 jsonlite_1.6 gtable_0.2.0
[29] magrittr_1.5 scales_1.0.0 cli_1.0.1 stringi_1.2.4
[33] bindrcpp_0.2.2 generics_0.0.2 tools_3.5.1 forcats_0.3.0
[37] glue_1.3.0 purrr_0.2.5 hms_0.4.2 yaml_2.2.0
[41] colorspace_1.3-2 memoise_1.1.0 bindr_0.1.1 haven_2.0.0
I have found that most, though by no means all, of these problems go away if I do the following:
Make sure I have only one instance of R open.
Set up both R and RStudio to run as administrator.
This is a little tricky because the instantiation of R I am running is usually opened by RStudio, not by me. For both RStudio and R, right-click on the shortcut, either on the desktop or the start menu. On the shortcut tab, select advanced. Then check the box for run as administrator. I also allow the R directory in program files to be modified by other programs, though I am not sure this was necessary. Right-click, properties, Security tab, select ALL APPLICATION PACKAGES, click on edit, check on Full control.
Set the R folder itself, and all its sub-folders, to shared. (Be
aware that this creates a security risk if you are on a shared
machine). Right-click on the R directory -- usually the one in your "My Documents" folder, not the one in Program Files -- and under the sharing tab, hit share.
Of the problems that remain when these things have been done, most seem to be caused by, usually short-term, problems with the repository copy. Such problems can disturb not only the corrupted (or whatever) package, but also its dependencies. Delete the folder of the problem package manually, restart R and, before loading any packages, switch to a different repository. Then install as usual

Cannot load package in R

Since yesterday, I cannot load some common used packages, e.g. reshape2 and stringr through library command.
Step to reproduce my problem:
1) Start a new session in Rstudio server (Version 0.99.467). I also kill the active-sessions in Rstudio server.
2) Run these codes
library(ggplot2)
library(stringr)
3) Get an error message:
Error in unloadNamespace(package) :
namespace ‘stringr’ is imported by ‘reshape2’ so cannot be unloaded
Show Traceback
Rerun with Debug
Error in library(stringr) : “stringr” version 0.6.2 cannot be unloaded.
But I don't get error if stringr is loaded before ggplot2. How should I fix this problem. Thanks for any suggestions.
I may have to update R to 3.2
My R session info:
R version 3.1.3 (2015-03-09)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu precise (12.04.5 LTS)
locale:
[1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C LC_TIME=en_AU.UTF-8 LC_COLLATE=en_AU.UTF-8
[5] LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8 LC_PAPER=en_AU.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ggplot2_1.0.0
loaded via a namespace (and not attached):
[1] colorspace_1.2-4 digest_0.6.4 grid_3.1.3 gtable_0.1.2 MASS_7.3-39 munsell_0.4.2 plyr_1.8.1
[8] proto_0.3-10 Rcpp_0.11.2 reshape2_1.4 scales_0.2.4 stringr_0.6.2 tools_3.1.3
I resolved my problem through updating R and all R packages.
I guess the problem is caused by old version of stringr or reshape2 as I install the same packages in different places (system folder or personal library). R tried to load a old version of package.
Even though it is a good practice to update R for me it was not necessary to solve the same issue. The problem can result from few .libPaths(). With .libPaths("C:/Program Files/R/R-3.4.3patched/library") you can point out one folder. After that just reinstall of packages, and it should work.
My code:
> .libPaths()
[1] "L:/My Documents/R/win-library/3.4" "C:/Program Files/R/R-3.4.3patched/library"
> .libPaths("C:/Program Files/R/R-3.4.3patched/library")
> packages<-c('ggplot2','data.table','knitr','xtable')
> install.packages(packages)
> lapply(packages,library,character.only=T)
Even though the issue is solved I hope it will help someone.

R `dev.new()` freezes

this just started to occur: when I type the command dev.new(), the window stays frozen, and I can't Ctrl+c to stop it. I have to kill the R process in another terminal. I am running 64-bit CentOS 6.7 and R 3.2.1. Here is the output from sessionInfo():
> sessionInfo()
R version 3.2.1 (2015-06-18)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS release 6.7 (Final)
locale:
[1] LC_CTYPE=en_US.utf8 LC_NUMERIC=C LC_TIME=en_US.utf8 LC_COLLATE=en_US.utf8 LC_MONETARY=en_US.utf8
[6] LC_MESSAGES=en_US.utf8 LC_PAPER=en_US.utf8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.utf8 LC_IDENTIFICATION=C
attached base packages:
[1] graphics grDevices utils datasets stats methods base
other attached packages:
[1] ggplot2_1.0.1 data.table_1.9.4 plyr_1.8.3 reshape2_1.4.1 vimcom_0.9-9 setwidth_1.0-4 colorout_1.1-0
loaded via a namespace (and not attached):
[1] Rcpp_0.12.0 digest_0.6.8 MASS_7.3-44 chron_2.3-47 grid_3.2.1 gtable_0.1.2 magrittr_1.5 scales_0.3.0
[9] stringi_0.5-5 proto_0.3-10 tools_3.2.1 stringr_1.0.0 munsell_0.4.2 colorspace_1.2-6
Has anybody come across this issue? Perhaps of relevance, but I was able to use a GUI application that creates plot devices as a cairoDevice, with no issues.
EDIT: A bit more info - when running R --vanilla, the same behavior occurs. Same with calling plot directly (e.g. plot(rnorm(1e2))), and making a call to ggplot.
EDIT 2: in case this wasn't confusing enough, I am able to plot without issue on my home system (where sessionInfo gives the same output, aside from some packages loaded via a namespace). I believe the same CentOS packages are installed, as well.
EDIT 3: to add a bit more info, in addition to the (RGtk2) GUI that I mentioned still works, I can call Cairo from the command line directly and plotting works without issue that way. So it seems to be specific to base plotting.
I had the same problem on SL6.7. This is not a R problem but rather the xorg-x11-server-Xorg update crashed it.
Just downgrade the package and restart your X-session and you can plot again.
~$ yum downgrade http://ftp.scientificlinux.org/linux/scientific/6.6/x86_64/updates/security/xorg-x11-server-Xorg-1.15.0-26.sl6.x86_64.rpm
To make it permanent disable the upgrade of the package in yum.conf
~$ echo "exclude=xorg-x11-server-Xorg" >> /etc/yum.conf
Actually, the issue with R was not really a bug in xorg-x11-server.
Its update (Fix backing store's Always mode) revealed a bug in the X11
module of R. More details can be found in R's bugzilla :
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=16497
(See comment 5 for details)
A couple of patches to fix the issue have been proposed.

Resources