Related
I work in a corporate environment that uses Microsoft Windows Defender Application Control (WDAC) to provide security. This blocks unsigned EXE and DLL files from being installed on devices. R packages which use DLLs fail to install. The workaround to this is provide an R installation from an approved central source which also copies over a default set of packages, such as tidyverse, data.table etc. to the R library. Users can continue to install additional packages which are built with native R, but run into issues if they try to install, build from source, or update packages with DLL files in.
Is there a way to check whether a package uses DLL files in advance of installation?
Output something like:-
check_dll(foo)
result: "This package and its dependencies have no DLL files. You can install this package"
check_dll(bar)
result: "bar does not have any DLL files, but one dependency, OOF, uses DLL files.
You have already have a version of OOF installed so it should be safe to install bar"
check_dll(foobar)
result: "foobar has a DLL. Do not attempt to install foobar".
check_dll(RABOOF)
result: "RABOOF does not have any DLL files, but one of it's dependencies,
foobar, does have a DLL file. Do not attempt to install RABOOF".
tools::package_dependencies() will list the package dependencies, but nothing else.
Downloading the zip file from CRAN and inspecting it for a libs/x64 folder with contents will work, but seems a heavyweight approach. Theoretically if a package has lots of dependencies this could result in downloading a lot of files unnecessarily.
Look for the NeedsCompilation field in the DESCRIPTION file. If it is "yes", there will be a DLL. If it is "no", there probably won't be. (If it is not there, the package wasn't built properly, so all bets are off.)
The test is not perfect, because packages can put DLLs into the inst folder to get them installed without compiling them, though CRAN isn't supposed to allow that: "Source packages may not contain any form of binary executable code." But packages like pak (mentioned in the comments) may be allowed to get around this rule, e.g. by downloading binaries, so the test isn't perfect. You will also need to put together a blacklist of packages that will fail your WDAC tests even though they claim not to need compilation, containing pak and others like it.
The NeedsCompilation field is included as a column of the result of available.packages(), so it is very easy to access without trying to install the package.
I have accepted the answer from user2554330 as the best solution. It makes use of the normal set of commands used for package management; and the matrix generated by available.packages() can be passed to tools::package_dependencies(), removing the need for multiple internet queries.
For completeness I am documenting another possible solution. A script could query the unofficial CRAN Github mirror https://docs.r-hub.io/#cranatgh and look for a /src directory in each package project.
I'd like to use Julia on a computer which is disconnected from the Internet.
Is there simple procedure to download a package and then install it offline?
Surely, its possible.
Pkg.dir() # => get you the package installation path
check the pkg.julialang.org/ address to get the right package and click on its github link, then you can download a zip archive from github.com and extract it into Pkg.dir()
BUT you may taking yourself into trouble
because you must do many optional things manually, e.g.:
rename folder to remove .jl
build steps
install all related packages
I think a better way is to install Pkgs on a connected machine and then copy Pkg.dir() contents from that machine, to your system. this approach would works well only if both machines are of the same architecture (cpuX os julia-version).
I am using R 2.13.0 with windows 7, after giving my user full privileges to the R folder (as described here).
This allows me to install new packages just fine.
However, when using update.packages(), to update existing packages, I keep getting the following error (for example, when updating the MASS package):
package 'MASS' successfully unpacked and MD5 sums checked
Warning: unable to move temporary installation
'C:\Program
Files\R\R-2.13.0\library\file6cae3bcf\MASS'
to 'C:\Program
Files\R\R-2.13.0\library\MASS'
Any suggestions on how to fix this?
p.s: Running R as an administrator or shifting the library location out of Program Files is not a solution (it's a hack - but I am looking for a solution)
I found that the problem indeed is the antivirus "real time file system protection". I do the following to fix the problem:
trace(utils:::unpackPkgZip, edit=TRUE)
I edit line 140 (line 142 in R 3.4.4):
Sys.sleep(0.5)
to:
Sys.sleep(2)
I seems like the antivirus stalls the creation of the package tmp dir. After changing it to 2 seconds the error is gone.
EDIT: to do this programmatically execute
trace(utils:::unpackPkgZip, quote(Sys.sleep(2)), at = which(grepl("Sys.sleep", body(utils:::unpackPkgZip), fixed = TRUE)))
(credits #DavidArenburg)
Just to update everyone, I (think that I) found out the source of the problem: antivirus.
The "real time file system protection" was blocking R from copying the files between folders once they were downloaded.
Upon adding the R directory to the exception list (coupled with adding user permission and installing R on D:\R), and the problem went away. With all of this work, I might as well switch to Linux (I should, really...)
(I updated my post with the above information: http://www.r-statistics.com/2011/04/how-to-upgrade-r-on-windows-7/)
I hope it will help someone in the future,
Tal
If you cannot turn off your antivirus, due to corporate policy for example, here is a workaround that I found. Debugging the unzip package function and then stepping through it gives the antivirus enough time to do its job without interfering. Use this command:
debug(utils:::unpackPkgZip)
install.packages("packageName")
and then step through the code (by pressing enter many times) when R starts debugging during the installation.
I found this solution here.
If you can just download the binary straight from CRAN. On windows when downloaded it will be a zip file. Now manually unzip this into the ..library/ folder of your R (.libPaths()). It worked for me on some packages.
I had this problem installing both swirl and dplyr. I am working on Windows 64-bit.
Warning: unable to move temporary installation
What I did is I accessed my temporary files on the C: drive, and opened my file extractor program and I extracted the files from the temp file in the C: drive to my R program files in the C: drive, by manually copying them. THIS WORKED FOR BOTH dpylr and swirl. Stoked!
Cheers,
Peach
Can you not use the lib.loc parameter to only update packages in your personal library (in user)?
There should be no way to enable a normal, non-augmented user to change files in the program files folder, so the only thing you can do (if you don't want to augment the user) is to have R not updating packages there.
A workaround is to avoid installing R in the program files folder (which may be more or less of a hack than just shifting the library location out of it, depending on your point of view).
Finally, if lib.loc doesn't cut it, you can look at the source code for update.packages and create your own customized version that will always avoid the common library location in program files.
I just met the same question, and the solution I found out was that you should install packages using the original R software (plus, you should choose the right mirror site, some of them are blocked). At first I used Rstudio to install packages and I got the same problem as you met. Hope this is helpful.
I have run into this error several times. In my own case, it is because our admins want us to use remote virtual disks (on Windows 7) for our files and everything is locked up tight as a drum. The only way I can use R packages is in a lib directory on that remote virtual disk. This wouldn't be a problem except that the network isn't always smooth and fast. Thus, when I need a package, especially one with several other packages in tow (e.g., MBESS), I either have to go through the get.packages() process multiple times until it finally finishes or make it IT's headache to do quick like the bunny for me. I can't always wait for IT.
I just went to the library folder (Windows XP) and deleted all fileXXXX folders. Reran the install an it is worked.
I had the same problem. Since the issue seems to be the antivirus blocking the transf of a downloaded file, I tried a different download method in the install.packages and it worked.
For example:
install.packages("stringr", method = "curl")
You must go into the properties of the R folder and change the security parameters. You can enable the option to write and modify for all users.
The error : "unable to move temporary installation" is basically arising due to any of the antivirus running on your system.
Try unzipping the downloaded file from the Temp folder into the default library path (you can get it by running .libPaths() in R session).
I'm using a MRAN and I was having so many versioning issues. Trying to work with tidyverse and ggplot2 and by upgrading to the latest version from Microsoft it solved all of my R-Studio versioning issues.
Version info:
Microsoft R Open 3.5.1
The enhanced R distribution from Microsoft
Default CRAN mirror snapshot taken on 2018-08-01.
Download Microsoft R Open 3.5.1
Whenever I try to install a package in R, I get the following error:
Error in readRDS(file) : unknown input format
This just started occurring after I had a system crash. I am running 32 bit R 2.13.0 under windows 7. I tried removing and re-installing R, but continue to get the error. Is there any way I can fix this without deleting everything (i.e. all the packages I've installed) and starting over?
Thanks
These are suggestions I have come across:
Delete your .Rhistory and .RData files in the directory in which you are running R.
Run update.packages()
Try and detect "bad files" in your library directories. You can do this in R
# List the library paths
# The issue is likely to be in the first directory
paths = .libPaths()
## Try and detect bad files
list.files(paths,
pattern = "^00LOCK*|*\\.rds$|*\\.RDS$",
full.names = TRUE)
## List files of size 0
l = list.files(paths, full.names = TRUE)
l[sapply(l, file.size) == 0]
Delete any files/directories highlighted. You could use file.remove() if you really wanted to.
Delete the directory in which you have stored your downloaded packages.
Only solution 3 worked for me.
Ref:
R-sig-Debian mailing list
Option 3 was a combination of answers provided by different people over the last few years, including Chunxiao Xu, Larry Hunsicker and Frank Harrell
Run find /usr/local/lib/R/site-library/ /usr/lib/R/library/ /usr/lib/R/site-library/ ~/.local/lib/ -iname '*rds' -a -size 0 and then delete the files found.
Chunxiao Xu and Rando Hinn's solution above worked for me, with a minor tweak.
First, change directories to your personal R package directory, then run: find -iname '*rds' -a -size 0
Delete the directories containing any 0 length files in the above list. Then reopen R (or RStudio) and reinstall the deleted packages.
You should now be able to list the packages again.
Chunxiao Xu's original suggestion above lists ALL of the R package directories. But the exact locations of these directories will vary from installation to installation, and their addresses will have to be edited for your installation.
Larry Hunsicker
Something simple to try: if it is an .rda file use load instead of readRDS. You can then save the loaded file as an .rds and try readRDS again.
Here is what I had to do to solve this error:
Open your Rstudio, go to install tab under packages
Note your install to library path.(where your package will be saved)
Go to the install library path.And hard delete the recent packages
Refresh you package details in Rstudio
This happened to me after I had to reboot my machine in the middle of installing several R packages. I removed the latest installed packages by physically removing the files in the appropriate library directory, reinstalled them, and all was well. I also had to remove a 00LOCK directory in the library directory.
You might have some problem with R packages .
See if the problem has occurred after you installed some packed or did you copied the packages.
What you can do is Copy all the packages from Library and save it somewhere, then Copy some packages on the Library and restart the R test it and see how it works you have to try with all the packages one by one .
I has the same issue and it was due to package problem.
I also had several problems opening an .RDS file, either "error reading from connection" or "unknown input format". I tried with several R versions, with version 3.4.2. the following worked:
(I accidentally uninstalled rstudio and installed it again, also installed older R version 3.4.2, I don't know if that changes anything)
set up a new working directory
I opened the RDS file directly instead of downloading it (so only saved to temp files), then I was able to open it in R
I moved it from temp files to my new working directory and used the Data_Essay_Account <- readRDS("C:/.../newdirectory/Data_Essay_Account.RDS") command and it worked.
Somehow properly downloading it and then moving it did not work, only after I had it in the temp files. Hard deleting packages did not work for me as I tried that initially.
Unlike the top few answers here, my issue was resolved not by doing anything with my installation of RStudio or cleaning local files, but by re-writing the RDS file.
I could confirm this because the file was stored in the cloud, and reading it from a variety of windows and mac laptops all failed, but similar RDS files from the same S3 bucket worked without a problem.
When I re-wrote the RDS file, the problem went away.
I checked for this issue on several forums and then I tried this and it worked for me:
1) Remove the ggplot2 package first
remove.packages("ggplot2")
2) Then Install the package again
install.packages("ggplot2")
3) restart rstudio and install your package again, hope this helps
Seems that the issue was with the required package (ggplot2 in my case) got corrupt and was hampering the installation of any new dependent package.
I had the same problem as the OP, but in my case the problem .rds files were not 0 bytes but had file size >0 and were full of NULL characters (at least that's what I saw when I opened them in Notepad++)
By using debugonce(loadNamespace) before loading the library that was giving me the error (in my example it was the VIM package) I eventually traced my problem to corrupted .rds files in the META folder of the forcats package, which was installed as a 5th generation import when I installed VIM
(In the rstudio environment pane)
__NamesSpacesLoading__ chr[1:5] "forcats" "haven" "rio" "car" "VIM"
I then fixed by VIM problem simply by removing and reinstalling forcats
Adapting the code in #csgillespie accepted answer, I could have found this by trying to read all the rds files as below
paths = .libPaths()
l <- list.files(paths,
pattern = "*\\.rds$",
ignore.case = T,
recursive = T,
full.names = TRUE)
checkRDS <- function(file) {
tryCatch({
readRDS(file)
"OK"
},
error = function(cond) {
return("Error")
})
}
l[sapply(l,checkRDS)=="Error"]
I was able to fix it by using .libPaths() and finding the 4.1 folder (my version of R was 4.1, so may differ). From there I closed R studio and deleted all the folders for each package.
I reinstalled all packages and found the packages which wouldn't download through the error messages. When I located the packages that didn't download I installed them by using: install.packages("package", type = "binary").
A bit long but solved all my issues.
My mirror was misconfigured. My mirror is Oregon State University because that is what I am closest too. Here is a list of mirrors. Call this code before installing any packages. More information in ?options.
mirror <- "https://ftp.osuosl.org/pub/cran/"
local({
r <- getOption("repos")
r["CRAN"] <- mirror
options(repos = r)
})
The workaround
Execute this command in the R console:
options(pkgType = "source")
You're good to go.
I am using R 2.13.0 with windows 7, after giving my user full privileges to the R folder (as described here).
This allows me to install new packages just fine.
However, when using update.packages(), to update existing packages, I keep getting the following error (for example, when updating the MASS package):
package 'MASS' successfully unpacked and MD5 sums checked
Warning: unable to move temporary installation
'C:\Program
Files\R\R-2.13.0\library\file6cae3bcf\MASS'
to 'C:\Program
Files\R\R-2.13.0\library\MASS'
Any suggestions on how to fix this?
p.s: Running R as an administrator or shifting the library location out of Program Files is not a solution (it's a hack - but I am looking for a solution)
I found that the problem indeed is the antivirus "real time file system protection". I do the following to fix the problem:
trace(utils:::unpackPkgZip, edit=TRUE)
I edit line 140 (line 142 in R 3.4.4):
Sys.sleep(0.5)
to:
Sys.sleep(2)
I seems like the antivirus stalls the creation of the package tmp dir. After changing it to 2 seconds the error is gone.
EDIT: to do this programmatically execute
trace(utils:::unpackPkgZip, quote(Sys.sleep(2)), at = which(grepl("Sys.sleep", body(utils:::unpackPkgZip), fixed = TRUE)))
(credits #DavidArenburg)
Just to update everyone, I (think that I) found out the source of the problem: antivirus.
The "real time file system protection" was blocking R from copying the files between folders once they were downloaded.
Upon adding the R directory to the exception list (coupled with adding user permission and installing R on D:\R), and the problem went away. With all of this work, I might as well switch to Linux (I should, really...)
(I updated my post with the above information: http://www.r-statistics.com/2011/04/how-to-upgrade-r-on-windows-7/)
I hope it will help someone in the future,
Tal
If you cannot turn off your antivirus, due to corporate policy for example, here is a workaround that I found. Debugging the unzip package function and then stepping through it gives the antivirus enough time to do its job without interfering. Use this command:
debug(utils:::unpackPkgZip)
install.packages("packageName")
and then step through the code (by pressing enter many times) when R starts debugging during the installation.
I found this solution here.
If you can just download the binary straight from CRAN. On windows when downloaded it will be a zip file. Now manually unzip this into the ..library/ folder of your R (.libPaths()). It worked for me on some packages.
I had this problem installing both swirl and dplyr. I am working on Windows 64-bit.
Warning: unable to move temporary installation
What I did is I accessed my temporary files on the C: drive, and opened my file extractor program and I extracted the files from the temp file in the C: drive to my R program files in the C: drive, by manually copying them. THIS WORKED FOR BOTH dpylr and swirl. Stoked!
Cheers,
Peach
Can you not use the lib.loc parameter to only update packages in your personal library (in user)?
There should be no way to enable a normal, non-augmented user to change files in the program files folder, so the only thing you can do (if you don't want to augment the user) is to have R not updating packages there.
A workaround is to avoid installing R in the program files folder (which may be more or less of a hack than just shifting the library location out of it, depending on your point of view).
Finally, if lib.loc doesn't cut it, you can look at the source code for update.packages and create your own customized version that will always avoid the common library location in program files.
I just met the same question, and the solution I found out was that you should install packages using the original R software (plus, you should choose the right mirror site, some of them are blocked). At first I used Rstudio to install packages and I got the same problem as you met. Hope this is helpful.
I have run into this error several times. In my own case, it is because our admins want us to use remote virtual disks (on Windows 7) for our files and everything is locked up tight as a drum. The only way I can use R packages is in a lib directory on that remote virtual disk. This wouldn't be a problem except that the network isn't always smooth and fast. Thus, when I need a package, especially one with several other packages in tow (e.g., MBESS), I either have to go through the get.packages() process multiple times until it finally finishes or make it IT's headache to do quick like the bunny for me. I can't always wait for IT.
I just went to the library folder (Windows XP) and deleted all fileXXXX folders. Reran the install an it is worked.
I had the same problem. Since the issue seems to be the antivirus blocking the transf of a downloaded file, I tried a different download method in the install.packages and it worked.
For example:
install.packages("stringr", method = "curl")
You must go into the properties of the R folder and change the security parameters. You can enable the option to write and modify for all users.
The error : "unable to move temporary installation" is basically arising due to any of the antivirus running on your system.
Try unzipping the downloaded file from the Temp folder into the default library path (you can get it by running .libPaths() in R session).
I'm using a MRAN and I was having so many versioning issues. Trying to work with tidyverse and ggplot2 and by upgrading to the latest version from Microsoft it solved all of my R-Studio versioning issues.
Version info:
Microsoft R Open 3.5.1
The enhanced R distribution from Microsoft
Default CRAN mirror snapshot taken on 2018-08-01.
Download Microsoft R Open 3.5.1