I have developed a solution using R and want to transfer it to the production server (CentOS 7) which has no Internet connection to install packages. To facilitate installation of packages, I used packrat to bundle the packages I used in my R script to the project.
Using packrat::bundle(), I have created a tar file of the project and moved the file to the server and untar the zip file.
According to a post in Blogger, once I open the project, When R is started from that directory, Packrat will do its magic and make sure the software environment is the same as on the source machine.
However, when I open the project in Server (using R-Studio Server 0.99), nothing happens and it throws error of unknown packages.
When manually execute the "packarat/init.R" file below error is thrown
Error in ensurePackageSymlink(source, target) :
Target '/home/R_Projects/prjName/packrat/lib-R/base' already exists and is not a symlink
Well, I found the problem and solve it. The symlink error is related to centOS (it is not related to R). I just simply removed all the folders inside the
/home/R_Projects/prjName/packrat/lib-R
Because these folder exist, the packrat is unable to create symlink with the same name inside the lib-R folder. If I remove them, it will create a link (shortcut) to the actual folder where the r package is located.
Hope it helps future readers.
Related
I have been given access to a beefy machine on which to run a large simulation.
I have developed the code in an RStudio project with renv. Renv makes a local copy of all the packages and stores versions thereof in a lock file.
The target machine (which runs Windows) does not have access to the internet. I have copied the project file, the code files, the renv folder (which includes all the local copies of the packages, the lock file, and the .RProfile file, to a folder on the target machine.
When I open the project on the target machine, the .RProfile executes source("renv/activate.R"). However, this fails to load the projects, instead giving me the following message
The following package(s) are missing their DESCRIPTION files:
... Long list of packages ...
These may be left over from a prior, failed installation attempt.
Consider removing or re-installing these packages.
Trouble is I can't reinstall them since this machine does not have access to the internet. I could manually go through each package and download the binaries on my work machine, then transfer them over to the target machine, then install them one by one, but this seems like a very painful thing to do.
Is there a way for me to convince renv, or R itself, to just use the packages in the local renv folder?
From the Cache section of https://rstudio.github.io/renv/articles/renv.html:
When using renv with the global package cache, the project library is instead formed as a directory of symlinks (or, on Windows, junction points) into the renv global package cache. Hence, while each renv project is isolated from other projects on your system, they can still re-use the same installed packages as required.
I think that implies trying to copy the renv folder means copying those junction points (which are something like shortcuts / symlinks), not the actual underlying folder.
It looks like you already have a solution, but another option would be to call renv::isolate() to ensure that your project doesn't link to packages within the global cache, and instead just maintains an isolated library directly.
In the end I just wrote an small script to copy the files over.
sourceFolder = "some/path"
targetFolder = "some/other/path"
subFolders = list.files(sourceFolder)
for (i in seq_along(subFolders)) {
subFolder = subFolders[i]
file.copy(
from = paste0(sourceFolder, subFolder),
to = targetFolder,
overwrite = TRUE,
recursive = TRUE
)
paste(subFolder) |> message()
}
I am working on a remote server using RStudio. This server has no access to the Internet. I would like to install the package "stringi." I have looked at this stackoverflow article, but whenever I use the command
install.packages("stringi_0.5-5.tar.gz",
configure.vars="ICUDT_DIR=/my/directory/for/icudt.zip")
It simply tries to access the Internet, which it cannot do. Up until now I have been using Tools -> Install Packages -> Install from Packaged Archive File. However, due to this error, I can no longer use this method.
How can I install this package?
If you have no internet access on local machines, you can build a distributable source package that includes all the required
ICU data files (for off-line use) by omitting some relevant lines in
the .Rbuildignore file. The following command sequence should do the trick:
wget https://github.com/gagolews/stringi/archive/master.zip -O stringi.zip
unzip stringi.zip
sed -i '/\/icu..\/data/d' stringi-master/.Rbuildignore
R CMD build stringi-master
Assuming the most recent development version is 1.3.1,
a file named stringi_1.3.1.tar.gz is created in the current working directory.
The package can now be installed (the source bundle may be propagated via
scp etc.) by executing:
R CMD INSTALL stringi_1.3.1.tar.gz
or by calling install.packages("stringi_1.3.1.tar.gz", repos=NULL),
from within an R session.
For a Linux machine the easiest way is from my point of view:
Download the release you need from Rexamine in tar.gz format to your local pc. In opposition to the version on CRAN it already contains the icu55\data\ folder.
Move the archive to your target linux machine without internet access
run R CMD INSTALL stringi-1.0-1.tar.gz (in case of release 1.0-1)
You provided the wrong value of configure.vars.
It indicates that you have to give the directory's name, not a final file name.
Correct your code to the following:
install.packages("stringi_0.5-5.tar.gz",
configure.vars="ICUDT_DIR=/my/directory/for/")
Regards,
Sean
Follow the steps below
Download icudt55l.zip seperately from server where you have internet access with
wget http://www.mini.pw.edu.pl/~gagolews/stringi/icudt55l.zip
Copy the downloaded packages to the server where you want to install stringi
Execute the following command
R CMD INSTALL --configure-vars='ICUDT_DIR=/tmp/ALL' stringi_1.1.6.tar.gz
icudt55l.zip is copied to /tmp/ALL
The suggestion from #gagolews almost worked for me. Here's what actually did the trick with RStudio.
Download the master.zip file that will save as stringi-master.zip.
Unzip the file onto your desktop. The unzipped folder should be stringi-master.
Edit the .Rbuildignore file by removing ^src/icu55/data and ^src/icu61/data or similar lines.
Move the folder from your desktop to the home directory of your server.
Create a New Project in RStudio with ~/stringi-master as the Existing Directory
From RStudio's menu, select Build and Build Source Package. (You may need to first select Configure Build Tools. For Project build tools choose Package then select OK.)
It should create a tar.gz file, in the following format: stringi_x.x.(x+1).tar.gz. For example, if the current version of stringi is 1.5.3, it will create version 1.5.4. (I received a few warnings that didn't seem to affect the outcome.)
Move the newly created package to your local repository. Update the repository index. And install the package.
I am using R 2.13.0 with windows 7, after giving my user full privileges to the R folder (as described here).
This allows me to install new packages just fine.
However, when using update.packages(), to update existing packages, I keep getting the following error (for example, when updating the MASS package):
package 'MASS' successfully unpacked and MD5 sums checked
Warning: unable to move temporary installation
'C:\Program
Files\R\R-2.13.0\library\file6cae3bcf\MASS'
to 'C:\Program
Files\R\R-2.13.0\library\MASS'
Any suggestions on how to fix this?
p.s: Running R as an administrator or shifting the library location out of Program Files is not a solution (it's a hack - but I am looking for a solution)
I found that the problem indeed is the antivirus "real time file system protection". I do the following to fix the problem:
trace(utils:::unpackPkgZip, edit=TRUE)
I edit line 140 (line 142 in R 3.4.4):
Sys.sleep(0.5)
to:
Sys.sleep(2)
I seems like the antivirus stalls the creation of the package tmp dir. After changing it to 2 seconds the error is gone.
EDIT: to do this programmatically execute
trace(utils:::unpackPkgZip, quote(Sys.sleep(2)), at = which(grepl("Sys.sleep", body(utils:::unpackPkgZip), fixed = TRUE)))
(credits #DavidArenburg)
Just to update everyone, I (think that I) found out the source of the problem: antivirus.
The "real time file system protection" was blocking R from copying the files between folders once they were downloaded.
Upon adding the R directory to the exception list (coupled with adding user permission and installing R on D:\R), and the problem went away. With all of this work, I might as well switch to Linux (I should, really...)
(I updated my post with the above information: http://www.r-statistics.com/2011/04/how-to-upgrade-r-on-windows-7/)
I hope it will help someone in the future,
Tal
If you cannot turn off your antivirus, due to corporate policy for example, here is a workaround that I found. Debugging the unzip package function and then stepping through it gives the antivirus enough time to do its job without interfering. Use this command:
debug(utils:::unpackPkgZip)
install.packages("packageName")
and then step through the code (by pressing enter many times) when R starts debugging during the installation.
I found this solution here.
If you can just download the binary straight from CRAN. On windows when downloaded it will be a zip file. Now manually unzip this into the ..library/ folder of your R (.libPaths()). It worked for me on some packages.
I had this problem installing both swirl and dplyr. I am working on Windows 64-bit.
Warning: unable to move temporary installation
What I did is I accessed my temporary files on the C: drive, and opened my file extractor program and I extracted the files from the temp file in the C: drive to my R program files in the C: drive, by manually copying them. THIS WORKED FOR BOTH dpylr and swirl. Stoked!
Cheers,
Peach
Can you not use the lib.loc parameter to only update packages in your personal library (in user)?
There should be no way to enable a normal, non-augmented user to change files in the program files folder, so the only thing you can do (if you don't want to augment the user) is to have R not updating packages there.
A workaround is to avoid installing R in the program files folder (which may be more or less of a hack than just shifting the library location out of it, depending on your point of view).
Finally, if lib.loc doesn't cut it, you can look at the source code for update.packages and create your own customized version that will always avoid the common library location in program files.
I just met the same question, and the solution I found out was that you should install packages using the original R software (plus, you should choose the right mirror site, some of them are blocked). At first I used Rstudio to install packages and I got the same problem as you met. Hope this is helpful.
I have run into this error several times. In my own case, it is because our admins want us to use remote virtual disks (on Windows 7) for our files and everything is locked up tight as a drum. The only way I can use R packages is in a lib directory on that remote virtual disk. This wouldn't be a problem except that the network isn't always smooth and fast. Thus, when I need a package, especially one with several other packages in tow (e.g., MBESS), I either have to go through the get.packages() process multiple times until it finally finishes or make it IT's headache to do quick like the bunny for me. I can't always wait for IT.
I just went to the library folder (Windows XP) and deleted all fileXXXX folders. Reran the install an it is worked.
I had the same problem. Since the issue seems to be the antivirus blocking the transf of a downloaded file, I tried a different download method in the install.packages and it worked.
For example:
install.packages("stringr", method = "curl")
You must go into the properties of the R folder and change the security parameters. You can enable the option to write and modify for all users.
The error : "unable to move temporary installation" is basically arising due to any of the antivirus running on your system.
Try unzipping the downloaded file from the Temp folder into the default library path (you can get it by running .libPaths() in R session).
I'm using a MRAN and I was having so many versioning issues. Trying to work with tidyverse and ggplot2 and by upgrading to the latest version from Microsoft it solved all of my R-Studio versioning issues.
Version info:
Microsoft R Open 3.5.1
The enhanced R distribution from Microsoft
Default CRAN mirror snapshot taken on 2018-08-01.
Download Microsoft R Open 3.5.1
Normally I install packages using:
install.packages("foo")
and a Repo over the internet. But I have a new machine now where I want to replicate the packages from my existing installation without having to pull everything off the internet all over again. (I've a ton of packages and slow internet access)
Both machines are Windows and run the same R version. (2.13.1)
Is there a way to do this? Closest I can get is I know I can install from local zip files using:
install.packages("pathtozip", repos = NULL)
But does R store all Zips somewhere? I found a few in locations like:
C:\Documents and Settings\foouser\Local Settings\Temp\RtmpjNKkyp\downloaded_packages
But not all.
Any tips?
The function .libPaths will give you a vector of all the libraries on your machine. Run this on your old machine to find all of them. You can simply copy all these files into the libraries on your new machine (run .libPaths on it too to find out where).
Alternatively, if you want to set up a real repository (i.e. basically a CRAN mirror) on your computer or on a network drive you can update, you can put binary or source packages into a folder and run tools::write_PACKAGES on that folder. You can them run install.packages using the contriburl argument and point it to your repository folder.
All packages that you have installed are stored in a folder called win-library\r-version, for example,
C:\Users\Ehsan\Documents\R\win-library\2.15 so, it is enough to copy all the folders inside 2.15 to the same folder in your new machine. because you have the same version of R you do not need to update them by update.packages().
On your original computer, run
write.csv(unique(data.frame(installed.packages())[,1]),"packages.csv",row.names=F)
Save this .csv into the working directory of your new computer, then run
install.packages(as.character(read.csv("packages.csv")[,1]))
You can check what your working directory is using getwd().
I am using R 2.13.0 with windows 7, after giving my user full privileges to the R folder (as described here).
This allows me to install new packages just fine.
However, when using update.packages(), to update existing packages, I keep getting the following error (for example, when updating the MASS package):
package 'MASS' successfully unpacked and MD5 sums checked
Warning: unable to move temporary installation
'C:\Program
Files\R\R-2.13.0\library\file6cae3bcf\MASS'
to 'C:\Program
Files\R\R-2.13.0\library\MASS'
Any suggestions on how to fix this?
p.s: Running R as an administrator or shifting the library location out of Program Files is not a solution (it's a hack - but I am looking for a solution)
I found that the problem indeed is the antivirus "real time file system protection". I do the following to fix the problem:
trace(utils:::unpackPkgZip, edit=TRUE)
I edit line 140 (line 142 in R 3.4.4):
Sys.sleep(0.5)
to:
Sys.sleep(2)
I seems like the antivirus stalls the creation of the package tmp dir. After changing it to 2 seconds the error is gone.
EDIT: to do this programmatically execute
trace(utils:::unpackPkgZip, quote(Sys.sleep(2)), at = which(grepl("Sys.sleep", body(utils:::unpackPkgZip), fixed = TRUE)))
(credits #DavidArenburg)
Just to update everyone, I (think that I) found out the source of the problem: antivirus.
The "real time file system protection" was blocking R from copying the files between folders once they were downloaded.
Upon adding the R directory to the exception list (coupled with adding user permission and installing R on D:\R), and the problem went away. With all of this work, I might as well switch to Linux (I should, really...)
(I updated my post with the above information: http://www.r-statistics.com/2011/04/how-to-upgrade-r-on-windows-7/)
I hope it will help someone in the future,
Tal
If you cannot turn off your antivirus, due to corporate policy for example, here is a workaround that I found. Debugging the unzip package function and then stepping through it gives the antivirus enough time to do its job without interfering. Use this command:
debug(utils:::unpackPkgZip)
install.packages("packageName")
and then step through the code (by pressing enter many times) when R starts debugging during the installation.
I found this solution here.
If you can just download the binary straight from CRAN. On windows when downloaded it will be a zip file. Now manually unzip this into the ..library/ folder of your R (.libPaths()). It worked for me on some packages.
I had this problem installing both swirl and dplyr. I am working on Windows 64-bit.
Warning: unable to move temporary installation
What I did is I accessed my temporary files on the C: drive, and opened my file extractor program and I extracted the files from the temp file in the C: drive to my R program files in the C: drive, by manually copying them. THIS WORKED FOR BOTH dpylr and swirl. Stoked!
Cheers,
Peach
Can you not use the lib.loc parameter to only update packages in your personal library (in user)?
There should be no way to enable a normal, non-augmented user to change files in the program files folder, so the only thing you can do (if you don't want to augment the user) is to have R not updating packages there.
A workaround is to avoid installing R in the program files folder (which may be more or less of a hack than just shifting the library location out of it, depending on your point of view).
Finally, if lib.loc doesn't cut it, you can look at the source code for update.packages and create your own customized version that will always avoid the common library location in program files.
I just met the same question, and the solution I found out was that you should install packages using the original R software (plus, you should choose the right mirror site, some of them are blocked). At first I used Rstudio to install packages and I got the same problem as you met. Hope this is helpful.
I have run into this error several times. In my own case, it is because our admins want us to use remote virtual disks (on Windows 7) for our files and everything is locked up tight as a drum. The only way I can use R packages is in a lib directory on that remote virtual disk. This wouldn't be a problem except that the network isn't always smooth and fast. Thus, when I need a package, especially one with several other packages in tow (e.g., MBESS), I either have to go through the get.packages() process multiple times until it finally finishes or make it IT's headache to do quick like the bunny for me. I can't always wait for IT.
I just went to the library folder (Windows XP) and deleted all fileXXXX folders. Reran the install an it is worked.
I had the same problem. Since the issue seems to be the antivirus blocking the transf of a downloaded file, I tried a different download method in the install.packages and it worked.
For example:
install.packages("stringr", method = "curl")
You must go into the properties of the R folder and change the security parameters. You can enable the option to write and modify for all users.
The error : "unable to move temporary installation" is basically arising due to any of the antivirus running on your system.
Try unzipping the downloaded file from the Temp folder into the default library path (you can get it by running .libPaths() in R session).
I'm using a MRAN and I was having so many versioning issues. Trying to work with tidyverse and ggplot2 and by upgrading to the latest version from Microsoft it solved all of my R-Studio versioning issues.
Version info:
Microsoft R Open 3.5.1
The enhanced R distribution from Microsoft
Default CRAN mirror snapshot taken on 2018-08-01.
Download Microsoft R Open 3.5.1