This is sort of related to a previous post of mine. I have the need to use the bigmemory library on my 32bit Windows PC to do some ugly matrix calculations. Unfortunately, it appears that the maintainers have temporarily ceased production of Windows binaries. I have Ubuntu on my home PC. I would really like to take the .tar.gz file and build it into a Windows binary that I can actually run at work. I realize there are more efficient ways, like installing RTools on the Windows device. However, our IT keeps our admin rights on lockdown, so I can never edit my PATH enviro variable. Could anyone provide some general guidance for doing this? Are there any tools I need to install on my Ubuntu PC above and beyond R?
I found similar questions, but nothing that thoroughly answered my questions.
Unless the package source is incompatible with current versions of R, you could use the R project's win-builder site to build a Windows binary. Quoting from the linked site, win-builder is a service:
intended for useRs who do not have Windows available for checking and building Windows binary packages.
As a convenience, Hadley Wickham's devtools package includes a utility function, build_win(), that you can use for this purpose. From ?build_win:
Works by building source package, and then uploading to http://win-builder.r-project.org/>. Once building is complete you'll receive a link to the built package in the email address listed in the maintainer field. It usually takes around 30 minutes.
Windows has four sets of environment variables (system, user, volatile and process sets). The first three sets are stored in the registry but the process set is not so even if they have locked down the registry its typically still possible to set the process environment variables (including the PATH) in a local process, i.e. on a temporary basis, so you might double check your assumptions that you can't modify anything. Its more likely that you can't modify the system variables and registry but can still modify the set in your local process. To check this from the Windows cmd line enter this:
set mytest=123
set mytest
and if the second line shows that mytest has the value 123 then you likely have all the permissions you need.
Furthermore anything you need to set is all handled automatically for you by R.bat in the batchfiles distribution so you don't have to set anything yourself.
Just ensure that Rtools and R are installed into the standard locations (you can tell them to skip the setting of any registry keys during the installation process), ensure R.bat is on your path or in current directory and run:
R.bat CMD INSTALL mypackage.tar.gz
without setting environment variables, registry keys or path.
If that does not work try Rpathset.bat also from the batchfiles which is not automatic like R.bat but on the other hand is extremely flexible since you must modify the SET statments in it to whatever you want.
There is a PDF document that comes with the batchfiles which gives more info.
Related
I recently began receiving warnings that prior installations of R packages cannot be removed when I try to re-install packages:
install.packages("gtools")
#> Warning: cannot remove prior installation of package ‘gtools’
#> Warning: restored ‘gtools’
I found solutions to this issue encouraging me to delete the packages manually from my library folder, which I could find with .libPaths(). However, (a) this seems like a way of addressing symptoms rather than the underlying issue (which remains unclear) and (b) there are two paths for seemingly different versions of R and I'm not sure which to delete from anyway:
.libPaths()
#> [1] "C:/Users/foo/Documents/R/win-library/4.1"
#> [2] "C:/Program Files/R/R-4.1.2/library"
How can I fix the problem so I don't have to manually delete package folders every time I want to re-install a package? If there is no alternative, do I need to delete the subdirectories for the package from one of those folders or both? FWIW, I'm working in RStudio.
The problem is that you have installed packages using different permissions. On Windows, you need elevated permissions to write to Program Files. At some point you (or an admin) probably used "Run as admin" to install gtools there, and now using regular permissions you can't delete that.
You should be able to delete the Users/foo copy, if you are running as user foo, but even that one may have had permissions changed. But I'd guess the issue is that gtools is in the Program files location.
The error message from R doesn't tell you which location it is trying to delete from, which is unfortunate. In fact, allowing installations of different versions in those two locations is a bad design feature in R that just leads to confusion, because you don't necessarily always use the same version each time you load packages. (The rule for which one you use is the first acceptable one found in the .libPaths list, but since you can change .libPaths, and since packages can load other packages, it's hard to predict which one you'll have loaded at any given time.)
To fix this, you can delete both copies (if you have two) and start over, but that's risky because other packages might be depending on gtools. If you are the only user on your computer, you could instead delete the entire "C:/Users/foo/Documents/R/win-library/4.1" library, and then do all your installs using "Run as admin", but that's also easy to mess up.
(On a Mac, that's effectively what happens, because most single user systems put the user in the "admin" group, so they can always install packages to the system location. It causes a lot less confusion, but some "purists" think the Windows way is better.)
So I don't have any good advice for you, but maybe this explains the situation, and you can work out for yourself the best way forward.
I want to follow the advice I've read and heard to have both a main library in R_HOME/library and a user library. I'm using W10 on a desktop machine (not important, except that it gives me a name by which to refer to it), and I can't make R use the user library.
I have succeeded in doing that on a W10 laptop: C:/R/R-4.0.2/library contains some 30 recommended packages, and C:/Users/[username]/Documents/R/win-library/4.0 con contains a much larger number of packages in my user library.
As I recall, and as I wrote down when I did an upgrade on a server, all you have to do to create a site-library is to create a directory called C:/R/R-4.0.2/site-library, and R will use that the next time it starts.
To create a user library, create the directory C:/Users/[username]/Documents/R/win-library/4.0.
That seemed to work on my laptop, for I have seemingly a working R library and a user library there.
That seemed to work on the server, too: I have a library and a site-library.
In both cases, .libPaths() shows the same libraries that I see with Dired on the disk.
I tried to do the same thing on the desktop machine, and i can't make it work.
I created a directory C:/Users/[username]/Documents/R/win-library/4.0, restarted R, and ran .libPaths(); the only directory that was listed was C:/R/R-4.0.2/library.
Because I thought the Documents in that path seemed odd, I tried it again using C:/Users/[username]/R/win-library/4.0, still with no success.
https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Managing-libraries seems pertinent, but I'm not sure how to interpret the output of Sys.getenv("R_LIBL_USER). I get "\\[toplevel]\[nextlevel]\Home$\[username]/R/win-library/4.0", which I presume is a long-winded way to get to /Home$/[username]/R/win-library/4.0 (aka C:/Users/[username]/R/win-library/4.0.
Suggestions? I've tried a number of other suggestions from SO, all to no avail.
I'm trying to use a linux server with R installed. Apparently the R system library has old versions of non-base packages installed like dplyr and testthat.
Because i don't have permission to edit the system library, i'm unable to update the packages.
My plan is to only use a user library, so I can controll the package versions myself. However i'm unable to remove the "/usr/lib64/R/library" folder from .libPaths(). I tried changing the environment variables R_LIBS_SITE and R_LIBS with the .Renviron and .Rprofile files to a different folder, but the /usr/lib64/R/library folder will always be present. Removing it with the command .libPaths(.libPaths()[1:2]) doesn't work either.
Is there a way to remove the system library from .libPaths(), so I'm not depending on the update policy of the server admin?
You can't remove the system library, because that's where the base packages live. They can't be installed anywhere else, and R won't work without them.
Best would be for you to get your sysadmin to update the system library. Those obsolete packages probably contain bugs.
If you can't do that, then run update.packages(instlib = "local") to install all the latest versions in the library named "local". (Substitute your own local lib name, of course.) This requires all your users to specify .libPaths("local") when they start, and some will likely forget, so it's not as good.
It might be easiest for you to just install a full copy of R in your own account. Then you'll have control of things, and anyone using your copy will get your library.
(There's a new release (3.5.3) coming in ten days; you might wait for that, or install one of the betas or RCs, which should be available now, then update again when the final release arrives.)
For me, it works to use
.libPaths(.libPaths()[2:1])
This will still search the system library, but only after it searches my personal library, so if I have a newer version, it uses that. Note: I used .libPaths()[2:1] not .libPaths()[1:2]
We recently decided to make Julia Language available on our cluster systems. The cluster system is not able to connect to the internet.
Is there any way to download all Julia packages and make them available for our different users to install and use them offline?
Another option that we have is a system that can connect to the internet temporarily, but it is always connected to the main cluster system. Is there any way to use this system as a mirror for the Julia packages or not?
We want to use "Julia 1.0.1".
our cluster operation system is: "CentOS 5.5
notes: I have seen the question asked before here, but it is for Julia 0.6 and a single package that will be copied by hand. I want that user uses the Pkg.add <pkgName> command but instead of the internet, the package manager gets the packages from our offline system.
Thank you for your help and time.
Caution:
Side effects are not known!
May please be tested properly before put into production!
a) Collect the required packages along with their dependent packages in compiled form, put them in folder, stdlib (for example: /opt/julia/julia-1.1.0/shared/julia/stdlib/v1.1/)
b) add stdlib path to environment variables, JULIA_DEPOT_PATH and JULIA_LOAD_PATH
The following is a crosspost of https://stackoverflow.com/a/74800608/18431399
PackageCompiler.jl seems like the best tool for using modern Julia (v1.8) on secure systems. The following approach requires a build server with the same architecture as the deployment server, something your institution probably already uses for developing containers, etc.
Build a sysimage with PackageCompiler's create_sysimage()
Upload the build (sysimage and depot) along with the Julia binaries to the secure system
Alias a script to julia, similar to the following example:
#!/bin/bash
set -Eeu -o pipefail
unset JULIA_LOAD_PATH
export JULIA_PROJECT=/Path/To/Project
export JULIA_DEPOT_PATH=/Path/To/Depot
export JULIA_PKG_OFFLINE=true
/Path/To/julia -J/Path/To/sysimage.so "$#"
I've been able to run a research pipeline on my institution's secure system, for which there is a public version of the approach.
The default R library, .Library, is normally not writeable under Windows.
You need to run R as Administrator. For new packages you can set and use a personal library, but this doesn't work when updating packages in the base setup (e.g. by update.packages()).
If you forget (or don't know you need) to run as Administrator, you get duplicate versions of the same packages, messing the installation.
I think one solution could be copying all packages to a personal library and disabling the default one. I know how to add a new library path to R, i.e. .libPaths ("my/path"), but how to remove the default library from .libPaths ()?
Update for non-Windows users
Some clarifications might help mostly non-Windows R users to understand the mentioned problem.
In Windows "Log on as Administrator" (or better as a user belonging to administrators' group) and "Run as Administrator" are quite different things.
In the former case you just give your credentials at logon, much like in Linux; in the latter you are already logged as a "superuser", but in order to carry out a potentially dangerous action, you have to ask an ad hoc permission to Windows (proving that it's you and not a malware acting).
That's said, programs (and developers), before accessing known Windows' protected objects (i.e. C:\Program Files folder), ask permission to the user to avoid being blocked by the OS.
Even when they don't ask (because they assume the knowledgeable user should give this permission in advance), failure to access is normally reported like "Permission denied to access to folder etc.".
As for R version 3.0.2, update.packages() involves one of the situations, where an elevated permission request should be triggered, because this might involve writing to protected program folders. Unfortunately R doesn't ask and cannot update the directory with old packages.
What about the second safe net: user notifications? While install.packages() gives messages like:
stop ... "'lib' element %s is not a writable directory" ...
and you get the idea of a permission problem, with others functions, such as update.packages(), you get:
warning ... "package '%s' in library '%s' will not be updated"
whose causes can be everything.
Can this scenario be even worse? Yes. Besides not asking for permission to write to "Program Folders", besides not issuing a notification of the permission error, update.packages(), when unable to update packages in protected folders, actually installs them to the personal user folder, but without notifying this. This is similar to what install.packages() does, except that the latter notifies and asks permission to do this.
So you end up with two versions of the same packages in different folders! Your calculations will be therefore dependent on library priorities.
Can this scenario be even worse? Yes. You are clever (or Google) enough to understand that you need to "Run as Administrator", when you want to update packages. You restart R as Administrator and hope this will fix everything. Not at all. R sees the updated packages in the personal library and does not act. So you remain with two versions of the same packages.
To solve this you have to detect duplicate packages and remove them manually, then restart R as administrator and update again (or write a script to do this).
Clearly the solution would be R conforming to Windows apps expected behaviour, or at least do nothing when prevented to act (instead of taking non-notified decisions).
In the meantime I think that totally disabling the default library (located in a protected area) would be a temporary workaround.
A final note. Packages and package updating are crucial for using R, so my humble opinion is that the topic should deserve specific careful attention even for less GNU-blessed systems like Windows.
One solution is to change R_LIBS environment variable. You can see for example this question.
But If you don't have admin rights, you can specify location when you load the package:
library(my_package, lib.loc="my/path")