The default R library, .Library, is normally not writeable under Windows.
You need to run R as Administrator. For new packages you can set and use a personal library, but this doesn't work when updating packages in the base setup (e.g. by update.packages()).
If you forget (or don't know you need) to run as Administrator, you get duplicate versions of the same packages, messing the installation.
I think one solution could be copying all packages to a personal library and disabling the default one. I know how to add a new library path to R, i.e. .libPaths ("my/path"), but how to remove the default library from .libPaths ()?
Update for non-Windows users
Some clarifications might help mostly non-Windows R users to understand the mentioned problem.
In Windows "Log on as Administrator" (or better as a user belonging to administrators' group) and "Run as Administrator" are quite different things.
In the former case you just give your credentials at logon, much like in Linux; in the latter you are already logged as a "superuser", but in order to carry out a potentially dangerous action, you have to ask an ad hoc permission to Windows (proving that it's you and not a malware acting).
That's said, programs (and developers), before accessing known Windows' protected objects (i.e. C:\Program Files folder), ask permission to the user to avoid being blocked by the OS.
Even when they don't ask (because they assume the knowledgeable user should give this permission in advance), failure to access is normally reported like "Permission denied to access to folder etc.".
As for R version 3.0.2, update.packages() involves one of the situations, where an elevated permission request should be triggered, because this might involve writing to protected program folders. Unfortunately R doesn't ask and cannot update the directory with old packages.
What about the second safe net: user notifications? While install.packages() gives messages like:
stop ... "'lib' element %s is not a writable directory" ...
and you get the idea of a permission problem, with others functions, such as update.packages(), you get:
warning ... "package '%s' in library '%s' will not be updated"
whose causes can be everything.
Can this scenario be even worse? Yes. Besides not asking for permission to write to "Program Folders", besides not issuing a notification of the permission error, update.packages(), when unable to update packages in protected folders, actually installs them to the personal user folder, but without notifying this. This is similar to what install.packages() does, except that the latter notifies and asks permission to do this.
So you end up with two versions of the same packages in different folders! Your calculations will be therefore dependent on library priorities.
Can this scenario be even worse? Yes. You are clever (or Google) enough to understand that you need to "Run as Administrator", when you want to update packages. You restart R as Administrator and hope this will fix everything. Not at all. R sees the updated packages in the personal library and does not act. So you remain with two versions of the same packages.
To solve this you have to detect duplicate packages and remove them manually, then restart R as administrator and update again (or write a script to do this).
Clearly the solution would be R conforming to Windows apps expected behaviour, or at least do nothing when prevented to act (instead of taking non-notified decisions).
In the meantime I think that totally disabling the default library (located in a protected area) would be a temporary workaround.
A final note. Packages and package updating are crucial for using R, so my humble opinion is that the topic should deserve specific careful attention even for less GNU-blessed systems like Windows.
One solution is to change R_LIBS environment variable. You can see for example this question.
But If you don't have admin rights, you can specify location when you load the package:
library(my_package, lib.loc="my/path")
Related
I recently began receiving warnings that prior installations of R packages cannot be removed when I try to re-install packages:
install.packages("gtools")
#> Warning: cannot remove prior installation of package ‘gtools’
#> Warning: restored ‘gtools’
I found solutions to this issue encouraging me to delete the packages manually from my library folder, which I could find with .libPaths(). However, (a) this seems like a way of addressing symptoms rather than the underlying issue (which remains unclear) and (b) there are two paths for seemingly different versions of R and I'm not sure which to delete from anyway:
.libPaths()
#> [1] "C:/Users/foo/Documents/R/win-library/4.1"
#> [2] "C:/Program Files/R/R-4.1.2/library"
How can I fix the problem so I don't have to manually delete package folders every time I want to re-install a package? If there is no alternative, do I need to delete the subdirectories for the package from one of those folders or both? FWIW, I'm working in RStudio.
The problem is that you have installed packages using different permissions. On Windows, you need elevated permissions to write to Program Files. At some point you (or an admin) probably used "Run as admin" to install gtools there, and now using regular permissions you can't delete that.
You should be able to delete the Users/foo copy, if you are running as user foo, but even that one may have had permissions changed. But I'd guess the issue is that gtools is in the Program files location.
The error message from R doesn't tell you which location it is trying to delete from, which is unfortunate. In fact, allowing installations of different versions in those two locations is a bad design feature in R that just leads to confusion, because you don't necessarily always use the same version each time you load packages. (The rule for which one you use is the first acceptable one found in the .libPaths list, but since you can change .libPaths, and since packages can load other packages, it's hard to predict which one you'll have loaded at any given time.)
To fix this, you can delete both copies (if you have two) and start over, but that's risky because other packages might be depending on gtools. If you are the only user on your computer, you could instead delete the entire "C:/Users/foo/Documents/R/win-library/4.1" library, and then do all your installs using "Run as admin", but that's also easy to mess up.
(On a Mac, that's effectively what happens, because most single user systems put the user in the "admin" group, so they can always install packages to the system location. It causes a lot less confusion, but some "purists" think the Windows way is better.)
So I don't have any good advice for you, but maybe this explains the situation, and you can work out for yourself the best way forward.
My problem is that I can't use R-studio at my work place as the IT does not support it . I want to use R and R-studio that installed on my personnel laptop on my company laptop ( using a modern browser which is behind firewall ) . Some of the options I am thinking of two two things
should I need to build a docker for R and R-studio (I see base images are already available) , I am mostly interested in basic R , Dplyr (haven ,xporter, and Reticulate ) packages .
Should I have to use a binder . I am not technical person and my programming skills are very limited can any one suggest me way .
What exactly are the difference between using Docker option vs Binder ?
I know I can use R-Studio online and get my work done but with the new paid account I am running out of project hours and very slow sometimes . Thanks in advance
Here are some examples beyond the modern RStudio MyBinder example:
https://github.com/fomightez/pythonista_skewedf
https://github.com/fomightez/r_phylogenetics_worshop
https://github.com/fomightez/chapter7/tree/master/binder
The modern RStudio MyBinder example has been set as a template on GitHub so you can use
The first one is for a special use of a package not on conda. And I started that one from square one.
The other two were converted from content by others to aid in making them Binder-ready.
You essentially list everything you need from conda in the environment.yml along with the appropriate channels. If you need special stuff not on conda, you need the other configuration files included there.
Getting everything working can take some iterations on adding things, letting the image get built, and testing your libraries are available. Although you seem to think your situation is not overly complex.
The binder launch badges you see are just images where you modify the URL to point the MyBinder federation site at your repository. Look at the URL and you should see the pattern where you put studio at the end of the URL pointing at your repo. The form at MyBinder.org site can help with this; however, most often it is easier to just adapt a working launch badge's code copied from elsewhere. The form isn't set up at this time for making the URLs for launching to RStudio.
Download anything useful your create in a running session. The sessions timeout after 10 minutes, although RStudio usually keeps them active.
Lack of Persistence and limited memory, storage, & power can be drawbacks. The inherent reproducibility and portability are advantages.
MyBinder.org doesn't work with private repos. If you have code you don't want to share, you can upload it to the temporary session, using the repo for specifying the environment. You could host a private binderhub that does allow the use of private git repositories; however, that is probably overkill for your use case and exceed your ability level at this time.
GitHub isn't the only place to host repositories that can be pointed at the MyBinder system. If you go to the MyBinder.org page and click where it says 'GitHub' on the left side of the top line of the form, you can see a list of the sources at which you can host a repository and point the system to build an image and launch a container with that specified image.
Building the image from a source repository takes some minutes the first time. Once the image is built though on the service, launch is typically less than 30 seconds. Each time you make a change on the source repo, a build is necessary. Some changes don't cause the new build to be as long as the initial one as some optimizing is done to only build what is necessary after a change. Keep in mind there are several members of the federation around the workd and if traffic on the internet gets sent to where the built image isn't yet available, it will be built from scratch again first.
The Holepunch project is out there to offer some help for users working in the R ecosystem; however, with the R-Conda system that is now integrated into MyBinder it is pretty much as easy to do it the way I described. Last I knew, the Holepunch route makes a Dockerfile that isn't as easy to troubleshoot as using the current the R-Conda system route. Dockerfiles are essentially a last ditch configuration file that MyBinder can handle. The reason being the other configuration files are much easier and don't require knowing Dockerfile syntax. MyBinder aims to offer the ability to take advantage of Docker offering containers with a specified environment without users needing to know anything about Docker.
There is a Binder Help category for posting to get help at the Jupyter Discourse Forum. Some other examples of posts already there may help you troubleshoot.
Notice of a common pitfall
Most of the the configuration files for making a repository Binder-ready are simply text and can be edited right in the GitHub browser interface, without need to git or even cloning the repo locally.
Last I knew, there are two exceptions to this. The postBuild and start configuration files have settings that allow them to be run as scripts and these get altered in a way they no longer work if you edit them via the GitHub browser interface. (This was my experience when last I tried. Your mileage may vary or things may have changed now.) To edit those, you have to have git available on a system you have and pull one from some other source. Then edit that on your machine that has git working & add it your repo and push it back up from your local computer.
(If this is a problem, you can post in the Jupyter Discourse Forum Binder help category and you and I could coordinate where I fork and edit those files in your repo to your specifications and then make a pull request to update your source of the fork with those changes.)
If you are using Jupyter notebooks extensively then it may make sense to use Binder
But if you simply want to use R and Rstudio, then all you need is docker. A good resource is
https://github.com/rocker-org/rocker
I've just installed julia for usage on a slurm cluster. Running a hello world job works well, so the installation was successful ... until installing a first package which gives some permissions issues. Script with command
Pkg.add("MAT")
or
Pkg.installed()
gives error message
ERROR: LoadError: SystemError (with /home/<my_user_name>/.julia/logs): mkdir: Permission denied
The same error appears if I start up julia command line from the user directory. Such message disappears when starting julia using sudo, but obviously cannot sudo cluster jobs.
I tried installing the pkg with sudo on the user account, then just using it non-sudo, but other error messages arise similar to those documented here.
https://github.com/JuliaLang/julia/issues/12876
On this page it's indicated to chown user MAT.ji , but that does not work. I tried removing and re-add the package but I'm just running in circles with the same error messages. I also got, at one point, error messages like EACCES similar to documented here
https://discourse.julialang.org/t/juliapro-pkg-installation-error-ioerror-unlink-permission-denied-eacces/35912
I'm a novice with permissions issues like this, so I could use some guidance on how to approach this problem. I'm not sure what to try, and in what order.
Permissions issues on clusters can be tricky.
If we are talking about a physical cluster, the simplest generic solution that you can probably get to work without involving your sysadmin is probably to just install your .julia somewhere where just about every process has filesystem permissions. Namely, global networked scratch (wherever that exactly is on your cluster).
This is arguably a good idea anyways, given that global scratch tends to be the fastest or one of the fastest networked filesystems around on most clusters, and every julia process is going to have to read from .julia when you start a job, so if that's on a fast parallel filesystem, so much the better. On the other hand, scratch tends to have a time limit, so you might want to keep a local copy around for when scratch/<yourusername>/.julia inevitably gets deleted.
In order for this to work well, you have tell Julia so that it knows where to look for .julia, and not to just make a new one when it doesn't find it in the default location (~). One relatively simple way to do this is with environment variables. You could do this manually, but I recommend instead putting something like the following in your ~/.bash_profile.
# Some julia-specific environment variables
# To make sure I call the Julia installation in global scratch
export JULIA_DEPOT_PATH="/path/to/your/global/scratch/username/.julia"
export JULIA_PROJECT="/path/to/your/global/scratch/username/.julia/environments/v1.6/Project.toml"
export JULIA_LOAD_PATH="/path/to/your/global/scratch/username/.julia/environments/v1.6/Project.toml:/path/to/your/julia/application/folder/possibly/also/in/scratch/julia-1.6.1/share/julia/stdlib/v1.6" # The second one may be tricky to find if you're using a cluster-provided julia, but you can always just download the latest julia release yourself and plop it in scratch too
export JULIA_BINDIR="/path/to/your/julia/application/folder/possibly/also/in/scratch/julia-1.6.1/bin" # This last line may not be necessary, but doesn't hurt to specify.
where julia versions and the actual path to your global scratch folder are adjusted as appropriate.
Looking to install R on my work machine, don't have the admin rights though and IT is not friendly to work with without business use-case so that's my last resort as it will likely fail.
I've attempted installing locally as this suggests.
https://support.rstudio.com/hc/en-us/community/posts/201152613-How-to-install-R-from-zip-without-admin-rights-
But no dice, it won't let me save the pkg in /usr/local/bin/: "Failed - Insufficient permissions".
Any one know of another way before I throw in the towel?
Cheers.
Asked IT for local rights. They said yes :)
This is sort of related to a previous post of mine. I have the need to use the bigmemory library on my 32bit Windows PC to do some ugly matrix calculations. Unfortunately, it appears that the maintainers have temporarily ceased production of Windows binaries. I have Ubuntu on my home PC. I would really like to take the .tar.gz file and build it into a Windows binary that I can actually run at work. I realize there are more efficient ways, like installing RTools on the Windows device. However, our IT keeps our admin rights on lockdown, so I can never edit my PATH enviro variable. Could anyone provide some general guidance for doing this? Are there any tools I need to install on my Ubuntu PC above and beyond R?
I found similar questions, but nothing that thoroughly answered my questions.
Unless the package source is incompatible with current versions of R, you could use the R project's win-builder site to build a Windows binary. Quoting from the linked site, win-builder is a service:
intended for useRs who do not have Windows available for checking and building Windows binary packages.
As a convenience, Hadley Wickham's devtools package includes a utility function, build_win(), that you can use for this purpose. From ?build_win:
Works by building source package, and then uploading to http://win-builder.r-project.org/>. Once building is complete you'll receive a link to the built package in the email address listed in the maintainer field. It usually takes around 30 minutes.
Windows has four sets of environment variables (system, user, volatile and process sets). The first three sets are stored in the registry but the process set is not so even if they have locked down the registry its typically still possible to set the process environment variables (including the PATH) in a local process, i.e. on a temporary basis, so you might double check your assumptions that you can't modify anything. Its more likely that you can't modify the system variables and registry but can still modify the set in your local process. To check this from the Windows cmd line enter this:
set mytest=123
set mytest
and if the second line shows that mytest has the value 123 then you likely have all the permissions you need.
Furthermore anything you need to set is all handled automatically for you by R.bat in the batchfiles distribution so you don't have to set anything yourself.
Just ensure that Rtools and R are installed into the standard locations (you can tell them to skip the setting of any registry keys during the installation process), ensure R.bat is on your path or in current directory and run:
R.bat CMD INSTALL mypackage.tar.gz
without setting environment variables, registry keys or path.
If that does not work try Rpathset.bat also from the batchfiles which is not automatic like R.bat but on the other hand is extremely flexible since you must modify the SET statments in it to whatever you want.
There is a PDF document that comes with the batchfiles which gives more info.