How to use R in HPC SSH frontend? - r

I used to use R in RStudio which make it easer to handle.
As I got a bigger data I got access to a HPC of the university through SSH interface (MobaXterm) which could be done using terminal as well.
My problem is my inability to access to the administration, which prevent me from installing packages. And I'm pretty sure that I need to change one of the libraries directory that I don't have access to it.
.libPaths()
[1] "/home/UserIsMe/R/x86_64-pc-linux-gnu-library/4.2"
[2] "/cm/shared/apps/R/4.2.1/lib64/R/library"
Is there any way to change the second directory to mine (like the first).
NB: I found a trouble even when I'm trying to install packages from linux's terminal.

Related

Non-manual solution to "cannot remove prior installation of package" when re-installing R packages

I recently began receiving warnings that prior installations of R packages cannot be removed when I try to re-install packages:
install.packages("gtools")
#> Warning: cannot remove prior installation of package ‘gtools’
#> Warning: restored ‘gtools’
I found solutions to this issue encouraging me to delete the packages manually from my library folder, which I could find with .libPaths(). However, (a) this seems like a way of addressing symptoms rather than the underlying issue (which remains unclear) and (b) there are two paths for seemingly different versions of R and I'm not sure which to delete from anyway:
.libPaths()
#> [1] "C:/Users/foo/Documents/R/win-library/4.1"
#> [2] "C:/Program Files/R/R-4.1.2/library"
How can I fix the problem so I don't have to manually delete package folders every time I want to re-install a package? If there is no alternative, do I need to delete the subdirectories for the package from one of those folders or both? FWIW, I'm working in RStudio.
The problem is that you have installed packages using different permissions. On Windows, you need elevated permissions to write to Program Files. At some point you (or an admin) probably used "Run as admin" to install gtools there, and now using regular permissions you can't delete that.
You should be able to delete the Users/foo copy, if you are running as user foo, but even that one may have had permissions changed. But I'd guess the issue is that gtools is in the Program files location.
The error message from R doesn't tell you which location it is trying to delete from, which is unfortunate. In fact, allowing installations of different versions in those two locations is a bad design feature in R that just leads to confusion, because you don't necessarily always use the same version each time you load packages. (The rule for which one you use is the first acceptable one found in the .libPaths list, but since you can change .libPaths, and since packages can load other packages, it's hard to predict which one you'll have loaded at any given time.)
To fix this, you can delete both copies (if you have two) and start over, but that's risky because other packages might be depending on gtools. If you are the only user on your computer, you could instead delete the entire "C:/Users/foo/Documents/R/win-library/4.1" library, and then do all your installs using "Run as admin", but that's also easy to mess up.
(On a Mac, that's effectively what happens, because most single user systems put the user in the "admin" group, so they can always install packages to the system location. It causes a lot less confusion, but some "purists" think the Windows way is better.)
So I don't have any good advice for you, but maybe this explains the situation, and you can work out for yourself the best way forward.

How can I create a user library for R on Windows 10?

I want to follow the advice I've read and heard to have both a main library in R_HOME/library and a user library. I'm using W10 on a desktop machine (not important, except that it gives me a name by which to refer to it), and I can't make R use the user library.
I have succeeded in doing that on a W10 laptop: C:/R/R-4.0.2/library contains some 30 recommended packages, and C:/Users/[username]/Documents/R/win-library/4.0 con contains a much larger number of packages in my user library.
As I recall, and as I wrote down when I did an upgrade on a server, all you have to do to create a site-library is to create a directory called C:/R/R-4.0.2/site-library, and R will use that the next time it starts.
To create a user library, create the directory C:/Users/[username]/Documents/R/win-library/4.0.
That seemed to work on my laptop, for I have seemingly a working R library and a user library there.
That seemed to work on the server, too: I have a library and a site-library.
In both cases, .libPaths() shows the same libraries that I see with Dired on the disk.
I tried to do the same thing on the desktop machine, and i can't make it work.
I created a directory C:/Users/[username]/Documents/R/win-library/4.0, restarted R, and ran .libPaths(); the only directory that was listed was C:/R/R-4.0.2/library.
Because I thought the Documents in that path seemed odd, I tried it again using C:/Users/[username]/R/win-library/4.0, still with no success.
https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Managing-libraries seems pertinent, but I'm not sure how to interpret the output of Sys.getenv("R_LIBL_USER). I get "\\[toplevel]\[nextlevel]\Home$\[username]/R/win-library/4.0", which I presume is a long-winded way to get to /Home$/[username]/R/win-library/4.0 (aka C:/Users/[username]/R/win-library/4.0.
Suggestions? I've tried a number of other suggestions from SO, all to no avail.

Don't use R system library

I'm trying to use a linux server with R installed. Apparently the R system library has old versions of non-base packages installed like dplyr and testthat.
Because i don't have permission to edit the system library, i'm unable to update the packages.
My plan is to only use a user library, so I can controll the package versions myself. However i'm unable to remove the "/usr/lib64/R/library" folder from .libPaths(). I tried changing the environment variables R_LIBS_SITE and R_LIBS with the .Renviron and .Rprofile files to a different folder, but the /usr/lib64/R/library folder will always be present. Removing it with the command .libPaths(.libPaths()[1:2]) doesn't work either.
Is there a way to remove the system library from .libPaths(), so I'm not depending on the update policy of the server admin?
You can't remove the system library, because that's where the base packages live. They can't be installed anywhere else, and R won't work without them.
Best would be for you to get your sysadmin to update the system library. Those obsolete packages probably contain bugs.
If you can't do that, then run update.packages(instlib = "local") to install all the latest versions in the library named "local". (Substitute your own local lib name, of course.) This requires all your users to specify .libPaths("local") when they start, and some will likely forget, so it's not as good.
It might be easiest for you to just install a full copy of R in your own account. Then you'll have control of things, and anyone using your copy will get your library.
(There's a new release (3.5.3) coming in ten days; you might wait for that, or install one of the betas or RCs, which should be available now, then update again when the final release arrives.)
For me, it works to use
.libPaths(.libPaths()[2:1])
This will still search the system library, but only after it searches my personal library, so if I have a newer version, it uses that. Note: I used .libPaths()[2:1] not .libPaths()[1:2]

how to clone an RStudio environment on a different machine

i've been getting up to speed using R of late, and am wondering what the most efficient way is to clone an RStudio environment, especially the package installations, from one machine to another. i'd like to be able to switch from my desktop machine to my laptop, but i am adding packages very frequently to the desktop as i work and would like a simple way to make sure the same packages get installed on the laptop.
any help much appreciated
ps. not everything i'm installing is from CRAN...some are packages taken from github
If you have more than a couple of machine to maintain with the same R configuration, I think you should consider setting up your own local R repository.
And I will just redirect you to another SO question here:
Creating a local R package repository
You can also find the most useful information in the R manual.
Once this is done, you just have to update the local R repository and the packages will be updated on all machines, Windows or Unix
You can just copy and paste the folders in the R libraries between machines. As long as it is the same operating system on both machines there should not be any problem. If you want it to be automatically synchronised then place the R libraries into something like dropbox so that adding or updating a package will automatically appear on either machine with the next sync.

Create R Windows Binary from .tar.gz linux

This is sort of related to a previous post of mine. I have the need to use the bigmemory library on my 32bit Windows PC to do some ugly matrix calculations. Unfortunately, it appears that the maintainers have temporarily ceased production of Windows binaries. I have Ubuntu on my home PC. I would really like to take the .tar.gz file and build it into a Windows binary that I can actually run at work. I realize there are more efficient ways, like installing RTools on the Windows device. However, our IT keeps our admin rights on lockdown, so I can never edit my PATH enviro variable. Could anyone provide some general guidance for doing this? Are there any tools I need to install on my Ubuntu PC above and beyond R?
I found similar questions, but nothing that thoroughly answered my questions.
Unless the package source is incompatible with current versions of R, you could use the R project's win-builder site to build a Windows binary. Quoting from the linked site, win-builder is a service:
intended for useRs who do not have Windows available for checking and building Windows binary packages.
As a convenience, Hadley Wickham's devtools package includes a utility function, build_win(), that you can use for this purpose. From ?build_win:
Works by building source package, and then uploading to http://win-builder.r-project.org/>. Once building is complete you'll receive a link to the built package in the email address listed in the maintainer field. It usually takes around 30 minutes.
Windows has four sets of environment variables (system, user, volatile and process sets). The first three sets are stored in the registry but the process set is not so even if they have locked down the registry its typically still possible to set the process environment variables (including the PATH) in a local process, i.e. on a temporary basis, so you might double check your assumptions that you can't modify anything. Its more likely that you can't modify the system variables and registry but can still modify the set in your local process. To check this from the Windows cmd line enter this:
set mytest=123
set mytest
and if the second line shows that mytest has the value 123 then you likely have all the permissions you need.
Furthermore anything you need to set is all handled automatically for you by R.bat in the batchfiles distribution so you don't have to set anything yourself.
Just ensure that Rtools and R are installed into the standard locations (you can tell them to skip the setting of any registry keys during the installation process), ensure R.bat is on your path or in current directory and run:
R.bat CMD INSTALL mypackage.tar.gz
without setting environment variables, registry keys or path.
If that does not work try Rpathset.bat also from the batchfiles which is not automatic like R.bat but on the other hand is extremely flexible since you must modify the SET statments in it to whatever you want.
There is a PDF document that comes with the batchfiles which gives more info.

Resources