Occasionally one wants to patch a function in a package, without recompiling the whole package.
For example, in Emacs ESS, the function install.packages() might get stuck if tcltk is not loaded. One might want to patch install.packages() in order to require tcltk before installation and unload it after the package setup.
A temp() patched version of install.packages() might be:
## Get original args without ending NULL
temp=rev(rev(deparse(args(install.packages)))[-1])
temp=paste(paste(temp, collapse="\n"),
## Add code to load tcltk
"{",
" wasloaded= 'package:tcltk' %in% search()",
" require(tcltk)",
## Add orginal body without braces
paste(rev(rev(deparse(body(install.packages))[-1])[-1]), collapse="\n"),
## Unload tcltk if it was not loaded before by user
" if(!wasloaded) detach('package:tcltk', unload=TRUE)",
"}\n",
sep="\n")
## Eval patched function
temp=eval(parse(text=temp))
# temp
Now we want to replace the original install.packages() and perhaps insert the code in Rprofile.
To this end it is worth nothing that:
getAnywhere("install.packages")
# A single object matching 'install.packages' was found
# It was found in the following places
# package:utils
# namespace:utils
# with value
#
# ... install.packages() source follows (quite lengthy)
That is, the function is stored inside the package/namespace of utils. This environment is sealed and therefore install.packages() should be unlocked before being replaced:
## Override original function
unlockBinding("install.packages", as.environment("package:utils"))
assign("install.packages", temp, envir=as.environment("package:utils"))
unlockBinding("install.packages", asNamespace("utils"))
assign("install.packages", temp, envir=asNamespace("utils"))
rm(temp)
Using getAnywhere() again, we get:
getAnywhere("install.packages")
# A single object matching 'install.packages' was found
# It was found in the following places
# package:utils
# namespace:utils
# with value
#
# ... the *new* install.packages() source follows
It seems that the patched function is placed in the right place.
Unfortunately, running it gives:
Error in install.packages(xxxxx) :
could not find function "getDependencies"
getDependencies() is a function inside the same utils package, but not exported; therefore it is not accessible outside its namespace.
Despite the output of getAnywhere("install.packages"), the patched install.packages() is still misplaced.
The problem is that we need to reload the utils library to obtain the desired effect, which also requires unloading other libraries importing it.
detach("package:stats", unload=TRUE)
detach("package:graphics", unload=TRUE)
detach("package:grDevices", unload=TRUE)
detach("package:utils", unload=TRUE)
library(utils)
install.packages() works now.
Of course, we need to reload the other libraries too. Given the dependencies, using
library(stats)
should reload everything. But there is a problem when reloading the graphics library, at least on Windows:
library(graphics)
# Error in FUN(X[[i]], ...) :
# no such symbol C_contour in package path/to/library/graphics/libs/x64/graphics.dll
Which is the correct way of (re)loading the graphics library?
Patching functions in packages is a low-level operation that should be avoided, because it may break internal assumptions of the execution environment and lead to unpredictable behavior/crashes. If there is a problem with tck/ESS (I didn't try to repeat that) perhaps it should be fixed or there may be a workaround. Particularly changing locked bindings is something to avoid.
If you really wanted to run some code at the start/end of say install.packages, you can use trace. It will do some of the low-level operations mentioned in the question, but the good part is you don't have to worry about fixing this whenever some new internals of R change.
trace(install.packages,
tracer=quote(cat("Starting install.packages\n")),
exit=quote(cat("Ending install packages.\n"))
)
Replace tracer and exit accordingly - maybe exit is not needed anyway, maybe you don't need to unload the package. Still, trace is a very useful tool for debugging.
I am not sure if that will solve your problem - if it would work with ESS - but in general you can also wrap install.packages in a function you define say in your workspace:
install.packages <- function(...) {
cat("Entry.\n")
on.exit(cat("Exit.\n"))
utils::install.packages(...)
}
This is the cleanest option indeed.
Related
I have written some functions to facilitate repeated tasks among my R projects. I am trying to use an environment to load them easily but also prevent them from appearing when I use ls() or delete them with rm(list=ls()).
As a dummy example I have an environment loader function in a file that I can just source from my current project and an additional file for each specialized environment I want to have.
currentProject.R
environments/env_loader.R
environments/colors_env.R
env_loader.R
.environmentLoader <- function(env_file, env_name='my_env') {
sys.source(env_file, envir=attach(NULL, name=env_name))
}
path <- dirname(sys.frame(1)$ofile) # this script's path
#
# Automatically load
.environmentLoader(paste(path, 'colors_env.R', sep='/'), env_name='my_colors')
colors_env.R
library(RColorBrewer) # this doesn't work
# Return a list of colors
dummyColors <- function(n) {
require(RColorBrewer) # This doesn't work
return(brewer.pal(n, 'Blues'))
}
CurrentProject.R
source('./environments/env_loader.R')
# Get a list of 5 colors
dummyColors(5)
This works great except when my functions require me to load a library. In my example, I need to load the RColorBrewer library to use the brewer.pal function in colors_env.R, but the way is now I just get an error Error in brewer.pal(n, "Blues") : could not find function "brewer.pal".
I tried just using library(RColorBrewer) or using require inside my dummyColors function or adding stuff like evalq(library("RColorBrewer"), envir=parent.env(environment())) to the colors_env.R file but it doesn't work. Any suggestions?
If you are using similar functions across projects, I would recommend creating an R package. It's essentially what you're doing in many ways, but you don't have reinvent a lot of the loading mechanisms, etc. Hadley Wickham's book R Packages is very good for this topic. It doesn't need to be a completely fully built out, CRAN ready sort of thing. You can just create a personal package with misc. functions you frequently use.
That being said, the solution for your specific question would be to explicitly use the namespace to call the function.
dummyColors <- function(n) {
require(RColorBrewer) # This doesn't work
return(RColorBrewer::brewer.pal(n, 'Blues'))
}
Create a package and then run it. Use kitten to build the boilerplate, copy your file to it, optionally build it if you want a .tar.gz file or omit that step if you don't need it and finally install it. Then test it out. We have assumed colors_env.R, shown in the question, is in current directory.
(Note that require should always be within an if so that if it does not load then the error is caught. If not within an if use library which will guarantee an error message in that case.)
# create package
library(devtools)
library(pkgKitten)
kitten("colors")
file.copy("colors_env.R", "./colors/R")
build("colors") # optional = will create colors_1.0.tar.gz
install("colors")
# test
library(colors)
dummyColors(5)
## Loading required package: RColorBrewer
## [1] "#EFF3FF" "#BDD7E7" "#6BAED6" "#3182BD" "#08519C"
What is the difference between require() and library()?
There's not much of one in everyday work.
However, according to the documentation for both functions (accessed by putting a ? before the function name and hitting enter), require is used inside functions, as it outputs a warning and continues if the package is not found, whereas library will throw an error.
Another benefit of require() is that it returns a logical value by default. TRUE if the packages is loaded, FALSE if it isn't.
> test <- library("abc")
Error in library("abc") : there is no package called 'abc'
> test
Error: object 'test' not found
> test <- require("abc")
Loading required package: abc
Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, :
there is no package called 'abc'
> test
[1] FALSE
So you can use require() in constructions like the one below. Which mainly handy if you want to distribute your code to our R installation were packages might not be installed.
if(require("lme4")){
print("lme4 is loaded correctly")
} else {
print("trying to install lme4")
install.packages("lme4")
if(require(lme4)){
print("lme4 installed and loaded")
} else {
stop("could not install lme4")
}
}
In addition to the good advice already given, I would add this:
It is probably best to avoid using require() unless you actually will be using the value it returns e.g in some error checking loop such as given by thierry.
In most other cases it is better to use library(), because this will give an error message at package loading time if the package is not available. require() will just fail without an error if the package is not there. This is the best time to find out if the package needs to be installed (or perhaps doesn't even exist because it it spelled wrong). Getting error feedback early and at the relevant time will avoid possible headaches with tracking down why later code fails when it attempts to use library routines
You can use require() if you want to install packages if and only if necessary, such as:
if (!require(package, character.only=T, quietly=T)) {
install.packages(package)
library(package, character.only=T)
}
For multiple packages you can use
for (package in c('<package1>', '<package2>')) {
if (!require(package, character.only=T, quietly=T)) {
install.packages(package)
library(package, character.only=T)
}
}
Pro tips:
When used inside the script, you can avoid a dialog screen by specifying the repos parameter of install.packages(), such as
install.packages(package, repos="http://cran.us.r-project.org")
You can wrap require() and library() in suppressPackageStartupMessages() to, well, suppress package startup messages, and also use the parameters require(..., quietly=T, warn.conflicts=F) if needed to keep the installs quiet.
Always use library. Never use require.
tl;dr: require breaks one of the fundamental rules of robust software systems: fail early.
In a nutshell, this is because, when using require, your code might yield different, erroneous results, without signalling an error. This is rare but not hypothetical! Consider this code, which yields different results depending on whether {dplyr} can be loaded:
require(dplyr)
x = data.frame(y = seq(100))
y = 1
filter(x, y == 1)
This can lead to subtly wrong results. Using library instead of require throws an error here, signalling clearly that something is wrong. This is good.
It also makes debugging all other failures more difficult: If you require a package at the start of your script and use its exports in line 500, you’ll get an error message “object ‘foo’ not found” in line 500, rather than an error “there is no package called ‘bla’”.
The only acceptable use case of require is when its return value is immediately checked, as some of the other answers show. This is a fairly common pattern but even in these cases it is better (and recommended, see below) to instead separate the existence check and the loading of the package. That is: use requireNamespace instead of require in these cases.
More technically, require actually calls library internally (if the package wasn’t already attached — require thus performs a redundant check, because library also checks whether the package was already loaded). Here’s a simplified implementation of require to illustrate what it does:
require = function (package) {
already_attached = paste('package:', package) %in% search()
if (already_attached) return(TRUE)
maybe_error = try(library(package, character.only = TRUE))
success = ! inherits(maybe_error, 'try-error')
if (! success) cat("Failed")
success
}
Experienced R developers agree:
Yihui Xie, author of {knitr}, {bookdown} and many other packages says:
Ladies and gentlemen, I've said this before: require() is the wrong way to load an R package; use library() instead
Hadley Wickham, author of more popular R packages than anybody else, says
Use library(x) in data analysis scripts. […]
You never need to use require() (requireNamespace() is almost always better)
?library
and you will see:
library(package) and require(package) both load the package with name
package and put it on the search list. require is designed for use
inside other functions; it returns FALSE and gives a warning (rather
than an error as library() does by default) if the package does not
exist. Both functions check and update the list of currently loaded
packages and do not reload a package which is already loaded. (If you
want to reload such a package, call detach(unload = TRUE) or
unloadNamespace first.) If you want to load a package without putting
it on the search list, use requireNamespace.
My initial theory about the difference was that library loads the packages whether it is already loaded or not, i.e. it might reload an already loaded package, while require just checks that it is loaded, or loads it if it isn't (thus the use in functions that rely on a certain package). The documentation refutes this, however, and explicitly states that neither function will reload an already loaded package.
Here seems to be the difference on an already loaded package.
While it is true that both require and library do not load the package. Library does a lot of other things before it checks and exits.
I would recommend removing "require" from the beginning of a function running 2mil times anyway, but if, for some reason I needed to keep it. require is technically a faster check.
microbenchmark(req = require(microbenchmark), lib = library(microbenchmark),times = 100000)
Unit: microseconds
expr min lq mean median uq max neval
req 3.676 5.181 6.596968 5.655 6.177 9456.006 1e+05
lib 17.192 19.887 27.302907 20.852 22.490 255665.881 1e+05
I have used several packages of R libraries for my study. All libraries charge together at the beginning of my code. And here is the problem. It turns out that I have done several tests with different functions that were already in the packages of R. However, in the final code I have not implemented all the functions I have tried. Therefore, I am loading libraries that I do not use.
Would there be any way to check the libraries to know if they really are necessary for my code?
Start by restarting R with a fresh environment, no libraries loaded. For this demonstration, I'm going to define two functions:
zoo1 <- function() na.locf(1:10)
zoo2 <- function() zoo::na.locf(1:10)
With no libraries loaded, let's try something:
codetools::checkUsage(zoo1)
# <anonymous>: no visible global function definition for 'na.locf'
codetools::checkUsage(zoo2)
library(zoo)
# Attaching package: 'zoo'
# The following objects are masked from 'package:base':
# as.Date, as.Date.numeric
codetools::checkUsage(zoo1)
Okay, so we know we can check a single function to see if it is abusing scope and/or using non-base functions. Let's assume that you've loaded your script full of functions (but not the calls to require or library), so let's do this process for all of them. Let's first unload zoo, so that we'll see a complaint again about our zoo1 function:
detach("package:zoo", unload=TRUE)
Now let's iterate over all functions:
allfuncs <- Filter(function(a) is.function(get(a)), ls())
str(sapply(allfuncs, function(fn) capture.output(codetools::checkUsage(get(fn))), simplify=FALSE))
# List of 2
# $ zoo1: chr "<anonymous>: no visible global function definition for 'na.locf'"
# $ zoo2: chr(0)
Now you know to look in the function named zoo1 for a call to na.locf. It'll be up to you to find in which not-yet-loaded package this function resides, but that might be more more reasonable, depending on the number of packages you are loading.
Some side-thoughts:
If you have a script file that does not have everything comfortably ensconced in functions, then just wrap all of the global R code into a single function, say bigfunctionfortest <- function() { as the first line and } as the last. Then source the file and run codetools::checkUsage(bigfunctionfortest).
Package developers have to go through a process that uses this, so that the Imports: and Depends: sections of NAMESPACE (another ref: http://r-pkgs.had.co.nz/namespace.html) will be correct. One good trick to do that will prevent "namespace pollution" is loading the namespace but not the package ... and though that may sound confusing, it often results in using zoo::na.locf for all non-base functions. This gets old quickly (especially if you are using dplyr and such, where most of your daily functions are non-base), suggesting those oft-used functions should be directly imported instead of just referenced wholly. If you're familiar with python, then:
# R
library(zoo)
na.locf(c(1,2,NA,3))
is analagous to
# fake-python
from zoo import *
na_locf([1,2,None,3])
(if that package/function exists). Then the non-polluting variant looks like:
# R
zoo::na.locf(c(1,2,NA,3))
# fake-python
import zoo
zoo.na_locf([1,2,None,3])
where the function's package (and/or subdir packaging) must be used explicitly. There is no ambiguity. It is explicit. This is by some/many considered "A Good Thing (tm)".
(Language-philes will likely say that library(zoo) and from zoo import * are not exactly the same ... a better way to describe what is happening is that they bring everything from zoo into the search path of functions, potentially causing masking as we saw in a console message earlier; while the :: functionality only loads the namespace but does not add it to the search path. Lots of things going on in the background.)
I am trying to install the r package sowas and unfortunately it is too old to implement in the new versions of r.
According to the author you can use the package using the source() function to gain access to the code but I have not been able to figure out how to do that.
Any help is appreciated.
Here is a link to the package I described as it is not a CRAN package: http://tocsy.pik-potsdam.de/wavelets/
The .zip file is a windows binary and as such it won't be too interesting. What you'll want to look at is the contents of the .tar.gz archive. You can extract those contents and then look at the code in the R subdirectory.
You could also update the package to work with new versions of R so that you can actually build and install the package. To do so you could unpack the .tar.gz as before but now you'll need to add a NAMESPACE file. This is just a plaintext file at the top of the package directory that has a form like:
export(createar)
export(createwgn)
export(criticalvaluesWCO)
export(criticalvaluesWSP)
export(cwt.ts)
export(plot.wt)
export(plotwt)
export(readmatrix)
export(readts)
export(rk)
export(wco)
export(wcs)
export(writematrix)
export(wsp)
Where you have an export statement for any function in the package that you actually want to be able to use. If a function isn't exported then the functions in the package still have access to that function but the user can't use it (as easily). Once you do that you should be able to build and install the package.
I took the liberty of doing some of this already. I haven't actually taken the time to figure out which functions are useful and should be exported and just assumed that if a help page was written for the function that it should be exported and if there wasn't a help page then I didn't export it. I used Rd2roxygen to convert the help pages to roxygen code (because that's how I roll) and had to do a little bit of cleanup after that but it seems to install just fine.
So if you have the devtools package installed you should actually be able to install the version I modified directly by using the following commands
library(devtools)
install_github("SOWAS", "Dasonk")
Personally I would recommend that you go the route of adding the NAMESPACE file and what not directly as then you'll have more control over the code and be more able to fix any problems that might occur when using the package. Or if you use git you could fork my repo and continue fixing things from there. Good luck.
If you want to see the source code of a particular function, then just type the name of the function without the braces and press enter. You will see the code.
For example type var in command prompt to see it's code.
> var
function (x, y = NULL, na.rm = FALSE, use)
{
if (missing(use))
use <- if (na.rm)
"na.or.complete"
else "everything"
na.method <- pmatch(use, c("all.obs", "complete.obs", "pairwise.complete.obs",
"everything", "na.or.complete"))
if (is.na(na.method))
stop("invalid 'use' argument")
if (is.data.frame(x))
x <- as.matrix(x)
else stopifnot(is.atomic(x))
if (is.data.frame(y))
y <- as.matrix(y)
else stopifnot(is.atomic(y))
.Call(C_cov, x, y, na.method, FALSE)
}
<bytecode: 0x0000000008c97980>
<environment: namespace:stats>
What is the difference between require() and library()?
There's not much of one in everyday work.
However, according to the documentation for both functions (accessed by putting a ? before the function name and hitting enter), require is used inside functions, as it outputs a warning and continues if the package is not found, whereas library will throw an error.
Another benefit of require() is that it returns a logical value by default. TRUE if the packages is loaded, FALSE if it isn't.
> test <- library("abc")
Error in library("abc") : there is no package called 'abc'
> test
Error: object 'test' not found
> test <- require("abc")
Loading required package: abc
Warning message:
In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, :
there is no package called 'abc'
> test
[1] FALSE
So you can use require() in constructions like the one below. Which mainly handy if you want to distribute your code to our R installation were packages might not be installed.
if(require("lme4")){
print("lme4 is loaded correctly")
} else {
print("trying to install lme4")
install.packages("lme4")
if(require(lme4)){
print("lme4 installed and loaded")
} else {
stop("could not install lme4")
}
}
In addition to the good advice already given, I would add this:
It is probably best to avoid using require() unless you actually will be using the value it returns e.g in some error checking loop such as given by thierry.
In most other cases it is better to use library(), because this will give an error message at package loading time if the package is not available. require() will just fail without an error if the package is not there. This is the best time to find out if the package needs to be installed (or perhaps doesn't even exist because it it spelled wrong). Getting error feedback early and at the relevant time will avoid possible headaches with tracking down why later code fails when it attempts to use library routines
You can use require() if you want to install packages if and only if necessary, such as:
if (!require(package, character.only=T, quietly=T)) {
install.packages(package)
library(package, character.only=T)
}
For multiple packages you can use
for (package in c('<package1>', '<package2>')) {
if (!require(package, character.only=T, quietly=T)) {
install.packages(package)
library(package, character.only=T)
}
}
Pro tips:
When used inside the script, you can avoid a dialog screen by specifying the repos parameter of install.packages(), such as
install.packages(package, repos="http://cran.us.r-project.org")
You can wrap require() and library() in suppressPackageStartupMessages() to, well, suppress package startup messages, and also use the parameters require(..., quietly=T, warn.conflicts=F) if needed to keep the installs quiet.
Always use library. Never use require.
tl;dr: require breaks one of the fundamental rules of robust software systems: fail early.
In a nutshell, this is because, when using require, your code might yield different, erroneous results, without signalling an error. This is rare but not hypothetical! Consider this code, which yields different results depending on whether {dplyr} can be loaded:
require(dplyr)
x = data.frame(y = seq(100))
y = 1
filter(x, y == 1)
This can lead to subtly wrong results. Using library instead of require throws an error here, signalling clearly that something is wrong. This is good.
It also makes debugging all other failures more difficult: If you require a package at the start of your script and use its exports in line 500, you’ll get an error message “object ‘foo’ not found” in line 500, rather than an error “there is no package called ‘bla’”.
The only acceptable use case of require is when its return value is immediately checked, as some of the other answers show. This is a fairly common pattern but even in these cases it is better (and recommended, see below) to instead separate the existence check and the loading of the package. That is: use requireNamespace instead of require in these cases.
More technically, require actually calls library internally (if the package wasn’t already attached — require thus performs a redundant check, because library also checks whether the package was already loaded). Here’s a simplified implementation of require to illustrate what it does:
require = function (package) {
already_attached = paste('package:', package) %in% search()
if (already_attached) return(TRUE)
maybe_error = try(library(package, character.only = TRUE))
success = ! inherits(maybe_error, 'try-error')
if (! success) cat("Failed")
success
}
Experienced R developers agree:
Yihui Xie, author of {knitr}, {bookdown} and many other packages says:
Ladies and gentlemen, I've said this before: require() is the wrong way to load an R package; use library() instead
Hadley Wickham, author of more popular R packages than anybody else, says
Use library(x) in data analysis scripts. […]
You never need to use require() (requireNamespace() is almost always better)
?library
and you will see:
library(package) and require(package) both load the package with name
package and put it on the search list. require is designed for use
inside other functions; it returns FALSE and gives a warning (rather
than an error as library() does by default) if the package does not
exist. Both functions check and update the list of currently loaded
packages and do not reload a package which is already loaded. (If you
want to reload such a package, call detach(unload = TRUE) or
unloadNamespace first.) If you want to load a package without putting
it on the search list, use requireNamespace.
My initial theory about the difference was that library loads the packages whether it is already loaded or not, i.e. it might reload an already loaded package, while require just checks that it is loaded, or loads it if it isn't (thus the use in functions that rely on a certain package). The documentation refutes this, however, and explicitly states that neither function will reload an already loaded package.
Here seems to be the difference on an already loaded package.
While it is true that both require and library do not load the package. Library does a lot of other things before it checks and exits.
I would recommend removing "require" from the beginning of a function running 2mil times anyway, but if, for some reason I needed to keep it. require is technically a faster check.
microbenchmark(req = require(microbenchmark), lib = library(microbenchmark),times = 100000)
Unit: microseconds
expr min lq mean median uq max neval
req 3.676 5.181 6.596968 5.655 6.177 9456.006 1e+05
lib 17.192 19.887 27.302907 20.852 22.490 255665.881 1e+05