I'm working on an R package at present and trying to follow the best practice guidelines provided by Hadley Wickham at http://r-pkgs.had.co.nz. As part of this, I'm aiming to have all of the package dependencies within the Imports section of the DESCRIPTION file rather than the Depends since I agree with the philosophy of not unnecessarily altering the global environment (something that many CRAN and Bioconductor packages don't seem to follow).
I want to use functions within the Bioconductor package rhdf5 within one of my package functions, in particular h5write(). The issue I've now run into is that it doesn't have its S3 methods declared as such in its NAMESPACE. They are declared using (e.g.)
export(h5write.default)
export(h5writeDataset.matrix)
rather than
S3method(h5write, default)
S3method(h5writeDataset, matrix)
The generic h5write is defined as:
h5write <- function(obj, file, name, ...) {
res <- UseMethod("h5write")
invisible(res)
}
In practice, this means that calls to rhdf5::h5write fail because there is no appropriate h5write method registered.
As far as I can see, there are three solutions to this:
Use Depends rather than Imports in the DESCRIPTION file.
Use library("rhdf5") or require("rhdf5") in the code for the relevant function.
Amend the NAMESPACE file for rhdf5 to use S3methods() rather than export().
All of these have disadvantages. Option 1 means that the package is loaded and attached to the global environment even if the relevant function in my package is never called. Option 2 means use of library in a package, which while again attaches the package to the global environment, and is also deprecated per Hadley Wickham's guidelines. Option 3 would mean relying on the other package author to update their package on Bioconductor and also means that the S3 methods are no longer exported which could in turn break other packages which rely on calling them explicitly.
Have I missed another alternative? I've looked elsewhere on StackOverflow and found the following somewhat relevant questions Importing S3 method from another package and
How to export S3 method so it is available in namespace? but nothing that directly addresses my issue. Of note, the key difference from the first of these two is that the generic and the method are both in the same package, but the issue is the use of export rather than S3method.
Sample code to reproduce the error (without needing to create a package):
loadNamespace("rhdf5")
rdhf5::h5write(1:4, "test.h5", "test")
Error in UseMethod("h5write") :
no applicable method for 'h5write' applied to an object of class
"c('integer', 'numeric')
Alternatively, there is a skeleton package at https://github.com/NikNakk/s3issuedemo which provides a single function demonstrateIssue() which reproduces the error message. It can be installed using devtools::install_github("NikNakk/s3issuedemo").
The key here is to import the specific methods in addition to the generic you want to use. Here is how you can get it to work for the default method.
Note: this assumes that the test.h5 file already exists.
#' #importFrom rhdf5 h5write.default
#' #importFrom rhdf5 h5write
#' #export
myFun <- function(){
h5write(1:4, "test.h5", "test")
}
I also have put up my own small package demonstrating this here.
Related
Currently I have in my package DESCRIPTION, a dependency on dbplyr:
Imports:
dbplyr,
dplyr
dbplyr is useful almost solely because of the S3 methods it defines: https://github.com/tidyverse/dbplyr/blob/main/NAMESPACE. The actual functions you call to use dbplyr are almost entirely from dplyr.
By putting dbplyr in my Imports, it should automatically get loaded, but not attached, which should be enough to register its S3 methods: https://r-pkgs.org/dependencies-mindset-background.html#sec-dependencies-attach-vs-load.
This seems to work fine, but whenever I R CMD check, it tells me:
N checking dependencies in R code (10.8s)
Namespace in Imports field not imported from: ‘dbplyr’
All declared Imports should be used.
Firstly, why does R CMD check even check this, considering that it often makes sense to load packages without importing them. Secondly, how am I supposed to satisfy R CMD check without loading things into my namespace that I don't want or need?
I am pretty sure two of your assumptions are false.
First, putting Imports: dbplyr into your DESCRIPTION file won't load it, so its methods won't be loaded from that alone. Basically the Imports field in the DESCRIPTION file just guarantees that dbplyr is available to be loaded when requested. If you import something via the NAMESPACE file, that will cause it to be loaded. If you evaluate dbplyr::something that will cause it to be loaded. Executing loadNamespace("dbplyr") is another way, and there are a few others. You may also load some other package that loads it.
Second, I think you have misinterpreted the error message. It isn't saying that you loaded it without importing it (though it would complain about that too), it is saying that it can't detect any use of it in your package, so maybe it shouldn't be a requirement for installing your package.
Unfortunately, the code to detect uses is fallible, so it sometimes misses uses. Examples I've heard about are:
if the package is only used in the default value for a function argument. This has been fixed in R-devel.
if the package is only used during the build to construct some object, e.g. code like someclass <- R6::R6Class( ... ) needs R6, but the check code won't see it because it looks at someclass, not at the source code that created it.
if the use of the package is hidden by specifying the name of the package in a character variable.
if the need for the package is indirect, e.g. you need to use ggplot2::geom_hex. That needs the hexbin package, but ggplot2 only declares it as "Suggested".
These examples come from this discussion: https://github.com/hadley/r-pkgs/issues/828#issuecomment-1421353457 .
The recommended workaround there is to create an object that refers to the imported package explicitly, e.g. putting the line
dummy_r6 <- function() R6::R6Class
into your package is enough to suppress the note without actually loading R6. (It will be loaded if you ever call this function.)
However, your requirement is stronger: you do need to make sure dbplyr is loaded if you want its methods to be used. I'd put something in your .onLoad() function that triggers the load. For example,
.onLoad <- function(lib, pkg) {
# Make sure the dbplyr methods are loaded
loadNamespace("dbplyr")
}
EDITED TO ADD: As pointed out in the comments, there's a bug in the check code that means it won't detect this as being a use of dbplyr. You really need to do both things, e.g.
.onLoad <- function(lib, pkg) {
# Make sure the dbplyr methods are loaded
loadNamespace("dbplyr")
# Work around bug in code checking in R 4.2.2 for use of packages
dummy <- function() dbplyr::across_apply_fns
}
The function used in the dummy construction is arbitrary; it probably doesn't even need to exist, but I chose one that does.
Writing an R-package I use name spaces to use functions from existing packages, e.g. raster::writeRaster(...).
However, I am wondering if functions from the base package have also be used like this, e.g. base::sum(...). This might end up in very confusing code parts:
foo[base::which(base::sapply(bar, function())]
No you don't need to reference base packages like this. You only need to reference non-base packages to ensure they are loaded into the function environment when functions from your package are run, either by using :: or #import in the Roxegen notes at the top of your script. See why you don't need to reference base packages below:
http://adv-r.had.co.nz/Environments.html
"Package namespaces keep packages independent. For example, if package A uses the base mean() function, what happens if package B creates its own mean() function? Namespaces ensure that package A continues to use the base mean() function, and that package A is not affected by package B (unless explicitly asked for)."(Hadley Wickham)
The only time you need to reference base:: is if the namespace for your package contains a package that has an alternative function of the same name.
I'm creating an R package that will use a single function from plyr. According to this roxygen2 vignette:
If you are using just a few functions from another package, the
recommended option is to note the package name in the Imports: field
of the DESCRIPTION file and call the function(s) explicitly using ::,
e.g., pkg::fun().
That sounds good. I'm using plyr::ldply() - the full call with :: - so I list plyr in Imports: in my DESCRIPTION file. However, when I use devtools::check() I get this:
* checking dependencies in R code ... NOTE
All declared Imports should be used:
‘plyr’
All declared Imports should be used.
Why do I get this note?
I am able to avoid the note by adding #importFrom dplyr ldply in the file that is using plyr, but then I end but having ldply in my package namespace. Which I do not want, and should not need as I am using plyr::ldply() the single time I use the function.
Any pointers would be appreciated!
(This question might be relevant.)
If ldply() is important for your package's functionality, then you do want it in your package namespace. That is the point of namespace imports. Functions that you need, should be in the package namespace because this is where R will look first for the definition of functions, before then traversing the base namespace and the attached packages. It means that no matter what other packages are loaded or unloaded, attached or unattached, your package will always have access to that function. In such cases, use:
#importFrom plyr ldply
And you can just refer to ldply() without the plyr:: prefix just as if it were another function in your package.
If ldply() is not so important - perhaps it is called only once in a not commonly used function - then, Writing R Extensions 1.5.1 gives the following advice:
If a package only needs a few objects from another package it can use a fully qualified variable reference in the code instead of a formal import. A fully qualified reference to the function f in package foo is of the form foo::f. This is slightly less efficient than a formal import and also loses the advantage of recording all dependencies in the NAMESPACE file (but they still need to be recorded in the DESCRIPTION file). Evaluating foo::f will cause package foo to be loaded, but not attached, if it was not loaded already—this can be an advantage in delaying the loading of a rarely used package.
(I think this advice is actually a little outdated because it is implying more separation between DESCRIPTION and NAMESPACE than currently exists.) It implies you should use #import plyr and refer to the function as plyr::ldply(). But in reality, it's actually suggesting something like putting plyr in the Suggests field of DESCRIPTION, which isn't exactly accommodated by roxygen2 markup nor exactly compliant with R CMD check.
In sum, the official line is that Hadley's advice (which you are quoting) is only preferred for rarely used functions from rarely used packages (and/or packages that take a considerable amount of time to load). Otherwise, just do #importFrom like WRE advises:
Using importFrom selectively rather than import is good practice and recommended notably when importing from packages with more than a dozen exports.
I am currently developing a plug-in for the R-Commander GUI. In this package I am using a great deal of other packages which I simply attached by using the Depends option in the description file.
I am however now switching them over to the Imports option and am experiencing some problems with it.
Because I want to use some functions not only internally in my own code, but also be able to print and use them in the script window of R Commander, I will also have to export them in the namespace.
Let's take for example the biclust package. This package has the following exports in its namespace:
# First a bunch of functions are exported (Note that the biclust function is not in here!)
export(drawHeatmap,drawHeatmap2,bubbleplot,...,heatmapBC)
# The classes are exported
exportClasses(BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
# Methods are exported
exportMethods(biclust,show,summary)
So when I library(biclust) in an R session, it works as intended, meaning I can use the biclust method/function in the R console.
Now this how my namespace file looks like (or at least the part of it relevant to this discussion)
# I select those functions I need and import them.
importFrom(biclust, drawHeatmap,...,biclustbarchart)
# I import all the classes
importClassesFrom(biclust,BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
# I import all the methods
importMethodsFrom(biclust,show,summary,biclust)
# I now export all of the previous again so I can use the doItAndPrint functionality in R Commander
export( drawHeatmap,...,biclustbarchart)
exportClasses(BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
exportMethods(biclust,show,summary)
However when I load in my own package now, it is not working as intended. While functions such as drawHeatmap are working, the biclust method/function can not be found.(Though I have clearly imported and exported the method.)
Seemingly the only way to get this working, is to put the biclust method also in the normal export() command.
export(biclust,drawHeatmap,...,biclustbarchart)
Could someone clarify what I am doing wrong or what is going on here? Why are the same exports working for the biclust package, but not for my own package?
The only description of your error is that "it is not working as intended", so the following is a little stab in the dark.
It's useful to distinguish between methods and the generics that they are associated with. Biclust makes available both, and they are tightly associated. importFrom(biclust, biclust) imports the generic and associated methods, importMethodsFrom(biclust, biclust) imports the biclust methods defined in the biclust package, and implicitly the generic(s) on which the methods are defined. These are functionally equivalent so far; I think the original intention of importMethodsFrom() was when pkgA defines a generic, pkgB defines methods on the generic, and pkgD wants to use the generic from pkgA and the methods on that generic defined in pkgA and pkgB -- import(pkgA, foo), importMethodsFrom(pkgB, foo).
On the other end, when you say exportMethods(foo), it instructs R to make foo methods defined in your package available for others to use. But there are no foo methods defined in your package, so nothing is exported (maybe this should generate an error, or the methods that you import should be exported again). On the other hand, export(foo) tells R to export the foo generic, which is available for export -- it's the symbol that you'd imported earlier. (You mention that you "put the biclust method also in the normal export()", but actually it is the generic (and any methods associated with it) available for export.) So exporting biclust, rather than methods defined on it, seems to be what you want to do.
Normally, I would say that importing and then re-exporting functions or generics defined in other packages is not the right thing to do -- biclust, not your package, provides and documents the generic, and biclust would probably belong in Depends: -- presumably, many other functions from biclust are typically used in conjunction with the generic. Perhaps your Rcommander GUI is an exception.
Even though Imports: implies additional work (in the NAMESPACE file), it is usually the case that packages belong as Imports: rather than Depends: -- it makes the code in your package much more robust (imported functions are found in the package name space, rather than on the search path that the user can easily modify) and reduces the likelihood that the user experiences name clashes between identical symbols defined in different packages.
Suppose I'm developing a package, called foo, which would like to use the description function from the memisc package. I don't want to import the whole memisc namespace because :
It is bad
memisc overrides the base aggregate.formula function, which breaks several things. For example, example(aggregate) would fail miserably.
The package includes the following files :
DESCRIPTION
Package: foo
Version: 0.0
Title: Foo
Imports:
memisc
Collate:
'foo.R'
NAMESPACE
export(bar)
importFrom(memisc,description)
R/foo.R
##' bar function
##'
##' #param x something
##' #return nothing
##' #importFrom memisc description
##' #export
`bar` <- function(x) {
description(x)
}
I'd think that using importFrom would not load the entire memisc namespace, but only namespace::description, but this is not the case. Starting with a vanilla R :
R> getS3method("aggregate","formula")
## ... function code ...
## <environment: namespace:stats>
R> library(foo)
R> getS3method("aggregate","formula")
## ... function code ...
## <environment: namespace:memisc>
R> example(aggregate)
## Fails
So, do you know how I can import the description function from memisc without getting aggregate.formula in my environment ?
You can't.
If you declare memisc in the Imports: field, the namespace will be loaded when the package is loaded and the exported objects will be findable by your package. (If you specify it in Depends:, the namespace will be loaded and attached to the search path which makes the exported objects findable by any code.)
Part of loading a namespace is registering methods with the generic. (I looked but couldn't find a canonical documentation that says this; I will appeal to the fact that functions are declared as S3 methods in the NAMESPACE file as evidence.) The defined methods are kept with the generic and have the visibility of the generic function (or, perhaps, the generic function's namespace).
Typically, a package will define a method either for a generic it creates or for a class it defines. The S3 object system does not have a mechanism for formally defining an S3 class (or what package created the class), but the general idea is that if the package defines functions which return an object with that class attribute (and is the only package that does), that class is that package's class. If either of these two conditions hold, there will not be a problem. If the generic is defined in the package, it can only be found if the package is attached; if the class is defined in the package, objects of that class would only exist (and therefore be dispatched on) if the package is attached and used.
In the memisc example, neither holds. The aggregate generic is defined in the stats package and the formula object is also defined in the stats package (based on that package defining as.formula, [.formula, etc.) Since it is neither memisc's generic nor memisc's object, the effects can be seen even (and the method dispatched to) if memisc is simply loaded but not attached.
For another example of this problem, but with reorder.factor, see Reordering factor gives different results, depending on which packages are loaded.
In general, it is not good practice to add methods to generics for which the package does not control either the object or the generic; doubly so if it overrides a method in a core package; and egregiously so if it is not a backwards compatible function to the existing function in the core packages.
For your example, you may be better off copying the code for memisc::describe into your package, although that approach has its own problems and caveats.
With the caveat that I'm not too familiar with the R environment and namespaces, nor whether this would work in a package — a workaround I've used in programming is to use :: to copy the function into my own function.
It may have unknown consequences of loading the whole package, as discussed in the comments to OP's question, but it seems to not attach the package's function names to R namespace and mask existing function names.
Example:
my_memisc_description <- memisc::description