I'm writing a package that imports (and uses) both SparkR::sql and dbplyr::sql. The other relevant questions involve unintentional collisions brought on my haphazard importing.
In my NAMESPACE I have:
importFrom(dbplyr, sql)
importFrom(SparkR, sql)
Both of these functions are used in the script and, being aware of the conflict, I'm sure to always prefix the package name:
dbplyr::sql(...)
SparkR::sql(...)
Nevertheless, I get an import replacement warning when building/checking the package:
Warning: replacing previous import ‘dbplyr::sql’ by ‘SparkR::sql’ when loading ‘my_pkg’
What I see in Writing R Extensions seems to be the following:
If a package only needs a few objects from another package it can use a fully qualified variable reference [foo::f] in the code instead of a formal import... This is slightly less efficient than a formal import and also loses the advantage of recording all dependencies in the NAMESPACE file (but they still need to be recorded in the DESCRIPTION file). Evaluating foo::f will cause package foo to be loaded, but not attached, if it was not loaded already—this can be an advantage in delaying the loading of a rarely used package.
Am I right that the takeaway of this / best practice is to:
Pick which function is used most often and add that to importFrom
Remove the "less-frequent" function from importFrom but keep that package in DESCRIPTION
Just use :: (perhaps preceded by require()) for the "less-frequent" function
I have always followed this advice:
It’s common for packages to be listed in Imports in DESCRIPTION, but not in NAMESPACE. In fact, this is what I recommend: list the package in DESCRIPTION so that it’s installed, then always refer to it explicitly with pkg::fun().
In your case:
Remove both importFrom
Keep both packages in Imports:
Use dbplyr::sql and SparkR::sql
My main motivation here is consistency: Even without any name clashes I want to always use the full name to make it clear when reading the code where some function comes from. If I do not use importForm and forget about using the full name at one place, R CMD ckeck will catch that. I value this code clarity higher than the (perceived) advantage of collecting dependencies in two places: DESCRIPTION and (more explicitly) NAMESPACE.
Related
Currently I have in my package DESCRIPTION, a dependency on dbplyr:
Imports:
dbplyr,
dplyr
dbplyr is useful almost solely because of the S3 methods it defines: https://github.com/tidyverse/dbplyr/blob/main/NAMESPACE. The actual functions you call to use dbplyr are almost entirely from dplyr.
By putting dbplyr in my Imports, it should automatically get loaded, but not attached, which should be enough to register its S3 methods: https://r-pkgs.org/dependencies-mindset-background.html#sec-dependencies-attach-vs-load.
This seems to work fine, but whenever I R CMD check, it tells me:
N checking dependencies in R code (10.8s)
Namespace in Imports field not imported from: ‘dbplyr’
All declared Imports should be used.
Firstly, why does R CMD check even check this, considering that it often makes sense to load packages without importing them. Secondly, how am I supposed to satisfy R CMD check without loading things into my namespace that I don't want or need?
I am pretty sure two of your assumptions are false.
First, putting Imports: dbplyr into your DESCRIPTION file won't load it, so its methods won't be loaded from that alone. Basically the Imports field in the DESCRIPTION file just guarantees that dbplyr is available to be loaded when requested. If you import something via the NAMESPACE file, that will cause it to be loaded. If you evaluate dbplyr::something that will cause it to be loaded. Executing loadNamespace("dbplyr") is another way, and there are a few others. You may also load some other package that loads it.
Second, I think you have misinterpreted the error message. It isn't saying that you loaded it without importing it (though it would complain about that too), it is saying that it can't detect any use of it in your package, so maybe it shouldn't be a requirement for installing your package.
Unfortunately, the code to detect uses is fallible, so it sometimes misses uses. Examples I've heard about are:
if the package is only used in the default value for a function argument. This has been fixed in R-devel.
if the package is only used during the build to construct some object, e.g. code like someclass <- R6::R6Class( ... ) needs R6, but the check code won't see it because it looks at someclass, not at the source code that created it.
if the use of the package is hidden by specifying the name of the package in a character variable.
if the need for the package is indirect, e.g. you need to use ggplot2::geom_hex. That needs the hexbin package, but ggplot2 only declares it as "Suggested".
These examples come from this discussion: https://github.com/hadley/r-pkgs/issues/828#issuecomment-1421353457 .
The recommended workaround there is to create an object that refers to the imported package explicitly, e.g. putting the line
dummy_r6 <- function() R6::R6Class
into your package is enough to suppress the note without actually loading R6. (It will be loaded if you ever call this function.)
However, your requirement is stronger: you do need to make sure dbplyr is loaded if you want its methods to be used. I'd put something in your .onLoad() function that triggers the load. For example,
.onLoad <- function(lib, pkg) {
# Make sure the dbplyr methods are loaded
loadNamespace("dbplyr")
}
EDITED TO ADD: As pointed out in the comments, there's a bug in the check code that means it won't detect this as being a use of dbplyr. You really need to do both things, e.g.
.onLoad <- function(lib, pkg) {
# Make sure the dbplyr methods are loaded
loadNamespace("dbplyr")
# Work around bug in code checking in R 4.2.2 for use of packages
dummy <- function() dbplyr::across_apply_fns
}
The function used in the dummy construction is arbitrary; it probably doesn't even need to exist, but I chose one that does.
I'm creating an R package that will use a single function from plyr. According to this roxygen2 vignette:
If you are using just a few functions from another package, the
recommended option is to note the package name in the Imports: field
of the DESCRIPTION file and call the function(s) explicitly using ::,
e.g., pkg::fun().
That sounds good. I'm using plyr::ldply() - the full call with :: - so I list plyr in Imports: in my DESCRIPTION file. However, when I use devtools::check() I get this:
* checking dependencies in R code ... NOTE
All declared Imports should be used:
‘plyr’
All declared Imports should be used.
Why do I get this note?
I am able to avoid the note by adding #importFrom dplyr ldply in the file that is using plyr, but then I end but having ldply in my package namespace. Which I do not want, and should not need as I am using plyr::ldply() the single time I use the function.
Any pointers would be appreciated!
(This question might be relevant.)
If ldply() is important for your package's functionality, then you do want it in your package namespace. That is the point of namespace imports. Functions that you need, should be in the package namespace because this is where R will look first for the definition of functions, before then traversing the base namespace and the attached packages. It means that no matter what other packages are loaded or unloaded, attached or unattached, your package will always have access to that function. In such cases, use:
#importFrom plyr ldply
And you can just refer to ldply() without the plyr:: prefix just as if it were another function in your package.
If ldply() is not so important - perhaps it is called only once in a not commonly used function - then, Writing R Extensions 1.5.1 gives the following advice:
If a package only needs a few objects from another package it can use a fully qualified variable reference in the code instead of a formal import. A fully qualified reference to the function f in package foo is of the form foo::f. This is slightly less efficient than a formal import and also loses the advantage of recording all dependencies in the NAMESPACE file (but they still need to be recorded in the DESCRIPTION file). Evaluating foo::f will cause package foo to be loaded, but not attached, if it was not loaded already—this can be an advantage in delaying the loading of a rarely used package.
(I think this advice is actually a little outdated because it is implying more separation between DESCRIPTION and NAMESPACE than currently exists.) It implies you should use #import plyr and refer to the function as plyr::ldply(). But in reality, it's actually suggesting something like putting plyr in the Suggests field of DESCRIPTION, which isn't exactly accommodated by roxygen2 markup nor exactly compliant with R CMD check.
In sum, the official line is that Hadley's advice (which you are quoting) is only preferred for rarely used functions from rarely used packages (and/or packages that take a considerable amount of time to load). Otherwise, just do #importFrom like WRE advises:
Using importFrom selectively rather than import is good practice and recommended notably when importing from packages with more than a dozen exports.
Title should be pretty clear I hope. I'm writing a package called forecasting, with imports for dplyr among other packages. With the imports written in to the DESCRIPTION file, I am able to force these other packages to be installed along with forecasting - is there an equivalent way to do this for the loading of the package? In other words, is there a way that when I load my package with library(forecasting), it automatically also loads dplyr and the other packages?
Thanks
Yes.
Re-read "Writing R Extensions". The Depends: forces both the initial installation as well as the loading of the depended-upon packages.
But these days you want Imports: along with importFrom() in the NAMESPACE file which is more fine-grained.
But first things first: get it working with Depends.
Edit:
Opps you're correct, the documentation I referenced is not a primary source. Perhaps this is better:
From the R documentation:
The ‘Depends’ field gives a comma-separated list of package names which this package depends on. Those packages will be attached before the current package when library or require is called.
and
The ‘Imports’ field lists packages whose namespaces are imported from (as specified in the NAMESPACE file) but which do not need to be attached. Namespaces accessed by the ‘::’ and ‘:::’ operators must be listed here, or in ‘Suggests’ or ‘Enhances’
Original:
From the R packages documentation:
Adding a package dependency here [the DESCRIPTION file] ensures that it’ll be installed. However, it does not mean that it will be attached along with your package (i.e., library(x)). The best practice is to explicitly refer to external functions using the syntax package::function(). This makes it very easy to identify which functions live outside of your package. This is especially useful when you read your code in the future.
I am currently developing a plug-in for the R-Commander GUI. In this package I am using a great deal of other packages which I simply attached by using the Depends option in the description file.
I am however now switching them over to the Imports option and am experiencing some problems with it.
Because I want to use some functions not only internally in my own code, but also be able to print and use them in the script window of R Commander, I will also have to export them in the namespace.
Let's take for example the biclust package. This package has the following exports in its namespace:
# First a bunch of functions are exported (Note that the biclust function is not in here!)
export(drawHeatmap,drawHeatmap2,bubbleplot,...,heatmapBC)
# The classes are exported
exportClasses(BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
# Methods are exported
exportMethods(biclust,show,summary)
So when I library(biclust) in an R session, it works as intended, meaning I can use the biclust method/function in the R console.
Now this how my namespace file looks like (or at least the part of it relevant to this discussion)
# I select those functions I need and import them.
importFrom(biclust, drawHeatmap,...,biclustbarchart)
# I import all the classes
importClassesFrom(biclust,BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
# I import all the methods
importMethodsFrom(biclust,show,summary,biclust)
# I now export all of the previous again so I can use the doItAndPrint functionality in R Commander
export( drawHeatmap,...,biclustbarchart)
exportClasses(BiclustMethod,Biclust,BCBimax,BCCC,BCXmotifs,BCSpectral,BCPlaid)
exportMethods(biclust,show,summary)
However when I load in my own package now, it is not working as intended. While functions such as drawHeatmap are working, the biclust method/function can not be found.(Though I have clearly imported and exported the method.)
Seemingly the only way to get this working, is to put the biclust method also in the normal export() command.
export(biclust,drawHeatmap,...,biclustbarchart)
Could someone clarify what I am doing wrong or what is going on here? Why are the same exports working for the biclust package, but not for my own package?
The only description of your error is that "it is not working as intended", so the following is a little stab in the dark.
It's useful to distinguish between methods and the generics that they are associated with. Biclust makes available both, and they are tightly associated. importFrom(biclust, biclust) imports the generic and associated methods, importMethodsFrom(biclust, biclust) imports the biclust methods defined in the biclust package, and implicitly the generic(s) on which the methods are defined. These are functionally equivalent so far; I think the original intention of importMethodsFrom() was when pkgA defines a generic, pkgB defines methods on the generic, and pkgD wants to use the generic from pkgA and the methods on that generic defined in pkgA and pkgB -- import(pkgA, foo), importMethodsFrom(pkgB, foo).
On the other end, when you say exportMethods(foo), it instructs R to make foo methods defined in your package available for others to use. But there are no foo methods defined in your package, so nothing is exported (maybe this should generate an error, or the methods that you import should be exported again). On the other hand, export(foo) tells R to export the foo generic, which is available for export -- it's the symbol that you'd imported earlier. (You mention that you "put the biclust method also in the normal export()", but actually it is the generic (and any methods associated with it) available for export.) So exporting biclust, rather than methods defined on it, seems to be what you want to do.
Normally, I would say that importing and then re-exporting functions or generics defined in other packages is not the right thing to do -- biclust, not your package, provides and documents the generic, and biclust would probably belong in Depends: -- presumably, many other functions from biclust are typically used in conjunction with the generic. Perhaps your Rcommander GUI is an exception.
Even though Imports: implies additional work (in the NAMESPACE file), it is usually the case that packages belong as Imports: rather than Depends: -- it makes the code in your package much more robust (imported functions are found in the package name space, rather than on the search path that the user can easily modify) and reduces the likelihood that the user experiences name clashes between identical symbols defined in different packages.
The "Writing R Extensions" manual provides the following guidance on when to use Imports or Depends:
The general rules are
Packages whose namespace only is needed to load the package using library(pkgname) must be listed in the ‘Imports’ field and not in the
‘Depends’ field.
Packages that need to be attached to successfully load the package using library(pkgname) must be listed in the ‘Depends’ field, only.
Can someone provide a bit more clarity on this? How do I know when my package only needs namespaces loaded versus when I need a package to be attached? What are examples of both? I think the typical package is just a collection of functions that sometimes call functions in other packages (where some bit of work has already been coded-up). Is this scenario 1 or 2 above?
Edit
I wrote a blog post with a section on this specific topic (search for 'Imports v Depends'). The visuals make it a lot easier to understand.
"Imports" is safer than "Depends" (and also makes a package using it a 'better citizen' with respect to other packages that do use "Depends").
A "Depends" directive attempts to ensure that a function from another package is available by attaching the other package to the main search path (i.e. the list of environments returned by search()). This strategy can, however, be thwarted if another package, loaded later, places an identically named function earlier on the search path. Chambers (in SoDA) uses the example of the function "gam", which is found in both the gam and mgcv packages. If two other packages were loaded, one of them depending on gam and one depending on mgcv, the function found by calls to gam() would depend on the order in which they those two packages were attached. Not good.
An "Imports" directive should be used for any supporting package whose functions are to be placed in <imports:packageName> (searched immediately after <namespace:packageName>), instead of on the regular search path. If either one of the packages in the example above used the "Imports" mechanism (which also requires import or importFrom directives in the NAMESPACE file), matters would be improved in two ways. (1) The package would itself gain control over which mgcv function is used. (2) By keeping the main search path clear of the imported objects, it would not even potentially break the other package's dependency on the other mgcv function.
This is why using namespaces is such a good practice, why it is now enforced by CRAN, and (in particular) why using "Imports" is safer than using "Depends".
Edited to add an important caveat:
There is one unfortunately common exception to the advice above: if your package relies on a package A which itself "Depends" on another package B, your package will likely need to attach A with a "Depends directive.
This is because the functions in package A were written with the expectation that package B and its functions would be attached to the search() path.
A "Depends" directive will load and attach package A, at which point package A's own "Depends" directive will, in a chain reaction, cause package B to be loaded and attached as well. Functions in package A will then be able to find the functions in package B on which they rely.
An "Imports" directive will load but not attach package A and will neither load nor attach package B. ("Imports", after all, expects that package writers are using the namespace mechanism, and that package A will be using "Imports" to point to any functions in B that it need access to.) Calls by your functions to any functions in package A which rely on functions in package B will consequently fail.
The only two solutions are to either:
Have your package attach package A using a "Depends" directive.
Better in the long run, contact the maintainer of package A and ask them to do a more careful job of constructing their namespace (in the words of Martin Morgan in this related answer).
Hadley Wickham gives an easy explanation (http://r-pkgs.had.co.nz/namespace.html):
Listing a package in either Depends or Imports ensures that it’s
installed when needed. The main difference is that where Imports just
loads the package, Depends attaches it. There are no other
differences. [...]
Unless there is a good reason otherwise, you should always list
packages in Imports not Depends. That’s because a good package is
self-contained, and minimises changes to the global environment
(including the search path). The only exception is if your package is
designed to be used in conjunction with another package. For example,
the analogue package builds on top of vegan. It’s not useful without
vegan, so it has vegan in Depends instead of Imports. Similarly,
ggplot2 should really Depend on scales, rather than Importing it.
Chambers in SfDA says to use 'Imports' when this package uses a 'namespace' mechanism and since all packages are now required to have them, then the answer might now be always use 'Imports'. In the past packages could have been loaded without actually having namespaces and in that case you would need to have used Depends.
Here is a simple question to help you decide which to use:
Does your package require the end user to have direct access to the functions of another package?
NO -> Imports (most common answer)
YES -> Depends
The only time you should use 'Depends' is when your package is an add-on or companion to another package, where your end user will be using functions from both your package and the 'Depends' package in their code. If your end user will only be interfacing with your functions, and the other package will only be doing work behind the scenes, then use 'Imports' instead.
The caveat to this is that if you add a package to 'Imports', as you usually should, your code will need to refer to functions from that package, using the full namespace syntax, e.g. dplyr::mutate(), instead of just mutate(). It makes the code a little clunkier to read, but it’s a small price to pay for better package hygiene.