this question is somewhat a follow up on this question.
Consider the following example
set.seed(1)
x <- cumsum(rnorm(10))
y <- stats::arima(x, order = c(1, 0, 0))
length(stats::fitted(y))
[1] 0
So far so good: zero is returned because R does not now how to use stats::fitted on an object of class Arima.
Next in my code, I need one function from the forecast package. I do not attach the package, I just load it using the ::notation.
In my code below I will load it directly using requireNamespace.
requireNamespace("forecast", quietly = TRUE)
length(stats::fitted(y))
[1] 10
And suddenly the same command returns a different result.
I understand why this happens (and I hope I am saying it correctly): by loading the forecastpackage a new method for the generic function fitted (namely fitted.Arima) is loaded into the namespace which results in a different outcome.
For me this behavior is quite annoying: is there any way to choose one specific method for fitted?
I read this chapter but did not figure out how to circumvent this problem.
I also tried to unload the forecast package from namespace, but no success:
unloadNamespace("forecast")
length(stats::fitted(y))
[1] 10
It seems that once I load the package I cannot use the old method of fitted.
I am wondering how to handle these situations.
EDIT
As pointed out in the comments after unloadNamespace("forecast") I get that
isNamespaceLoaded("forecast")
[1] FALSE
But methods fitted still includes fitted.Arima.
#CalumYou is exactly right in pointing out that unloading a namespace will not remove S3 methods registered for an S3 generic defined in another package. Here, in case you are interested, is a more detailed look at how and why that is the case.
When the forecast package is loaded, all of the methods that it defines are "registered" in data bases in a variety of different namespaces. The rule R follows is that a method gets registered in the namespace of the package that defines its S3 generic. Since the fitted() generic is defined in stats, that's where the new methods defined by forecast get registered, in an environment called .__S3MethodsTable__.. Detaching or unloading forecast leaves the stats package untouched (probably an overall wise design decision, if you think about it), with the unfortunate consequence that the fitted.Arima method (along with many others) remain registered in its .__S3MethodsTable__.
To see that this is so, have a look at the following:
isNamespaceLoaded("forecast")
## [1] FALSE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
## [1] "fitted.default" "fitted.isoreg" "fitted.kmeans"
## [4] "fitted.nls" "fitted.smooth.spline"
## Loading the forecast namespace registers new 'fitted' methods ...
requireNamespace("forecast", quietly = TRUE)
isNamespaceLoaded("forecast")
## [1] TRUE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
## [1] "fitted.ar" "fitted.Arima" "fitted.arma"
## [4] "fitted.bats" "fitted.default" "fitted.ets"
## [7] "fitted.fracdiff" "fitted.garch" "fitted.gls"
## [10] "fitted.glsStruct" "fitted.gnls" "fitted.gnlsStruct"
## [13] "fitted.isoreg" "fitted.kmeans" "fitted.lagwalk"
## [16] "fitted.lme" "fitted.lmeStruct" "fitted.lmList"
## [19] "fitted.modelAR" "fitted.nlmeStruct" "fitted.nls"
## [22] "fitted.nnetar" "fitted.quantmod" "fitted.smooth.spline"
## [25] "fitted.tbats" "fitted.tslm" "fitted.values.quantmod"
## ... which are left behind even when the forecast namespace is unloaded
unloadNamespace("forecast")
isNamespaceLoaded("forecast")
## [1] FALSE
ls(stats:::.__S3MethodsTable__., pattern = "fitted")
## [1] "fitted.ar" "fitted.Arima" "fitted.arma"
## [4] "fitted.bats" "fitted.default" "fitted.ets"
## [7] "fitted.fracdiff" "fitted.garch" "fitted.gls"
## [10] "fitted.glsStruct" "fitted.gnls" "fitted.gnlsStruct"
## [13] "fitted.isoreg" "fitted.kmeans" "fitted.lagwalk"
## [16] "fitted.lme" "fitted.lmeStruct" "fitted.lmList"
## [19] "fitted.modelAR" "fitted.nlmeStruct" "fitted.nls"
## [22] "fitted.nnetar" "fitted.quantmod" "fitted.smooth.spline"
## [25] "fitted.tbats" "fitted.tslm" "fitted.values.quantmod"
(For a related question and answer, see here.)
I found this thread from R devel. Brian Ripley (of R Core) says:
Unloading a namespace does not unregister its methods (and
registration has no stack, so there is no way R knows what was there
before).
The thread then notes that ?unloadNamespace points you at ?detach:
See the comments in the help for detach about some issues with
unloading and reloading name spaces.
which does eventually say the following (emphasis mine)
If a package has a namespace, detaching it does not by default unload
the namespace (and may not even with unload = TRUE), and detaching
will not in general unload any dynamically loaded compiled code
(DLLs). Further, registered S3 methods from the namespace will not be
removed.
My understanding is therefore that while loading a namespace (such as by using ::) registers S3 methods, these methods are never linked to the namespace that they were loaded from and so unloading the name space cannot also unregister the methods. The only way to clear them from methods() would be to restart R.
As RolandASc noted, you can choose to call the default method by using stats:::fitted.default if you want to avoid dispatch to fitted.Arima.
Related
Because it is on the list of Internal Generic Functions, I know that $ is an internal generic function. To my knowledge, this means that it cannot be extended using S3. Despite this, it is well-known that $ behaves differently for tibbles as it does data frames, implying that the developer of the tibble package have done what I believed to be impossible. How was this achieved? I tried to get the code for $.tibble up in my IDE, but tibble::"$".tibble returned nothing.
You can change behavior for internal generics if the first parameter is an object (is.object(x)==TRUE). A tibble is an object. Most user created S3 classes are. Your problem before was you were trying to change the behavior for a matrix and a matrix is not an "object" so you cannot change the dispatch for internal generics for such objects.
Use the following to find examples in the base of R and in any loaded packages. (If the tibble package were loaded it would also list any $ methods in it although be aware that the class name of a tibble is not tibble .)
methods("$")
## [1] $,envRefClass-method $,refObjectGenerator-method
## [3] $.bibentry* $.DLLInfo
## [5] $.package_version $.person*
getAnywhere("$.bibentry")
## ...snip...
library(tibble)
tib <- tibble()
class(tib)
## [1] "tbl_df" "tbl" "data.frame"
getAnywhere("$.tbl_df")
## ..snip...
Here are some more examples assuming you have installed the relevant packages:
zoo:::"$.zoo"
proto:::"$.proto"
gsubfn:::"$.fn"
dyn:::"$.dyn"
or:
https://github.com/cran/zoo/blob/master/R/zoo.R
https://github.com/hadley/proto/blob/master/R/proto.R
https://github.com/cran/gsubfn/blob/master/R/fn.R
https://github.com/cran/dyn/blob/master/R/dyn.R
A key consideration is that the part after the $ is not evaluated even if you extend it.
I wanted to filter a data set based on some conditions. When I looked at the help for filter function the result was:
filter {stats} R Documentation
Linear Filtering on a Time Series
Description
Applies linear filtering to a univariate time series or to each series separately of a multivariate time series.
After searching on web I found the filter function I needed i.e. from dplyr package. How can R have two functions with same name. What am I missing here?
At the moment the R interpreter would dispatch a call to filter to the dplyr environment, at least if the class of the object were among the avaialble methods:
methods(filter)
[1] filter.data.frame* filter.default* filter.sf* filter.tbl_cube* filter.tbl_df* filter.tbl_lazy*
[7] filter.ts*
As you can see there is a ts method, so if the object were of that class, the interpreter would instead deliver the x values to it. However, it appears that the authors of dplyr have blocked that mechanism and instead put in a warning function. You would need to use:
getFromNamespace('filter', 'stats')
function (x, filter, method = c("convolution", "recursive"),
sides = 2L, circular = FALSE, init = NULL)
{ <omitting rest of function body> }
# same result also obtained with:
stats::filter
R functions are contained in namespaces, so a full designation of a function would be: namespace_name::function_name. There is a hierarchy of namespace containers (actually "environments" in R terminology) arranged along a search path (which will vary depending on the order in which packages and their dependencies have been loaded). The ::-infix-operator can be used to specify a namespace or package name that is further up the search path than might be found in the context of the calling function. The function search can display the names of currently loaded packages and their associated namespaces. See ?search Here's mine at the moment (which is a rather bloated one because I answer a lot of questions and don't usually start with a clean systems:
> search()
[1] ".GlobalEnv" "package:kernlab" "package:mice" "package:plotrix"
[5] "package:survey" "package:Matrix" "package:grid" "package:DHARMa"
[9] "package:eha" "train" "package:SPARQL" "package:RCurl"
[13] "package:XML" "package:rnaturalearthdata" "package:rnaturalearth" "package:sf"
[17] "package:plotly" "package:rms" "package:SparseM" "package:Hmisc"
[21] "package:Formula" "package:survival" "package:lattice" "package:remotes"
[25] "package:forcats" "package:stringr" "package:dplyr" "package:purrr"
[29] "package:readr" "package:tidyr" "package:tibble" "package:ggplot2"
[33] "package:tidyverse" "tools:rstudio" "package:stats" "package:graphics"
[37] "package:grDevices" "package:utils" "package:datasets" "package:methods"
[41] "Autoloads"
At the moment I can find instances of 3 versions of filter using the help system:
?filter
# brings this up in the help panel
Help on topic 'filter' was found in the following packages:
Return rows with matching conditions
(in package dplyr in library /home/david/R/x86_64-pc-linux-gnu-library/3.5.1)
Linear Filtering on a Time Series
(in package stats in library /usr/lib/R/library)
Objects exported from other packages
(in package plotly in library /home/david/R/x86_64-pc-linux-gnu-library/3.5.1)
I'm using sma from package Smooth and it's all working fine. Just curious why when I call methods(sma) it doesn't give any methods:
> methods(sma)
no methods found
methods function returns:
List all available methods for a S3 and S4 generic function, or all methods for
an S3 or S4 class.
hence there are no such methods available for sma function.
Please compare with e.g. plot.
head(methods(plot), 20)
Output:
[1] "plot,ANY,ANY-method" "plot,ca.jo,missing-method" "plot,ca.po,missing-method" "plot,color,ANY-method"
[5] "plot,timeDate,ANY-method" "plot,ur.df,missing-method" "plot,ur.ers,missing-method" "plot,ur.kpss,missing-method"
[9] "plot,ur.pp,missing-method" "plot,ur.sp,missing-method" "plot,ur.za,missing-method" "plot.acf"
[13] "plot.ACF" "plot.ar" "plot.Arima" "plot.arma"
[17] "plot.armasubsets" "plot.augPred" "plot.bats" "plot.chobTA"
I was working in the mirt package in R and noticed that I couldn't use mirt:: or mirt::: to call the coef or residuals functions. From what I can tell this is a S3 to S4 difference (magic fingers & hand waving).
Which brings me to the question, how do you call a specific R function within it's package when it's coded in S4?
After
> library(mirt)
Loading required package: stats4
Loading required package: lattice
I see
> methods(coef)
[1] coef,ANY-method coef,DiscreteClass-method
[3] coef,MixedClass-method coef,mle-method
[5] coef,MultipleGroupClass-method coef,SingleGroupClass-method
[7] coef,summary.mle-method coef.aov*
[9] coef.Arima* coef.default*
[11] coef.listof* coef.nls*
see '?methods' for accessing help and source code
I guess you have an instance of one of the classes, e.g., 'DiscreteClass'. You can select the method with
selectMethod("coef", signature="DiscreteClass")
or maybe more naturally
selectMethod("coef", class(obj))
where obj is an instance of the object you're interested in. But you shouldn't have to call a specific method; this should be taken care of -- what's the problem you're actually experiencing.
An R namespace acts as the immediate environment for all functions in its associated package. In other words, when function bar() from package foo calls another function, the R evaluator first searches for the other function in <environment: namespace:foo>, then in "imports.foo", <environment: namespace:base>, <environment: R_GlobalEnv>, and so on down the search list returned by typing search().
One nice aspect of namespaces is that they can make packages act like better citizens: unexported functions in <environment: namespace:foo> and functions in imports:foo are available only: (a) to functions in foo; (b) to other packages that import from foo; or (c) via fully qualified function calls like foo:::bar().
Or so I thought until recently...
The behavior
This recent SO question highlighted a case in which a function well-hidden in its package's namespace was nonetheless found by a call to a seemingly unrelated function:
group <- c("C","F","D","B","A","E")
num <- c(12,11,7,7,2,1)
data <- data.frame(group,num)
## Evaluated **before** attaching 'gmodels' package
T1 <- transform(data, group = reorder(group,-num))
## Evaluated **after** attaching 'gmodels
library(gmodels)
T2 <- transform(data, group = reorder(group,-num))
identical(T1, T2)
# [1] FALSE
Its immediate cause
#Andrie answered the original question by pointing out that gmodels imports from the the package gdata, which includes a function reorder.factor that gets dispatched to inside the second call to transform(). T1 differs from T2 because the first is calculated by stats:::reorder.default() and the second by gdata:::reorder.factor().
My question
How is it that in the above call to transform(data, group=reorder(...)), the dispatching mechanism for reorder finds and then dispatches to gdata:::reorder.factor()?
(An answer should include an explanation of the scoping rules that lead from a call involving functions in the stats and base packages to a seemingly well-hidden method in gdata.)
Further possibly helpful details
Neither gdata:::reorder.factor, nor the gdata package as a whole are explicitly imported by gmodels. Here are the import* directives in gmodels' NAMESPACE file:
importFrom(MASS, ginv)
importFrom(gdata, frameApply)
importFrom(gdata, nobs)
There are no methods for reorder() or transform() in <environment: namespace:gmodels>, nor in "imports:gmodels":
ls(getNamespace("gmodels"))
ls(parent.env(getNamespace("gmodels")))
Detaching gmodels does not revert reorder()'s behavior: gdata:::reorder.factor() still gets dispatched:
detach("package:gmodels")
T3 <- transform(data, group=reorder(group,-num))
identical(T3, T2)
# [1] TRUE
reorder.factor() is not stored in the list of S3 methods in the base environment:
grep("reorder", ls(.__S3MethodsTable__.))
# integer(0)
R chat threads from the last couple of days include some additional ideas. Thanks to Andrie, Brian Diggs, and Gavin Simpson who (with others) should feel free to edit or add possibly impt. details to this question.
I'm not sure if I correctly understand your question, but the main point is that group is character vector while data$group is factor.
After attaching gmodels, the call for reorder(factor) calls gdata:::reorder.factor.
so, reorder(factor(group)) calls it.
In transform, the function is evaluated within the environment of the first argument, so in T2 <- transform(data, group = reorder(group,-num)), group is factor.
UPDATED
library attaches the import packages into loaded namespace.
> loadedNamespaces()
[1] "RCurl" "base" "datasets" "devtools" "grDevices" "graphics" "methods"
[8] "stats" "tools" "utils"
> library(gmodels) # here, namespace:gdata is loaded
> loadedNamespaces()
[1] "MASS" "RCurl" "base" "datasets" "devtools" "gdata" "gmodels"
[8] "grDevices" "graphics" "gtools" "methods" "stats" "tools" "utils"
Just in case, the reorder generic exists in namespace:stats:
> r <- ls(.__S3MethodsTable__., envir = asNamespace("stats"))
> r[grep("reorder", r)]
[1] "reorder" "reorder.default" "reorder.dendrogram"
And for more details
The call of reorder will search the S3generics in two envs:
see ?UseMethod
first in the environment in which the generic function is called, and then in the registration data base for the environment in which the generic is defined (typically a namespace).
then, loadNamespace registers the S3 functions to the namespace.
So , in your case, library(gmodels) -> loadNamespace(gdata) -> registerS3Methods(gdata).
After this, you can find it by:
> methods(reorder)
[1] reorder.default* reorder.dendrogram* reorder.factor*
Non-visible functions are asterisked
However, as the reorder.factor is not attached on your search path, you cannot access it directly:
> reorder.factor
Error: object 'reorder.factor' not found
Probably this is whole scenario.