I am using tbats function from forecast package in R. tbats.components function enables us to extract different components from a model object. However, I was not able to find any solution to extract trend component after I make the forecast. That is, I am looking for solutions to get trend forecast component.
Below are the codes examples:
library(forecast)
fit <- tbats(USAccDeaths)
the following is used to extract components
comp = tbats.components(fit);
trend_comp = comp[, 'level']
Making prediction, and this would give the forecast on target variable.
pred = forecast(fit, h = 10)
My question would be: is it possible to get trend component from pred? i.e. trend forecast.
We can use tbats.components
library(forecast)
out <- tbats.components(fit)
out[, "slope"]
It also depends on how the model was created.
data
fit <- tbats(USAccDeaths, use.trend = TRUE)
Related
Is it possible to calculate prediction intervals from a tidymodels stacked model?
Working through the example from the stacks() package here yields the stacked frog model (which can be downloaded here for reprex) and the testing data:
data("tree_frogs")
tree_frogs <- tree_frogs %>%
filter(!is.na(latency)) %>%
select(-c(clutch, hatched))
set.seed(1)
tree_frogs_split <- initial_split(tree_frogs)
tree_frogs_train <- training(tree_frogs_split)
tree_frogs_test <- testing(tree_frogs_split)
I tried to run something like this:
pi <- predict(tree_frogs_model_st, tree_frogs_test, type = "pred_int")
but this gives an error:
Error in UseMethod("stack_predict") : no applicable method for 'stack_predict' applied to an object of class "NULL"
Reading the documentation of stacks() I also tried passing "pred_int" in the opts list:
pi <- predict(tree_frogs_model_st, tree_frogs_test, opts = list(type = "pred_int"))
but this just gives: opts is only used with type = raw and was ignored.
For reference, I am trying to do a similar thing that is done in Ch.19 of Tidy Modeling with R book
lm_fit <- fit(lm_wflow, data = Chicago_train)
predict(lm_fit, Chicago_test, type = "pred_int")
which seems to work fine for a single model fit like lm_fit, but apparently not for a stacked model?
Am I missing something? Is it not possible to calculate prediction intervals for stacked models for some reason?
This is very difficult to do.
Even if glmnet produced a prediction interval, it would be a significant underestimate since it doesn’t know anything about the error in each of the ensemble members.
We would have to get the standard error of prediction from all of the models to compute it for the stacking model. A lot of these models don’t/can’t generate that standard error.
The alternative is the use bootstrapping to get the interval but you would have to bootstrap each model a large number of times to get the overall prediction interval.
My goal: I want to understand a time series, a strongly auto-regressive one (ACF and PACF output told me that) and make a forecast.
So what I did was I first transformed my data into a ts, then decomposed the time series, checked its stationarity (the series wasn't stationary). Then I conducted a log transformation and found an Arima model that fits the data best - I checked the accuracy with accuracy(x) - I selected the model with the accuracy output closest to 0.
Was this the correct procedure? I'm new to statistics and R and would appreciate some criticism if that wasn't correct.
When building the Arima model I used the following code:
ARIMA <- Arima(log(mydata2), order=c(2,1,2), list(order=c(0,1,1), period=12))
The result I received was a log function and the data from the past (the data I used to build the model) wasn't displayed in the diagram. So then to transform the log into the original scale I used the following code:
ARIMA_FORECAST <- forecast(ARIMA, h=24, lambda=0)
Is that correct? I found it somewhere on the web and don't really understand it.
Now my main question: How can I plot the original data and the ARIMA_FORECAST in one diagram? I mean displaying it the way the forecasts are displayed if no log transformation is undertaken - the forecast should be displayed as the extension of the data from the past, confidence intervals should be there too.
The simplest approach is to set the Box-Cox transformation parameter $\lambda=0$ within the modelling function, rather than take explicit logarithms (see https://otexts.org/fpp2/transformations.html). Then the transformation will be automatically reversed when the forecasts are produced. This is simpler than the approach described by #markus. For example:
library(forecast)
# estimate an ARIMA model to log data
ARIMA <- auto.arima(AirPassengers, lambda=0)
# make a forecast
ARIMA_forecast <- forecast(ARIMA)
# Plot forecasts and data
plot(ARIMA_forecast)
Or if you prefer ggplot graphics:
library(ggplot2)
autoplot(ARIMA_forecast)
The package forecast provides the functions autolayer and geom_forecast that might help you to draw the desired plot. Here is an example using the AirPassengers data. I use the function auto.arima to estimate the model.
library(ggplot2)
library(forecast)
# log-transform data
dat <- log(AirPassengers)
# estimate an ARIMA model
ARIMA <- auto.arima(dat)
# make a forecast
ARIMA_forecast <- forecast(ARIMA, h = 24, lambda = 0)
Since your data is of class ts you can use the autoplot function from ggplot2 to plot your original data and add the forecast with the autolayer function from forecast.
autoplot(AirPassengers) + forecast::autolayer(ARIMA_forecast)
The result is shown below.
I'm trying to use an ARIMA model in R to forecast data. A slice of my time series looks like this:
This is just a slice of time for you get a sense of it. I have daily data from 2010 to 2015.
I want to forecast this into the future. I'm using the forecast library, and my code looks like this:
dt = msts(data$val, seasonal.periods=c(7, 30))
fit = auto.arima(dt)
plot(forecast(fit, 300))
This results in:
This model isn't good or interesting. My seasonal.periods were defined by me because I expect to see weekly and monthly seasonality, but the result looks the same with no seasonal periods defined.
Am I missing something? Very quickly the forecast predictions change very, very little from point to point.
Edit:
To further show what I'm talking about, here's a concrete example. Let's say I have the following fake dataset:
x = 1:500
y = 0.5*c(NA, head(x, -1)) - 0.4*c(NA, NA, head(x, -2)) + rnorm(500, 0, 5)
This is an AR(2) model with coefficients 0.5 and 0.4. Plotting this time series yields:
So I create an ARIMA model of this and plot the forecast results:
plot(forecast(auto.arima(y), 300))
And the results are:
Why can't the ARIMA function learn this obvious model? I don't get any better results if I use the arima function and force it to try an AR(2) model.
auto.arima does not handle multiple seasonal periods. Use tbats for that.
dt = msts(data$val, seasonal.periods=c(7, 30))
fit = tbats(dt)
plot(forecast(fit, 300))
auto.arima will just use the largest seasonal period and try to do the best it can with that.
Is there any way to find the arima parameters for a hts forecast ?
My forecast is something like this:
myts_f <- forecast(myts, h=78, fmethod = "arima", method = "tdfp")
hts is: http://cran.r-project.org/web/packages/hts
Thanks,
Regards
A separate ARIMA model is estimated for every series in the hierarchy. For large hierarchies, that can involve thousands or even millions of models. There is no point in storing all the resulting information when you only want forecasts.
If you really care about the individual models, then fit them explicitly to all series. You can get the matrix of every series (included aggregated series) using aggts(myts). For example:
y <- aggts(myts)
models <- list()
for(i in 1:ncol(y))
models[[i]] <- auto.arima(y[,i])
Hello I use the package forecast in order to do times-series prevision. I would like to know how to un-log a series on the final forecast plot. With the forecast package I don't know how to un-log my series. Here is an example:
library(forecast)
data <- AirPassengers
data <- log(data) #with this AirPassengers data not nessesary to LOG but with my private data it is...because of some high picks...
ARIMA <- arima(data, order = c(1, 0, 1), list(order = c(12,0, 12), period = 1)) #Just a fake ARIMA in this case...
plot(forecast(ARIMA, h=24)) #but my question is how to get a forecast plot according to the none log AirPassenger data
So the image is logged. I want to have the same ARIMA modell but witht the none loged data.
It is not necessary to use the hack proposed by #ndoogan. forecast.Arima has built-in facilities for undoing transformations. The following code will do what is required:
fc <- forecast(ARIMA, h=24, lambda=0)
Better still, build the transformation into the model itself:
ARIMA <- Arima(data, order=c(1,0,1), list(order=c(1,0,1),period=12)), lambda=0)
fc <- forecast(ARIMA, h=24)
Note that you need to use the Arima function from the forecast package to do this, not the arima function from the stats package.
#Hemmo is correct that this back-transformation will not give the mean of the forecast distribution, and so is not the optimal MSE forecast. However, it will give the median of the forecast distribution, and so will give the optimal MAE forecast.
Finally, the fake model used by #Swiss12000 makes little sense as the seasonal part has frequency 1, and so is confounded with the non-seasonal part. I think you probably meant the model I've used in the code above.
The problem with #ndoogan's answer is that logarithm is not a linear transformation. Which means that E[exp(y)] != exp(E[y]). Jensen's inequality gives actually that E[exp(y)] >= exp(E[y]). Here's a simple demonstration:
set.seed(1)
x<-rnorm(1000)
mean(exp(x))
[1] 1.685356
exp(mean(x))
[1] 0.9884194
Here's a case concerning the prediction:
# Simulate AR(1) process
set.seed(1)
y<-10+arima.sim(model=list(ar=0.9),n=100)
# Fit on logarithmic scale
fit<-arima(log(y),c(1,0,0))
#Simulate one step ahead
set.seed(123)
y_101_log <- fit$coef[2]*(1-fit$coef[1]) +
fit$coef[1]*log(y[100]) + rnorm(n=1000,sd=sqrt(fit$sigma2))
y_101<-exp(y_101_log) #transform to natural scale
exp(mean(y_101_log)) # This is exp(E(log(y_101)))
[1] 5.86717 # Same as exp(predict(fit,n.ahead=1)$pred)
# differs bit because simulation
mean(y_101) # This is E(exp(log(y_101)))=E(y_101)
[1] 5.904633
# 95% Prediction intervals:
#Naive way:
pred<-predict(fit,n.ahead=1)
c(exp(pred$pred-1.96*pred$se),exp(pred$pred+1.96*pred$se))
pred$pred pred$pred
4.762880 7.268523
# Correct ones:
quantile(y_101,probs=c(0.025,0.975))
2.5% 97.5%
4.772363 7.329826
This also provides a solution to your problem in general sense:
Fit your model
Simulate multiple samples from that model (for example one step ahead predictions as above)
For each simulated sample, make the inverse transformation to get the values in original scale
From these simulated samples you can compute the expected value as a ordinary mean, or if you need confidence intervals, compute empirical quantiles.
This is a bit of a hack, but it seems to do what you want. Based on your fitted model ARIMA:
fc<-forecast(ARIMA,h=24)
fc$mean<-exp(fc$mean)
fc$upper<-exp(fc$upper)
fc$lower<-exp(fc$lower)
fc$x<-exp(fc$x)
Now plot it
plot(fc)