I have various financial data that I am trying to merge into an xts object so I can perform multiple statistical analyses. I am having difficulty, however, with dates when moving from the original data to a zoo object to an xts object.
For instance, I read in some hedge fund return data, change the report date variable using the ymd function from the lubridate package, create a zoo object, then just as a check create a timeSeries object. All seems to be OK, but I continue to get an error when I attempt to create the xts object, as shown below:
hfIndexes$ReportDt <- ymd(hfIndexes$ReportDt)
hfIndexesZoo <- zoo(hfIndexes,order.by="ReportDt")
hfIndexesTimeSeries <- as.timeSeries(hfIndexesZoo)
hfIndexesXTS <- as.xts(hfIndexesZoo)
Error in xts(coredata(x), order.by = order.by, frequency = frequency, :
order.by requires an appropriate time-based object
What do I need to do to ensure that I have the correct time-based object to create the desired xts object?
Consider this answer: https://stackoverflow.com/a/4297342/3253015
order.by is an argument needed in xts objects. As we are dealing with timeseries, you can consider it to be one, that creates a frame of sorts, into which the data is put. So you tell as.xts that the data you want inside is spaced out by the time-based object given in order.by.
Related
I am exploring Facebook's prophet package in R. I have noticed that the package sets all dates to the GMT time zone with no way (that I am aware of) to set the time zone when calling the function.
My question is: given that the prophet algorithm returns a list of lists with several different POSIXt or POSIXct objects, is there a way to change all of these objects at once to the same time zone without having to change all objects individually?
For instance, if I have an object created by the prophet function called prophet.model, I can update history.dates or start from the GMT timezone to the "Europe/Paris" time zone using:
library(prophet)
history <- data.frame(ds = seq(as.Date('2015-01-01'), as.Date('2016-01-01'), by = 'd'),
y = sin(1:366/200) + rnorm(366)/10)
prophet.model <- prophet(history)
attr(prophet.model$start,"tzone") <- "Europe/Paris"
attr(prophet.model$history.dates,"tzone") <- "Europe/Paris"
Can this be done instead for the whole object prophet.model at once without having to make individual calls to attr?
Data is something like this:
df <- tribble(
~y,~timestamp
18.74682, 1500256800,
19.00424, 1500260400,
18.86993, 1500264000,
18.74960, 1500267600,
18.99854, 1500271200,
18.85443, 1500274800,
18.78031, 1500278400,
18.97948, 1500282000,
18.86576, 1500285600,
18.55633, 1500289200,
18.79052, 1500292800,
18.74790, 1500296400,
18.62743, 1500300000,
19.04696, 1500303600,
18.97851, 1500307200,
18.70956, 1500310800,
18.92302, 1500314400,
18.91465, 1500318000,
18.61556, 1500321600,
19.03535, 1500325200 )
I'm trying to apply hybridModel on timeseries data to perform ensemble.Below is my code:
library(tidyquant)
library(forecast)
library(timetk)
library(sweep)
library(forecastHybrid)
df <- mutate(df, timestamp = as_datetime(timestamp))
tk_ts_df <- tk_ts(df, start = 1, freq = 3600, silent = TRUE)
fit <- hybridModel(tk_ts_df)
On fitting timeseries object tk_ts_df (ts object) to hybridModel; it's giving error : "The time series must be numeric and may not be a matrix or dataframe object."
But on link: https://cran.r-project.org/web/packages/forecastHybrid/vignettes/forecastHybrid.html
It's clearly mentioned : The workhorse function of the package is hybridModel(), a function that combines several component models from the “forecast” package. At a minimum, the user must supply a ts or numeric vector for y
Please suggest what I'm doing wrong.
The "forecastHybrid" requires that the input timeseries is a numeric vector or ts type. While the "timekit" package does return a ts object, it also adds additional attributes that are not in regular ts objects so input checks failed.
See discussion here. and the fixing commit here.
The latest version from Github incorporating the fix can be downloaded with
devtools::install_github("ellisp/forecastHybrid/pkg")
I need to use 'PerformanceAnalytics' package of R and to use this package, I understand that I need to convert the data into xts data, which is actually a panel data. Following this forum's suggestion I have done the following:
library(foreign)
RNOM <- read.dta("Return Panel without missing.dta")
RNOM_list<-split(RNOM,RNOM$gvkey)
xts_list<-lapply(RNOM_list,function(x)
{out<-xts(x[,-1],order.by=as.Date(x$datadate,format="%d/%m/%Y")) })
It gives me RNOM_list and xts_list.
After this, can some please help me to estimate the monthly returns using the function Return.calculate and lapply and save the output generated as an addition variable in my original data-set for regression analysis? Subsequently, I also need to estimate VaR, ES and semi-sd.
The data can be downloaded here. Note, prccm is the monthly closing price in the data and gvkey is the firm ID.
An efficient way to achieve this goal is to covert the Panel Data (long format) into wide format using 'reshape2' package. After performing the estimations, convert it back to long format or panel data format. Here is an example:
library(foreign)
library(reshape2)
dd <- read.dta("DDA.dta") // DDA.dta is Stata data; keep only date, id and variable of interest (i.e. three columns in total)
wdd<-dcast(dd, datadate~gvkey) // gvkey is the id
require(PerformanceAnalytics)
wddxts <- xts(wdd[,-1],order.by=as.Date(wdd$datadate,format= "%Y-%m-%d"))
ssd60A<-rollapply(wddxts,width=60,SemiDeviation,by.column=TRUE,fill=NA) // e.g of rolling window calculation
ssd60A.df<-as.data.frame(ssd60A.xts) // convert dataframe to xts
ssd60A.df$datadate=rownames(ssd60A.df) // insert time index
lssd60A.df<-melt(ssd60A.df, id.vars=c('datadate'),var='gvkey') // convert back to panel format
write.dta(lssd60A.df,"ssd60A.dta",convert.factors = "string") // export as Stata file
Then simply merge it with the master database to perform some regression.
I am struggling to convert a zoo objects to a ts object.
I have a huge data.frame "test" with quarterly hour data, which looks like this:
date <- c("2010-07-04 09:45:00", "2010-07-04 10:00:00", "2010-07-04 10:15:00", "2010-07-04 10:30:00", "2010-07-04 10:45:00", "2010-07-04 11:00:00")
nrv <- c("-147.241", "-609.778", "-432.289", "-340.418", "-73.96" , "-533.108")
tt <- c("3510.7", "3608.5", "3835.7", "4003.7", "4018.8", "4411.9")
test <- data.frame(date,nrv,tt)
test
I want to make some predictions (mostly ARIMA) and thought the forecastpackage would be a good idea for that.
First of I formated the data away from characters.
test$date <- strptime(test$date,format="%Y-%m-%d %H:%M")
test$nrv <- as.numeric(as.character(test$nrv))
test$tt <- as.numeric(as.character(test$tt))
str(test) #date is POSIXlt object
Since I needed to do an interpolation and construct lags, I also used the zoo package using the date variable as index, which worked great. The `zoo package was recommended to me while dealing with time series data.
library(zoo)
test.zoo <- zoo(test[,2:3],test[,1])
test.zoo #date is now the Index and and the zoo objects works nicely
But then I realized that forecasting only seems to work with ts objects. (Is that true?)
When I tried to convert the zoo object to a ts object, my time index disappeared. I think this might be due to not using a proper frequency. However I am somewhat lost as to what would be a working frequency for this dataset and with ts objects in general.
test.ts <- as.ts(test.zoo)
test.ts
How do I convert this zoo object back to a ts object I can use for forecasting?
Thanks!
The forecast package only works with ts objects as you suspected.
You can use test.ts with the forecast package. For example
plot(forecast(test.ts[,1]))
I had the same problem and solved it by using zooreg function.
step1: use zooreg to transform zoo object to non-zoo but ts alike objet
step2: use ts function to transform further to ts object
I'm trying to automate some seasonal adjustment with the x12 package. To do this I need a ts object. However, I do not need a simple ts object, but one whose start date and frequency has been set. For any given series I could type that, but I will be feeding a mix of monthly or weekly data in. I can get the data from a quantmod as an xta object, but can't seem to figure out how to extract the frequency from the xts.
Here is some sample code that works the the whole way through, but I would like to pull the frequency info from the xts, rather than explicitly set it:
getSymbols("WILACR3URN",src="FRED", from="2000-01-01") # get data as an XTS
lax <- WILACR3URN #shorten name
laxts <- ts(lax$WILACR3URN, start=c(2000,1), frequency=12) #explicitly it works
plot.ts(laxts)
x12out <- x12(laxts,x12path="c:\\x12arima\\x12a.exe",transform="auto", automdl=TRUE)
laxadj <- as.ts(x12out$d11) # extract seasonally adjusted series
Any suggestions? Or is it not possible and I should determine/feed the frequency explicitly?
Thanks
This is untested for this specific case, but try using xts::periodicity for the frequency:
freq <- switch(periodicity(lax)$scale,
daily=365,
weekly=52,
monthly=12,
quarterly=4,
yearly=1)
And use the year and mon elements of POSIXlt objects to calculate the start year and month.
pltStart <- as.POSIXlt(start(lax))
Start <- c(pltStart$year+1900,pltStart$mon+1)
laxts <- ts(lax$WILACR3URN, start=Start, frequency=freq)
plot.ts(laxts)
The xts::periodicity suggestion was helpful to me. I've also found the following approach using xts::convertIndex works well for monthly and quarterly data. It is untested for weekly data.
require("quantmod")
require("dplyr")
getSymbols("WILACR3URN",src="FRED", from="2000-01-01") # get data as an XTS
lax <- WILACR3URN #shorten name
laxts <- lax %>%
convertIndex("yearmon") %>% # change index of xts object
as.ts(start = start(.), end = end(.)) # convert to ts
plot.ts(laxts)