I have a data set containing the energy usage by day (date) from 01 Jan 2016 through to 07 Nov 2017 on a daily basis. One of the fields therein is a flag for non working day (nwd) with values of 0 and 1 indicating whether or not this is a working day.
The structure of the data looks like this :-
Date,usage,avgtemp,nwd
2016-01-01,28.5,105986,1
2016-01-02,29.2,105548,1
.
.
.
2017-11-07,98457,23.5,0
I created a data frame with these values - no problems. I then created 2 other data frames, one with nwd = 1 and other with nwd = 1 for the data set for non working and working days respectively.
I am trying to generate a time series (using zoo or xts package - I am open to either) for each of these 2 data frames so that I can then do the non stationarity tests (adf/pp) on them and then do the arima modelling to build a forecast model of the usage.
Can I use a time series for such data sets where the data is not quite regular because each of these series will have gaps - the work day series may have less than 5 continuous days in a week if there are holidays in between. The same would apply to the non working day series.
I cannot summarize this at a weekly level as I need to forecast them at a daily level and possibly at the half hourly level subsequently. I might even want to do 'ardl' modelling later using 'avgtemp' as one of the regressors.
P.S.
Found a post which to some extent is similar to mine but I can't seem to get it going based on the responses there :-
how to convert data frame into time series in R
Related
Are there tools in R that simplify analysis of lagged and disparate time series. For example:
Daily values that only occur on weekdays (no entry on weekends or holidays)
vs
Bi-annual values
What I'm seeking is ways to:
Complete the missing daily values (with interpolated, or last value rolled forward, etc.)
Look for correlation between daily values and the bi-annual value (only the values that came before the bi-annual event)
As an example:
10-year treasury note interest rate (daily on non-holiday weekdays) as "X" and i-bond fixed rate as "Y" (set May 1/Nov 1)
Any suggestions appreciated.
I've built a test dataset manually for "x" and used functions in zoo to populate the missing values (interpolated), but I'm hoping for a less "brute-force" method for looking at analyzing the disparate time series. I've used lag functions in the past, but those were on matching interval time series.
What Jon commented is what I had in mind:
expand a weekday time series to full week using missing value function(s) in zoo
Sample the daily value - say April 15 for the May 1, Oct 15 for Nov 1
Ideally be able to automate - say loop through April 1-30, Oct 1-30 to look for highest RSqr for the model of choice (linear, polynomial, etc.)
Not have to build discrete datasets for each of the above - but if that is what is required I can do it programmatically - I've done that with stock data in the past. I was looking for a more efficient means of selecting the datasets ad hoc during the analysis.
I don't have code to post, because I'm clueless as to the feature/function that would make the date selection I'm after possible (at least in R).
Thanks for the input so far. It has already been useful in helping me look at alternative methods to achieve what I'm after.
I am trying to analyze this time series data from the Wooldridge Econometrics book containing weekly data on the New York Stock Exchange, beginning in the year 1976 January and ending in 1989.
I have never worked with the ts() function before but I understand the general grammar already. What I have difficulties with, is how I should define the frequencies since every 4th year has 366 instead of 365 days. In the book is already stated that for holidays or weekends, the following day was used, when the stock exchange was open.
So how do I exactly deal with this problem of creating a time series object?
Here is a screenshot of the first rows of the data frame:
data frame of nyse
I have daily sales data between 1/1/2017 and 10/15/2017. My first question is that I was trying to set the time series with the following command:
marketing = ts(df$Total_Sales_Spend, start=c(2017,1,1), frequency = 365)
However, when I try to decompose the TS, I am getting the following error:
time-series has no or less than 2 periods
I understand that it's because I don't have at least two years of data. So in this case what can I do?
Also, I'd like to forecast the daily sales numbers for the second half of October (16 days). How may I do that in this case? I wasn't able to set it up with zoo and arima.
Thank you.
I have a daily rainfall data for 36 years. I want to analyze the time series, but my data is still in the form of frame data, how I change the frame data into time series. My data is a variable, how to unify the year number with the date and month, so the data is only in one column
You could use a time series package for that, such as fpp i.e. install.packages('fpp'). Since you don't give an example code, I can't really help you properly with it but it's quite easy.
ts(your_data, start =, frequency = ) At start = you put the year or month where you'd start and at frequency = you'd put e.g. 36 since you talk about 36 years.
You might want to check out https://robjhyndman.com/. He has an online (free) book available that walks you through the use of his package as well as providing useful information with respect to time series analysis.
Hope this helps.
I'm having trouble doing time series for my data set. Most examples have quarterly or monthly frequencies but my issue comes with data that is collect annually or every two years. Consider my code:
data<-data.frame(year=seq(1978,2012,2), number=runif(18,100,500))
time<-ts(data$number, start=1978, frequency=.5)
decomp<-decompose(time)
Error in decompose(time) : time series has no or less than 2 periods
How do I make R recognize time series values from data that is collected over an annual basis? Thanks!
Seasonal decomposition only makes sense with intra-yearly data, because you have seasons within years. So, trying to calculate seasonal effects with decompose on data collected every two years you get the error.