I am having some data with a time column expressed in week.year and a corresponding unit that was measured in that week.
Week-Year Units
01.2020 39.12727273
02.2020 33.34545455
03.2020 118.7181818
04.2020 83.71818182
05.2020 58.56985
. .
52.2020 89.54651534
I have to create a ts object which takes these Week-Year values as input.
The reason for requiring this step is- there are sometimes values missing for certain weeks so using an auto generated time scale (start=, end=, frequency=) will mess up the readings.
Is there any way of achieving it? or is there any way to accommodate such a situation?
R novice here, would really appreciate some guidance. :)
Assuming the input is the data frame DF shown reproducibly in the Note at the end, convert it to a zoo object and then use as.ts to create a ts series with frequency 52.
library(zoo)
week <- as.integer(DF[[1]])
year <- as.numeric(sub("...", "", DF[[1]]))
z <- zoo(DF[[2]], year + (week - 1) / 52)
tt <- as.ts(z)
tt
## Time Series:
## Start = c(2020, 1)
## End = c(2020, 52)
## Frequency = 52
## [1] 39.12727 33.34545 118.71818 83.71818 58.56985 NA NA
## [8] NA NA NA NA NA NA NA
## [15] NA NA NA NA NA NA NA
## [22] NA NA NA NA NA NA NA
## [29] NA NA NA NA NA NA NA
## [36] NA NA NA NA NA NA NA
## [43] NA NA NA NA NA NA NA
## [50] NA NA 89.54652
frequency(tt)
## [1] 52
class(tt)
## [1] "ts"
Note
Lines <- " Week-Year Units
01.2020 39.12727273
02.2020 33.34545455
03.2020 118.7181818
04.2020 83.71818182
05.2020 58.56985
52.2020 89.54651534"
DF <- read.table(text = Lines, header = TRUE, colClasses = c("character", NA))
Related
I'm trying to convert a column of time (which I imported from Excel) that R has converted into a decimal/character string back into hh:mm:ss time. I have seen many good answers (using library chron, for example), but I keep getting these errors:
My data:
> head(env$Time, 10)
[1] "0.41736111111111113" "0.6020833333333333" "0.45" "0.47222222222222227" "0.5131944444444444"
[6] "0.51250000000000007" "0.47361111111111115" "0.44791666666666669" "0.35138888888888892" "0.45277777777777778"
times(env$Time)
Error in convert.times(times., fmt) : format h:m:s may be incorrect
In addition: Warning message:
In unpaste(times, sep = fmt$sep, fnames = fmt$periods, nfields = 3) :
8173 entries set to NA due to wrong number of fields
chron(times(env$Time))
Error in convert.times(times., fmt) : format h:m:s may be incorrect
In addition: Warning message:
In unpaste(times, sep = fmt$sep, fnames = fmt$periods, nfields = 3) :
8173 entries set to NA due to wrong number of fields
strptime(env$Time, format = "%H:%M:%S")
[1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[38] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
Found this answer here: How to express a decimal time in HH:MM
x <- as.numeric(env$Time) # Store time variable as numeric vector
env$Time2 <- sub(":00$", "", round(times(x), "min")) # Run this code to save as new column in dataframe (note, don't need to divide by 24 if decimal is fraction of a day like my data is
I want to convert those dates as this format "1995-01", "1995-02", etc.
Here is some of my data
Date Change
1 January-1995 0.01417476
2 February-1995 0.01427050
3 March-1995 0.01556348
4 April-1995 0.01644737
5 May-1995 0.01603727
6 June-1995 0.01627500
7 July-1995 0.01557800
8 August-1995 0.01429773
9 September-1995 0.01344300
10 October-1995 0.01334667
11 November-1995 0.01328429
12 December-1995 0.01345368
13 January-1996 0.01293091
14 February-1996 0.01301762
15 March-1996 0.01289048
16 April-1996 0.01268476
17 May-1996 0.01287364
18 June-1996 0.01253400
19 July-1996 0.01254591
20 August-1996 0.01271238
21 September-1996 0.01245700
22 October-1996 0.01201636
23 November-1996 0.01191300
24 December-1996 0.01195600
I tried this :
date <- as.Date(Data$Date,format="%B/%Y")
and this
date <- as.Date(paste0("01/", Data$Date),format = "%m/%Y")
But it juste return me
[1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
I'm stuck!
The last option can be modified to include the %d for day when we are pasteing the day. Also, in format, specify the correct delimiter and %B- (Full month name) where as %m- month as decimal number)
as.Date(paste0("01/", Data$Date),format = "%d/%B-%Y")
Or use lubridate
library(lubridate)
my(Data$Date)
my("January-1995")
[1] "1995-01-01"
Here is another base R option, but with output of type string (not Date)
x <- c("January-1995", "February-1996")
paste0(
gsub("\\D", "", x),
"-",
sprintf("%02d", match(gsub("-.*", "", x), month.name))
)
which gives
[1] "1995-01" "1996-02"
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 5 years ago.
Improve this question
dates <- as.Date(dli$Dates)
class(dates)
[1] "Date"
dates
[1] "2016-01-01" "2016-01-02" "2016-01-03" "2016-01-04" "2016-01-05" "2016-01-06"
[7] "2016-01-07" "2016-01-08" "2016-01-09" "2016-01-10" "2016-01-11" "2016-01-12"
[13] "2016-01-13" "2016-01-14" "2016-01-15" "2016-01-16" "2016-01-17" "2016-01-18"
[19] "2016-01-19" "2016-01-20" "2016-01-21" "2016-01-22" "2016-01-23" "2016-01-24"
[25] "2016-01-25" "2016-01-26" "2016-01-27" "2016-01-28" "2016-01-29" "2016-01-30"
[31] "2016-01-31" "2016-02-01" "2016-02-02" "2016-02-03" "2016-02-04" "2016-02-05"
[37] "2016-02-06" "2016-02-07" "2016-02-08" "2016-02-09" "2016-02-10" "2016-02-11"
This is my date format , so i need to convert it into "2016-month-day"
I am getting NA values
dates <- as.Date(dli$Dates,"%d/%b/%Y")
class(dates)
[1] "Date"
dates
[1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[31] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[61] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[91] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[121] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
can you give any suggestions
Thanks in Advance
Good practice is to store date in R as YYYY-MM-DD, and your strings already seem to be at the good format, but :
The format you're passing to as.Date must describe what the strings contain, not what you're expecting as an output.
"%d/%b/%Y" stands for "day as a number (0-31) slash abbreviated month slash 4-digit year", and your strings format are "4-digit year - month as a number - day as a number".
If you want to format the date, you need to call format :
> date <- "2016-01-01"
> date <- as.Date(date, format = "%Y-%m-%d")
> date
[1] "2016-01-01"
> format(date, "%d/%b/%Y")
[1] "01/jan/2016"
To obtain your required format i.e., 2016-month-day , you can use format function once you have converted vector of strings to Date type.
I hope below code snippet clears your doubt.
> d = c("2016-02-08","2016-02-18","2015-02-08","2016-02-02")
> class(d)
[1] "character"
> d = as.Date(d)
> class(d)
[1] "Date"
> d = format(d,"%Y-%b-%d")
> d
[1] "2016-Feb-08" "2016-Feb-18" "2015-Feb-08" "2016-Feb-02"
Format function converts the date type objects into the required format. Refer to this link for more information on date type formatting.
If you just want to render your dates in this format, then use format:
x <- as.Date("2016-01-01")
format(x, "%Y %b %a %d")
[1] "2016 Jan Fri 01"
There is a separation of concerns here. If you already have your date information stored in R as date types, then you need not change anything internally to extract further information from those dates.
Demo
You would use as.Date() to convert between dates saved as character and Date objects.
If you want to change the format of a Date object, you can use format().
You have specified "2016-month-day" as the desired format of the dates in the question, but in the code you provide you are using "%d/%b/%Y". The way this works is: the % indicates that the next character will be a conversion specification, everything else (e.g. (- or /) will be used for finding / adding delimiter to the date representation. (see ?strptime for details).
So in your case, just use
dates <- format(dli$Dates, format = "%Y-%b-%d")
to get the result specified in the text of the question:
[1] "2016-Jan-01" "2016-Jan-02" "2016-Jan-03" "2016-Jan-04" "2016-Jan-05"
or this:
dates <- format(dli$Dates, format = "%Y/%b/%d")
to get what you have used in the code snipped:
[1] "2016/Jan/01" "2016/Jan/02" "2016/Jan/03" "2016/Jan/04" "2016/Jan/05"
You can use the package lubridate to convert to a date with ymd then format it for the way you want it displayed
Dates_df <- mutate(Dates, dli = format(ymd(dli), "%Y/%b/%d")
(I use dplyr here, I assume you have other variables in Dates)
without dplyr if you just want to keep the dates in a vector:
Dates_vec <- format(ymd(Dates$dli), "%Y/%b/%d")
when creating an xtsobject from a data.frame I seem to lose some data (approximately 3000 data lost over 33 000).
My dataset is as follow: (with the time being day-month-year, EU format)
> head(mesdonnees)
time value
1 05-03-2006 04:07 NA
2 05-03-2006 04:17 NA
3 05-03-2006 04:27 NA
4 05-03-2006 04:37 NA
5 05-03-2006 04:47 NA
6 05-03-2006 04:57 NA
Due to the format I had to extract the different parts of the date (at least I couldn't get as.POSIXct to work with this format).
Here is how I did it:
# Extract characters and define as S....
Syear <- substr(mesdonnees$time, 7,10)
Smonth <- substr(mesdonnees$time, 4,5)
Sday <- substr(mesdonnees$time, 1, 2)
#Gather all parts and use "-" as sep
datetext <- paste(Syear, Smonth, Sday, sep="-")
#define format of each part of the string
formatdate<-as.POSIXct(datetext, format="%Y-%m-%d", tz = "GMT")
I then try to create my xtswith...
xtsdata <- xts(mesdonnees$value, order.by = formatdate, tz = "GMT")
... but when doing this I get some quite weird results: the first value is in 1900
> head(xtsdata)
[,1]
1900-01-04 NA
2006-03-05 NA
2006-03-05 NA
2006-03-05 NA
2006-03-05 NA
2006-03-05 NA
and many (3000) dates are not kept:
> xtsdata[30225:30233,]
[,1]
2006-12-31 0
2006-12-31 0
2006-12-31 0
2006-12-31 0
<NA> NA
<NA> NA
<NA> NA
<NA> NA
<NA> NA
When looking at what should be the same line in both my data.frameand my xtsI can see that the lines are offset (I had the date format changed in the xts object creation):
> mesdonnees[25617,]
time value
25617 08-11-2006 23:51 0
> xtsdata[25617,]
[,1]
2006-11-25 0.27
How is it that my data are offset? I tried changing the tz but it doesn't affect it. I removed all duplicates using the dyplr package, it doesn't affect the xts results either. Thank you for your help !
After changing my xts code to the one suggested by Joshua:
xtsdata <- xts(mesdonnees$value, order.by = as.POSIXct(mesdonnees$time, tz = "GMT", format = "%d-%m-%Y %H:%M"))
... my data show properly for the "last" part, but I now have a different problem. The first 2300 data show the following results when doing (using xtsdata[1500,] (or any row < 2300) displays the same results)
> view(xtsdata):
0206-06-30 23:08:00 NA
0206-06-30 23:18:00 NA
0206-06-30 23:28:00 NA
1900-01-04 12:00:00 NA
2006-03-05 04:07:00 NA
2006-03-05 04:17:00 NA
I noticed this error before and thought it was due to the date format; maybe it is not? Also, when looking at the xtsdata I do not get the same results for the same row (the last rows are correct thought):
> mesdonnees[2360,]
time value
2360 23-03-2006 03:09 NA
> xtsdata[2360,]
[,1]
2006-03-05 09:07:00 NA
As requested:
> str(mesdonnees)
'data.frame': 32556 obs. of 2 variables:
$ time : chr "05-03-2006 04:07" "05-03-2006 04:17" "05-03-2006 04:27" "05-03-2006 04:37" ...
$ value: num NA NA NA NA NA NA NA NA NA NA ...
And if needed:
An ‘xts’ object on 0206-06-01 00:09:00/2006-12-31 23:29:00 containing:
Data: num [1:32556, 1] NA NA NA NA NA NA NA NA NA NA ...
Indexed by objects of class: [POSIXct,POSIXt] TZ: GMT
xts Attributes:
NULL
The problem is that you only include the date portion of the timestamp in datetext and formatdate, but your data have dates and times.
You also do not need to do all the string subsetting. You can achive the same result by specifying the format argument in your as.POSIXct call.
xtsdata <- xts(mesdonnees$value,
as.POSIXct(mesdonnees$times, "GMT", format = "%d-%m-%Y %H:%M")
I wonder how I can make use of an already existing date field when creating a ts in R.
Sometimes you simply have a date before you have a ts object, e.g.
x <- as.Date("2008-01-01") + c(30,60,90,120,150)
# add some data to it
df = data.frame(datefield=x,test=1:length(x))
Now, is there a way to use the datefield of the df to as an index when creating a ts object? Because:
ts(df$test,start=c(2008,1,2),frequency=12)
(obviuously) completely ignores the date information I already have. Making use of ts methods like acf is the reason why I´d like to make it a ts object. I typcically use monthly an quarterly time series...
You don't necessarily need to create new types of objects from scratch; you can always coerce to other classes, including ts as you need to. zoo or xts are arguably to most useful and intuitive but there are others. Here is your example, cast as a zoo object, which we then coerce to class ts for use in acf().
## create the data
x <- as.Date("2008-01-01") + c(30,60,90,120,150)
df = data.frame(datefield=x,test=1:length(x))
## load zoo
require(zoo)
## convert to a zoo object, with order given by the `datefield`
df.zoo <- with(df, zoo(test, order.by = x))
## or to a regular zoo object
df.zoo2 <- with(df, zooreg(test, order.by = x))
Now we can easily go to a ts object using the as.ts() method:
> as.ts(df.zoo)
Time Series:
Start = 13920
End = 14040
Frequency = 0.0333333333333333
[1] 1 2 3 4 5
> ## zooreg object:
> as.ts(df.zoo2)
Time Series:
Start = 13909
End = 14029
Frequency = 1
[1] 1 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[21] NA NA NA NA NA NA NA NA NA NA 2 NA NA NA NA NA NA NA NA NA
[41] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[61] 3 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[81] NA NA NA NA NA NA NA NA NA NA 4 NA NA NA NA NA NA NA NA NA
[101] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
[121] 5
Notice the two ways in which the objects are represented (although we could have made the zooreg version the same as the standard zoo object by setting the frequency argument to 0.03333333):
> as.ts(with(df, zooreg(test, order.by = datefield,
+ frequency = 0.033333333333333)))
Time Series:
Start = 13920.0000000001
End = 14040.0000000001
Frequency = 0.033333333333333
[1] 1 2 3 4 5
We can use the zoo/zooreg object in acf() and it will get the correct lags (daily observations but every 30 days):
acf(df.zoo)
Whether this is intuitive to you or not depends on how you view the time series. We can do the same thing in terms of a 30-day interval via:
acf(coredata(df.zoo))
where we use coredata() to extract the time series itself, ignoring the date information.
I don't know exactly what you're trying to do, but acf also works on simple vectors, given off course it represents a regular time series (i.e. even spaced). Otherwise the result is just bollocks.
>acf(df$test)
Regarding the ts object :
The "dates" you see are just from the print.ts function, so they're not inherent to the ts object. The ts object has no date information in it. You can set the option calender=FALSE to get the standard print out of the ts object.
> ts(df$test,start=2008,frequency=12)
Jan Feb Mar Apr May
2008 1 2 3 4 5
> print(ts(df$test,start=2008,frequency=12),calendar=F)
Time Series:
Start = c(2008, 1)
End = c(2008, 5)
Frequency = 12
[1] 1 2 3 4 5
Now, the vector you construct looks like :
> x
[1] "2008-01-31" "2008-03-01" "2008-03-31" "2008-04-30" "2008-05-30"
which is or isn't regular, depending on how you see it. If you extract the months, then you have 1 observation for january, 2 for march, 1 for april...: not regular. You have an observation every 30 days : regular. If you have an observation every 30 days, you shouldn't bother about the dates as 365 is not dividable through 30. Hence, one year you'll have 12 observations, another one you'll have 13 observations. So you can't set the frequency in ts in a consequent correct way.
So I'd refrain from using a ts all together, as James already indicated in the comments.
If you want:
Use the date information you already have
Easily set the frequency to a desired value
End up with a ts object
You can start with an xts object, add a frequency attribute, and then convert to ts:
library("xts")
my_xts <- xts(df$test, df$datefield)
attr(my_xts, 'frequency') <- 12 # Set the frequency
my_ts <- as.ts(my_xts)
The resulting ts object will have the specified period and will have the correct dates assigned to each data point.