I have dates encoded in a weekly time format (European convention >> 01 through 52/53, e.g. "2016-48") and would like to standardize them to a POSIX date:
require(magrittr)
(x <- as.POSIXct("2016-12-01") %>% format("%Y-%V"))
# [1] "2016-48"
as.POSIXct(x, format = "%Y-%V")
# [1] "2016-01-11 CET"
I expected the last statement to return "2016-12-01" again. What am I missing here?
Edit
Thanks to Dirk, I was able to piece it together:
y <- sprintf("%s-1", x)
While I still don't get why this doesn't work
(as.POSIXct(y, format = "%Y-%V-%u"))
# [1] "2016-01-11 CET"
this does
(as.POSIXct(y, format = "%Y-%U-%u")
# [1] "2016-11-28 CET"
Edit 2
Oh my, I think using %V is a very bad idea in general:
as.POSIXct("2016-01-01") %>% format("%Y-%V")
# [1] "2016-53"
Should this be considered to be on a "serious bug" level that requires further action?!
Sticking to either %U or %W seems to be the right way to go
as.POSIXct("2016-01-01") %>% format("%Y-%U")
# [1] "2016-00"
Edit 3
Nope, not quite finished/still puzzled: the approach doesn't work for the very first week
(x <- as.POSIXct("2016-01-01") %>% format("%Y-%W"))
# [1] "2016-00"
as.POSIXct(sprintf("%s-1", x), format = "%Y-%W-%u")
# [1] NA
It does for week 01 as defined in the underlying convention when using %U or %W (so "week 2", actually)
as.POSIXct("2016-01-1", format = "%Y-%W-%u")
# [1] "2016-01-04 CET"
As I have to deal a lot with reporting by ISO weeks, I've created the ISOweek package some years ago.
The package includes the function ISOweek2date() which returns the date of a given weekdate (year, week of the year, day of week according to ISO 8601). It's the inverse function to date2ISOweek().
With ISOweek, your examples become:
library(ISOweek)
# define dates to convert
dates <- as.Date(c("2016-12-01", "2016-01-01"))
# convert to full ISO 8601 week-based date yyyy-Www-d
(week_dates <- date2ISOweek(dates))
[1] "2016-W48-4" "2015-W53-5"
# convert back to class Date
ISOweek2date(week_dates)
[1] "2016-12-01" "2016-01-01"
Note that date2ISOweek() requires a full ISO week-based date in the format yyyy-Www-d including the day of the week (1 to 7, Monday to Sunday).
So, if you only have year and ISO week number you have to create a character string with a day of the week specified.
A typical phrase in many reports is, e.g., "reporting week 31 ending 2017-08-06":h
yr <- 2017
wk <- 31
ISOweek2date(sprintf("%4i-W%02i-%1i", yr, wk, 7))
[1] "2017-08-06"
Addendum
Please, see this answer for another use case and more background information on the ISOweek package.
Related
I have a vector of date strings in the form month_name-2_digit_year i.e.
a = rbind("April-21", "March-21", "February-21", "January-21")
I'm trying to convert that vector into a vector of date objects. I'm aware this question is very similar to this: Convert non-standard date format to date in R posted some years ago, but unfortunately, it has not answered my question.
I have tried the following as.Date() calls to do this, but it just returns a vector of NA. I.e.
b = as.Date(a, format = "%B-%y")
b = as.Date(a, format = "%B%y")
b = as.Date(a, "%B-%y")
b = as.Date(a, "%B%y")
I'm also attempted to do it using the convertToDate function from the openxlsx package:
b = convertToDate(a, format = "%B-%y")
I have also tried all the above but using a single character string rather than a vector, but that produced the same issue.
I'm a little lost as to why this isn't working, as this format has worked in reverse earlier in my script (that is, I had a date object already in dd-mm-yyyy format and converted it to month_name-yy using %B-%y). Is there another way to go from string to date when the string is a non-standard (anything other than dd-mm-yyy or mm-dd-yy if you're in the US) date format?
For the record my R locales are all UK and english.
Thanks in advance.
A Date must have all three of day, month and year. Convert to yearmon class which requires only month and year and then to Date as in (1) and (2) below or add the day as in (3).
(1) and (3) give first of month and (2) gives the end of the month.
(3) uses only functions from base R.
Also consider not converting to Date at all but just use yearmon objects instead since they directly represent a year and month which is what the input represents.
library(zoo)
# test input
a <- c("April-21", "March-21", "February-21", "January-21")
# 1
as.Date(as.yearmon(a, "%B-%y"))
## [1] "2021-04-01" "2021-03-01" "2021-02-01" "2021-01-01"
# 2
as.Date(as.yearmon(a, "%B-%y"), frac = 1)
## [1] "2021-04-30" "2021-03-31" "2021-02-28" "2021-01-31"
# 3
as.Date(paste(1, a), "%d %B-%y")
## [1] "2021-04-01" "2021-03-01" "2021-02-01" "2021-01-01"
In addition to zoo, which #G. Grothendieck mentioned, you can also use clock or lubridate.
clock supports a variable precision calendar type called year_month_day. In this case you'd want "month" precision, then you can set the day to whatever you'd like and convert back to Date.
library(clock)
x <- c("April-21", "March-21", "February-21", "January-21")
ymd <- year_month_day_parse(x, format = "%B-%y", precision = "month")
ymd
#> <year_month_day<month>[4]>
#> [1] "2021-04" "2021-03" "2021-02" "2021-01"
# First of month
as.Date(set_day(ymd, 1))
#> [1] "2021-04-01" "2021-03-01" "2021-02-01" "2021-01-01"
# End of month
as.Date(set_day(ymd, "last"))
#> [1] "2021-04-30" "2021-03-31" "2021-02-28" "2021-01-31"
The simplest solution may be to use lubridate::my(), which parses strings in the order of "month then year". That assumes that you want the first day of the month, which may or may not be correct for you.
library(lubridate)
x <- c("April-21", "March-21", "February-21", "January-21")
# Assumes first of month
my(x)
#> [1] "2021-04-01" "2021-03-01" "2021-02-01" "2021-01-01"
I have a list of days in the format "day of the year" obtained by applying lubridate::yday() function to a list of dates. For instance, from the following dates (mm-dd-yyyy format):
01-01-2015
01-02-2015
...
by applying yday() you get
1
2
...
Is there a function that can do the reverse given the yday output and the year? Ie, from a yday value and a year, get back to a date in the mm-dd-yyyy format?
To do this with the lubridate package, you use the parse_date_time() function with j as the order = argument.
Example:
my.yday <- "1"
parse_date_time(x = my.yday, orders = "j")
# [1] "2021-01-01 UTC"
The default appears to be the current year. If you want to specify the year just add it in!
my.yday <- "1"
parse_date_time(x = paste(1995, my.yday), orders = "yj")
# [1] "1995-01-01 UTC"
Note: make sure you supply the julian day as character, not numeric. If numeric it will fail when over 3 digits (weird bug that they don't plan to fix)
Any sequence added to a Date() type creates a new Date() sequence with just that offset.
Witness:
R> as.Date("2016-01-01") + 0:9
[1] "2016-01-01" "2016-01-02" "2016-01-03"
[4] "2016-01-04" "2016-01-05" "2016-01-06"
[7] "2016-01-07" "2016-01-08" "2016-01-09"
[10] "2016-01-10"
R> as.Date("2016-01-01") + 100:109
[1] "2016-04-10" "2016-04-11" "2016-04-12"
[4] "2016-04-13" "2016-04-14" "2016-04-15"
[7] "2016-04-16" "2016-04-17" "2016-04-18"
[10] "2016-04-19"
R>
So once again a so-called lubridate question as nothing to do with that package but simply requires to know how the base R types function.
> yday ("1990-03-17") - 1 + as.Date ("1990-01-01")
[1] "1990-03-17"
Just as round out the answer by #DirkEddelbuettel, since I was having the same issue and its been 3 years.
In order to get the desired mm-dd-yyyy for a given dataset where you have only the year and the calendar day as a result of lubridate::yday().
You just need to offset a Date() type object for that given year starting at january 1st, by what your yday() output is subtracted by one.
Then if you want to get the month or day within that month you can use lubridate's month() and day() functions to pull those parts.
(I assume you have the year since that will impact the calendar date b/c of leap years, messing up your month/day assignment. If not, any year will do)
library(dplyr)
library(magrittr)
#Example dataset with years on/around leap year
my_df <- data.frame(
year = c(2010, 2011, 2012, 2013, 2014, 2015),
my_yday = c(150, 150, 150, 150, 150, 150)
)
#skip straight back to the yyyy-mm-dd format
my_df %>% mutate(new_date = as.Date(paste0(year, "-01-01")) + (my_yday - 1))
#Get every component
my_df %>%
mutate(
new_day = lubridate::day(as.Date(str_c(year, "-01-01")) + (my_yday - 1)),
new_month = lubridate::month(as.Date(paste0(year, "-01-01")) + (my_yday - 1)),
new_date = as.POSIXct(str_c(new_month, new_day, year, sep = "/"),
format = "%m/%d/%y"))
This question already has an answer here:
R conversion of week number (UK) to POSIXct issue
(1 answer)
Closed 9 years ago.
I have a POSIXct variable with the value "2012-04-15 16:49:36 CEST". The format function returns the year, week of the year and the weekday in decimal numbers, for this example 2012 15 0. The description of the format for those less familiar with it:
%Y: Year with century.
%W: Week of the year as decimal number
(00–53) using Monday as the first day of week (and typically with the
first Monday of the year as day 1 of week 1). The UK convention.
%w: Weekday as decimal number (0–6, Sunday is 0).
Then, I try to convert the values back to a POSIXct variable and something unexpected happens. When I read the values, the functions seems to interpret a wrong date (2012-04-08). However, the surprise comes when I do the same procedure with a second example using Sys.time() and it works as expected. Can someone explain me why it does not work in the first example?
(TS <- structure(1334501376, class = c("POSIXct", "POSIXt")))
(TS_YWw <- format(TS,format="%Y %W %w"))
as.POSIXct(TS_YWw,format="%Y %W %w")
(TS <- Sys.time())
(TS_YWw <- format(TS,format="%Y %W %w"))
as.POSIXct(TS_YWw,format="%Y %W %w")
Output
> (TS <- structure(1334501376, class = c("POSIXct", "POSIXt")))
[1] "2012-04-15 16:49:36 CEST"
> (TS_YWw <- format(TS,format="%Y %W %w"))
[1] "2012 15 0"
> as.POSIXct(TS_YWw,format="%Y %W %w")
[1] "2012-04-08 CEST"
>
> (TS <- Sys.time())
[1] "2013-05-16 15:27:44 CEST"
> (TS_YWw <- format(TS,format="%Y %W %w"))
[1] "2013 19 4"
> as.POSIXct(TS_YWw,format="%Y %W %w")
[1] "2013-05-16 CEST"
By the way, I ran the code on a Windows XP 32bit machine with R 2.15.3. Thank you all!
It seems like a bug. Below I create a sequence of the days in 2012 (dtimes) and convert to strings and back again using the '%Y %W %w' format. The two series are compared and the head output shows which datetimes weren't preserved in the conversion. There's an obvious weekly pattern. Note also that as.POSIXct('2012 0 0', '%Y %W %w') returns NA.
dtimes <- seq(as.POSIXct('2012-1-1'), as.POSIXct('2013-1-1'), by=as.difftime(1, units='days'))
convert.YWw <- function(dtime) {
fmt <- "%Y %W %w"
string <- format(dtime, format=fmt)
as.POSIXct(string, format=fmt)
}
converted <- lapply(dtimes, convert.YWw)
preserved <- dtimes == converted
dtimes.and.converted <- mapply(function(d, c) c(dtime=d, convert=c), dtimes, converted, SIMPLIFY=FALSE)
head(dtimes.and.converted[! preserved])
# [[1]]
# NULL
#
# [[2]]
# dtime convert
# "2012-01-08 EST" "2012-01-01 EST"
#
# [[3]]
# dtime convert
# "2012-01-15 EST" "2012-01-08 EST"
#
# [[4]]
# dtime convert
# "2012-01-22 EST" "2012-01-15 EST"
#
# [[5]]
# dtime convert
# "2012-01-29 EST" "2012-01-22 EST"
#
# [[6]]
# dtime convert
# "2012-02-05 EST" "2012-01-29 EST"
I have week-date data in the form yyyy-ww where wwis the week number in two digits. The data span 2007-01 to 2010-30. The week counting convention is ISO 8601, which as you can see here on Wikipedia's "Week number" article, occasionally reaches 53 weeks in a year. For example 2009 had 53 weeks by this system, see the week numbers in this ISO 8601 calendar. (See other years; as per the Wikipedia article, 53rd weeks are fairly rare.)
Basically I want to read the week date in, convert it to a Date object and save this to a separate column in a data.frame. As a test, I reconverted the Date objects to yyyy-ww formats by format([Date-object], format = "%Y-%W", and this threw up an error at 2009-53. That week fails to be interpreted as a date by R. This is very odd, as other years which do not have a 53rd week (in ISO 8601 standard) are converted fine, such as 2007-53, whereas other years that also do not have a 53rd week (in ISO 8601 standard) also fail, such as 2008-53
The following minimal example demonstrates the issue.
Minimal example:
dates <- c("2009-50", "2009-51", "2009-52", "2009-53", "2010-01", "2010-02")
as.Date(x = paste(dates, 1), format = "%Y-%W %w")
# [1] "2009-12-14" "2009-12-21" "2009-12-28" NA "2010-01-04"
# [6] "2010-01-11"
other.dates <- c("2007-53", "2008-53", "2009-53", "2010-53")
as.Date(x = paste(other.dates, 1), format = "%Y-%W %w")
# [1] "2007-12-31" NA NA NA
The question is, how do I get R to accept week numbers in ISO 8601 format?
Note: This question summarises a problem I have been struggling with for a few hours. I have searched and found various helpful posts such as this, but none solved the problem.
The package ISOweek manages ISO 8601 style week numberings, converting to and from Date objects in R. See ISOweek for more. Continuing the example dates above, we first need to modify the formatting a bit. They must be in form yyyy-Www-w rather than yyyy-ww, i.e. 2009-W53-1. The final digit identifies which day of the week to use in identifying the week, in this case it is the Monday. The week number must be two-digit.
library(ISOweek)
dates <- c("2009-50", "2009-51", "2009-52", "2009-53", "2010-01", "2010-02")
other.dates <- c("2007-53", "2008-53", "2009-53", "2010-53")
dates <- sub("(\\d{4}-)(\\d{2})", "\\1W\\2-1", dates)
other.dates <- sub("(\\d{4}-)(\\d{2})", "\\1W\\2-1", other.dates)
## Check:
dates
# [1] "2009-W50-1" "2009-W51-1" "2009-W52-1" "2009-W53-1" "2010-W01-1"
# [6] "2010-W02-1"
(iso.date <- ISOweek2date(dates)) # deal correctly
# [1] "2009-12-07" "2009-12-14" "2009-12-21" "2009-12-28" "2010-01-04"
# [6] "2010-01-11"
(iso.other.date <- ISOweek2date(other.dates)) # also deals with this
# [1] "2007-12-31" "2008-12-29" "2009-12-28" "2011-01-03"
## Check that back-conversion works:
all(date2ISOweek(iso.date) == dates)
# [1] TRUE
## This does not work for the others, since the 53rd week of
## e.g. 2008 is back-converted to the first week of 2009, in
## line with the ISO 6801 standard.
date2ISOweek(iso.other.date) == other.dates
# [1] FALSE FALSE TRUE FALSE
I'm probably doing something stupid and not seeing it, but:
> strptime("201101","%Y%m")
[1] NA
From help strptime:
%Y Year with century
%m Month as decimal number (01–12)
Just paste a day field (say, "01") that you ignore:
R> shortdate <- "201101"
R> as.Date(paste(shortdate, "01", sep=""), "%Y%m%d")
[1] "2011-01-01"
R>
I prefer as.Date() for dates and strptime() for POSIXct objects, i.e. dates and times.
You can then convert the parsed Date object into a POSIXlt object to retrieve year and month:
R> mydt <- as.Date(paste(shortdate, "01", sep=""), "%Y%m%d")
R> myp <- as.POSIXlt(mydt)
R> c(myp$year, myp$mon)
[1] 111 0
R>
This is standard POSIX behaviour with years as "year - 1900" and months as zero-indexed.
Edit seven years later: For completeness, and as someone just upvoted this, the functions in my anytime package can help:
R> anytime::anydate("201101") ## returns a Date
[1] "2011-01-01"
R> anytime::anytime("201101") ## returns a Datetime
[1] "2011-01-01 CST"
R>
The use a different parser (from Boost Date_time which is more generous and imputes the missing day (or day/hour/minute/second in the second case).