Okay so here is a subtle "quirk" in the r as.Date function converting from a POSIXct with a timezone, which I am wondering if it is a bug.
> as.POSIXct("2013-03-29", tz = "Europe/London")
[1] "2013-03-29 GMT"
> as.Date(as.POSIXct("2013-03-29", tz = "Europe/London"))
[1] "2013-03-29"
So far no problem, but.....
> as.POSIXct("2013-04-01", tz = "Europe/London")
[1] "2013-04-01 BST"
> as.Date(as.POSIXct("2013-04-01", tz = "Europe/London"))
[1] "2013-03-31"
Anybody seen this? Is this a bug or another quirk? April fools?
The default time zone for as.Date.POSIXct is "UTC" (see the help page). Try as.Date(as.POSIXct("2013-04-01", tz = "Europe/London"),tz = "Europe/London").
Related
Here is my example.
test <- as.POSIXct(as.Date("2019-11-01"), tz = "UTC")
test
It prints:
[1] "2019-10-31 19:00:00 CDT"
It looks like it ignored tz parameter:
attr(test, "tzone")
returns NULL.
Why is it coming with "19" hours and not 00? How can I make it 00 hours and take UTC?
UPDATE
Here is even better case:
test_2 <- as.POSIXct("2019-11-01 00:00:00", tz = "UTC")
str(test_2)
attr(test_2, "tzone")
strftime(test_2, "%H")
It generates:
POSIXct[1:1], format: "2019-11-01"
[1] "UTC"
[1] "19"
Now it looks like parameter tz is not ignored, but hour is 19, but why?
We can use with_tz from lubridate
library(lubridate)
test1 <- with_tz(as.Date("2019-11-01"), tzone = 'UTC')
attr(test1, 'tzone')
#[1] "UTC"
Also, as.POSIXct can be directly applied
test2 <- as.POSIXct("2019-11-01", tz = "UTC")
test2
#[1] "2019-11-01 UTC"
attr(test2, 'tzone')
#[1] "UTC"
With strftime, use the option for tz
strftime(test2, "%H", tz = 'UTC')
#[1] "00"
If we check the strftime, it is doing a second conversion with as.POSIXlt and then with format gets the formatted output
strftime
function (x, format = "", tz = "", usetz = FALSE, ...)
format(as.POSIXlt(x, tz = tz), format = format, usetz = usetz,
...)
According to ?as.POSIXlt
tz - time zone specification to be used for the conversion, if one is required. System-specific (see time zones), but "" is the current time zone, and "GMT" is UTC (Universal Time, Coordinated). Invalid values are most commonly treated as UTC, on some platforms with a warning.
as.POSIXlt(test2)$hour
#[1] 0
as.POSIXlt(test2, tz = "")$hour
#[1] 20
The "" uses the Sys.timezone by default
as.POSIXlt(test2, tz = Sys.timezone())$hour
#[1] 20
timestamp = 1491800340000
I'm having trouble with some date manipulation in R. The timestamp above is:
2017-04-10T04:59:00.000 GMT
2017-04-09T23:59:00.000 America/Bogota (Local time)
I want to round it to 2017-04-09T00:00:00.000 GMT because my daily aggregations are set to 00:00 GMT.
How can I do that?
Here's what I tried:
> Sys.timezone()
[1] "America/Bogota"
> timestamp = 1491800340000
> date = strptime(timestamp / 1000, "%s");
[1] "2017-04-09 23:59:00 COT"
> midnightLocal = trunc(date, "day");
[1] "2017-04-09 COT"
> midnightUTC = strptime(format(midnightLocal, "%Y-%m-%d"), "%Y-%m-%d", tz = "UTC");
[1] "2017-04-09 UTC"
> truncatedtimestamp = as.integer(format(midnightUTC, "%s"));
[1] 1491714000
which is 2017-04-09T05:00:00.000 GMT (not midnight as I expected). Looks like I failed to specify the timezone somewhere?
I tried many things like POSIXct but did not succeed.
Any hint is appreciated!
Cheers
ps: I'd prefer not to install any package
A little trickery:
timestamp = 1491800340000
ts <- as.POSIXct(timestamp / 1000, origin = "1970-01-01 00:00:00 GMT")
ts2 <- as.Date(trunc(ts, "day"))
attr(ts2, "tzone") <- "GMT"
format(ts2, "%Y-%m-%d %H:%M:%S %Z") # to prove it's midnight
# [1] "2017-04-09 00:00:00 UTC"
class(ts2)
# [1] "Date"
From here you have a couple of options: a little brute-force (numeric conversion) or perhaps the more time-friendly/safe way.
Brute-force numeric:
ts3a <- as.numeric(ts2) * 60*60*24
ts3a
# [1] 1491696000
as.POSIXct(ts3a, origin = "1970-01-01 00:00:00 GMT", tz = "GMT")
# [1] "2017-04-09 GMT"
Time-friendly/safe:
ts3b <- as.POSIXct(ts2)
attr(ts3b, "tzone") <- "GMT"
ts3b
# [1] "2017-04-09 GMT"
(Since they are POSIXct, it's showing the date only because it is midnight; you can easily prove it's correct.)
I am puzzled with this result:
a = "2008-03-03 12:30:38"
#I convert to POSIXct and set the timezone
dt = as.POSIXct(a, format="%Y-%m-%d %H:%M:%S", tz='Europe/Paris')
dt
[1] "2008-03-03 12:30:38 CET"
unclass(dt)
[1] 1204543838
attr(,"tzone")
[1] "Europe/Paris"
#I want to come back to POSIXct
as.POSIXct(unclass(dt), origin='1970-01-01', tz='Europe/Paris')
[1] "2008-03-03 11:30:38 CET"
I would have expected to get back the date-time a, what is wrong here ?
As it says in ?as.POSIXct, the origin is in tz="GMT".
You can use .POSIXct instead:
.POSIXct(unclass(dt), tz='Europe/Paris')
# [1] "2008-03-03 12:30:38 CET"
You may want to check the attribute:
isdst
To see if there is some Daylight Savings conversion going on in there somewhere.
This page from the R manual on Date-Time Classes may be useful
I need to use as.Date on the index of a zoo object. Some of the dates are in BST and so when converting I lose a day on (only) these entries. I don't care about one hour's difference or even the time part of the date at all, I just want to make sure that the dates displayed stay the same. I'm guessing this is not very hard but I can't manage it. Can somebody help please?
class(xtsRet)
#[1] "xts" "zoo"
index(xtsRet)
#[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
class(index(xtsRet))
#[1] "POSIXt" "POSIXct"
index(xtsRet) <- as.Date(index(xtsRet))
index(xtsRet)
#[1] "2007-07-30" "2007-08-30" "2007-09-29" "2007-10-31"
Minimally reproducible example (not requiring zoo package):
my_date <- as.POSIXct("2007-04-01") # Users in non-UK timezone will need to
# do as.POSIXct("2007-04-01", "Europe/London")
my_date
#[1] "2017-04-01 BST"
as.Date(my_date)
#[1] "2017-03-31"
Suppose we have this sample data:
library(zoo)
x <- as.POSIXct("2000-01-01", tz = "GMT")
Then see if any of these are what you want:
# use current time zone
as.Date(as.character(x, tz = ""))
# use GMT
as.Date(as.character(x, tz = "GMT"))
# set entire session to GMT
Sys.setenv(TZ = "GMT")
as.Date(x)
Also try "BST" in place of "GMT" and note the article on dates and times in R News 4/1 .
You can offset the POSIX objects so its not based around midnight. 1 hour (3600 secs) should be sufficient:
d <- as.POSIXct(c("2007-07-31","2007-08-31","2007-09-30","2007-10-31"))
d
[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
as.Date(d)
[1] "2007-07-30" "2007-08-30" "2007-09-29" "2007-10-31"
as.Date(d+3600)
[1] "2007-07-31" "2007-08-31" "2007-09-30" "2007-10-31"
I would suggest using as.POSIXlt to convert to a date object, wrapped in as.Date:
d <- as.POSIXct(c("2007-07-31","2007-08-31","2007-09-30","2007-10-31"))
d
[1] "2007-07-31 BST" "2007-08-31 BST" "2007-09-30 BST" "2007-10-31 GMT"
as.Date(as.POSIXlt(d))
[1] "2007-07-31" "2007-08-31" "2007-09-30" "2007-10-31"
Achieves the same thing as the +3600 above, but slightly less of a hack
I'm trying to convert a yearmon date (from the zoo package) to a POSIXct in the UTC timezone.
This is what I tried to do:
> as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
[1] "2010-01-01 01:00:00 CET"
I get the same when I convert a Date:
> as.POSIXct(as.Date("2010-01-01"),tz="UTC")
[1] "2010-01-01 01:00:00 CET"
The only way to get it to work is to pass a character as an argument:
> as.POSIXct("2010-01-01", tz="UTC")
[1] "2010-01-01 UTC"
I looked into the documentation of DateTimeClasses, tzset and timezones. My /etc/localtime is set to Europe/Amsterdam. I couldn't find a way to set the tz to UTC, other than setting the TZ environment variable:
> Sys.setenv(TZ="UTC")
> as.POSIXct(as.Date("2010-01-01"),tz="UTC")
[1] "2010-01-01 UTC"
Is it possible to directly set the timezone when creating a POSIXct from a yearmon or Date?
Edit:
I checked the functions as.POSIXct.yearmon. This one passes to the as.POSIXct.Date.
> zoo:::as.POSIXct.yearmon
function (x, tz = "", ...)
as.POSIXct(as.Date(x), tz = tz, ...)
<environment: namespace:zoo>
So like Joshua says the timezone gets lost in the as.POSIXct.Date. For now I'll use Richies suggestion to set the tzone by hand using:
attr(x, "tzone") <- 'UTC'
This solves the issue of the lost tzone, which is only used for presentation and not internally like Grothendieck and Dwin suggested.
This is because as.POSIXct.Date doesn't pass ... to .POSIXct.
> as.POSIXct.Date
function (x, ...)
.POSIXct(unclass(x) * 86400)
<environment: namespace:base>
You are setting the timezone correctly in your code. The problem you are perceiving is only at the output stage. POSIX values are all referenced to UTC/GMT. Dates are assumed to be midnight times. Midnight UTC is 1 AM CET ( which is apparently where you are).
> as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
[1] "2009-12-31 19:00:00 EST" # R reports the time in my locale's timezone
> dtval <- as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
> format(dtval, tz="UTC") # report the date in UTC note it is the correct date ... there
[1] "2010-01-01"
> format(dtval, tz="UTC", format="%Y-%m-%d ")
[1] "2010-01-01 " # use a format string
> format(dtval, tz="UTC", format="%Y-%m-%d %OS3")
[1] "2010-01-01 00.000" # use decimal time
See ?strptime for many, many other format possibilities.
In the help page ?as.POSIXct, for the tz argument it says
A timezone specification to be used
for the conversion, if one is
required. System-specific (see time
zones), but ‘""’ is the current
timezone, and ‘"GMT"’ is UTC
(Universal Time, Coordinated).
Does as.POSIXct(as.yearmon("2010-01-01"), tz="GMT") work for you?
After more perusal of the documentation, in the details section we see:
Dates without times are treated as
being at midnight UTC.
So in your example, the tz argument is ignored. If you use as.POSIXlt it is easier to see what happens with the timezone. The following should all give the same answer, with UTC as the timezone.
unclass(as.POSIXlt(as.yearmon("2010-01-01")))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "UTC"))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "GMT"))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "CET"))
In fact, since you are using as.yearmon (which strips the time out) you will never get to set the timezone. Compare, e.g.,
unclass(as.POSIXlt(as.yearmon("2010-01-01 12:00:00"), tz = "CET"))
unclass(as.POSIXlt("2010-01-01 12:00:00", tz = "CET"))
This seems to be an oddity with the date/time "POSIXct" class methods. Try formatting the "Date" or "yearmon" variable first so that as.POSIXct.character rather than as.POSIXct.{Date, yearmon} is dispatched:
Date
> d <- as.Date("2010-01-01")
> as.POSIXct(format(d), tz = "UTC")
[1] "2010-01-01 UTC"
yearmon
> library(zoo)
> y <- as.yearmon("2010-01")
> as.POSIXct(format(y, format = "%Y-%m-01"), tz = "UTC")
[1] "2010-01-01 UTC"
> # or
> as.POSIXct(format(as.Date(y)), tz = "UTC")
[1] "2010-01-01 UTC"