There is something I don't understand. I simply try to convert a date in an other time zone. The date is in the 8601-ISO format. I followed this.
pb.txt <- "2012-09-11T21:23:20Z"
pb.date <- as.POSIXct(pb.txt, tz="UTC")
format(pb.date, tz="WEST",usetz=TRUE)
[1] "2012-09-11 WEST"
Why only the date appears and not anymore the timestamp ?
I tried also :
pb.date <- as.POSIXct(pb.txt, origin=ISOdatetime(2012,09,11,21,23,20))
format(pb.date, tz="WEST",usetz=TRUE)
[1] "2012-09-10 22:00:00 WEST
It's better, but the timestamp is rounded. How to convert perfectly an 8601-ISO datetime ?
Use the correct format:
as.POSIXct(pb.txt, "%Y-%m-%dT%H:%M:%S", tz="UTC")
[1] "2012-09-11 21:23:20 UTC"
In addition to #JoshuaUlrich answer, don't use daylight saving time timezone: use regular time zone, the system will convert automatically if the day chosen falls during summer time.
Given pb.date <- as.POSIXct(pb.txt, "%Y-%m-%dT%H:%M:%S", tz="UTC") as per Joshua's answer, this fails:
format(pb.date, tz="WEST",usetz=TRUE)
[1] "2012-09-11 21:23:20 UTC"
but this doesn't:
format(pb.date, tz="WET", usetz=TRUE)
[1] "2012-09-11 22:23:20 WEST"
Related
I have a data set containing the following date, along with several others
03/12/2017 02:17:13
I want to put the whole data set into a data table, so I used read_csv and as.data.table to create DT which contained the date/time information in date.
Next I used
DT[, date := as.POSIXct(date, format = "%m/%d/%Y %H:%M:%S")]
Everything looked fine except I had some NA values where the original data had dates. The following expression returns an NA
as.POSIXct("03/12/2017 02:17:13", format = "%m/%d/%Y %H:%M:%S")
The question is why and how to fix.
Just use functions anytime() or utctime() from package anytime
R> library(anytime)
R> anytime("03/12/2017 02:17:13")
[1] "2017-03-12 01:17:13 CST"
R>
or
R> utctime("03/12/2017 02:17:13")
[1] "2017-03-11 20:17:13 CST"
R>
The real crux is that time did not exists in North America due to DST. You could parse it as UTC as UTC does not observer daylight savings:
R> utctime("03/12/2017 02:17:13", tz="UTC")
[1] "2017-03-12 02:17:13 UTC"
R>
You can express that UTC time as Mountain time, but it gets you the previous day:
R> utctime("03/12/2017 02:17:13", tz="America/Denver")
[1] "2017-03-11 19:17:13 MST"
R>
Ultimately, you (as the analyst) have to provide as to what was measured. UTC would make sense, the others may need adjustment.
My solution is below but ways to improve appreciated.
The explanation for the NA is that in the mountain time zone in the US, that date and time is in the window of the switch to daylight savings where the time doesn't exist, hence NA. While the time zone is not explicitly specified, I guess R must be picking it up from the computer's time, which is in "America/Denver"
The solution is to explicitly state the date/time string is in UTC and then convert back as follows:
time.utc <- as.POSIXct("03/12/2017 02:17:13", format = "%m/%d/%Y %H:%M:%S", tz = "UTC")
> time.utc
[1] "2017-03-12 02:17:13 UTC"
>
Next, add 6 hours to the UTC time which is the difference between UTC and MST
time.utc2 <- time.utc + 6 * 60 * 60
> time.utc2
[1] "2017-03-12 08:17:13 UTC"
>
Now convert to America/Denver time using daylight savings.
time.mdt <- format(time.utc2, usetz = TRUE, tz = "America/Denver")
> time.mdt
[1] "2017-03-12 01:17:13 MST"
>
Note that this is in standard time, because daylight savings doesn't start until 2 am.
If you change the original string from 2 am to 3 am, you get the following
> time.mdt
[1] "2017-03-12 03:17:13 MDT"
>
The hour between 2 and 3 is lost in the change from standard to daylight savings but the data are now correct.
My day starts at 2016-03-02 00:00:00. Not 2016-03-02 00:00:01.
How do I get the beginning of the day in POSIXct in local time?
My confusing probably comes from the fact that R sees this as the end-date of 2016-03-01? Given that R uses an ISO 8601?
For example if I try to find the beginning of the day using Sys.Date():
as.POSIXct(Sys.Date(), tz = "CET")
"2016-03-01 01:00:00 CET"
Which is not correct - but are there other ways?
I know I can hack my way out using a simple
as.POSIXct(paste(Sys.Date(), "00:00:00", sep = " "), tz = "CET")
But there has to be a more correct way to do this? Base R preferred.
It's a single command---but you want as.POSIXlt():
R> as.POSIXlt(Sys.Date())
[1] "2016-03-02 UTC"
R> format(as.POSIXlt(Sys.Date()), "%Y-%m-%d %H:%M:%S")
[1] "2016-03-02 00:00:00"
R>
It is only when converting to POSIXct happens that the timezone offset to UTC (six hours for me) enters:
R> as.POSIXct(Sys.Date())
[1] "2016-03-01 18:00:00 CST"
R>
Needless to say by wrapping both you get the desired type and value:
R> as.POSIXct(as.POSIXlt(Sys.Date()))
[1] "2016-03-02 UTC"
R>
Filed under once again no need for lubridate or other non-Base R packages.
Notwithstanding that you understandably prefer base R, a "smart way," for certain meaning of "smart," would be:
library(lubridate)
x <- floor_date(Sys.Date(),"day")
> format(x,"%Y-%m-%d-%H-%M-%S")
[1] "2016-03-02-00-00-00"
From ?floor_date:
floor_date takes a date-time object and rounds it down to the nearest
integer value of the specified time unit.
Pretty handy.
Your example is a bit unclear.
You are talking about a 1 minute difference for the day start, but your example shows a 1 hour difference due to the timezone.
You can try
?POSIXct
to get the functionality explained.
Using Sys.Date() withing POSIXct somehow overwrites your timezone setting.
as.POSIXct(Sys.Date(), tz="EET")
"2016-03-01 01:00:00 CET"
While entering a string gives you
as.POSIXct("2016-03-01 00:00:00", tz="EET")
"2016-03-01 EET"
It looks like 00:00:00 is actually the beginning of the day. You can conclude it from the results of the following 2 inequalities
as.POSIXct("2016-03-02 00:00:02 CET")>as.POSIXct("2016-03-02 00:00:01 CET")
TRUE
as.POSIXct("2016-03-02 00:00:01 CET")>as.POSIXct("2016-03-02 00:00:00 CET")
TRUE
So somehow this is a timezone issue. Notice that 00:00:00 is automatically removed from the as.POSIXct result.
as.POSIXct("2016-03-02 00:00:00 CET")
"2016-03-02 CET"
How can I convert local DateTime in the following format "12/31/2014 6:42:52 PM" to UTC in R? I tried this
as.POSIXct(as.Date("12/31/2014 6:42:52 PM", format="%m/%d/%Y %H:%M:%S"),tz="UTC")
but it doesn't seem to be valid.
If you want to shift a datetime from your current timezone to UTC, you need to
import in your local timezone, then just shift the display timezone to "UTC". e.g.: in Australian EST I am UTC+10.
out <- as.POSIXct("12/31/2014 6:42:52 PM", format="%m/%d/%Y %H:%M:%S")
out
#"2014-12-31 06:42:52 EST"
#(Australian Eastern Standard Time)
as.numeric(out)
#[1] 1419972172
Now shift the timezone for display purposes:
attr(out, "tzone") <- "UTC"
out
#[1] "2014-12-30 20:42:52 UTC"
# display goes 10 hours backwards as I'm UTC+10
as.numeric(out)
#[1] 1419972172
Note that this doesn't affect the underlying numeric data (seconds since 1970-01-01), it only changes what is displayed.
I am puzzled with this result:
a = "2008-03-03 12:30:38"
#I convert to POSIXct and set the timezone
dt = as.POSIXct(a, format="%Y-%m-%d %H:%M:%S", tz='Europe/Paris')
dt
[1] "2008-03-03 12:30:38 CET"
unclass(dt)
[1] 1204543838
attr(,"tzone")
[1] "Europe/Paris"
#I want to come back to POSIXct
as.POSIXct(unclass(dt), origin='1970-01-01', tz='Europe/Paris')
[1] "2008-03-03 11:30:38 CET"
I would have expected to get back the date-time a, what is wrong here ?
As it says in ?as.POSIXct, the origin is in tz="GMT".
You can use .POSIXct instead:
.POSIXct(unclass(dt), tz='Europe/Paris')
# [1] "2008-03-03 12:30:38 CET"
You may want to check the attribute:
isdst
To see if there is some Daylight Savings conversion going on in there somewhere.
This page from the R manual on Date-Time Classes may be useful
I'm trying to convert a yearmon date (from the zoo package) to a POSIXct in the UTC timezone.
This is what I tried to do:
> as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
[1] "2010-01-01 01:00:00 CET"
I get the same when I convert a Date:
> as.POSIXct(as.Date("2010-01-01"),tz="UTC")
[1] "2010-01-01 01:00:00 CET"
The only way to get it to work is to pass a character as an argument:
> as.POSIXct("2010-01-01", tz="UTC")
[1] "2010-01-01 UTC"
I looked into the documentation of DateTimeClasses, tzset and timezones. My /etc/localtime is set to Europe/Amsterdam. I couldn't find a way to set the tz to UTC, other than setting the TZ environment variable:
> Sys.setenv(TZ="UTC")
> as.POSIXct(as.Date("2010-01-01"),tz="UTC")
[1] "2010-01-01 UTC"
Is it possible to directly set the timezone when creating a POSIXct from a yearmon or Date?
Edit:
I checked the functions as.POSIXct.yearmon. This one passes to the as.POSIXct.Date.
> zoo:::as.POSIXct.yearmon
function (x, tz = "", ...)
as.POSIXct(as.Date(x), tz = tz, ...)
<environment: namespace:zoo>
So like Joshua says the timezone gets lost in the as.POSIXct.Date. For now I'll use Richies suggestion to set the tzone by hand using:
attr(x, "tzone") <- 'UTC'
This solves the issue of the lost tzone, which is only used for presentation and not internally like Grothendieck and Dwin suggested.
This is because as.POSIXct.Date doesn't pass ... to .POSIXct.
> as.POSIXct.Date
function (x, ...)
.POSIXct(unclass(x) * 86400)
<environment: namespace:base>
You are setting the timezone correctly in your code. The problem you are perceiving is only at the output stage. POSIX values are all referenced to UTC/GMT. Dates are assumed to be midnight times. Midnight UTC is 1 AM CET ( which is apparently where you are).
> as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
[1] "2009-12-31 19:00:00 EST" # R reports the time in my locale's timezone
> dtval <- as.POSIXct(as.yearmon("2010-01-01"), tz="UTC")
> format(dtval, tz="UTC") # report the date in UTC note it is the correct date ... there
[1] "2010-01-01"
> format(dtval, tz="UTC", format="%Y-%m-%d ")
[1] "2010-01-01 " # use a format string
> format(dtval, tz="UTC", format="%Y-%m-%d %OS3")
[1] "2010-01-01 00.000" # use decimal time
See ?strptime for many, many other format possibilities.
In the help page ?as.POSIXct, for the tz argument it says
A timezone specification to be used
for the conversion, if one is
required. System-specific (see time
zones), but ‘""’ is the current
timezone, and ‘"GMT"’ is UTC
(Universal Time, Coordinated).
Does as.POSIXct(as.yearmon("2010-01-01"), tz="GMT") work for you?
After more perusal of the documentation, in the details section we see:
Dates without times are treated as
being at midnight UTC.
So in your example, the tz argument is ignored. If you use as.POSIXlt it is easier to see what happens with the timezone. The following should all give the same answer, with UTC as the timezone.
unclass(as.POSIXlt(as.yearmon("2010-01-01")))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "UTC"))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "GMT"))
unclass(as.POSIXlt(as.yearmon("2010-01-01"), tz = "CET"))
In fact, since you are using as.yearmon (which strips the time out) you will never get to set the timezone. Compare, e.g.,
unclass(as.POSIXlt(as.yearmon("2010-01-01 12:00:00"), tz = "CET"))
unclass(as.POSIXlt("2010-01-01 12:00:00", tz = "CET"))
This seems to be an oddity with the date/time "POSIXct" class methods. Try formatting the "Date" or "yearmon" variable first so that as.POSIXct.character rather than as.POSIXct.{Date, yearmon} is dispatched:
Date
> d <- as.Date("2010-01-01")
> as.POSIXct(format(d), tz = "UTC")
[1] "2010-01-01 UTC"
yearmon
> library(zoo)
> y <- as.yearmon("2010-01")
> as.POSIXct(format(y, format = "%Y-%m-01"), tz = "UTC")
[1] "2010-01-01 UTC"
> # or
> as.POSIXct(format(as.Date(y)), tz = "UTC")
[1] "2010-01-01 UTC"