I would like to ask the R gurus to comment of the following:
as.POSIXct("05/11/1998 09:35", "%m/%d/%Y %H:%M",tz="EST") - as.POSIXct("1998-05-11 09:35:00 EST")
Time difference of 1 hours
Shouldn't it be zero since dates are the same?
Thanks.
According to ?strptime (which ?as.POSIXct points to) the format= argument should be
A character string. The default for the ‘format’ methods is
‘"%Y-%m-%d %H:%M:%S"’ if any element has a time component
which is not midnight, and ‘"%Y-%m-%d"’ otherwise. If
‘options("digits.secs")’ is set, up to the specified number
of digits will be printed for seconds.
The time "1998-05-11 09:35:00 EST" has a format of "%Y-%m-%d %H:%M:%S %Z". However %Z can only be used for output (see ?strptime)
If you provide the tz= argument to the second call, it will work as expected
> as.POSIXct("05/11/1998 09:35", "%m/%d/%Y %H:%M",tz="EST") - as.POSIXct("1998-05-11 09:35:00 EST", tz="EST")
Time difference of 0 secs
It is worth noting that
'EST' is a time zone used in Canada _without_ daylight saving time, and not
‘EST5EDT’ nor (Australian) Eastern Standard Time.)
(see ?timezone)
Related
I would like to use R for time series analysis. I want to make a time-series model and use functions from the packages timeDate and forecast.
I have intraday data in the CET time zone (15 minutes data, 4 data points per hour). On March 31st daylight savings time is implemented and I am missing 4 data points of the 96 that I usually have. On October 28th I have 4 data points too many as time is switched back.
For my time series model I always need 96 data points, as otherwise the intraday seasonality gets messed up.
Do you have any experiences with this? Do you know an R function or a package that would be of help to automat such data handling - something elegant?
Thank you!
I had a similar problem with hydrological data from a sensor. My timestamps were in UTC+1 (CET) and did not switch to daylight saving time (UTC+2, CEST). As I didn't want my data to be one hour off (which would be the case if UTC were used) I took the %z conversion specification of strptime. In ?strptime you'll find:
%z Signed offset in hours and minutes from UTC, so -0800 is 8 hours
behind UTC.
For example: In 2012, the switch from Standard Time to DST occured on 2012-03-25, so there is no 02:00 on this day. If you try to convert "2012-03-25 02:00:00" to a POSIXct-Object,
> as.POSIXct("2012-03-25 02:00:00", tz="Europe/Vienna")
[1] "2012-03-25 CET"
you don't get an error or a warning, you just get date without the time (this behavior is documented).
Using format = "%z" gives the desired result:
> as.POSIXct("2012-03-25 02:00:00 +0100", format="%F %T %z", tz="Europe/Vienna")
[1] "2012-03-25 03:00:00 CEST"
In order to facilitate this import, I wrote a small function with appropriate defaults values:
as.POSIXct.no.dst <- function (x, tz = "", format="%Y-%m-%d %H:%M", offset="+0100", ...)
{
x <- paste(x, offset)
format <- paste(format, "%z")
as.POSIXct(x, tz, format=format, ...)
}
> as.POSIXct.no.dst(c("2012-03-25 00:00", "2012-03-25 01:00", "2012-03-25 02:00", "2012-03-25 03:00"))
[1] "2012-03-25 00:00:00 CET" "2012-03-25 01:00:00 CET" "2012-03-25 03:00:00 CEST"
[4] "2012-03-25 04:00:00 CEST"
If you don't want daylight saving time, convert to a timezone that doesn't have it (e.g. GMT, UTC).
times <- .POSIXct(times, tz="GMT")
Here is getting the daylight savings time offset -
e.g. Central Daylight Savings time
> Sys.time()
"2015-08-20 07:10:38 CDT" # I am at America/Chicago daylight time
> as.POSIXct(as.character(Sys.time()), tz="America/Chicago")
"2015-08-20 07:13:12 CDT"
> as.POSIXct(as.character(Sys.time()), tz="UTC") - as.POSIXct(as.character(Sys.time()), tz="America/Chicago")
Time difference of -5 hours
> as.integer(as.POSIXct(as.character(Sys.time()), tz="UTC") - as.POSIXct(as.character(Sys.time()), tz="America/Chicago"))
-5
Some inspiration was from
Converting time zones in R: tips, tricks and pitfalls
I have dates and times stored in two columns. The first has the date as "20180831." The time is stored as the number of seconds from midnight; 3am would be stored as 10,800.
I need a combined date time column and am having a hard time with something that should be simple.
I can get the dates in no problem but lubridate "hms" interprets the time field as a period, not a 'time' per se.
I tried converting the date to posix.ct format and then using that as the origin for the time field but posix.ct does not set the time for midnight, instead it sets it for either 1800 or 1900 hours depending on the date. I need it set to midnight for all rows, I don't want any daylight savings time adjustment.
Here's the code:
First I made a function because there are several date and time fields I have to do this for.
mkdate<-function(x){
a<-as.Date(as.character(x),format='%Y%m%d')
a<-as.POSIXct(a)
return(a)
}
df$date<-mkdate(df$date) #applies date making function to date field
df$datetime<-as.POSIXct(df$time,origin=df$date)
I'm sure this has to do with time zones. I'm in Central time zone and I have experimented with adding the "tz" specification into these commands in both the mkdate function and in the time code creating "datetime" column.
I've tried:
tz="America/Chicago"
tz="CST"
tz="UTC"
Help would be much appreciated!
Edited with example:
x<-c(20180831,20180710,20160511,20170105,20180101) #these are dates.
as.POSIXct(as.Date(as.character(x),format="%Y%m%d"))
Above code converts dates to seconds from the Jan 1 1970. I could convert this to numeric and add my 'seconds' value to this field BUT it is not correct. This is what I see instead as the output:
[1] "2018-08-30 19:00:00 CDT" "2018-07-09 19:00:00 CDT" "2016-05-10 19:00:00 CDT" "2017-01-04 18:00:00 CST" "2017-12-31 18:00:00 CST"
Look at the first date - it should be 8/31 but instead it is 8/30. Somewhere in there there is a timezone adjustment taking place. It's moving the clock back 5 or 6 hours because I am on central time. The first entry should be 2018-08-31 00:00:00. I would then convert it to numeric and add the seconds field on and convert back to POSIXct format. I've tried including tz specification all over the place with no luck.
Sys.getlocale("LC_TIME")
returns "English_United States.1252"
I believe the following does what you want.
My locale is the following, so the results are different from yours.
Sys.getlocale("LC_TIME")
#[1] "Portuguese_Portugal.1252"
The difference will be due to the daylight savings time, the summer hour.
As for your problem, all you have to do is to remeber that the objects of class "POSIXct are coded as the number of seconds since an origin, and that origin is usually the midnight of 1970-01-01. So you have to add your seconds since midnight to the seconds of as.Date.
x <- "20180831"
xd <- mkdate(x)
y <- 10800
as.POSIXct(as.integer(xd) + y, origin = "1970-01-01")
#[1] "2018-08-31 04:00:00 BST"
as.POSIXct(as.integer(xd) + y, origin = "1970-01-01", tz = "America/Chicago")
#[1] "2018-08-30 22:00:00 CDT"
There are very many ways to do this:
mktime = function(a, b)modifyList(strptime(a, '%Y%m%d'), list(sec = as.numeric(gsub(',', '', b))))
mktime("20180831",'10,800')
[1] "2018-08-31 03:00:00 PDT"
mktime('20180301','10800')
[1] "2018-03-01 03:00:00 PST"
mktime('20180321','10800')
[1] "2018-03-21 03:00:00 PDT"
Looking at the above code, it does not adjust for the daylight saving time. Irrespective of the date, the seconds still show that it Is 3 AM, including the dates when ST-->DT. This will also take into consideration, your LOCAL timezone.
Unix time is 1435617000.
as.Date(1435617000,origin="01-01-1970")
[1] "3930586-11-23"
Which is wrong. I'm trying to (a) get the correct date, which, per epoch converter is GMT: Mon, 29 Jun 2015 22:30:00 GMT.
How do I get R to tell me the month, day, year, hour, minute & second? Thank you.
I think the reason why that happen is because as.Date converts arguments to class date objects. In this case you do not need a date but a class POSIXct object because your input, the x vector, contains other informations that as.Date is not able to manage. Another problem that even with the right function could appear, is that if when you do not specify the right time zone with the tz argument (except the case where your time zone is the same as the original time).
The following code does the job.
x <- 1435617000
as.POSIXct(x, origin = "1970-01-01", tz ="GMT")
[1] "2015-06-29 22:30:00 GMT"
Use as.Date
Just in the case you wanted only the date but you have a complete Unix time like x, you have to just divide by 86400 (which is the number of seconds in a day!) to get only the right date.
as.Date(x/86400L, origin = "1970-01-01")
[1] "2015-06-29"
Another important detail
The origin argument has to be supplied with YYYY-MM-DD and not like you did DD-MM-YYYY I am not sure but I think that the former is the only accepted and correct way.
I am trying to convert UTC time to local standard time. I have found many functions which convert to Local Daylight Time, but I was not successful in getting the standard time. Right now, I have the following code which converts to local daylight time at my specific timezone:
pb.date <- as.POSIXct(date,tz="UTC")
format(pb.date, tz="timeZone",usetz=TRUE)
I would appreciate any help.
First, POSIXct date-times are always UCT internally. The print.POSIXt and format.POSIXt methods will appropriately make the TZ shift on output from their internal representations:
pb.date <- as.POSIXct(Sys.Date())
Sys.Date()
#[1] "2015-07-09"
So that was midnight of the current date in Greenwich:
format(pb.date, tz="America/Los_Angeles",usetz=TRUE)
#[1] "2015-07-08 17:00:00 PDT"
When it's midnight in Greenwich, it's 5PM Daylight Time in the previous day on the Left Coast of the US. You need to use the correct character values for your TZ (and your OS) both of which at the moment are unspecified.
The US Pacific timezone is 8 hours behind GMT (in winter months) so you can use a timezone that is Standard/Daylight-agnostic:
> format(pb.date,usetz=TRUE, tz="Etc/GMT+8")
[1] "2015-07-08 16:00:00 GMT+8"
(Notice the reversal of + with "behind" and - with "ahead".)
I know this question has an accepted answer, but in case anyone comes along and this can help. I needed a function to convert UTC times to MTN time (Server is in UTC, we operate in MTN).
Not sure why, but needed to force it to UTC/GMT first and the convert it to MTN. However it does work
mtn_ts = function(utcTime){
library(lubridate)
toTz = "us/mountain"
utcTime = force_tz(utcTime,tzone= 'GMT')
dt = as.POSIXct(format(utcTime,tz = toTz,origin ='GMT', usetz=TRUE))
dt = force_tz(dt,tzone= toTz)
return(dt)
}
mtn_ts(as.POSIXct("2021-09-27 14:48:51.000000000"))
I am trying to convert UTC time to local standard time. I have found many functions which convert to Local Daylight Time, but I was not successful in getting the standard time. Right now, I have the following code which converts to local daylight time at my specific timezone:
pb.date <- as.POSIXct(date,tz="UTC")
format(pb.date, tz="timeZone",usetz=TRUE)
I would appreciate any help.
First, POSIXct date-times are always UCT internally. The print.POSIXt and format.POSIXt methods will appropriately make the TZ shift on output from their internal representations:
pb.date <- as.POSIXct(Sys.Date())
Sys.Date()
#[1] "2015-07-09"
So that was midnight of the current date in Greenwich:
format(pb.date, tz="America/Los_Angeles",usetz=TRUE)
#[1] "2015-07-08 17:00:00 PDT"
When it's midnight in Greenwich, it's 5PM Daylight Time in the previous day on the Left Coast of the US. You need to use the correct character values for your TZ (and your OS) both of which at the moment are unspecified.
The US Pacific timezone is 8 hours behind GMT (in winter months) so you can use a timezone that is Standard/Daylight-agnostic:
> format(pb.date,usetz=TRUE, tz="Etc/GMT+8")
[1] "2015-07-08 16:00:00 GMT+8"
(Notice the reversal of + with "behind" and - with "ahead".)
I know this question has an accepted answer, but in case anyone comes along and this can help. I needed a function to convert UTC times to MTN time (Server is in UTC, we operate in MTN).
Not sure why, but needed to force it to UTC/GMT first and the convert it to MTN. However it does work
mtn_ts = function(utcTime){
library(lubridate)
toTz = "us/mountain"
utcTime = force_tz(utcTime,tzone= 'GMT')
dt = as.POSIXct(format(utcTime,tz = toTz,origin ='GMT', usetz=TRUE))
dt = force_tz(dt,tzone= toTz)
return(dt)
}
mtn_ts(as.POSIXct("2021-09-27 14:48:51.000000000"))