If this question has been asked before, please downvote and direct me. I have been looking through SO, but it seems no one has had the need for a non-midnight start time i.e. everyone wants to know how to convert seconds from a specific midnight value.
I'm trying to convert my second values to a data value. What I have are seconds from the time 2017-05-21 22:00.
I tried using the as.POSIXct() function, however it only seem to take Y-m-d into account and disregards if I write h:m after it.
e.g file$date = as.POSIXct(file$Time,origin = "2017-05-21 22:00") gives me
Time date
1 0.00 2017-05-22 00:00:00
I have found if I use
file$Time = file$Time-3600*4
file$date = as.POSIXct(file$Time,origin = "2017-05-22")
for some reason gives me the correct output which is of course
Time date
1 0.00 2017-05-21 22:00:00
Any idea on how to do this more elegantly?
Also, if you have a clue on why that gives me the correct output, I'm all ears.
You can simply try as.POSIXct to convert your starting time and then keep on adding seconds. as:
as.POSIXct("2017-05-21 22:00:00", format = "%Y-%m-%d %H:%M:%S")
#[1] "2017-05-21 22:00:00 BST"
as.POSIXct("2017-05-21 22:00:00", format = "%Y-%m-%d %H:%M:%S") + 1
#[1] "2017-05-21 22:00:01 BST"
as.POSIXct("2017-05-21 22:00:00", format = "%Y-%m-%d %H:%M:%S") + 100
#[1] "2017-05-21 22:01:40 BST"
as.POSIXct("2017-05-21 22:00:00", format = "%Y-%m-%d %H:%M:%S") + 300
#[1] "2017-05-21 22:05:00 BST
You can even specify time-zone using tz parameter as:
as.POSIXct("2017-05-21 22:00:00", tz = "UTC", format = "%Y-%m-%d %H:%M:%S")
#[1] "2017-05-21 22:00:00 UTC"
as.POSIXct("2017-05-21 22:00:00", tz = "UTC", format = "%Y-%m-%d %H:%M:%S") + 96
#[1] "2017-05-21 22:01:36 UTC"
Have a look at lubridate...
library(lubridate)
ymd_hm("2017-05-21 22:00") + seconds(1.01)
So in your case it would be something like
file$date <- ymd_hm("2017-05-21 22:00") + seconds(file$Time)
Related
This question already has answers here:
How can I keep midnight (00:00h) using strptime() in R?
(2 answers)
Closed 3 years ago.
I have had a good hunt around and sure this has to have been answered before but I cant seem to find any help!
I have a series of times in a data frame, some of which have the following time stamp in the following format:
Date <- '2018-10-10'
Time <- '00:00:00'
When I use the strptime function it returns only the date, it removes the 00:00:00, see below:
datetime <- strptime(paste(Date,Time),
format = "%Y-%m-%d %H:%M:%S",
tz = 'GMT')
> datetime
[1] "2018-10-10 GMT"
if for example it was Time <- 00:00:01 it would return
> datetime
[1] "2018-10-10 00:00:01 GMT"
Does anyone know a way of ensuring the output for 00:00:00 instances are displayed. Desired output:
"2018-10-10 00:00:00 GMT"
Many thanks!!
Jim
When you type datetime and hit <Enter>, R will use a/the suitable print method to display datetime. Just because datetime returns "2018-10-10 GMT" doesn't mean that datetime has forgotten about the seconds.
To ensure a consistent format of your POSIXlt object, you could use format
format(datetime, "%Y-%m-%d %H:%M:%S", usetz = T)
#[1] "2018-10-10 00:00:00 GMT"
Similar for case 2
Date <- '2018-10-10'
Time <- '00:00:01'
datetime <- strptime(paste(Date,Time), format = "%Y-%m-%d %H:%M:%S", tz = 'GMT')
format(datetime, "%Y-%m-%d %H:%M:%S", usetz = T)
#[1] "2018-10-10 00:00:01 GMT"
Sample data
Date <- '2018-10-10'
Time <- '00:00:00'
datetime <- strptime(paste(Date,Time), format = "%Y-%m-%d %H:%M:%S", tz = 'GMT')
I feel like I'm missing something obvious here. I'm importing some data which is stored as HH:MM:SS. I'm trying to convert this to POSIXct and manually specify the origin as the date the data was collected.
datIn$TimeComplete <- as.POSIXct(datIn$Time, format="%H:%M:%S", origin="2000-01-01", tz="CET")
The output of this registers the HH:MM:SS correctly but says the day is 2019-03-05 (today) and I can't seem to convince it to do anything different.
You are misunderstanding the concept of origin. Origin is there to help convert numbers to dates. Those numbers represent seconds so you need the origin in order to add those seconds to the origin and get the datetime object. For example,
as.POSIXct(60, tz = "GMT", origin = '2015-03-05')
#[1] "2015-03-05 00:01:00 GMT"
as.POSIXct(3600, tz = "GMT", origin = '2015-07-05')
#[1] "2015-07-05 01:00:00 GMT"
What you are trying to do can be easily achieved by pasting the desired date to your times and converting to datetime, i.e.
as.POSIXct(paste0('2000-01-01 ', '11:03:15'), format = "%Y-%m-%d %H:%M:%S", origin = "", tz = "CET")
#[1] "2000-01-01 11:03:15 CET"
This looks like trivial issue but I couldn't make it work. I need simply convert midnight into POSIXct format but also with hours, minutes and seconds, just like that:
nextDay_t <- strptime(paste0(as.character(Sys.Date() + 1)," 00:00:00"), format = "%Y-%m-%d %H:%M:%S")
nextDay_t <- format(nextDay_t, "%Y-%m-%d %H:%M:%S")
nextDay_t <- as.POSIXct(nextDay_t, format='%Y-%m-%d %H:%M:%S', tz="EST")
But still have only "2018-12-11 EST" instead of "2018-12-11 00:00:00 EST". Is there anything I'm missing in my code?
Your code seems on the right track. The hour/minute/second components are in fact still there after calling strptime. They just do not show up automatically by default when inspecting the object.
You may try the following call to format, which includes these components, as well as the time zone (%Z):
nextDay_t <- strptime(paste0(as.character(Sys.Date() + 1)," 00:00:00"), format = "%Y-%m-%d %H:%M:%S")
nextDay_t
format(nextDay_t,"%Y/%m/%d %H:%M:%S %Z") # include hour/minute/second and time zone
[1] "2018-12-11 CET"
[1] "2018/12/11 00:00:00 CET"
I am struggling to convert date time format "%Y-%m-%d %H:%M:%S" to "%Y-%m-%d %H:%M:%S.sss" in an R data.frame? Note that I want seconds with fractional seconds.
I'm not exactly sure what you're asking. You can format fractional seconds with e.g. "%OS3".
From ?strptime
Specific to R is ‘%OSn’, which for output gives the seconds
truncated to ‘0 <= n <= 6’ decimal places (and if ‘%OS’ is not
followed by a digit, it uses the setting of
‘getOption("digits.secs")’, or if that is unset, ‘n = 0’).
Further, for ‘strptime’ ‘%OS’ will input seconds including
fractional seconds. Note that ‘%S’ does not read fractional parts
on output.
Example:
ss <- "2018-08-22 21:30:00.5"
format(as.POSIXct(ss, format = "%Y-%m-%d %H:%M:%OS"), format = "%Y-%m-%d %H:%M:%OS3")
#[1] "2018-08-22 21:30:00.500"
Or for a sample data.frame
df <- data.frame(
date = c("2018-08-22 21:30:00", "2018-08-22 22:00:00", "2018-08-22 22:30:00"))
transform(df, new.date = format(
as.POSIXct(date, format = "%Y-%m-%d %H:%M:%OS"),
format = "%Y-%m-%d %H:%M:%OS3"))
# date new.date
#1 2018-08-22 21:30:00 2018-08-22 21:30:00.000
#2 2018-08-22 22:00:00 2018-08-22 22:00:00.000
#3 2018-08-22 22:30:00 2018-08-22 22:30:00.000
Hi I have a character vector (rr) that is several million in length, and it represents time and date stamps in the format %Y-%m-%d %H:%M:%S recorded in Australia/Sydney.
How do get a POSIXct object (quickly) that represents this.
I have found fastPOSIXct in the fasttime package, but for this to be accurate, it requires the original character string to be in GMT/UTC, (which mine is not) and then converted back into the correct timezone using the tz arguement...
> head(rr)
[1] "2009-05-01 10:01:00" "2009-05-01 10:02:00" "2009-05-01 10:03:00" "2009-05-01 10:04:00"
[5] "2009-05-01 10:05:00" "2009-05-01 10:06:00"
> as.POSIXct(head(rr),tz="Australia/Sydney")
[1] "2009-05-01 10:01:00 EST" "2009-05-01 10:02:00 EST" "2009-05-01 10:03:00 EST"
[4] "2009-05-01 10:04:00 EST" "2009-05-01 10:05:00 EST" "2009-05-01 10:06:00 EST"
The above line takes ages if doing it on the full set of data...so any speed improvements would be appreciated. Thanks.
Inspired by Dirk's answer to this qn, I made this wrapper for handling a whole bunch of dates across the year:
fastPOSIXct_generic <- function(x, mytz = "America/New_York")
{
# Caution, read: ?DateTimeClasses
stopifnot(is.character(x))
times_UTC <- fastPOSIXct(x, tz='UTC')
num_times <- as.numeric(times_UTC)
t1 <- as.POSIXct(x[1], tz = mytz)
t2 <- as.POSIXct(x[1], tz = "UTC")
offset <- as.numeric(difftime(t1, t2, units = "secs"))
daylightoffset <- as.POSIXlt(t1)$isdst
# For this first 'time' in t1 and t2, remove possible impact of losing one hour by setting clocks one hour forward during summer months:
offset <- offset + daylightoffset * 3600
num_times <- num_times + offset
new_num_times <- as.POSIXct(num_times, tz = mytz, origin = '1970-01-01')
new_num_times2 <- new_num_times - as.POSIXlt(new_num_times)$isdst * 3600
return(new_num_times2)
}
# Test Sydney time
mm <- as.POSIXct(c("2015-03-15 15:00:00", "2015-4-10 15:00:00", "2014-10-01 15:00:00", "2015-10-15 15:00:00"), tz = "Australia/Sydney")
# "2015-03-15 15:00:00 AEDT" "2015-04-10 15:00:00 AEST" "2014-10-01 15:00:00 AEST" "2015-10-15 15:00:00 AEDT"
aus_stamps <- as.character(mm)
aus_back <- fastPOSIXct_generic(x = aus_stamps, mytz = "Australia/Sydney")
#"2015-03-15 15:00:00 AEDT" "2015-04-10 15:00:00 AEST" "2014-10-01 15:00:00 AEST" "2015-10-15 15:00:00 AEDT"
identical(mm, aus_back)
# TRUE
My use cases are nearly always UTC to America/New_York, where so far it has seemed to work fine. I don't know whether it works correctly for other time zones; just the cases where dst has time go forward an hour.
Here is one approach:
i) Lie to fasttime() and pretend the data was UTC, use to parse the data into a vector x
ii) Compute an offset to UTC using your first data point:
R> d1 <- "2009-05-01 10:01:01" ## or use `head(rr,1)`
R> t1 <- as.POSIXct(d1,tz="Australia/Sydney")
R> t2 <- as.POSIXct(d1,tz="UTC")
R> offset <- as.numeric(difftime(t2, t1, units="secs"))
R> offset
[1] 36000
iii) Apply the offset value to your data -- that is a quick addition as POSIXct really is a numeric type with (fractional) seconds (since epoch) as its unit.