I'm trying to parse the value
1454181695067-0800
As a datetime object in R. This value is milliseconds since epoch.
If I truncate the timezone and the milliseconds part of the value I can parse, i.e.
> as.POSIXlt(1454181695, origin = "1970-01-01")
[1] "2016-01-30 11:21:35 PST"
However, is it possible to do it all in one function call, which includes the milliseconds and handles the -0800 portion correctly?
I'd prefer to do it with the default R functionality.
With your timestamp as a character:
x="1454181695067-0800"
Then split it with read.fwf and construct:
> bits = read.fwf(textConnection(x),widths=c(10,3,1,2,2))
> as.POSIXlt(bits$V1+bits$V2/1000, origin="1970-01-01") + ifelse(bits$V3=="-",-1,1)* (bits$V4*60*60+bits$V5*60)
[1] "2016-01-30 11:21:35 GMT"
Although the milliseconds don't show, they are there. If you call that thing t0 then:
> as.numeric(t0 - as.integer(t0))
[1] 0.06699991
Related
I am working in R and need to change the timestamp from what I believe is nanosecond precision to either microsecond precision or millisecond precision (I believe it needs to be milliseconds or only three digits past the decimal).
Example of two of the timestamps
"2019-03-02D00:00:12.214841000"
Part of the difficulty is I don't think there is a package like lubridate to handle it. I'm not sure if I need to use a regular expression to extract the seconds and then transform the nanoseconds to milliseconds. I'm open to any suggestions.
Also, how do you recommend dealing with the D? I was thinking I should use gsub("D", "-", df$timestamp) and maybe then a package like lubridate could parse the timestamp even with the nanosecond precision?
You can use the library nanotime which is related to integer64(really high precision float)
library(nanotime)
x<-nanotime("2019-03-02T00:00:12.214841000+00:00")
As you can see, you need to change D for T and add 00:00to the end, but that is easyly done as symbolrush showed you.
x<-nanotime(paste0(gsub("D", "T", "2019-03-02D00:00:12.214841000"), "+00:00"))
See more here:
http://dirk.eddelbuettel.com/code/nanotime.html
You can use as.POSIXct after gsub("D", " ", x):
as.POSIXct(gsub("D", " ", "2019-03-02D00:00:12.214841000"))
You can still work with millisecond precision afterwards:
dt <- as.POSIXct(gsub("D", " ", "2019-03-02D00:00:12.214841000"))
dt
[1] "2019-03-02 00:00:12 CET"
for(i in 1:1000) dt <- dt - 0.001
dt
[1] "2019-03-02 00:00:11 CET"
If you want to display those milliseconds you can use format:
format(dt, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.214"
format(dt - 1E-3, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.213"
format(dt - 10E-3, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.204"
I'm trying to convert a 12-hour timestamp to a POSIXct object in R. For some reason it strips away the seconds after the conversion.
## timestamp
chk = '17-MAY-16 04.51.34.000000000 PM'
## convert
as.POSIXct(chk, format = '%d-%b-%y %I.%M.%S.%OS %p', tz = 'America/New_York')
[1] "2016-05-17 16:51:00 EDT"
Am I doing something incorrectly?
It does not strip the seconds. It simply adheres to a default for printing and formatting which does not include subseconds.
Witness an example that
actually has subsecond entries
runs in a session with options(digits.secs) set correctly
corrects one error you had in the format string
Demo:
R> options(digits.secs=6) # important to tell R we want subsecs
R> input <- '17-MAY-16 04.51.34.123456 PM'
R> as.POSIXct(input, '%d-%b-%y %I.%M.%OS %p', tz = 'America/New_York')
[1] "2016-05-17 16:51:34.123456 EDT"
R>
If we reset digits.secs=0 it falls back to whole seconds only (which is after all a good default for many settings, though one may argue that %0S could override it...)
R> options(digits.secs=0) # reset
R> as.POSIXct(input, '%d-%b-%y %I.%M.%OS %p', tz = 'America/New_York')
[1] "2016-05-17 16:51:34 EDT"
R>
Also note the small change to the format string. Don't use both %S and %OS.
I have almost finished my script but I have a problem with my dates format.
I installed lubridate package used the as_date function, but it doesn't give me what I want (a date).
"time" is my variable, I put its description below.
I do not put my entire script since the concern is only about this format question (and it implies a huge netcdf file impossible to download)
Could you help me please ?
class(time)
[1] "array"
head(time)
[1] 3573763200 3573774000 3573784800 3573795600 3573806400 3573817200
tunits
$long_name
[1] "time in seconds (UT)"
$standard_name
[1] "time"
$units
[1] "seconds since 1900-01-01T00:00:00Z"
$axis
[1] "T"
$time_origin
[1] "01-JAN-1900 00:00:00"
$conventions
[1] "relative number of seconds with no decimal part"
#conversion
date = as_date(time,tz="UTC",origin = "1900-01-01")
head(date)
[1] "-5877641-06-23" "-5877641-06-23" "-5877641-06-23" "-5877641-06-23"
[5] "-5877641-06-23" "-5877641-06-23"
Time is in seconds since 01/01/1900. Converting a value in time to an actual date would work as follows, using the seconds methods in lubridate:
lubridate::ymd("1900-01-01") + lubridate::seconds(3573763200)
You can vectorize it:
lubridate::ymd("1900-01-01") + lubridate::seconds(time)
as_date() calculates the date using the number of days since the origin.
What you are looking for seems to be as_datetime() also from the lubridate package which calculates the date using the number of seconds since the origin. In your example this would be:
time <- c(3573763200,3573774000,3573784800,3573795600,3573806400,3573817200)
date <- as_datetime(time, tz = "UTC", origin = "1900-01-01") %>% date()
Using a dplyr pipe and the date() function from lubridate to extract the date from the as_datetime() function.
date <- as_date(time/(24*60*60), tz = "UTC", origin = "1900-01-01")
date
I am working in R and need to change the timestamp from what I believe is nanosecond precision to either microsecond precision or millisecond precision (I believe it needs to be milliseconds or only three digits past the decimal).
Example of two of the timestamps
"2019-03-02D00:00:12.214841000"
Part of the difficulty is I don't think there is a package like lubridate to handle it. I'm not sure if I need to use a regular expression to extract the seconds and then transform the nanoseconds to milliseconds. I'm open to any suggestions.
Also, how do you recommend dealing with the D? I was thinking I should use gsub("D", "-", df$timestamp) and maybe then a package like lubridate could parse the timestamp even with the nanosecond precision?
You can use the library nanotime which is related to integer64(really high precision float)
library(nanotime)
x<-nanotime("2019-03-02T00:00:12.214841000+00:00")
As you can see, you need to change D for T and add 00:00to the end, but that is easyly done as symbolrush showed you.
x<-nanotime(paste0(gsub("D", "T", "2019-03-02D00:00:12.214841000"), "+00:00"))
See more here:
http://dirk.eddelbuettel.com/code/nanotime.html
You can use as.POSIXct after gsub("D", " ", x):
as.POSIXct(gsub("D", " ", "2019-03-02D00:00:12.214841000"))
You can still work with millisecond precision afterwards:
dt <- as.POSIXct(gsub("D", " ", "2019-03-02D00:00:12.214841000"))
dt
[1] "2019-03-02 00:00:12 CET"
for(i in 1:1000) dt <- dt - 0.001
dt
[1] "2019-03-02 00:00:11 CET"
If you want to display those milliseconds you can use format:
format(dt, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.214"
format(dt - 1E-3, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.213"
format(dt - 10E-3, "%Y-%m-%d %H:%M:%OS3")
[1] "2019-03-02 00:00:11.204"
I'm this has been asked before, but I just can't find the exact answer.
If I have a number which represents milliseconds since midnight, say 34200577, how do I turn this into an R time?
Construct a 'baseline time' at midnight, add the given millisecond once converted to seconds and interpret as a time:
R> as.POSIXct(as.numeric(ISOdatetime(2013,8,22,0,0,0)) + 34200577/1e3,
+ origin="1970-01-01")
[1] "2013-08-22 09:30:00.576 CDT"
R>
In fact, the shorter
R> ISOdatetime(2013,8,22,0,0,0) + 34200577/1e3
[1] "2013-08-22 09:30:00.576 CDT"
R>
works as well as ISOdatetime() returns a proper time object which operates in fractional seconds so we just apply the given offset.
This appears to be correct as
R> 34200577/1e3 # seconds
[1] 34200.6
R> 34200577/1e3/60 # minutes
[1] 570.01
R> 34200577/1e3/60/60 # hours
[1] 9.50016
R>
POSIXct uses 1970 as the origin of its time scale(measured in seconds.)
> time= as.POSIXct(34200577/1000 , origin=Sys.Date() )
> time
[1] "2013-08-22 02:30:00 PDT"
Note the discrepancy in results between Dirk's and my method. The POSIX times are input as assumed to occur in UCT, so there appeared the addition 8 hours for my location in UCT-8.
> difftime( as.POSIXct(34200577/1000 , origin=Sys.Date() ) , Sys.Date() )
Time difference of 9.50016 hours
You could get the time since midnight with:
format( as.POSIXct(34200577/1000 , origin=Sys.Date(), tz="UCT" ),
format="%H:%M:%S")
[1] "09:30:00"
A little "gottcha" which I think is worth pointing out...
In R 3.1.2 on windows 64 bit I get the following results for Dirk's example
> ISOdatetime(2013,8,22,0,0,0) + 34200577/1e3
[1] "2013-08-22 09:30:00 BST"
Note the lack of fractional seconds. This is due to the option setting for "digits.secs"
> getOption("digits.secs")
NULL
Setting this option as follows gives the expected result:
> options(digits.secs=3)
> ISOdatetime(2013,8,22,0,0,0) + 34200577/1e3
[1] "2013-08-22 09:30:00.576 BST"
As you can probably guess, this is to do with the formatting of output, not the actual values we get from our date arithmetic. See ?strptime and ?options for the documentation on this.