Here's a weird artifact. I'm converting tens of thousands of character vectors into a datetime class, like so:
alles$DateTime=as.POSIXct(alles$roughdate, tz="EST",format="%Y%m%d.%H%M%S.%OS")
Pretty straight forward. The character string (alles$roughdate) is in the format YYYYMMDD.HHMMSS.ss with the .ss being milliseconds. The above code works, as would be expected. However, if the milliseconds equal .61, it returns an NA instead of a date time value.
This isn't too bad, but when dealing with tens of thousands cells, a few hundred are always returned as NA. Milliseconds always .61, doesn't matter what the rest of the date is. I do need those dates.
I've tried isolating those files and then merging the two data frames together again, but that doesn't seem to work. All of my dates are suddenly NA.
Any thoughts?
Example
vec <- c("20150101.010101.60", "20150101.010101.61", "20150101.010101.62")
as.POSIXlt(vec, tz="EST", format="%Y%m%d.%H%M%S.%OS")
#[1] "2015-01-01 01:01:60 EST" NA "2015-01-01 01:01:01 EST"
If you change the format for the time part to be %H%M%OS instead of %H%M%S.%OS, it seems to parse correctly. You may have to adjust your options so see this:
as.POSIXlt(vec, tz = "EST", format = "%Y%m%d.%H%M%OS")
#[1] "2015-01-01 01:01:01 EST" "2015-01-01 01:01:01 EST"
#[3] "2015-01-01 01:01:01 EST"
options(digits.secs = 2)
as.POSIXlt(vec, tz = "EST", format = "%Y%m%d.%H%M%OS")
# [1] "2015-01-01 01:01:01.60 EST" "2015-01-01 01:01:01.61 EST"
# [3] "2015-01-01 01:01:01.62 EST"
Related
I would like to convert a string to time. I have a time field where the string has only four digits and a letter (A or P). There is no colon between the digits showing it is a time. I would like to convert the string, which is 12 hours, to a 24 hour time so I can drop the A and P.
Here is an example:
time = c("1110A", "1120P", "0420P", "0245P")
I'm looking for a time class that loos like this:
Answer= c('11:10', '23:20', '16:20', '14:45')
Any help would be greatly appreciated.
You can use the function strptime to create dates from strings after making one small change to your strings.
time <- c("1110A", "1120P", "0420P", "02:45P")
time <- gsub(":", "", time)
time <- strptime(x = paste0(time, "m"), format = "%I%M%p")
paste is needed for strptime to parse with the format that we've given it. %I is an hour (00-24), %M is the minute and %p is for parsing AM/PM.
Once it's parsed as a date, you can use format for pretty printing, or use the normal operators on it like +, -, diff, etc....
strptime gives you a lot of flexibility when parsing dates, but sometimes you have to try a few things when dates are not in a standard format.
We could also use the lubridate functions to parse the format after pasteing the date
library(lubridate)
library(glue)
ymd_hm(glue("2018-01-01 {time}M"))
#[1] "2018-01-01 11:10:00 UTC" "2018-01-01 23:20:00 UTC"
#[3] "2018-01-01 16:20:00 UTC" "2018-01-01 14:45:00 UTC"
In your question, you say that you want to be able to subtract these times. I think it makes the most sense to convert it to a POSIXct object. If you want a specific day/month/year you need to append it to your string like below, otherwise you can not specify one and it will assume the date is today:
date2 = as.POSIXct(paste0("01-01-2018 ", time, "m"), format = "%m-%d-%Y %I%M%p")
date2
#[1] "2018-01-01 11:10:00 EST" "2018-01-01 23:20:00 EST" "2018-01-01 16:20:00 EST" "2018-01-01 14:45:00 EST"
I'm converting a string vector to date format with as.POSIXct().
Here is the strange thing:
as.POSIXct("2017-03-26 03:00:00.000",format="%Y-%m-%d %H")
#Gives
"2017-03-26 03:00:00 CEST"
#While
as.POSIXct("2017-03-26 02:00:00.000",format="%Y-%m-%d %H")
#Outputs
NA
This is really confusing and frustrating. It seem like the function really doesn't like the specific time:
02:00:00.000
We can specify the %T for time. In the format, there are minutes, seconds and millseconds. So, the %H is only matching the hour part
as.POSIXct("2017-03-26 02:00:00.000",format="%Y-%m-%d %T")
[1] "2017-03-26 02:00:00 EDT"
Or to take care of the milliseconds as well
as.POSIXct("2017-03-26 02:00:00.000",format="%Y-%m-%d %H:%M:%OS")
#[1] "2017-03-26 02:00:00 EDT"
Or using lubridate
library(lubridate)
ymd_hms("2017-03-26 02:00:00.000")
This was a daylight savings issue, the time:
"2017-03-26 02:00:00.000" does not exist in Sweden as we lost an hour this date when changing to "summer time".
I am having some problems with as.POSIXct().
Giving a vector as an example:
Dates <- c("2015-01 00","2015-01 01","2015-01 02","2015-01 03","2015-01 04","2015-01 05")
As you can see I have Information on the year, month and hour (but not on the day of the months).
I do not know what I am doing wrong but using as.POSIXct(Dates,"%Y-%m %H",tz="GMT")only creates a bunch of NA's.
Any solution is highly apperciated.
We can insert a day, e.g. first day of the month (as Date needs day as well) with sub and then use as.POSIXct
as.POSIXct(sub("\\s", "-01 ", Dates), format = "%Y-%m-%d %H")
#[1] "2015-01-01 00:00:00 EST" "2015-01-01 01:00:00 EST" "2015-01-01 02:00:00 EST" "2015-01-01 03:00:00 EST" "2015-01-01 04:00:00 EST"
#[6] "2015-01-01 05:00:00 EST"
I have a Time column in my df with value 1.01.2016 0:00:05. I want it without the seconds and therefore used df$Time <- as.POSIXct(df$Time, format = "%d.%m.%Y :%H:%M", tz = "Asia/Kolkata"). But I get NA value. What is the problem here?
I suspect there are two things working here: the storage of a time object (POSIXt), and the representation of that object.
The string you present is (I believe) not a proper POSIXt (whether POSIXct or POSIXlt) object for R, which means it is just a character string. In that case, you can remove it with:
gsub(':[^:]*$', '', '1.01.2016 0:00:05')
# [1] "1.01.2016 0:00"
However, that is still just a string, not a date or time object. If you parse it into a time-object that R knows about:
as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
# [1] "2016-01-01 00:00:05 IST"
then you now have a time object that R knows something about ... and it defaults to representing it (printing it on the console) with seconds-precision. Typically, all that is available to change for the console-printing is the precision of the seconds, as in
options("digits.secs")
# $digits.secs
# NULL
Sys.time()
# [1] "2018-06-26 18:21:06 PDT"
options("digits.secs"=3)
Sys.time()
# [1] "2018-06-26 18:21:10.090 PDT"
then you can get more. But alas, I do know think there is an R-option to say "always print my POSIXt objects in this way". So your only choice is (at the point where you no longer need it to be a time-like object) to change it back into a string with no time-like value:
x <- as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
x
# [1] "2016-01-01 00:00:05 IST"
?strptime
# see that day-of-month can either be "%d" for 01-31 or "%e" for 1-31
format(x, format="%e.%m.%Y %H:%M")
# [1] " 1.01.2016 00:00"
(This works equally well for a vector.)
Part of me suggests convert to POSIXt and back to string as opposed to my gsub example because using as.POSIXct will tell you when the string does not match the date-time-like object you are expecting, whereas gsub will happily do something wrong or nothing.
Try asPOSIXlt:
> test <- "1.01.2016 0:00:05"
> as.POSIXlt(test, "%d.%m.%Y %H:%M:%S", tz="Asia/Kolkata")
[1] "2016-01-01 00:00:05 IST"
I'm trying to calculate a date - 1 (basically the day before the date) in R and when it converts it to a POSIXct it seems to subtract another date?
The column is of type POSIXct:
class(df$Date)
[1] "POSIXct" "POSIXt"
Here's the initial value:
> df[12,"Date"]
[1] "2016-03-09 EST"
If I just do as.Date and subtract one it works fine:
as.Date(df[12,"Date"]-1, tz="EST")
[1] "2016-03-08"
But I'm saving it back to the same column so it converts is back to as.POSIXct automatically (I think). And then I end up with March 7 in that column. And 7 pm. If I type it out here I get this:
as.POSIXct(as.Date(df[12,"Date"]-1, tz="EST"))
[1] "2016-03-07 19:00:00 EST"
I've tried using America/New York for the tz. I've tried the as.Date around just the df[12,"Date"] or around the whole thing including the -1... I have no clue what to do!
Thanks!
Use as.difftime to take away amounts specified in units of time. It will work consistently regardless of your data being in Date or POSIXct formats. E.g.:
This fails as you described:
x <- as.POSIXct(c("2016-03-09","2016-03-10"), tz="US/Eastern")
#[1] "2016-03-09 EST" "2016-03-10 EST"
x[1] <- as.Date(x[1]-1, tz="US/Eastern")
#[1] "2016-03-07 19:00:00 EST" "2016-03-10 00:00:00 EST"
This works:
x <- as.POSIXct(c("2016-03-09","2016-03-10"), tz="US/Eastern")
#[1] "2016-03-09 EST" "2016-03-10 EST"
x[1] <- x[1] - as.difftime(1, units="days")
#[1] "2016-03-08 EST" "2016-03-10 EST"
If you don't want to do what Frank mentioned in the comments,
You should consider using strptime instead of as.POSIXct
To ensure it returns in a POSIXct format use:
strptime(df[12,"Date"]-1,tz="EST",format="%Y-%m-%d")