My question comes from this question. The question had the following character string.
x <- "2007-02-01 00:00:00"
y <- "02/01/2007 00:06:10"
If you try to convert this string to date-class object, something funny happens.
This is a sample from #nrusell's answer.
as.POSIXct(x,tz=Sys.timezone())
[1] "2007-02-01 EST"
as.POSIXct(y,format="%m/%d/%Y %H:%M:%S",tz=Sys.timezone())
[1] "2007-02-01 00:06:10 EST"
As you see, 00:00:00 disappears from the first example. #Richard Scriven left the following example in our discussion using lubridate.
dt <- as.POSIXct("2007-02-01 00:00:00")
hour(dt) <- hour(dt)+1
dt
[1] "2007-02-01 01:00:00 EST"
hour(dt) <- hour(dt)-1
dt
[1] "2007-02-01 EST"
Once again, 00:00:00 disappears. Why does R avoid keeping 00:00:00 in date-class object after conversion? How can we keep 00:00:00?
It is just the print that remove the precision if the time part of a date is a midnight. This is literlay explained in ??strftime help, specially the format parameter:
A character string. The default is "%Y-%m-%d %H:%M:%S" if any
component has a time component which is not midnight, and "%Y-%m-%d"
otherwise
One idea is to redefine the S3 method print for POSIXct object:
print.POSIXct <- function(x,...)print(format(x,"%Y-%m-%d %H:%M:%S"))
Now for your example if your print your x date(with midnight part) you get:
x <- "2007-02-01 00:00:00"
x <- as.POSIXct(x,tz=Sys.timezone())
x
[1] "2007-02-01 00:00:00"
Related
I have a Time column in my df with value 1.01.2016 0:00:05. I want it without the seconds and therefore used df$Time <- as.POSIXct(df$Time, format = "%d.%m.%Y :%H:%M", tz = "Asia/Kolkata"). But I get NA value. What is the problem here?
I suspect there are two things working here: the storage of a time object (POSIXt), and the representation of that object.
The string you present is (I believe) not a proper POSIXt (whether POSIXct or POSIXlt) object for R, which means it is just a character string. In that case, you can remove it with:
gsub(':[^:]*$', '', '1.01.2016 0:00:05')
# [1] "1.01.2016 0:00"
However, that is still just a string, not a date or time object. If you parse it into a time-object that R knows about:
as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
# [1] "2016-01-01 00:00:05 IST"
then you now have a time object that R knows something about ... and it defaults to representing it (printing it on the console) with seconds-precision. Typically, all that is available to change for the console-printing is the precision of the seconds, as in
options("digits.secs")
# $digits.secs
# NULL
Sys.time()
# [1] "2018-06-26 18:21:06 PDT"
options("digits.secs"=3)
Sys.time()
# [1] "2018-06-26 18:21:10.090 PDT"
then you can get more. But alas, I do know think there is an R-option to say "always print my POSIXt objects in this way". So your only choice is (at the point where you no longer need it to be a time-like object) to change it back into a string with no time-like value:
x <- as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
x
# [1] "2016-01-01 00:00:05 IST"
?strptime
# see that day-of-month can either be "%d" for 01-31 or "%e" for 1-31
format(x, format="%e.%m.%Y %H:%M")
# [1] " 1.01.2016 00:00"
(This works equally well for a vector.)
Part of me suggests convert to POSIXt and back to string as opposed to my gsub example because using as.POSIXct will tell you when the string does not match the date-time-like object you are expecting, whereas gsub will happily do something wrong or nothing.
Try asPOSIXlt:
> test <- "1.01.2016 0:00:05"
> as.POSIXlt(test, "%d.%m.%Y %H:%M:%S", tz="Asia/Kolkata")
[1] "2016-01-01 00:00:05 IST"
I have several variables that exist in the following format:
/Date(1353020400000+0100)/
I want to convert this format to ddmmyyyy. I found this solution for the same problem using php, but I don't know anything about php, so I'm unable to convert that solution to what I need, which is a solution that I can use in R.
Any suggestions?
Thanks.
If the format is milliseconds since the epoch then anytime() or as.POSIXct() can help you:
R> anytime(1353020400000/1000)
[1] "2012-11-15 17:00:00 CST"
R> anytime(1353020400.000)
[1] "2012-11-15 17:00:00 CST"
R>
anytime() converts to local time, which is Chicago for me. You would have to deal with the UTC offset separately.
Base R can do it too, but you need the dreaded origin:
R> as.POSIXct(1353020400.000, origin="1970-01-01")
[1] "2012-11-15 17:00:00 CST"
R>
As far as I can tell from the linked question, this is milliseconds since the epoch:
x <- "/Date(1353020400000+0100)/"
spl <- strsplit(x, "[()+]")
as.POSIXct(as.numeric(sapply(spl,`[[`,2)) / 1000, origin="1970-01-01", tz="UTC")
#[1] "2012-11-15 23:00:00 UTC"
If you want to pick up the timezone difference as well, here's an attempt:
x <- "/Date(1353020400000+0100)/"
spl <- strsplit(x, "(?=[+-])|[()]", perl=TRUE)
tzo <- sapply(spl, function(x) paste(x[3:4],collapse="") )
dt <- as.POSIXct(as.numeric(sapply(spl,`[[`,2)) / 1000, origin="1970-01-01", tz="UTC")
as.POSIXct(paste(format(dt), tzo), tz="UTC", format = '%F %T %z')
#[1] "2012-11-15 22:00:00 UTC"
The package lubridate can come to the rescue as follows:
as.Date("1970-01-01") + lubridate::milliseconds(1353020400000)
Read: Number of milliseconds since epoch (= 1. January 1970, UTC + 0)
A parsing function can now be made using regular expressions:
parse.myDate <- function(text) {
num <- as.numeric(stringr::str_extract(text, "(?<=/Date\\()\\d+"))
as.Date("1970-01-01") + lubridate::milliseconds(num)
}
finally, format the Date with
format(theDate, "%d/%m/%Y %H:%M")
If you also need the time zone information, you can use this instead:
parse.myDate <- function(text) {
parts <- stringr::str_match(text, "^/Date\\((\\d+)([+-])(\\d{4})\\)/$")
as.POSIXct(as.numeric(parts[,2])/1000, origin = "1970-01-01", tz = paste0("Etc/GMT", parts[,3], as.integer(parts[,4])/100))
}
I have two columns of differently formatted date strings that I need to make the same format,
the first is in the form:
vt_dev_date = "6/20/2016 7:45"
the second is in the form
vt_other = "2016-06-14 20:21:29.0"
If could get them both in the same form down to the minute that would be great. I have tried
strptime(vt_dev_date,format = "%Y-%m-%d %H:%M")
strptime(vt_other,"%Y-%m-%d %H:%M")
and for the second one, it works and I get
"2016-06-14 20:21:00 EDT"
But for the first string, it seems that because the month and hour are not padded with zeros, none of the formating tricks will work, becuase if I try
test_string <- "06/20/2016 07:45"
strptime(test_string,format = "%m/%d/%Y %H:%M")
[1] "2016-06-20 07:45:00 EDT"
It works, but I dont think going through every row in the column and padding each date is a great option. Any help would be appreciated.
Thanks,
josh
How about using lubridate , as follows :
library(lubridate)
x <- c("6/20/2016 7:45","2016-06-14 20:21:29.0")
> x
[1] "6/20/2016 7:45" "2016-06-14 20:21:29.0"
> parse_date_time(x, orders = c("mdy hm", "ymd hms"))
[1] "2016-06-20 07:45:00 UTC" "2016-06-14 20:21:29 UTC"
>
I'm trying to calculate a date - 1 (basically the day before the date) in R and when it converts it to a POSIXct it seems to subtract another date?
The column is of type POSIXct:
class(df$Date)
[1] "POSIXct" "POSIXt"
Here's the initial value:
> df[12,"Date"]
[1] "2016-03-09 EST"
If I just do as.Date and subtract one it works fine:
as.Date(df[12,"Date"]-1, tz="EST")
[1] "2016-03-08"
But I'm saving it back to the same column so it converts is back to as.POSIXct automatically (I think). And then I end up with March 7 in that column. And 7 pm. If I type it out here I get this:
as.POSIXct(as.Date(df[12,"Date"]-1, tz="EST"))
[1] "2016-03-07 19:00:00 EST"
I've tried using America/New York for the tz. I've tried the as.Date around just the df[12,"Date"] or around the whole thing including the -1... I have no clue what to do!
Thanks!
Use as.difftime to take away amounts specified in units of time. It will work consistently regardless of your data being in Date or POSIXct formats. E.g.:
This fails as you described:
x <- as.POSIXct(c("2016-03-09","2016-03-10"), tz="US/Eastern")
#[1] "2016-03-09 EST" "2016-03-10 EST"
x[1] <- as.Date(x[1]-1, tz="US/Eastern")
#[1] "2016-03-07 19:00:00 EST" "2016-03-10 00:00:00 EST"
This works:
x <- as.POSIXct(c("2016-03-09","2016-03-10"), tz="US/Eastern")
#[1] "2016-03-09 EST" "2016-03-10 EST"
x[1] <- x[1] - as.difftime(1, units="days")
#[1] "2016-03-08 EST" "2016-03-10 EST"
If you don't want to do what Frank mentioned in the comments,
You should consider using strptime instead of as.POSIXct
To ensure it returns in a POSIXct format use:
strptime(df[12,"Date"]-1,tz="EST",format="%Y-%m-%d")
Seeing that others can't reproduce this Any speculation about system settings that might cause what I'm seeing would be appreciated. This is on a work PC configured by IT, but I will compare with my personal install this evening and then update the question.
Using base R, I'm trying to read in date and time, convert to numeric, and then convert back to date time. The problem I'm running into is a + 5 hour shift that gets introduced, I think due to timezone defaults.
From a previous question, an example of date time to numeric was provided:
Change from date and hour format to numeric format
> x <- as.POSIXct("9/27/2011 3:33:00 PM", format="%m/%d/%Y %H:%M:%S %p")
> x
[1] "2011-09-27 03:33:00 EDT"
> y <- as.numeric(x)
[1] 1317108780
*Typo in above code fixed
When I try to bring this back to date time, I get:
> z <- as.POSIXct(y, origin="1970-01-01")
> z
[1] "2011-09-27 08:33:00 EDT"
I tried some variants, including specifying time zones explicitly, but am consistently getting this shift.
I think it is just a problem of specifying time zones :
x <- as.POSIXct("9/27/2011 15:33:00", format="%m/%d/%Y %H:%M:%S")
> as.POSIXct(as.numeric(x), origin="1970-01-01",tz="EST") # as.numeric(x)=1317130380
[1] "2011-09-27 08:33:00 EST"
but :
x <- as.POSIXct("9/27/2011 15:33:00", format="%m/%d/%Y %H:%M:%S",tz="EST")
> as.POSIXct(as.numeric(x), origin="1970-01-01",tz="EST") # as.numeric(x)=1317155580
[1] "2011-09-27 15:33:00 EST"
remark : I simplified 03:33:00 PM in 15:33:00