can anyone tell me why R give such outcome below:
> as.POSIXct("2013-01-01 08:00")
[1] "2013-01-01 08:00:00 HKT"
> as.Date(as.POSIXct("2013-01-01 08:00"))
[1] "2013-01-01"
> as.POSIXct("2013-01-01 07:00")
[1] "2013-01-01 07:00:00 HKT"
> as.Date(as.POSIXct("2013-01-01 07:00"))
[1] "2012-12-31"
Shouldn't it be 2013-01-01 after converting POSIXct to Date for 2013-01-01 07:00, is there any way to change the cutoff from 08:00 to 00:00?
Update #1
I found the following can fix my problem, but in a less neat way
> as.Date(as.character(as.POSIXct("2013-01-01 07:00")))
[1] "2013-01-01"
The problem here is timezones - you can see you're in "HKT". Try:
as.Date(as.POSIXct("2013-01-01 07:00", 'GMT'))
[1] "2013-01-01"
From ?as.Date():
["POSIXct" is] converted to days by ignoring the time after midnight
in the representation of the time in specified timezone, default UTC
Use the time zone parameter of as.Date:
as.Date(as.POSIXct("2013-01-01 07:00",tz="Hongkong"))
#[1] "2012-12-31"
as.Date(as.POSIXct("2013-01-01 07:00",tz="Hongkong"),tz="Hongkong")
#[1] "2013-01-01"
In fact, I recommend always using the tz parameter when using date-time converting functions. There are other nasty surprises, e.g. with daylight saving time.
This happens as documented and previously explained when contemporaneous UTC time is before (your third example) or after midnight on your POSIXct date. To see the math for yourself, inspect as.Date.POSIXct at the console. The math under the default tz="UTC" is clear. In the non-default case, R essentially calls as.Date.POSIXlt, and the date-travel does not occur. In fact, if you had started with the lt object you would not have had this problem:
> as.Date(as.POSIXlt("2013-01-01 07:00", tz = "Hongkong"))
[1] "2013-01-01"
The easiest work-around is to call as.Date with tz="" to force using the less offending as.Date.POSIXlt algorithm:
> as.Date(as.POSIXct("2013-01-01 07:00"), tz = "")
[1] "2013-01-01"
Related
can anyone tell me why R give such outcome below:
> as.POSIXct("2013-01-01 08:00")
[1] "2013-01-01 08:00:00 HKT"
> as.Date(as.POSIXct("2013-01-01 08:00"))
[1] "2013-01-01"
> as.POSIXct("2013-01-01 07:00")
[1] "2013-01-01 07:00:00 HKT"
> as.Date(as.POSIXct("2013-01-01 07:00"))
[1] "2012-12-31"
Shouldn't it be 2013-01-01 after converting POSIXct to Date for 2013-01-01 07:00, is there any way to change the cutoff from 08:00 to 00:00?
Update #1
I found the following can fix my problem, but in a less neat way
> as.Date(as.character(as.POSIXct("2013-01-01 07:00")))
[1] "2013-01-01"
The problem here is timezones - you can see you're in "HKT". Try:
as.Date(as.POSIXct("2013-01-01 07:00", 'GMT'))
[1] "2013-01-01"
From ?as.Date():
["POSIXct" is] converted to days by ignoring the time after midnight
in the representation of the time in specified timezone, default UTC
Use the time zone parameter of as.Date:
as.Date(as.POSIXct("2013-01-01 07:00",tz="Hongkong"))
#[1] "2012-12-31"
as.Date(as.POSIXct("2013-01-01 07:00",tz="Hongkong"),tz="Hongkong")
#[1] "2013-01-01"
In fact, I recommend always using the tz parameter when using date-time converting functions. There are other nasty surprises, e.g. with daylight saving time.
This happens as documented and previously explained when contemporaneous UTC time is before (your third example) or after midnight on your POSIXct date. To see the math for yourself, inspect as.Date.POSIXct at the console. The math under the default tz="UTC" is clear. In the non-default case, R essentially calls as.Date.POSIXlt, and the date-travel does not occur. In fact, if you had started with the lt object you would not have had this problem:
> as.Date(as.POSIXlt("2013-01-01 07:00", tz = "Hongkong"))
[1] "2013-01-01"
The easiest work-around is to call as.Date with tz="" to force using the less offending as.Date.POSIXlt algorithm:
> as.Date(as.POSIXct("2013-01-01 07:00"), tz = "")
[1] "2013-01-01"
I am trying to get the last minute of yesterday using Sys.Date() in Posix time.
force_tz(as.POSIXlt(Sys.Date()-1), tz = 'America/New_York') + 86399
# [1] "2018-01-12 23:59:59 EST"
CORRECT
force_tz(as.POSIXct(Sys.Date()-1), tz = 'America/New_York') + 86399
# [1] "2018-01-12 15:59:59 EST"
INCORRECT
Sys.Date()
# [1] "2018-01-13"
Why does as.Posixct and as.Posixlt return two different values using Sys.Date() and why is the difference 8 hours even after applying force_tz from lubridate ?
As ever, debugonce is your friend. Running debugonce(force_tz), you can see that the difference in output comes from when force_tz hits the branches checking first is.POSIXct(time) (in which case the default tzone = "" is applied); in the POSIXlt case, the default branch is hit, where as.POSIXct is applied to time and tz(time) (which comes out as UTC for a POSIXlt object) is used as the time zone.
This comes down to something subtle happening; from ?as.POSIXlt.Date:
Dates without times are treated as being at midnight UTC.
Hence
tz(as.POSIXlt(Sys.Date()-1))
# [1] "UTC"
But
tz(as.POSIXct(Sys.Date()-1))
# [1] ""
What's peculiar is this can't be overridden -- as.POSIXlt.Date doesn't accept a tz argument:
formals(as.POSIXlt.Date)
# $x
# $...
If you want to use POSIXct, how about the following?
force_tz(as.POSIXct(sprintf('%s 00:00:00', Sys.Date())), 'America/New_York') - 1L
# [1] "2018-01-12 23:59:59 EST"
My day starts at 2016-03-02 00:00:00. Not 2016-03-02 00:00:01.
How do I get the beginning of the day in POSIXct in local time?
My confusing probably comes from the fact that R sees this as the end-date of 2016-03-01? Given that R uses an ISO 8601?
For example if I try to find the beginning of the day using Sys.Date():
as.POSIXct(Sys.Date(), tz = "CET")
"2016-03-01 01:00:00 CET"
Which is not correct - but are there other ways?
I know I can hack my way out using a simple
as.POSIXct(paste(Sys.Date(), "00:00:00", sep = " "), tz = "CET")
But there has to be a more correct way to do this? Base R preferred.
It's a single command---but you want as.POSIXlt():
R> as.POSIXlt(Sys.Date())
[1] "2016-03-02 UTC"
R> format(as.POSIXlt(Sys.Date()), "%Y-%m-%d %H:%M:%S")
[1] "2016-03-02 00:00:00"
R>
It is only when converting to POSIXct happens that the timezone offset to UTC (six hours for me) enters:
R> as.POSIXct(Sys.Date())
[1] "2016-03-01 18:00:00 CST"
R>
Needless to say by wrapping both you get the desired type and value:
R> as.POSIXct(as.POSIXlt(Sys.Date()))
[1] "2016-03-02 UTC"
R>
Filed under once again no need for lubridate or other non-Base R packages.
Notwithstanding that you understandably prefer base R, a "smart way," for certain meaning of "smart," would be:
library(lubridate)
x <- floor_date(Sys.Date(),"day")
> format(x,"%Y-%m-%d-%H-%M-%S")
[1] "2016-03-02-00-00-00"
From ?floor_date:
floor_date takes a date-time object and rounds it down to the nearest
integer value of the specified time unit.
Pretty handy.
Your example is a bit unclear.
You are talking about a 1 minute difference for the day start, but your example shows a 1 hour difference due to the timezone.
You can try
?POSIXct
to get the functionality explained.
Using Sys.Date() withing POSIXct somehow overwrites your timezone setting.
as.POSIXct(Sys.Date(), tz="EET")
"2016-03-01 01:00:00 CET"
While entering a string gives you
as.POSIXct("2016-03-01 00:00:00", tz="EET")
"2016-03-01 EET"
It looks like 00:00:00 is actually the beginning of the day. You can conclude it from the results of the following 2 inequalities
as.POSIXct("2016-03-02 00:00:02 CET")>as.POSIXct("2016-03-02 00:00:01 CET")
TRUE
as.POSIXct("2016-03-02 00:00:01 CET")>as.POSIXct("2016-03-02 00:00:00 CET")
TRUE
So somehow this is a timezone issue. Notice that 00:00:00 is automatically removed from the as.POSIXct result.
as.POSIXct("2016-03-02 00:00:00 CET")
"2016-03-02 CET"
Seeing that others can't reproduce this Any speculation about system settings that might cause what I'm seeing would be appreciated. This is on a work PC configured by IT, but I will compare with my personal install this evening and then update the question.
Using base R, I'm trying to read in date and time, convert to numeric, and then convert back to date time. The problem I'm running into is a + 5 hour shift that gets introduced, I think due to timezone defaults.
From a previous question, an example of date time to numeric was provided:
Change from date and hour format to numeric format
> x <- as.POSIXct("9/27/2011 3:33:00 PM", format="%m/%d/%Y %H:%M:%S %p")
> x
[1] "2011-09-27 03:33:00 EDT"
> y <- as.numeric(x)
[1] 1317108780
*Typo in above code fixed
When I try to bring this back to date time, I get:
> z <- as.POSIXct(y, origin="1970-01-01")
> z
[1] "2011-09-27 08:33:00 EDT"
I tried some variants, including specifying time zones explicitly, but am consistently getting this shift.
I think it is just a problem of specifying time zones :
x <- as.POSIXct("9/27/2011 15:33:00", format="%m/%d/%Y %H:%M:%S")
> as.POSIXct(as.numeric(x), origin="1970-01-01",tz="EST") # as.numeric(x)=1317130380
[1] "2011-09-27 08:33:00 EST"
but :
x <- as.POSIXct("9/27/2011 15:33:00", format="%m/%d/%Y %H:%M:%S",tz="EST")
> as.POSIXct(as.numeric(x), origin="1970-01-01",tz="EST") # as.numeric(x)=1317155580
[1] "2011-09-27 15:33:00 EST"
remark : I simplified 03:33:00 PM in 15:33:00
I have a POSIXct object and would like to change it's tz attribute WITHOUT R to interpret it (interpret it would mean to change how the datetime is displayed on the screen).
Some background: I am using the fasttime package from S.Urbanek, which take strings and cast it to POSIXct very quickly. Problem is that the string should represent a datetime in "GMT" and it's not the case of my data.
I end up with a POSIXct object with tz=GMT, in reality it is tz=GMT+1, if I change the timezone with
attr(datetime, "tzone") <- "Europe/Paris";
datetime <- .POSIXct(datetime,tz="Europe/Paris");
then it will be "displayed" as GMT+2 (the underlying value never change).
EDIT: Here is an example
datetime=as.POSIXct("2011-01-01 12:32:23.234",tz="GMT")
attributes(datetime)
#$tzone
#[1] "GMT"
datetime
#[1] "2011-01-01 12:32:23.233 GMT"
How can I change this attribute without R to interpret it aka how can I change tzone and still have datetime displayed as "2011-01-01 12:32:23.233" ?
EDIT/SOLUTION, #GSee's solution is reasonably fast, lubridate::force_tz very slow
datetime=rep(as.POSIXct("2011-01-01 12:32:23.234",tz="GMT"),1e5)
f <- function(x,tz) return(as.POSIXct(as.numeric(x), origin="1970-01-01", tz=tz))
> system.time(datetime2 <- f(datetime,"Europe/Paris"))
user system elapsed
0.01 0.00 0.02
> system.time(datetime3 <- force_tz(datetime,"Europe/Paris"))
user system elapsed
5.94 0.02 5.98
identical(datetime2,datetime3)
[1] TRUE
To change the tz attribute of a POSIXct variable it is not best practice to convert to character or numeric and then back to POSIXct. Instead you could use the force_tz function of the lubridate package
library(lubridate)
datetime2 <- force_tz(datetime, tzone = "CET")
datetime2
attributes(datetime2)
EDITED:
My previous solution was passing a character value to origin (i.e.origin="1970-01-01"). That only worked here because of a bug (#PR14973) that has now been fixed in R-devel.
origin was being coerced to POSIXct using the tz argument of the as.POSIXct call, and not "GMT" as it was documented to do. The behavior has been changed to match the documentation which, in this case, means that you have to specify your timezone for both the origin and the as.POSIXct call.
datetime
#[1] "2011-01-01 12:32:23.233 GMT"
as.POSIXct(as.numeric(datetime), origin=as.POSIXct("1970-01-01", tz="Europe/Paris"),
tz="Europe/Paris")
#[1] "2011-01-01 12:32:23.233 CET"
This will also works in older versions of R.
An alternative to the lubridate package is via conversion to and back from character type:
recastTimezone.POSIXct <- function(x, tz) return(
as.POSIXct(as.character(x), origin = as.POSIXct("1970-01-01"), tz = tz))
(Adapted from GSee's answer)
Don't know if this is efficient, but it would work for time zones with daylight savings.
Test code:
x <- as.POSIXct('2003-01-03 14:00:00', tz = 'Etc/UTC')
x
recastTimezone.POSIXct(x, tz = 'Australia/Melbourne')
Output:
[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 14:00:00 AEDT" # Nothing is changed apart from the time zone.
Output if I replaced as.character() by as.numeric() (as GSee had done):
[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 15:00:00 AEDT" # An hour is added.