I got a Unix timestamp from a GMT-based database which I want to convert to datetime format. I use R in Germany.
Unix timestamp from GMT-based database: 1525732148
I convert it with:
install.packages("anytime")
library(anytime)
anytime(as.numeric(as.character(1525732148)))
Why do I get same date time, independent of system-date?
anytime(as.numeric(as.character(1525732148, tz="GMT")))
anytime(as.numeric(as.character(1525732148, tz="CET")))
Both give me as result: "2018-05-07 22:29:08 UTC"
I expected different results, because of different timezones.
The problem is as follows:
> as.character(1525732148, tz="GMT")
[1] "1525732148"
This is what anytime gets as input. The tz parameter is supposed to be passed to anytime, not to as.character, which swallows without warning any additional parameters it does not recognise. Try anytime(1525732148, tz="GMT").
> anytime::anytime(1525732148, tz="GMT")
[1] "2018-05-07 22:29:08 GMT"
> anytime::anytime(1525732148, tz="CET")
[1] "2018-05-08 00:29:08 CEST"
See: anytime
Related
I have a timestamp in character in the following format with a suffixed Z:
"2022-03-01T00:00:00Z"
I wanted to retrieve the day in numbers and the time in minutes (should return 0 from the above example).
I tried the execute the following code but my output only retrieve the year-month-day and not the time format.
time_stamp <- as.POSIXct(participant1_data$timestamp[1], format="%Y-%m-%dT%H:%M")
str(time_stamp)
Output: POSIXct[1:1], format: "2022-03-01"
Is there anything wrong with the code that does not retrieve the time format?
As noted in comments, use tz="UTC", otherwise the "Z" == UTC (zulu) information gets lost, also see this answer.
If time is exactly midnight, the output is omitted.
as.POSIXct('2022-03-01T00:00:00Z', tz="UTC", "%Y-%m-%dT%H:%M:%OSZ")
# [1] "2022-03-01 UTC"
as.POSIXct('2022-03-01T11:11:11Z', tz="UTC", "%Y-%m-%dT%H:%M:%OSZ")
# [1] "2022-03-01 11:11:11 UTC"
You have now read in the time correctly and got "POSIXct" class. To achieve a different output from it, you need to format it to character with the desired format using strftime which allows to specify the correct timezone tz=, e.g. "CET".
strftime(as.POSIXct('2022-03-01T00:00:00Z', tz="UTC", "%Y-%m-%dT%H:%M:%OSZ"),
'%F %R', tz='CET')
# [1] "2022-03-01 01:00"
strftime(as.POSIXct('2022-03-01T11:11:11Z', tz="UTC", "%Y-%m-%dT%H:%M:%OSZ"),
'%F %R', tz='CET')
# [1] "2022-03-01 12:11"
Or if I understand "day in numbers and the time in minutes correctly, you maybe want this:
strftime(as.POSIXct('2022-03-01T11:11:11Z', tz="UTC", "%Y-%m-%dT%H:%M:%OSZ"),
'%d %M', tz='CET')
# [1] "01 11"
Type help('strftime') or short ?strftime into the R console and read the help page for possible in/output options.
I have a large dataset with time stamps. The original data contained the time format like this:
Short-weekday, day abbreviated-month year HH:MM:SS
e.g.: Thu, 25 Nov 2021 12:40:47
When the data were saved as csv, these dates were turned into numbers and were read in as integers into R. I would like to convert these integers back to dates and time in R to continue to work with them properly.
Here are some of my data for try-out.
df <- c(1633867200, 1633874400, 1633885200, 1633869155, 1633874402, 1633885202)
This is what I tried so far with the error messages
> openxlsx::convertToDateTime(df)
Error in as.POSIXlt.character(x, tz, ...) :
character string is not in a standard unambiguous format
> as.Date(as.character(df,format = "%a, %d %b %Y %H:%M:%S", origin = "1900-01-01"))
Error in charToDate(x) :
character string is not in a standard unambiguous format
I feel I am not taking into account the structure of the integer properly (thus, the error message), but I am not sure what else to try.
Does anyone have any idea?
Thanks a lot in advance!
The numbers look like unix epoch, which can be converted to date with the origin 1970-01-01
as.POSIXct(df, format="%s", origin="1970-01-01")
[1] "2021-10-10 15:32:50 CEST" "2021-10-10 17:32:50 CEST"
[3] "2021-10-10 20:32:50 CEST" "2021-10-10 16:05:25 CEST"
[5] "2021-10-10 17:32:52 CEST" "2021-10-10 20:32:52 CEST"
I have a Time column in my df with value 1.01.2016 0:00:05. I want it without the seconds and therefore used df$Time <- as.POSIXct(df$Time, format = "%d.%m.%Y :%H:%M", tz = "Asia/Kolkata"). But I get NA value. What is the problem here?
I suspect there are two things working here: the storage of a time object (POSIXt), and the representation of that object.
The string you present is (I believe) not a proper POSIXt (whether POSIXct or POSIXlt) object for R, which means it is just a character string. In that case, you can remove it with:
gsub(':[^:]*$', '', '1.01.2016 0:00:05')
# [1] "1.01.2016 0:00"
However, that is still just a string, not a date or time object. If you parse it into a time-object that R knows about:
as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
# [1] "2016-01-01 00:00:05 IST"
then you now have a time object that R knows something about ... and it defaults to representing it (printing it on the console) with seconds-precision. Typically, all that is available to change for the console-printing is the precision of the seconds, as in
options("digits.secs")
# $digits.secs
# NULL
Sys.time()
# [1] "2018-06-26 18:21:06 PDT"
options("digits.secs"=3)
Sys.time()
# [1] "2018-06-26 18:21:10.090 PDT"
then you can get more. But alas, I do know think there is an R-option to say "always print my POSIXt objects in this way". So your only choice is (at the point where you no longer need it to be a time-like object) to change it back into a string with no time-like value:
x <- as.POSIXct("1.01.2016 0:00:05", format = "%d.%m.%Y %H:%M:%S", tz = "Asia/Kolkata")
x
# [1] "2016-01-01 00:00:05 IST"
?strptime
# see that day-of-month can either be "%d" for 01-31 or "%e" for 1-31
format(x, format="%e.%m.%Y %H:%M")
# [1] " 1.01.2016 00:00"
(This works equally well for a vector.)
Part of me suggests convert to POSIXt and back to string as opposed to my gsub example because using as.POSIXct will tell you when the string does not match the date-time-like object you are expecting, whereas gsub will happily do something wrong or nothing.
Try asPOSIXlt:
> test <- "1.01.2016 0:00:05"
> as.POSIXlt(test, "%d.%m.%Y %H:%M:%S", tz="Asia/Kolkata")
[1] "2016-01-01 00:00:05 IST"
I do not understand how the "ymd" function from the library "lubridate" works in R. I am trying to build a feature which converts the date correctly without having to specify the format. I am checking for the minimum number of NA's occurring as a result of dmy(), mdy() and ymd() functions.
So ymd() is giving NA sometimes and sometimes not for the same Date value. Are there any other functions or packages in R, which will help me get over this problem.
> data$DTTM[1:5]
[1] "4-Sep-06" "27-Oct-06" "8-Jan-07" "28-Jan-07" "5-Jan-07"
> ymd(data$DTTM[1])
[1] NA
Warning message:
All formats failed to parse. No formats found.
> ymd(data$DTTM[2])
[1] "2027-10-06 UTC"
> ymd(data$DTTM[3])
[1] NA
Warning message:
All formats failed to parse. No formats found.
> ymd(data$DTTM[4])
[1] "2028-01-07 UTC"
> ymd(data$DTTM[5])
[1] NA
Warning message:
All formats failed to parse. No formats found.
>
> ymd(data$DTTM[1:5])
[1] "2004-09-06 UTC" "2027-10-06 UTC" "2008-01-07 UTC" "2028-01-07 UTC"
[5] "2005-01-07 UTC"
Thanks
#user1317221_G has already pointed out that you dates are in day-month-year format, which suggests that you should use dmy instead of ymd. Furthermore, because your month is in %b format ("Abbreviated month name in the current locale"; see ?strptime), your problem may have something to do with your locale. The month names you have seem to be English, which may differ from how they are spelled in the locale you are currently using.
Let's see what happens when I try dmy on the dates in my locale:
date_english <- c("4-Sep-06", "27-Oct-06", "8-Jan-07", "28-Jan-07", "5-Jan-07")
dmy(date_english)
# [1] "2006-09-04 UTC" NA "2007-01-08 UTC" "2007-01-28 UTC" "2007-01-05 UTC"
# Warning message:
# 1 failed to parse.
"27-Oct-06" failed to parse. Let's check my time locale:
Sys.getlocale("LC_TIME")
# [1] "Norwegian (Bokmål)_Norway.1252"
dmy does not recognize "oct" as a valid %b month in my locale.
One way to deal with this issue would be to change "oct" to the corresponding Norwegian abbreviation, "okt":
date_nor <- c("4-Sep-06", "27-Okt-06", "8-Jan-07", "28-Jan-07", "5-Jan-07" )
dmy(date_nor)
# [1] "2006-09-04 UTC" "2006-10-27 UTC" "2007-01-08 UTC" "2007-01-28 UTC" "2007-01-05 UTC"
Another possibility is to use the original dates (i.e. in their original 'locale'), and set the locale argument in dmy. Exactly how this is done is platform dependent (see ?locales. Here is how I would do it in Windows:
dmy(date_english, locale = "English")
[1] "2006-09-04 UTC" "2006-10-27 UTC" "2007-01-08 UTC" "2007-01-28 UTC" "2007-01-05 UTC"
Using the guess_formats function in the lubridate package would be the closest to what you are after.
library(lubridate)
x <- c("4-Sep-06", "27-Oct-06","8-Jan-07" ,"28-Jan-07","5-Jan-2007")
format <- guess_formats(x, c("mdY", "BdY", "Bdy", "bdY", "bdy", "mdy", "dby"))
strptime(x, format)
HTH
from the documentation on ymd on page 70
As long as the order of formats is
correct, these functions will parse dates correctly even when the input vectors contain differently
formatted dates
ymd() expects year-month-day, you have day-month-year
x <- c("2009-01-01", "2009-01-02", "2009-01-03")
ymd(x)
maybe you need something like
y <- c("4-Sep-06", "27-Oct-06", "8-Jan-07", "28-Jan-07", "5-Jan-07" )
as.POSIXct(y, format = "%d-%b-%y")
PS the reason I think you get NAs for some is that you only have a single digit for year and ymd doesn't know what to do with that, but it works when you have two digits for year e.g. "27-Oct-06" "28-Jan-07" but fails for "5-Jan-07" etc
I have a POSIXct object and would like to change it's tz attribute WITHOUT R to interpret it (interpret it would mean to change how the datetime is displayed on the screen).
Some background: I am using the fasttime package from S.Urbanek, which take strings and cast it to POSIXct very quickly. Problem is that the string should represent a datetime in "GMT" and it's not the case of my data.
I end up with a POSIXct object with tz=GMT, in reality it is tz=GMT+1, if I change the timezone with
attr(datetime, "tzone") <- "Europe/Paris";
datetime <- .POSIXct(datetime,tz="Europe/Paris");
then it will be "displayed" as GMT+2 (the underlying value never change).
EDIT: Here is an example
datetime=as.POSIXct("2011-01-01 12:32:23.234",tz="GMT")
attributes(datetime)
#$tzone
#[1] "GMT"
datetime
#[1] "2011-01-01 12:32:23.233 GMT"
How can I change this attribute without R to interpret it aka how can I change tzone and still have datetime displayed as "2011-01-01 12:32:23.233" ?
EDIT/SOLUTION, #GSee's solution is reasonably fast, lubridate::force_tz very slow
datetime=rep(as.POSIXct("2011-01-01 12:32:23.234",tz="GMT"),1e5)
f <- function(x,tz) return(as.POSIXct(as.numeric(x), origin="1970-01-01", tz=tz))
> system.time(datetime2 <- f(datetime,"Europe/Paris"))
user system elapsed
0.01 0.00 0.02
> system.time(datetime3 <- force_tz(datetime,"Europe/Paris"))
user system elapsed
5.94 0.02 5.98
identical(datetime2,datetime3)
[1] TRUE
To change the tz attribute of a POSIXct variable it is not best practice to convert to character or numeric and then back to POSIXct. Instead you could use the force_tz function of the lubridate package
library(lubridate)
datetime2 <- force_tz(datetime, tzone = "CET")
datetime2
attributes(datetime2)
EDITED:
My previous solution was passing a character value to origin (i.e.origin="1970-01-01"). That only worked here because of a bug (#PR14973) that has now been fixed in R-devel.
origin was being coerced to POSIXct using the tz argument of the as.POSIXct call, and not "GMT" as it was documented to do. The behavior has been changed to match the documentation which, in this case, means that you have to specify your timezone for both the origin and the as.POSIXct call.
datetime
#[1] "2011-01-01 12:32:23.233 GMT"
as.POSIXct(as.numeric(datetime), origin=as.POSIXct("1970-01-01", tz="Europe/Paris"),
tz="Europe/Paris")
#[1] "2011-01-01 12:32:23.233 CET"
This will also works in older versions of R.
An alternative to the lubridate package is via conversion to and back from character type:
recastTimezone.POSIXct <- function(x, tz) return(
as.POSIXct(as.character(x), origin = as.POSIXct("1970-01-01"), tz = tz))
(Adapted from GSee's answer)
Don't know if this is efficient, but it would work for time zones with daylight savings.
Test code:
x <- as.POSIXct('2003-01-03 14:00:00', tz = 'Etc/UTC')
x
recastTimezone.POSIXct(x, tz = 'Australia/Melbourne')
Output:
[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 14:00:00 AEDT" # Nothing is changed apart from the time zone.
Output if I replaced as.character() by as.numeric() (as GSee had done):
[1] "2003-01-03 14:00:00 UTC"
[1] "2003-01-03 15:00:00 AEDT" # An hour is added.