I am trying to convert a supposedly UNIX epoch (obtained from GPS) to date object in R. I tried to use the following code, but I got an unrealistic answer as a result (shown below). Could anybody tell me what type of UNIX this is and how to convert it ?
x <- 2.01604*10^13 # Unix epoch I need to convert
as.POSIXct(as.numeric(as.character(x)), origin="1970-01-01", tz="Asia/Shanghai")
> > "640827-08-24 07:06:40 CST" # result
Related
I used this script
df?timestamp< as.POSIXct(df$timestamp,tz="",format="%Y%m%d%H%M%S",origin = "1970-01-01")
then i get NA's.
The number i have is 43466.10 and the result i should get is 2019-01-01 02:17:26.
What am i supposed to do?
In fact, 43466.10 is not a valid Unix epoch time. See https://en.wikipedia.org/wiki/Unix_time for more information.
This number looks like an Excel date/time value. If that truly is, then "2019-01-01 02:17:26" is actually 43466.0954398148, and the origin you should use is "1899-12-30", not "1970-01-01".
You cannot use as.POSIXct directly in this case because UNIX epoch time is in counting seconds, but an Excel date/time value is in counting days.
You can use the package openxlsx to do this conversion. Try this:
# install.packages("openxlsx")
df$timestamp <- openxlsx::convertToDateTime(df$timestamp, origin = "1899-12-30")
I have a data frame called RequisitionHistory2 with a variable called RequisitionDateTime and the levels are factors which look like 4/30/2019 14:16 I would like to split this into RequisitionDate and RequisitionTime in a datetime format.
I tried this code, but this still does not solve my issue with needing to split these into their own columns. The code also did not work as I got the error below.
mutate(When = as.POSIXct(RequisitionHistory2, format="%m/%d/%. %H:%M %p"))
Error in as.POSIXct.default(RequisitionHistory2, format = "%m/%d/%. %H:%M %p") : do not know how to convert 'RequisitionHistory2' to class “POSIXct”
I would like to have the variable RequisitionDateTime split into RequisitionDate and another variable RequisitionTime in the dataframe RequisitionHistory2. Any help is greatly appreciated!
Do not convert factors to datetime directly. You will need to convert it to a character first and then use a datetime function.
as.Date(as.character("10/25/2018"), format = "%m/%d/%Y")
would work for your date example.
library(lubridate)
mutate(df,When = mdy_hm(RequisitionHistory2))
If your datetime is in 4/30/2019 14:16 format
Note that as.POSIXct() works only on datetimes already in ISO 8601 format. I wrote a blog post about this and I think would be helpful for you to check out:
https://jackylam.io/tutorial/uber-data/
The anytime package ON CRAN directly converts from many formats, including factor and ordered to dates and datetime objects. It also heuristically tries a number of viable formats so that you do not need a format string. See the README at GitHub for an introduction, there is also a vignette
Your example works:
R> library(anytime)
R> anytime(as.factor("4/30/2019 14:16"))
[1] "2019-04-30 14:16:00 CDT"
R> anytime(as.factor("4/3/2019 14:16:17"), useR=TRUE)
[1] "2019-04-03 14:16:17 CDT"
R>
However, the underlying (Boost C++) parser does not like single digit days or month so you may need to flip back to R's parser via useR=TRUE as I did on the second example.
Trying to convert a full column of Unix into date time in R. I've been able to use
as.POSIXlt(x, origin = "1970-01-01")
to convert Unix. But now I'm trying to convert a whole column of Unix into date time using:
as.POSIXct(x$time, origin="1970-01-01")
Which gives me:
Error in as.POSIXlt.character(as.character(x), ...) :
character string is not in a standard unambiguous format`
What is the best way to be doing this?
as.POSIXct(as.numeric(x$time), origin="1970-01-01")
I have date format in following format in a data frame:
Jan-85
Apr-99
1-Nov
Feb-96
When I see the typeof(df$col) I get the answer as "integer".
Actually when I see the format in excel it is in m/d/yyyy format. I was trying to convert this to date format in R. All my efforts yielded NA.
I tried parse_date_time function. I tried as.date along with as.character. I tried as.POSIXct but everything is giving me NA.
My trials were as follows and everything was a failure:
as.Date.numeric(df$col,"m%d%Y")
transform(df$col, as.Date(as.character(df$col), "%m%d%Y"))
as.Date(df$col,"m%d%Y")
as.POSIXct.numeric(as.character(loan_new$issue_d), format="%Y%m%d")
as.POSIXct.date(as.character(df$col), format="%Y%m%d")
mdy(df$col)
parse_date_time(df$col,c("mdy"))
How can I convert this to date format? I have used lubridate package for parse_date_time and mdy package.
dput output is below
Label <- factor(c("Apr-08",
"Apr-09", "Apr-10", "Apr-11", "Aug-07", "Aug-08", "Aug-09", "Aug-10",
"Aug-11", "Dec-07", "Dec-08", "Dec-09", "Dec-10", "Dec-11", "Feb-08",
"Feb-09", "Feb-10", "Feb-11", "Jan-08", "Jan-09", "Jan-10", "Jan-11",
"Jul-07", "Jul-08", "Jul-09", "Jul-10", "Jul-11", "Jun-07", "Jun-08",
"Jun-09", "Jun-10", "Jun-11", "Mar-08", "Mar-09", "Mar-10", "Mar-11",
"May-08", "May-09", "May-10", "May-11", "Nov-07", "Nov-08", "Nov-09",
"Nov-10", "Nov-11", "Oct-07", "Oct-08", "Oct-09", "Oct-10", "Oct-11",
"Sep-07", "Sep-08", "Sep-09", "Sep-10", "Sep-11"))
NA is typically what you get when you misspecify the format. Which is what you do. That said, if your data is really looking like the first example you gave, it's impossible to simply convert this to a date. You have two different formats, one being month-year and the other day-month.
If your updated date (i.e. Dec-11) is the correct format, then you use the format argument of as.Date like this:
date <- "Dec-11"
as.Date(date, format = "%b-%d")
# [1] "2017-12-11"
Or on your example data:
as.Date(Label, format = "%b-%d")
# [1] "2017-04-08" "2017-04-09" "2017-04-10" "2017-04-11" "2017-08-07" "2017-08-08"
# [7] "2017-08-09" "2017-08-10" "2017-08-11" "2017-12-07" "2017-12-08" "2017-12-09"
If you want to convert something like Jan-85, you have to decide which day of the month that date should have. Say we just take the first of each month, then you can do:
x <- "Jan-85"
xd <- paste0("1-",x)
as.Date(xd, "%d-%b-%y")
# [1] "1985-01-01"
More information on the format codes can be found on ?strptime
Note that R will automatically add this year as the year. It has to, otherwise it can't specify the date. In case you do not have a day of the month (eg like Jan-85), conversion to a date is impossible because the underlying POSIX algorithms don't have all necessary information.
Also keep in mind that this only works when your locale is set to english. Otherwise you have a big chance your OS won't recognize the month abbreviations correctly. To do so, do eg:
Sys.setlocale(category = "LC_TIME", locale = "English_United Kingdom")
You can later set it back to the original one if you must, or restart your R session to reset the locale settings.
note: Please check carefully which locale notations are valid for your OS. The above example works on Windows, but is not guaranteed on either Linux or Mac.
Why you see integer
The fact that these string values are of integer type, is due to the fact that R automatically convert character vectors to factors when reading in a data frame. So typeof() returns integer because that's the internal representation of a factor.
All of these dates that I’ve manipulated in Execute R module in Azure Machine Learning write out as blank in the output – that is, these date columns exist, but there is no value in those columns.
The source variables which contain date information that I’m reading into the data frame have two different date formats. They are as follows:
usage$Date1=c(‘8/6/2015’ ‘8/20/2015’ ‘7/9/2015’)
usage$Date2=c(‘4/16/2015 0:00’, ‘7/1/2015 0:00’, ‘7/1/2015 0:00’)
I inspected the log file in AML, and AML can't find the local time zone.
The log file warnings specifically:
[ModuleOutput] 1: In strptime(x, format, tz = tz) :
[ModuleOutput] unable to identify current timezone 'C':
[ModuleOutput] please set environment variable 'TZ' [ModuleOutput]
[ModuleOutput] 2: In strptime(x, format, tz = tz) : unknown timezone 'localtime'
I referred to another answer regarding setting default time zone for strptime here
unknown timezone name in R strptime/as.POSIXct
I changed my code to explicitly define the global environment time variable.
Sys.setenv(TZ='GMT')
####Data frame usage cleanup, format and labeling
usage<-as.data.frame(usage)
usage$Date1<-as.character(usage$Date1)
usage$Date1<-as.POSIXct(usage$Date1, "%m/%d/%Y",tz="GMT")
usage$Date1<-format(usage$Date1, "%m/%d/%Y")
usage$Date1<-as.Date(usage$Date1, "%m/%d/%Y")
usage<-as.data.frame(usage)
usage$Date2<- as.POSIXct(usage$Date2, "%m/%d/%Y",tz="GMT")
usage$Date2<- format(usage$Date2,"%m/%d/%Y")
usage$Date2<-as.Date(usage$Date2, "%m/%d/%Y")
usage<-as.data.frame(usage)
The problem persists -as a result AzureML does not write these variables out, rather writing out these columns as blanks.
(This code works in R studio, where I presume the local time is taken from the system.)
After reading two blog posts on this problem, it seems that Azure ML doesn't support some date time formats:
http://blogs.msdn.com/b/andreasderuiter/archive/2015/02/03/troubleshooting-error-1000-rpackage-library-exception-failed-to-convert-robject-to-dataset-when-running-r-scripts-in-azure-ml.aspx
http://www.mikelanzetta.com/2015/01/data-cleaning-with-azureml-and-r-dates/
So I tried to convert to POSIXct before sending it to the output stream, which I've done as follows:
tenantusage$Date1 = as.POSIXct(tenantusage$Date1 , "%m/%d/%Y",tz = "EST5EDT");
tenantusage$Date2 = as.POSIXct(tenantusage$Date2 , "%m/%d/%Y",tz = "EST5EDT");
But encounter the same problem. The information in these variables refuses to write out to the output. Date1 and Date2 columns are blank.
Please advise!
thanks
Hi SingingData and SochiX,
Sorry to hear about this source of frustration! I find that the following variation on SingingData's code sample works for me (tested in a CRAN 3.1.0 module):
usage <- data.frame(list(Date1 = c('8/6/2015', '8/20/2015', '7/9/2015'),
Date2 = c('4/16/2015 0:00', '7/1/2015 0:00', '7/1/2015 0:00')))
usage$Date1 <- as.POSIXlt(usage$Date1, "%m/%d/%Y",tz="GMT")
usage$Date2 <- as.POSIXlt(usage$Date2, "%m/%d/%Y",tz="GMT")
usage$Date1 <- format(usage$Date1, "%m/%d/%Y")
usage$Date2 <- format(usage$Date2,"%m/%d/%Y")
usage$Date1 <- as.Date(usage$Date1, "%m/%d/%Y")
usage$Date2 <- as.Date(usage$Date2, "%m/%d/%Y")
maml.mapOutputPort("usage");
I've used as.POSIXlt() instead of as.POSIXct(). I hope that this helps unblock your work in R.