I am working with a sqlite database table. I have pulled the data into R using the RSQLite package. One of the columns holds a date. Sqlite is storing it as a Real number, the number of days since noon in Greenwich on November 24, 4714 B.C. (e.g.1264896000). Any ideas on how to convert this to a valid date in R? I tried the following
as.POSIXct(1264896000,origin = "-4714-11-24")
However, this doesn't work as the character string in not in a standard form. Any ideas?
I tested my theory that your claim about the origin was unlikely. The theory that these are POSIX date-times (origin= 1970-01-01 and times in seconds) seems supported by experiment.
> as.POSIXct(1264896000,origin = "1970-01-01")
[1] "2010-01-30 16:00:00 PST"
Related
I have two columns, DATE_A and DATE_B.
I need to find how much time is between the two dates.
Usually, I would use JULIANDAY() and subtract one date from another, but the output is null because of the "+0000" part.
Below you'll find an example of values contained in the two columns:
DATE_A - '2022-05-12 00:16:17.553 +0000'
DATE_B - '2022-06-02 00:02:01.158 +0000'
Please tell me what '+0000' means and how can I find the time elapsed between the two dates.
+0000 is the offset from UTC this time represents in hours and minutes. For example, here in the US Pacific it's daylight savings time and we're 7 hours behind UTC so we're -0700. 2022-05-12 08:00:00+0000 and 2022-05-12 01:00:00-0700 are the same point in time.
SQLite will accept a slightly different format. There has to be the : separator between hours and minutes.
2022-05-12 00:16:17.553 +00:00
^
You'll have to change the format. Use your programming language's date and time functions.
See "Time Values" in SQLite Date and Time Functions for valid formats.
I have a column of dates with the following format in excel: MM/DD/YY AM or MM/DD/YY PM and was able to parse this date after importing with readxl::read_excel.
parse_date_time(x, '%m/%d/%y %p', tz = "UTC")
Now, if I wanted to bring in MM/DD/YY HH:MM PM instead, the import comes in as a number. For example.
"3/16/20 3:00 PM" becomes 43906.625 after import.
One solution would be to import date columns as strings, however, I have 50 columns in the file and don't want to hard code each column type. Is there a way to get the date and time from this numerical value instead (i.e. 43906.625)?
Excel uses a "day-integer" format. R uses "seconds-integer" for time and "day-integer" for Date, so depending on which class you are converting to, you need to accommodate a day of seconds (86,400). It is also worth knowing that Excel uses an "origin" from 1899 (the year).
as.POSIXct(43906.625 * 86400, origin = "1899-12-30", tz = "UTC")
# [1] "2020-03-16 15:00:00 UTC"
As a bit of history: the reason that it's "1899-12-30" and not, say "1899-12-31" (end of the day?) or something else is mentioned in a blog post from 2013:
For Excel on Windows, the origin date is December 30, 1899 for dates after 1900. (Excel’s designer thought 1900 was a leap year, but it was not.) For Excel on Mac, the origin date is January 1, 1904.
https://www.r-bloggers.com/date-formats-in-r/
I don't know the canonical reference for this, and the website from which R-Bloggers borrowed/scraped that article from is not responsive. I would much prefer to have still-active and more-canonical references for this assertion (that engineers mis-identified the leap-year).
I have a table Orders with Order_Date datatype is smalldatetime and my Order_Date Format is 01/10/2018 10:00:00 PM
Now I want to filter data between 01/10/2018 04:00:00 PM AND 02/10/2018 04:00:00 AM
What I tried
SELECT distinct(Order_No),Order_Date from Orders WHERE Order_Date BETWEEN '01/10/2018 04:00:00 PM' and '02/10/2018 04:00:00 AM'
This query is showing only 01/10/2018 Data but I want the data BETWEEN 01/10/2018 04:00:00 PM and 02/10/2018 04:00:00 AM
Is there any way to get the data from today 4PM To Next Day 4AM?
First off, sqlite does not have actual date/time types. It's a simple database with only a few types. Your smalldatetime column actually has NUMERIC affinity (See the affinity rules).
For Sqlite's builtin functions to be able to understand them, date and times can be stored as numbers or text; numbers are either the number of seconds since the Unix epoch, or a Julian day. Text strings can be one of a number of formats; see the list in the docmentation. All these have the additional advantage that, when compared to other timestamps in the same format, they can be properly sorted.
You seem to be using text strings like '01/10/2018 04:00:00 PM'. This is not one of the formats that sqlite date and time functions understand, and it doesn't sort naturally, so you can't use it in comparisons aside from testing equality. Plus it's ambiguous: Is it October 1, or January 10? Depending on where you're from you'll have a different interpretation of it.
If you change your timestamp format to a better one like (Assuming October 1) '2018-10-01 16:00:00', you'll be able to sort and compare ranges, and use it with sqlite functions.
SAS documentation states the following for data and datetime values:
SAS time value: is a value representing the number of seconds since midnight of the current day. SAS time values are between 0 and 86400.
SAS datetime value: is a value representing the number of seconds between January 1, 1960 and an hour/minute/second within a specified date.
I'm willing to convert the following date and hour values with R, I have a big doubt for the hour (datetime) conversion, which one of the "HH:MM:SS" values within R_hour1 and R_hour2 is correct ?
I have to separate columns, SAS date = 20562 and SAS hour = 143659, in my table
R: R_date <- as.Date(as.integer(20562), origin="1960-01-01"); R_date
[1] "2016-04-18"
R: R_hour1 <- as.POSIXct(143659, origin = R_date); R_hour1
[1] "2016-04-19 17:54:19 CEST"
R: R_hour2 <- as.POSIXct(143659, origin = "1960-01-01"); R_hour2
[1] "1960-01-02 16:54:19 CET"
Similar to R, SAS Date and DateTime values can have whatever origin you wish them to. The default formats have a default (1/1/1960 for both), but you can use the datetime field to mean any origin you wish, and it will generally still work perfectly well with any of the datetime functions (though it will not display properly unless you write a custom format). It is very possible to have a different origin, as you show above with R_hour1.
As such, you would have to ask the person who generated the data what the meaning of the field is and what its origin should be.
I use rmongodb to query a MongoDB. I connect to the DB which works nicely (require(rmongodb); mongo <- mongo.create("foo")) and I am generally able to get stuff out of the database. I just don't know what to do about the date formats..
TIME <- strptime("2013-11-11 15:00",format="%Y-%m-%d %H:%M",tz="CET")
query = mongo.bson.buffer.create()
mongo.bson.buffer.append(query, "timestamp", TIME)
query = mongo.bson.from.buffer(query)
when I look at this query it says:
timestamp : 9 1198930688
So mongo.bson.buffer.append has properly recognized that timestamp is a date class and does some conversion -- which I don't understand. This is not UNIX time and I would not really care if the values returned from the database weren't in this format as well. I'm particularly puzzled because quite some of these numeric date values are negative while all my dates are from 2013... Some more examples:
# 2013-10-10 12:15 --> -1579369312
# 2013-10-10 12:30 --> -1578469312
# 2013-11-10 12:30 --> 1103530688
So basically my question is: How can I convert this funny date format (1198930688) back to POSIXct?
Thanks a lot!
skr
Try
myTIME <- mongo.bson.value( query, "timestamp" )
myTIME
[1] "2013-11-11 15:00:00 CET"