I don't like how SQLAlchemy treats datetime values with sqlite databases. It stores the values in plain string format. At the same time sqlite recommends using julianday for storing datetime values.
Is there an easy way to change the SQLAlchemy's behaviour here?
PS. Shall I worry about it? May be noone is dealing with julianday just because it's not necessary?
Actually it is not SQLAlchemy that stores dates as plain strings; SQLite itself does not support date types. It's important that you understand that from the outset; Sqlite does provide some functions for dealing with dates, but those are dates stored as text. That's why SQLAlchemy does some magic in transforming the dates to and from python's datetime type: Per the SQLAlchemy's documentation:
SQLite does not have built-in DATE,
TIME, or DATETIME types, and pysqlite
does not provide out of the box
functionality for translating values
between Python datetime objects and a
SQLite-supported format. SQLAlchemy’s
own DateTime and related types provide
date formatting and parsing
functionality when SQlite is used. The
implementation classes are SLDateTime,
SLDate and SLTime. These types
represent dates and times as ISO
formatted strings, which also nicely
support ordering. There’s no reliance
on typical “libc” internals for these
functions so historical dates are
fully supported.
As for using the Julian calendar as opposed to the Gregorian calendar. Are you sure you want that? Might you mean Gregorian dates?
Related
How do I compare dates in SQLite database which are stored in DD-MMM-YYYY format e.g. 10-OCT-2017?
I want to compare dates and select rows of particular date and adjacent dates.
Short answer:
Painfully. You must convert the datestring, particularly the character month, into a numeric value, then you can compare the date values.
... Or more easily, by storing your dates using a format like “2018-01-01” which can be natively compared within the SQL via SQLite date functions.
Longer Answer
This page shows all the SQLite SQL date-time functions available. None of them produce or manipulate character month values. So your options are:
1) Select a group of records via SQL into a dataset; and then use your programming language to convert the date values in the dataset to a format whose values are comparable; then compare them.
This would have poor performance for anything except very small data queries, but would probably be reasonably simple to implement, and have no data conversion necessary. (To specifically answer the question you asked, this is the best solution for an app selecting few and small datasets.)
2) Create a SQLite function to do the date format conversion. To do that, you use SQLite's C API. See more how-to discussion here.
I haven't done this myself, and I wouldn’t recommend this route just due to the learning curve, but also due to the type & purpose of SQLite and its capabilities. (To specifically answer the question you asked, this is the best solution for not few and not small datasets.)
3) (This option does not answer your specific question) Convert your stored date values to a natively SQL comparable format. Like: “2018-01-01”. Then you can use the SQLite date-time functions for adjacent date comparisons.
Sample:
select mySQLTableDate, myOtherSQLTableDate
date(mySQLTableDate,'+1 day'), -- YYYY-DD-MM stored format
date(strftime('%Y-%m-%d', mySQLTableDate),'+1 day') -- many stored formats
from mySQLTable
where select mySQLTableDate = date(myOtherSQLTableDate,'+1 day')
Answering your question in terms of the goal rather than the specific question :) , my recommendation is to Use This Solution, especially if you are scanning a lot of data. See more about SQLite Date types here, but for dates with no time, I just store them as the string “2018-01-01”. I’m working with .js, and this is very simple to convert to/from a .js Date object.
According to this article, it's best to convert Date into Epoch time in order to use it is range query in DocumentDB. However, as recently the range query on Sting values has been added to DocumentDB, it is necessary to do convert date-time to epoch (as long as all date-time values have the same format and are in UTC format)?
This is similar to this question, where the accepted answer suggests using strings as you point out.
But to answer your question more specifically, DocumentDB cannot store JavaScript Date objects because it only stores pure JSON and Date is not a part of the JSON spec. So, you (or your client API) needs to do something with Date objects. By default, the node.js and .NET clients will convert Date objects to ISO-8601 formatted strings so using strings is actually a bit easier than Epoch. Just send the Date object to the database. The one trick to keep in mind here is that it's not converted back into a Date object when you read it. It comes back as a string. You have to do the conversion yourself. In JavaScript, this is easy. Just call new Date(yourDateString). Not sure about .NET or the other platforms.
I’m creating Android apps and need to save date/time of the creation record. The SQLite docs say, however, "SQLite does not have a storage class set aside for storing dates and/or times" and it's "capable of storing dates and times as TEXT, REAL, or INTEGER values".
Is there a technical reason to use one type over another? And can a single column store dates in any of the three formats from row to row?
I will need to compare dates later. For instance, in my apps I will show all records that are created between date A until date B. I am worried that not having a true DATETIME column could make comparisons difficult.
SQlite does not have a specific datetime type. You can use TEXT, REAL or INTEGER types, whichever suits your needs.
Straight from the DOCS
SQLite does not have a storage class set aside for storing dates and/or times. Instead, the built-in Date And Time Functions of SQLite are capable of storing dates and times as TEXT, REAL, or INTEGER values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November 24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these formats and freely convert between formats using the built-in date and time functions.
SQLite built-in Date and Time functions can be found here.
One of the powerful features of SQLite is allowing you to choose the storage type. Advantages/disadvantages of each of the three different possibilites:
ISO8601 string
String comparison gives valid results
Stores fraction seconds, up to three decimal digits
Needs more storage space
You will directly see its value when using a database browser
Need for parsing for other uses
"default current_timestamp" column modifier will store using this format
Real number
High precision regarding fraction seconds
Longest time range
Integer number
Lowest storage space
Quick operations
Small time range
Possible year 2038 problem
If you need to compare different types or export to an external application, you're free to use SQLite's own datetime conversion functions as needed.
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS"). REAL as Julian
day numbers, the number of days since noon in Greenwich on November
24, 4714 B.C. according to the proleptic Gregorian calendar. INTEGER
as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these
formats and freely convert between formats using the built-in date and
time functions.
Having said that, I would use INTEGER and store seconds since Unix epoch (1970-01-01 00:00:00 UTC).
For practically all date and time matters I prefer to simplify things, very, very simple... Down to seconds stored in integers.
Integers will always be supported as integers in databases, flat files, etc. You do a little math and cast it into another type and you can format the date anyway you want.
Doing it this way, you don't have to worry when [insert current favorite database here] is replaced with [future favorite database] which coincidentally didn't use the date format you chose today.
It's just a little math overhead (eg. methods--takes two seconds, I'll post a gist if necessary) and simplifies things for a lot of operations regarding date/time later.
Store it in a field of type long. See Date.getTime() and new Date(long)
The program I am currently designing use the dd/mm/yyyy date format, while Sqlite standard format is yyyy-mm-dd. My program make use of quite a lot of date calculations using julianday('yyyy-mm-dd'). I know I could convert the dd/mm/yyyy format to yyyy-mm-dd by using SUBSTR(X,Y) manipulation or by using the code of the language I am designing the db front-end; but i wish to avoid those. Any Idea?
You should always store dates (and timestamps) using native date format that is provided by database engine for following reasons:
Native formats permit native date arithmetic functions to work.
Native formats permit indexes to be consistently applicable, so you can use date comparisons efficiently and use operators like BETWEEN.
Native formats take less space to store on disk. For SQLite, storing date as real number of days from 4174 BC or as integer number of seconds since Jan 1st, 1970 takes 8 bytes. For your representation, it will take at least 10 bytes.
While SQLite does not really have true native date/datetime type (which is big omission in my opinion), it does have 3 permissible formats: TEXT, REAL or INTEGER that are still treated (to some extent) as native datetime formats, and all advantages outlined above still apply.
When you need to display dates in your application, you should use libraries provided by your scripting or other programming languages that know how to display dates in desired format.
In other words, use database to store, compare and retrieve data, and use your application to render it in desired format.
Just getting into SQLite and I understand it does not use datatypes the same way as other languages.
I'm building a database and it has to store date and time quite a lot. And I've read a lot about the date and time functions, etc, but I just want to make sure that in my CREATE script I shouldn't have anything other than
BirthDate TEXT
DateTime TEXT
I'm not sure exactly what your question is, but I think the below excerpt may prove to be useful (taken from http://www.sqlite.org/datatype3.html):
SQLite does not have a storage class set aside for storing dates
and/or times. Instead, the built-in Date And Time Functions of SQLite
are capable of storing dates and times as TEXT, REAL, or INTEGER
values:
TEXT as ISO8601 strings ("YYYY-MM-DD HH:MM:SS.SSS").
REAL as Julian day numbers, the number of days since noon in Greenwich on November
24, 4714 B.C. according to the proleptic Gregorian calendar.
INTEGER as Unix Time, the number of seconds since 1970-01-01 00:00:00 UTC.
Applications can chose to store dates and times in any of these
formats and freely convert between formats using the built-in date and
time functions.