I'm familiar with how type affinity works in SQLite: You can declare column types as anything you want, and all that matters is whether the type name contains "INT", "CHAR", "FLOA", etc. But is there a commonly-used convention on what type names to use?
For example, if you have an integer column, is it better to distinguish between TINYINT, SMALLINT, MEDIUMINT, and BIGINT, or just declare everything as INTEGER?
So far, I've been using the following:
INTEGER
REAL
CHAR(n) -- for strings with a known fixed with
VARCHAR(n) -- for strings with a known maximum width
TEXT -- for all other strings
BLOB
BOOLEAN
DATE -- string in "YYYY-MM-DD" format
TIME -- string in "HH:MM:SS" format
TIMESTAMP -- string in "YYYY-MM-DD HH:MM:SS" format
(Note that the last three are contrary to the type affinity.)
I would recommend not using self-defined types. I have observed in version 3.5.6 that types not already defined could sometimes cause an INSERT command to be refused. Maybe 1 out of 1000. I don't know if this was addressed since.
In any case, there is no sizing advantage in typing a column TINYINT or SMALLINT. The only advantage would be outside SQLite, for either parsing your column types with another program or to satisfy your personal need for tidiness. So I strongly recommend using the base types defined by SQLite and sticking to those.
Since SQLite is typeless, use whatever types make it easier for you to see what the schema looks like. Or you can match the types to your codebase.
I'm going to go with Kevin on this one. In short, knock yourself out. Make up brand new areas of mathematics if it suits your schema. Use the classnames of your ORM. Or name every type (except the PRIMARY KEY INTEGER ones) for ex-girlfriends. In the end SQLite is more about how you access and use the data.
Related
After reading https://sqlite.org/datatype3.html which states
"SQLite does not have a storage class set aside for storing dates
and/or times."
but able to run this
CREATE TABLE User (ID INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, BORN_ON DATE NULL)
and then see it in "DB Browser for SQL" like this:
I start to wonder if SQLite does support Date type of it is just "faking" the support using other types. And even if so why the DB Browser see it as a Date? Any meta info stored inside the DB?
SQLite does not fake Date with Numerics.
There is no Date data type in SQLite.
In Datatypes In SQLite Version 3 it is explained clearly that:
SQLite uses a more general dynamic type system
Instead of data types there are 5 Storage Classes: NULL, INTEGER, REAL, TEXT and BLOB.
Also:
Any column in an SQLite version 3 database, except an INTEGER PRIMARY
KEY column, may be used to store a value of any storage class.
So when you use Date as the data type of a column in the CREATE TABLE statement you are not restricted to store in it only date-like values. Actually you can store anything in that column.
Tools like "DB Browser for SQLite" and others may offer various data types to select from to define a column when you create the table.
The selection of the data type that you make is not restrictive, but it is rather indicative of what type of data you want to store in a column.
In fact, you can create a table without even declaring the data types of the columns:
CREATE TABLE tablename(col1, col2)
or use fictional data types:
CREATE TABLE tablename(col1 somedatatype, col2 otherdatatype)
and insert values of any data type:
INSERT INTO tablename(col1, col2) VALUES
(1, 'abc'),
('XYZ', '2021-01-06'),
(null, 3.5)
Based on what Colonel Thirty Two suggested (read more on the page) it seems that when you declare a field as Date its affinity will be numeric.
So SQLite "fakes" Date with Numerics.
And even if so why the DB Browser see it as a Date? Any meta info stored inside the DB?
Yes, it simply stores the type name used when the column was created. The linked page calls it "declared type". In this case you get NUMERIC affinity (DATE is even given as one of the examples in 3.1.1) and it behaves like any other column with this affinity:
A column with NUMERIC affinity may contain values using all five storage classes. When text data is inserted into a NUMERIC column, the storage class of the text is converted to INTEGER or REAL (in order of preference) if the text is a well-formed integer or real literal, respectively. If the TEXT value is a well-formed integer literal that is too large to fit in a 64-bit signed integer, it is converted to REAL. For conversions between TEXT and REAL storage classes, only the first 15 significant decimal digits of the number are preserved. If the TEXT value is not a well-formed integer or real literal, then the value is stored as TEXT. For the purposes of this paragraph, hexadecimal integer literals are not considered well-formed and are stored as TEXT. (This is done for historical compatibility with versions of SQLite prior to version 3.8.6 2014-08-15 where hexadecimal integer literals were first introduced into SQLite.) If a floating point value that can be represented exactly as an integer is inserted into a column with NUMERIC affinity, the value is converted into an integer. No attempt is made to convert NULL or BLOB values.
A string might look like a floating-point literal with a decimal point and/or exponent notation but as long as the value can be expressed as an integer, the NUMERIC affinity will convert it into an integer. Hence, the string '3.0e+5' is stored in a column with NUMERIC affinity as the integer 300000, not as the floating point value 300000.0.
So if you insert dates looking like e.g. "2021-01-05" they will be stored as strings. But
you can also insert strings which don't look like dates.
if you insert "20210105" it will be stored as the number 20210105.
You can use CHECK constraints to prevent inserting non-date strings.
See also https://sqlite.org/lang_datefunc.html which says what (string and number) formats date/time functions expect.
I don't know what is the difference between SQLite NVARCHAR and NVARCHAR2 column.
I know that NVARCHAR is a Unicode-only text column, but what about NVARCHAR2?
There is a difference. In a way...
Here´s the thing:
As Lasse V. Karlsen says, SQLite does not act on the types you mentioned nor does it restrict the length by an argument passed in like in NVARCHAR(24) (but you could do check constraints to restrict length).
So why are these available in SQLite Expert (and other tools)?
This info will be saved in the database schema (please check https://www.sqlite.org/datatype3.html#affinity and http://www.sqlite.org/pragma.html#pragma_table_info) So should you bother to set these when creating a SQLite db as it will not be used by SQLite?
Yes if you will be using any tool to generate code from the schema! Maybe somebody will ask you to transfer the db to MSSQL, then there are some great tools that will use the schema and will map your SQLite to MSSQL in a blink. Or maybe you will use some .NET tool to map the tables into POCO classes, and these can also use the schema to map to the correct type and they will also use the restrictions and transfer these into data annotations on the properties that the columns map to. And EntityFramework 7 will have support built in for SQLite and their code generation will surely make use of the schema.
There is no difference.
SQLite does not operate with strict data types like that, it has "storage classes".
If you check the official documentation you'll find this rule, one of five used to determine which storage class to assign to a column from the data type you specify:
If the declared type of the column contains any of the strings "CHAR", "CLOB", or "TEXT" then that column has TEXT affinity. Notice that the type VARCHAR contains the string "CHAR" and is thus assigned TEXT affinity.
There are 5 rules in total but rule 2 covers NVARCHAR and NVARCHAR2 and both will assign the storage class TEXT to the column.
SEL * FROM TABLE WHERE a=10
VS
SEL * FROM TABLE WHERE a='10'
Here a is BIGINT, Explain plan does not show any difference, how teradata handles this and any difference in these query ?
Teradata automatically applies a datatype conversion if you compare different datatypes (usually but not always).
Whenever a string is compared to a number the string will be converted to a FLOAT, which is the most flexible numeric format.
In your case this conversion was already done by the parser, so the optimizer didn't know 10 was s string before.
If you do it the other way:
SEL * FROM TABLE WHERE a=10 -- column a is a character
you can spot this cast in explain:
"(table.last_name (FLOAT, FORMAT '-9.99999999999999E-999'))= 1.00000000000000E 001"
Sometimes this automatic conversion is convenient, but in a case like that it's really bad: No index can be used and all existing statistics are lost. So you better know you datatypes :-)
This (FLOAT, FORMAT '-9.99999999999999E-999')) in Explain is one of the first things I check if a query performs badly.
I use sqlite3 C/C++ API to retrieve rows from a table using SELECT query. I don't see any sqlite3_column_timestamp() to retrieve a timestamp column value after sqlite3_step().. How to get timestamp values ?
SQLite does not have a special timestamp data type.
When you want to use any of SQLite's date and time functions, you have to store timestamps in one of the formats supported by them, i.e., a string like YYYY-MM-DD HH:MM:SS or HH:MM:SS, a julian date number, or a Unix timestamp number.
You can declare a table column type as DATETIME, but SQLite will just ignore that type; SQLite always allows to put values of any type in any column. Such a declaration would be useful only as documentation.
The column/value accessors will only have types corresponding to the data types they support directly (NULL, INTEGER, REAL, TEXT, BLOB).
You would use the TEXT access to get/set the column value of dates.
There are some helper functions within SQL that they provide that let you to handle them in your queries.
I am not familiar with SQLite Manager, but I would assume that it is only reporting the data type that the table was declared with.
When parsing CREATE statements, sqlite understands the intention of many well supported datatypes and automatically maps them to what is appropriate for its internal storage structure. VARCHAR would be mapped to TEXT, for instance. I assume the column was declared DATETIME and sqlite just internally mapped it to TEXT.
When creating a table in SQLite3, I get confused when confronted with all the possible datatypes which imply similar contents, so could anyone tell me the difference between the following data-types?
INT, INTEGER, SMALLINT, TINYINT
DEC, DECIMAL
LONGCHAR, LONGVARCHAR
DATETIME, SMALLDATETIME
Is there some documentation somewhere which lists the min./max. capacities of the various data-types? For example, I guess smallint holds a larger maximum value than tinyint, but a smaller value than integer, but I have no idea of what these capacities are.
SQLite, technically, has no data types, there are storage classes in a manifest typing system, and yeah, it's confusing if you're used to traditional RDBMSes. Everything, internally, is stored as text. Data types are coerced/converted into various storage locations based on affinities (ala data types assigned to columns).
The best thing that I'd recommend you do is to :
Temporarily forget everything you used to know about standalone database datatypes
Read the above link from the SQLite site.
Take the types based off of your old schema, and see what they'd map to in SQLite
Migrate all the data to the SQLite database.
Note: The datatype limitations can be cumbersome, especially if you add time durations, or dates, or things of that nature in SQL. SQLite has very few built-in functions for that sort of thing. However, SQLite does provide an easy way for you to make your own built-in functions for adding time durations and things of that nature, through the sqlite3_create_function library function. You would use that facility in place of traditional stored procedures.
The difference is syntactic sugar. Only a few substrings of the type names matter as for as the type affinity is concerned.
INT, INTEGER, SMALLINT, TINYINT → INTEGER affinity, because they all contain "INT".
LONGCHAR, LONGVARCHAR → TEXT affinity, because they contain "CHAR".
DEC, DECIMAL, DATETIME, SMALLDATETIME → NUMERIC, because they don't contain any of the substrings that matter.
The rules for determining affinity are listed at the SQLite site.
If you insist on strict typing, you can implement it with CHECK constraints:
CREATE TABLE T (
N INTEGER CHECK(TYPEOF(N) = 'integer'),
Str TEXT CHECK(TYPEOF(Str) = 'text'),
Dt DATETIME CHECK(JULIANDAY(Dt) IS NOT NULL)
);
But I never bother with it.
As for the capacity of each type:
INTEGER is always signed 64-bit. Note that SQLite optimizes the storage of small integers behind-the-scenes, so TINYINT wouldn't be useful anyway.
REAL is always 64-bit (double).
TEXT and BLOB have a maximum size determined by a preprocessor macro, which defaults to 1,000,000,000 bytes.
Most of those are there for compatibility. You really only have integer, float, text, and blob. Dates can be stored as either a number (unix time is integer, microsoft time is float) or as text.
NULL. The value is a NULL value.
INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.
REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number.
TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE).
BLOB. The value is a blob of data, stored exactly as it was input.
As an addition to answer from dan04, if you want to blindly insert a NUMERIC other than zero represented by a TEXT but ensure that text is convertible to a numeric:
your_numeric_col NUMERIC CHECK(abs(your_numeric_col) <> 0)
Typical use case is in a query from a program that treats all data as text (for uniformity & simplicity, since SQLite already does so). The nice thing about this is that it allows constructs like this:
INSERT INTO table (..., your_numeric_column, ...) VALUES (..., some_string, ...)
which is convenient in case you're using placeholders because you don't have to handle such non-zero numeric fields specially. An example using Python's sqlite3 module would be,
conn_or_cursor.execute(
"INSERT INTO table VALUES (" + ",".join("?" * num_values) + ")",
str_value_tuple) # no need to convert some from str to int/float
In the above example, all values in str_value_tuple will be escaped and quoted as strings when passed to SQlite. However, since we're not checking explicitly the type via TYPEOF but only convertibility to type, it will still work as desired (i.e., SQLite will either store it as a numeric or fail otherwise).