SEL * FROM TABLE WHERE a=10
VS
SEL * FROM TABLE WHERE a='10'
Here a is BIGINT, Explain plan does not show any difference, how teradata handles this and any difference in these query ?
Teradata automatically applies a datatype conversion if you compare different datatypes (usually but not always).
Whenever a string is compared to a number the string will be converted to a FLOAT, which is the most flexible numeric format.
In your case this conversion was already done by the parser, so the optimizer didn't know 10 was s string before.
If you do it the other way:
SEL * FROM TABLE WHERE a=10 -- column a is a character
you can spot this cast in explain:
"(table.last_name (FLOAT, FORMAT '-9.99999999999999E-999'))= 1.00000000000000E 001"
Sometimes this automatic conversion is convenient, but in a case like that it's really bad: No index can be used and all existing statistics are lost. So you better know you datatypes :-)
This (FLOAT, FORMAT '-9.99999999999999E-999')) in Explain is one of the first things I check if a query performs badly.
Related
I found a method json_insert in the json section of the SQLite document. But it seems to be not working in the way that I expected.
e.g. select json_insert('[3,2,1]', '$[3]', 4) as result;
The result column returns '[3,2,1,4]', which is correct.
But for select json_insert('[3,2,1]', '$[1]', 4) as result;
I am expecting something like '[3,2,4,1]' to be returned, instead of '[3,2,1]'.
Am I missing something ? I don't see there is an alternative method to json_insert.
P.S. I am playing it on https://sqlime.org/#demo.db, the SQLite version is 3.37.2.
The documentation states that json_insert() will not overwrite values ("Overwrite if already exists? - No"). That means you can't insert elements in the middle of the array.
My interpretation: The function is primarily meant to insert keys into an object, where this kind of behavior makes more sense - not changing the length of an array is a sacrifice for consistency.
You could shoehorn it into SQLite by turning the JSON array into a table, appending your element, sorting the result, and turning it all back into a JSON array:
select json_group_array(x.value) from (
select key, value from json_each('[3,2,1]')
union
select 1.5, 4 -- 1.5 = after 1, before 2
order by 1
) x
This will produce '[3,2,4,1]'.
But you can probably see that this won't scale, and even if there was a built-in function that did this for you, it wouldn't scale, either. String manipulation is slow. It might work well enough for one-offs, or when done infrequently.
In the long run, I would recommend properly normalizing your database structure instead of storing "non-blob" data in JSON blobs. Manipulating normalized data is much easier than manipulating JSON, not to mention faster by probably orders of magnitude.
SQLite doesn't really have date columns. You can store your dates as ISO-8601 strings, or as the integer number of seconds since the epoch, or as Julian day numbers. In the table I'm using, I want my dates to be human-readable, so I've chosen to use ISO-8601 strings.
Suppose I want to query all the records with dates after today. The ISO-8601 strings will sort properly, so I should be able to use string comparison with the ISO-8601 string for today's date.
However, I see no way to do the comparison using the F# SqlProvider type provider. I'm hoping that this is just a reflection of my lack of knowledge of F# query expressions.
For instance, I can't do:
query {
for calendarEntry in dataContext.``[main].[calendar_entries]`` do
where (calendarEntry.date >= System.DateTime.Today.ToString("yyyy-MM-dd hh:mm:ss"))
... }
I get:
The binary operator GreaterThanOrEqual is not defined for the types 'System.String' and 'System.String'.
I also can't do any variation of:
query {
for calendarEntry in dataContext.``[main].[calendar_entries]`` do
where (calendarEntry.date.CompareTo(System.DateTime.Today.ToString("yyyy-MM-dd hh:mm:ss")) >= 0)
... }
I get:
Unsupported expression. Ensure all server-side objects appear on the left hand side of predicates. The In and Not In operators only support the inline array syntax.
Anyone know how I might do string comparisons in the where clause? It seems that my only option for filtering inside the query is to store seconds-since-epoch in the database and use integer comparisons.
This was a temporary bug with old SQLProvider version and it should be working now. If not, please open a new issue to the GitHub repository: https://github.com/fsprojects/SQLProvider
When creating a table in SQLite3, I get confused when confronted with all the possible datatypes which imply similar contents, so could anyone tell me the difference between the following data-types?
INT, INTEGER, SMALLINT, TINYINT
DEC, DECIMAL
LONGCHAR, LONGVARCHAR
DATETIME, SMALLDATETIME
Is there some documentation somewhere which lists the min./max. capacities of the various data-types? For example, I guess smallint holds a larger maximum value than tinyint, but a smaller value than integer, but I have no idea of what these capacities are.
SQLite, technically, has no data types, there are storage classes in a manifest typing system, and yeah, it's confusing if you're used to traditional RDBMSes. Everything, internally, is stored as text. Data types are coerced/converted into various storage locations based on affinities (ala data types assigned to columns).
The best thing that I'd recommend you do is to :
Temporarily forget everything you used to know about standalone database datatypes
Read the above link from the SQLite site.
Take the types based off of your old schema, and see what they'd map to in SQLite
Migrate all the data to the SQLite database.
Note: The datatype limitations can be cumbersome, especially if you add time durations, or dates, or things of that nature in SQL. SQLite has very few built-in functions for that sort of thing. However, SQLite does provide an easy way for you to make your own built-in functions for adding time durations and things of that nature, through the sqlite3_create_function library function. You would use that facility in place of traditional stored procedures.
The difference is syntactic sugar. Only a few substrings of the type names matter as for as the type affinity is concerned.
INT, INTEGER, SMALLINT, TINYINT → INTEGER affinity, because they all contain "INT".
LONGCHAR, LONGVARCHAR → TEXT affinity, because they contain "CHAR".
DEC, DECIMAL, DATETIME, SMALLDATETIME → NUMERIC, because they don't contain any of the substrings that matter.
The rules for determining affinity are listed at the SQLite site.
If you insist on strict typing, you can implement it with CHECK constraints:
CREATE TABLE T (
N INTEGER CHECK(TYPEOF(N) = 'integer'),
Str TEXT CHECK(TYPEOF(Str) = 'text'),
Dt DATETIME CHECK(JULIANDAY(Dt) IS NOT NULL)
);
But I never bother with it.
As for the capacity of each type:
INTEGER is always signed 64-bit. Note that SQLite optimizes the storage of small integers behind-the-scenes, so TINYINT wouldn't be useful anyway.
REAL is always 64-bit (double).
TEXT and BLOB have a maximum size determined by a preprocessor macro, which defaults to 1,000,000,000 bytes.
Most of those are there for compatibility. You really only have integer, float, text, and blob. Dates can be stored as either a number (unix time is integer, microsoft time is float) or as text.
NULL. The value is a NULL value.
INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.
REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number.
TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE).
BLOB. The value is a blob of data, stored exactly as it was input.
As an addition to answer from dan04, if you want to blindly insert a NUMERIC other than zero represented by a TEXT but ensure that text is convertible to a numeric:
your_numeric_col NUMERIC CHECK(abs(your_numeric_col) <> 0)
Typical use case is in a query from a program that treats all data as text (for uniformity & simplicity, since SQLite already does so). The nice thing about this is that it allows constructs like this:
INSERT INTO table (..., your_numeric_column, ...) VALUES (..., some_string, ...)
which is convenient in case you're using placeholders because you don't have to handle such non-zero numeric fields specially. An example using Python's sqlite3 module would be,
conn_or_cursor.execute(
"INSERT INTO table VALUES (" + ",".join("?" * num_values) + ")",
str_value_tuple) # no need to convert some from str to int/float
In the above example, all values in str_value_tuple will be escaped and quoted as strings when passed to SQlite. However, since we're not checking explicitly the type via TYPEOF but only convertibility to type, it will still work as desired (i.e., SQLite will either store it as a numeric or fail otherwise).
I'm familiar with how type affinity works in SQLite: You can declare column types as anything you want, and all that matters is whether the type name contains "INT", "CHAR", "FLOA", etc. But is there a commonly-used convention on what type names to use?
For example, if you have an integer column, is it better to distinguish between TINYINT, SMALLINT, MEDIUMINT, and BIGINT, or just declare everything as INTEGER?
So far, I've been using the following:
INTEGER
REAL
CHAR(n) -- for strings with a known fixed with
VARCHAR(n) -- for strings with a known maximum width
TEXT -- for all other strings
BLOB
BOOLEAN
DATE -- string in "YYYY-MM-DD" format
TIME -- string in "HH:MM:SS" format
TIMESTAMP -- string in "YYYY-MM-DD HH:MM:SS" format
(Note that the last three are contrary to the type affinity.)
I would recommend not using self-defined types. I have observed in version 3.5.6 that types not already defined could sometimes cause an INSERT command to be refused. Maybe 1 out of 1000. I don't know if this was addressed since.
In any case, there is no sizing advantage in typing a column TINYINT or SMALLINT. The only advantage would be outside SQLite, for either parsing your column types with another program or to satisfy your personal need for tidiness. So I strongly recommend using the base types defined by SQLite and sticking to those.
Since SQLite is typeless, use whatever types make it easier for you to see what the schema looks like. Or you can match the types to your codebase.
I'm going to go with Kevin on this one. In short, knock yourself out. Make up brand new areas of mathematics if it suits your schema. Use the classnames of your ORM. Or name every type (except the PRIMARY KEY INTEGER ones) for ex-girlfriends. In the end SQLite is more about how you access and use the data.
My table contains Birthdate field which has datatype as datetime.
I want to get all records having birthday today.
How can I get it?
Try this query:
SELECT * FROM mytable
WHERE strftime('%m-%d', 'now') = strftime('%m-%d', birthday)
Having a special datetime type has always seemed like unnecessary overhead to me, integers are fast, flexible, and use less space.
For general datetime values use Unix Epoch timestamps. Easy to work with, extremely flexible, as well as timezone (and even calender!) agnostic. (I recently wrote an article on using them, which I really have to plug...)
That said, if you're only interested in dates in the Gregorian calendar you may want to use a large integer in the following format: YYYYMMDD, eg 19761203. For you particular usage you could even create a four digit integer like MMDD, say 1703 — that's got to result in fast selects!
SQLite has very poor support for storing dates. You can use the method suggested by Nick D above but bear in mind that this query will result in full table scan since dates are not indexed correctly in SQLite (actually SQLite does not support dates as a built-in type at all).
If you really want to do a fast query then you'll have to add a separate (integral) column for storing the birth day (1-31) and attach an index for it in the database.
If you only want to compare dates then you can add a single (INTEGER) column that will store the date UTC value (but this trick won't allow you to search for individual date components easily).
Good Luck