In the SQLite documentation it says:
The parent key of a foreign key constraint is not allowed to use the
rowid. The parent key must used named columns only.
The parent key must be a named column or columns in the parent table,
not the rowid.
But does that apply to an alias of the rowid? For example in SQLite if you have a INTEGER PRIMARY KEY column then that column is essentially an alias of the rowid:
With one exception noted below, if a rowid table has a primary key
that consists of a single column and the declared type of that column
is "INTEGER" in any mixture of upper and lower case, then the column
becomes an alias for the rowid. Such a column is usually referred to
as an "integer primary key".
(Exception omitted; not relevant here)
There is a similar question here:
sql - Why does referencing a SQLite rowid cause foreign key mismatch? - Stack Overflow
If I take that example and modify it to use the alias (my integer primary key column) it appears to work:
sqlite> CREATE TABLE foo(a INTEGER PRIMARY KEY, name);
sqlite> create table bar(foo_rowid REFERENCES foo(a));
sqlite> INSERT INTO foo VALUES( NULL, "baz" );
sqlite> select * from foo;
a name
---------- ----------
1 baz
sqlite> INSERT INTO bar (foo_rowid) VALUES(1);
sqlite> select * from bar;
foo_rowid
----------
1
sqlite>
But is it legal to reference an alias of the rowid? Thanks.
If the internal rowid is not a named column, it might not keep the same values after a VACUUM, which would break the foreign key references.
If the rowid is named, it is guaranteed to keep its values.
Using an INTEGER PRIMARY KEY as the parent of a foreign key is allowed, and common.
Related
I have those two tables implementing some inheritance relationship via the Class Table Inheritance pattern:
pragma foreign_keys = ON;
create table foo(foo_id integer primary key);
create table bar(foo_id integer primary key references foo(foo_id));
Let's populate foo:
insert into foo values (1), (3), (4);
Now I can insert 3 into bar:
insert into bar values(3); -- no error
I cannot insert 2:
insert into bar values(2); -- Error: FOREIGN KEY constraint failed
However, what suprises me is that NULL values can be used to generate new keys:
insert into bar values(NULL); -- OK, 4 inserted
insert into bar values(NULL); -- FOREIGN KEY constraint failed
This behavior seems rather odd. When I try the same thing in MySQL, I am greeted with a
ERROR 1048 (23000): Column 'foo_id' cannot be null
which is what I would expect.
I find this behavior particularly dangerous when inserting new rows in bar with a subquery:
insert into bar (foo_id) values ((select foo_if from foo where ...))
which could end up silently inserting random rows into bar when there is no match in foo, instead of returning an error.
Is this behavior compliant with the SQL standard, in what scenario could it be useful, and more importantly, is there a way this behavior could be changed to match MySQL's?
EDIT
Here is an illustration of the problem in a perhaps more striking (and scary) fashion:
pragma foreign_keys = ON;
create table people(people_id integer primary key, name text not null);
insert into people (name) values ("Mom"), ("Jack the Ripper");
create table family_member(people_id integer primary key references people(people_id));
insert into family_member values ((select people_id from people where name = "Mom"));
insert into family_member values ((select people_id from people where name = "Dad")); -- silent error here
select name from family_member inner join people using (people_id);
-- uh-oh, Jack the Ripper is now part of my family
So I found an anwer to this problem, which may be a bit surprising for people not familiar with SQLite.
It turns out that a column that is declared integer primary key is automatically filled with an unused integer if it is not given a value.
A somewhat arcane way around this in SQLite is to use declare the column int primary key instead, which prevents the implicit automatic handling.
However, we are not over yet, because another distinctive behavior of SQLite is to allow NULL primary keys -- in our example above, insert into bar values (NULL) would insert a new row with a NULL primary key.
To explicitely ban primary keys from having NULL values, it must be declared not null:
pragma foreign_keys = ON;
create table foo(foo_id integer primary key);
create table bar(foo_id int primary key not null references foo(foo_id));
insert into foo values (1), (3), (4);
insert into bar values (3);
insert into bar values (2); -- Error: FOREIGN KEY constraint failed
insert into bar values (NULL); -- Error: NOT NULL constraint failed: bar.foo_id
This is probably the way primary key pointing to external keys should be declared by default in SQLite.
Ie, are the following two SQL statements equivalent in SQLite?
CREATE TABLE posts (
id INTEGER PRIMARY KEY
);
CREATE TABLE posts (
id INTEGER PRIMARY KEY ASC
);
Yes they are.
There is no need to specify ASC and beware that if you were to specify DESC, then NO they are then not equivalent (see 4 below) as id INTEGER PRIMARY KEY DESC is an exclusion to the column being an alias of the rowid column as per :-
The exception mentioned above is that if the declaration of a column
with declared type "INTEGER" includes an "PRIMARY KEY DESC" clause, it
does not become an alias for the rowid and is not classified as an
integer primary key. This quirk is not by design. It is due to a bug
in early versions of SQLite. But fixing the bug could result in
backwards incompatibilities. Hence, the original behavior has been
retained (and documented) because odd behavior in a corner case is far
better than a compatibility break.
ROWIDs and the INTEGER PRIMARY KEY
You can use id INTEGER, PRIMARY KEY(id, DESC), but still the order defaults to ASC when retrieving the column as it is an alias of the rowid (see 5 below )
Perhaps consider the following :-
DROP TABLE IF EXISTS posts1;
CREATE TABLE posts1 (
id INTEGER PRIMARY KEY
);
DROP TABLE IF EXISTS posts2;
CREATE TABLE posts2 (
id INTEGER PRIMARY KEY ASC
);
DROP TABLE IF EXISTS posts3;
CREATE TABLE posts3 (
id INTEGER PRIMARY KEY DESC
);
DROP TABLE IF EXISTS posts4;
CREATE TABLE posts4 (
id INTEGER, PRIMARY KEY (id DESC)
);
INSERT INTO posts1 VALUES(null),(null),(null);
INSERT INTO posts2 VALUES(null),(null),(null);
INSERT INTO posts3 VALUES(null),(null),(null);
INSERT INTO posts4 VALUES(null),(null),(null);
SELECT * FROM sqlite_master WHERE name LIKE '%posts%';
SELECT * FROM posts1;
SELECT * FROM posts2;
SELECT * FROM posts3;
SELECT * FROM posts4;
Results
1
The query SELECT * FROM sqlite_master WHERE name LIKE '%posts%'; results in :-
As you can see posts3 is significantly different as the index sqlite_autoindex_posts3_1 has been created
The others do not have a specific index created as the id column is an alias of the rowid column
The data for rowid tables is stored as a B-Tree structure containing
one entry for each table row, using the rowid value as the key. This
means that retrieving or sorting records by rowid is fast. Searching
for a record with a specific rowid, or for all records with rowids
within a specified range is around twice as fast as a similar search
made by specifying any other PRIMARY KEY or indexed value.
ROWIDs and the INTEGER PRIMARY KEY
2
The query SELECT * FROM posts1; results in :-
3
The query SELECT * FROM posts2;, confirms the initial YES answer as per :-
4
The query SELECT * FROM posts3;, may be a little confusing, but shows that id INTEGER PRIMARY KEY DESC does not result in an alias of the rowid and in the case of no value or null being inserted into the column, the value is null rather than an auto generated value. There is no UNIQUE constraint conflict (as nulls are considered as being different values).
5
The query SELECT * FROM posts4; produces the same result as for 1 and 2 even though id INTEGER, PRIMARY KEY (id DESC) was used. Confirming that even if DESC is applied via the column definition that the sort order is still defaults to ASC (unless the ORDER BY clause is used).
Note that this peculiarity is specific to the rowid column or an alias thereof.
See both https://www.sqlite.org/lang_createtable.html#rowid and https://www.sqlite.org/lang_createindex.html for a more complete answer. Shawn's link is specific to INTEGER PRIMARY KEY which matches the example code, but the more general question is not answered explicitly in either location, but can be deduced by reading both.
Under SQL Data Constraints, the first link says
In most cases, UNIQUE and PRIMARY KEY constraints are implemented by creating a unique index in the database. (The exceptions are INTEGER PRIMARY KEY and PRIMARY KEYs on WITHOUT ROWID tables.)
The CREATE INDEX page explains that originally the sort order was ignored and all indices were generated in ascending order. Only as of version 3.3.0 is the DESC order "understood". But even that description is somewhat vague, however altogether it is apparent that ASC is the default.
I am using the DB Browser for SQLite to try and figure this out. I've opened Northwind.sqlite and in it it shows me the following for a table:
CREATE TABLE `Order Details` (
`OrderID` int,
`ProductID` int,
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`)
);
However, in the Sql Server Northwind OrderID and ProductID are foreign keys, not primary keys. Does this work differently in SQLite? And if so, how do the relationships work?
thanks - dave
The above will create a table that has no FOREIGN keys but 2 indexes.
One a largely hidden index according to rowid.
The other, PRIMARY KEY(OrderID,ProductID) will be an index according to the combination of OrderId and ProductID.
some things about rowid (aka id)
rowid is an automatically created column called rowid (it can also be referenced using oid or rowid (case independent)) and if present is really the primary key.
rowid will be a unique signed integer using up to 64 bits. The lowest value and also the first value will be 1, the highest value being 9223372036854775807.
In later versions of SQLite 3.8.2 on the WITHOUT ROWID keyword was added to allow suppression of the rowid column/index (your Order Details table may benefit being a without rowid table).
if a column is defined with the type INTEGER PRIMARY KEY or INTEGER PRIMARY KEY AUTOINCREMENT then that column (there can only be 1 such column per table) is an alias of for the rowid column.
AUTOINCREMENT introduces a rule that when inserting a row the rowid must be greater than any that exist or existed.
It DOES NOT guarantee that the rowid will monotonically increase, although generally the id will (even without AUTOINCREMENT (perhaps the most misused/misunderstood keyword in SQLite)).
Without AUTOINCREMENT SQlite may find a lower rowid and use that, but not until a rowid of 9223372036854775807 has been reached.
AUTOINCREMENT, if a rowid of 9223372036854775807 has been reached will is an SQLITE_FULL exception.
AUTOINCREMENT results in overheads (e.q. a table named sqlite_sequence is then maintained recording the highest given sequence number). The documentation recommends that it not be used unless required, which is rarely the case.
Some limited testing I did resulted in an 8-12% greater processing time for AUTOINCREMENT. What are the overheads of using AUTOINCREMENT for SQLite on Android?
For more about rowid see SQLite Autoincrement and also Clustered Indexes and the WITHOUT ROWID Optimization
Coding PRIMARY KEY (if not on an INTEGER column i.e. not an alias of rowid) implies a UNIQUE constraint. It is not saying/checking that the value or any of the values in a clustered index exists in any other table.
Note null is not considered to be the same value, so in your Order Details table it is possible to have any combination of the values as null.
Coding a FOREIGN KEY introduces a constraint that the referenced value(s) must exist in the respective table/column. Additionally :-
Usually, the parent key of a foreign key constraint is the primary key
of the parent table. If they are not the primary key, then the parent
key columns must be collectively subject to a UNIQUE constraint or
have a UNIQUE index. If the parent key columns have a UNIQUE index,
then that index must use the collation sequences that are specified in
the CREATE TABLE statement for the parent table.
SQLite Foreign Key Support
Considering all of this you may want to do make some changes to the Order Details table :-
You could make it a WITHOUT ROWID table.
You could make both the OrderID and the ProductID columns NOT NULL.
You could add FOREIGN KEY's to both the OrderID and the ProductID columns.
So perhaps you could have :-
CREATE TABLE `Order Details` (
`OrderID` int NOT NULL REFERENCES `Orders` (`OrderId`), -- ADDED NOT NULL and FKEY
`ProductID` int NOT NULL REFERENCES `Products`(`ProductId`) , -- ADDED NOT NULL and FKEY
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`)
)
WITHOUT ROWID -- ADDED WITHOUT ROWID
;
The above uses column constraints
Alternately, utilising TABLE constraints, you could do :-
CREATE TABLE `Order Details` (
`OrderID` int NOT NULL, -- ADDED NOT NULL
`ProductID` int NOT NULL, -- ADDED NOT NULL
`UnitPrice` float ( 26 ),
`Quantity` int,
`Discount` float ( 13 ),
PRIMARY KEY(`OrderID`,`ProductID`),
FOREIGN KEY (`OrderId`) REFERENCES `Orders`(`OrderId`), -- ADDED FKEY AS TABLE CONSTRAINT
FOREIGN KEY (`ProductID`) REFERENCES `Products`(`ProductID`) -- ADDED FKEY AS TABLE CONSTRAINT
)
WITHOUT ROWID -- ADDED WITHOUT ROWID
;
Both have the same outcome, the only difference being where the FOREIGN KEY constraints are defined.
Both the above assumes that the referenced tables are Orders and Products.
Does SQLite indeed have a limitation that it is not possible to retrieve the name of a foreign key? I am asking because I couldn't find this limitation mentioned anywhere in their documentation.
For example, I run the following script:
CREATE TABLE
users (
id INTEGER NOT NULL PRIMARY KEY,
first_name TEXT NOT NULL,
last_name TEXT NOT NULL
) ;
CREATE TABLE
orders (
id INTEGER NOT NULL PRIMARY KEY,
user_id INTEGER NOT NULL,
CONSTRAINT fk_users FOREIGN KEY (user_id) REFERENCES users(id)
) ;
Now I would like to check that the key "fk_users" was created indeed, so I run the following PRAGMA:
PRAGMA foreign_key_list(orders);
I would expect to see the name of my foreign key in the first column, but I am seeing some "0" value instead. Moreover, if I create multiple foreign keys with custom names, they are all called either "0" or "1".
Is this indeed a limitation of SQLite, or am I missing something?
There is no mechanism to extract the constraint name.
The table sqlite_master stores a CREATE command in the column "sql". You could query that command and do some parsing to extract the name of the foreign key. An example for a combined foreign key that works for me:
SELECT sql FROM sqlite_master WHERE name = 'song'
yields
CREATE TABLE "song" (
"songid" INTEGER,
"songartist" TEXT,
"songalbum" TEXT,
"songname" TEXT,
CONSTRAINT "fk__song_album" FOREIGN KEY ("songartist", "songalbum") REFERENCES "album" ("albumartist", "albumname")
)
and contains the name "fk__song_album" of the foreign key.
If one alters the foreign key with a query, the content of the sql column is modified/updated:
The text in the sqlite_master.sql column is a copy of the original CREATE statement text that created the object, except normalized as described above and as modified by subsequent ALTER TABLE statements. The sqlite_master.sql is NULL for the internal indexes that are automatically created by UNIQUE or PRIMARY KEY constraints.
https://www.sqlite.org/fileformat2.html
Extra tip:
In order to see the foreign key information in Navicat (Lite) ... right click on a table and choose "Design table". Then select the foreign keys tab.
I get a syntax error near AUTOINCREMENT. What is the cause of this error?
CREATE TABLE person (
id INTEGER NOT NULL AUTOINCREMENT,
name TEXT NOT NULL
);
CREATE TABLE department (
id INTEGER NOT NULL AUTOINCREMENT,
name TEXT NOT NULL,
FOREIGN KEY (leader) REFERENCES person(id)
);
According to SQLite FAQ you have to declare either a INTEGER PRIMARY KEY or INTEGER PRIMARY KEY AUTOINCREMENT column to achieve that.
In SQLite you need not to specify AUTOINCREMENT if you are specifying a column as Primary Key...
SQLite AUTOINCREMENT : You Should Avoid Using It
Unless you create a table specifying the WITHOUT ROWID option, you get an implicit auto increment column called rowid.
The rowid column store 64-bit signed integer that uniquely identifies a row within the table.
It's an easy solution. Just use AUTOINCREMENT instead of AUTO_INCREMENT