The full-text search extensions of SQLite (FTS3, FTS4, FTS5) create shadow tables. E.g., if I create FTS5-table
CREATE VIRTUAL TABLE test USING fts5(textData);
that will also automatically create several real (non-virtual) tables, with test_content among them, which (apparently) stores the actual data inserted into the original virtual table test.
I know SQLite authors suggest that these shadow tables "should not be accessed directly by the user". But it's not clear from the docs if there are no any guarantee at all about behavior of these tables, or this advise concerns primarily attempts to directly INSERT or UPDATE on them. But what are the risks of reading from these tables?
Specifically - I need another (regular) table to have a FOREIGN KEY which references rowid of the virtual FTS5 table:
CREATE VIRTUAL TABLE test USING fts5(textData);
CREATE TABLE myTable (col1 INTEGER REFERENCES test(rowid));
I couldn't find hints on that in the docs, but my own experiments showed that foreign key just doesn't work here - I still can delete records from the test table even though they are referenced from myTable. However, if instead I do
CREATE VIRTUAL TABLE test USING fts5(textData);
CREATE TABLE myTable (col1 INTEGER REFERENCES test_content(id));
then everything seems to work as needed - I can't delete records from the virtual test table if they are referenced from myTable.
This works because, as follows from direct inspection, the rowid value of the test table is always equal to the id column value of the test_content table. Even when I specify rowid manually like in INSERT INTO test (rowid, value) VALUES (424242, 'foobar'); - then a new row appears in the test_content with the corresponding content and the corresponding id equal to 424242 (even if no preceding records were ever in the table).
So basically, my question is - is this correspondence between the rowid of an FTS5-table and the id of the corresponding <name>_content shadow table guaranteed to hold? Or might this break in some cases or in future versions?
Related
For example, let say DB has foreign key A.b_id -> B.id with SET NULL on delete.
If record with some B.id get deleted, all b_id references will be set to NULL.
But if A already contains record where A.b_id has value that is not in B.id (it was inserted without foreign keys support), is there a way to force SQLite DB check foreign keys and set to NULL such data?
In fact, in first place I'm solving an DB upgrading task.
On start app checks if internal DB (resource) has higher version than user DB.
If so it backups user DB, copies internal empty DB to user storage. Than turns off foreign keys support and fills new DB with data from backup, inserting automatically in loop table by table for all columns with same name. Turns on foreign keys support back.
Everything works fine, but if in some table in old DB there is no foreign key constrain previously, while new DB has one, the data will be inserted as is and link can point nowhere (possibly wrong links is unavoidable and not related to question).
Yes, I understand a way to insert without turning off foreign keys support, but it would need knowledge of tables dependencies order that I would like to avoid.
Thanks for any help in advance!
Although I don't know of a way that automatically will set to NULL all orphaned values of a column in a table that (should) reference another column in another table, there is a way to get a report of all these cases and then act accordingly.
This is the PRAGMA statement foreign_key_check:
PRAGMA schema.foreign_key_check;
or for a single table check:
PRAGMA schema.foreign_key_check(table-name);
From the documenation:
The foreign_key_check pragma checks the database, or the table called
"table-name", for foreign key constraints that are violated. The
foreign_key_check pragma returns one row output for each foreign key
violation. There are four columns in each result row. The first column
is the name of the table that contains the REFERENCES clause. The
second column is the rowid of the row that contains the invalid
REFERENCES clause, or NULL if the child table is a WITHOUT ROWID
table. The third column is the name of the table that is referred to.
The fourth column is the index of the specific foreign key constraint
that failed. The fourth column in the output of the foreign_key_check
pragma is the same integer as the first column in the output of the
foreign_key_list pragma. When a "table-name" is specified, the only
foreign key constraints checked are those created by REFERENCES
clauses in the CREATE TABLE statement for table-name.
Check a simplified demo of the way to use this PRAGMA statement, or its function counterpart pragma_foreign_key_check().
You can get a list of the rowids of all the problematic rows of each table.
In your case, you can execute an UPDATE statement that will set to NULL all the orphaned b_ids:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check() WHERE "table" = 'A')
This also works in later versions of SQLite:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check('A'))
but it does not seem to work up to SQLite 3.27.0
In sqlite the query:
select * from [....]
is often implemented to grab all the info associated with pertinent entries.
But, as this question asks, rowid is not one of the items returned with select all.
Why is rowid not automatically returned with a select all search?
All RDBMS (relational database management systems) implement some way to distinguish between rows in a table regardless of what keys are defined by the user. Otherwise there would be no way for the system to distinguish between rows for internal processing. Other systems also do not reveal their internal record-keeping metadata in * queries and only include user-defined columns. The fact that sqlite documentation reveals the existence of rowid, or even that one can explicitly request the rowid does not change that it is still essentially a special metadata column, so there is really no reason for it to be included in SELECT * queries.
If you rely on such an integer primary key and/or insist that it appears in the full list of columns returned by SELECT *, then you can add an alias for the rowid by defining an explicit integer primary key. See sqlite CREATE TABLE docs for details. Here is just one example from the documentation that creates a valid rowid alias:
CREATE TABLE t(x INTEGER PRIMARY KEY ASC, y, z);
Similar to this question and this solution for PostgreSQL (in particular "INSERT missing FK rows at the same time"):
Suppose I am making an address book with a "Groups" table and a "Contact" table. When I create a new Contact, I may want to place them into a Group at the same time. So I could do:
INSERT INTO Contact VALUES (
"Bob",
(SELECT group_id FROM Groups WHERE name = "Friends")
)
But what if the "Friends" Group doesn't exist yet? Can we insert this new Group efficiently?
The obvious thing is to do a SELECT to test if the Group exists already; if not do an INSERT. Then do an INSERT into Contacts with the sub-SELECT above.
Or I can constrain Group.name to be UNIQUE, do an INSERT OR IGNORE, then INSERT into Contacts with the sub-SELECT.
I can also keep my own cache of which Groups exist, but that seems like I'm duplicating functionality of the database in the first place.
My guess is that there is no way to do this in one query, since INSERT does not return anything and cannot be used in a subquery. Is that intuition correct? What is the best practice here?
My guess is that there is no way to do this in one query, since INSERT
does not return anything and cannot be used in a subquery. Is that
intuition correct?
You could use a Trigger and a little modification of the tables and then you could do it with a single query.
For example consider the folowing
Purely for convenience of producing the demo:-
DROP TRIGGER IF EXISTS add_group_if_not_exists;
DROP TABLE IF EXISTS contact;
DROP TABLE IF EXISTS groups;
One-time setup SQL :-
CREATE TABLE IF NOT EXISTS groups (id INTEGER PRIMARY KEY, group_name TEXT UNIQUE);
INSERT INTO groups VALUES(-1,'NOTASSIGNED');
CREATE TABLE IF NOT EXISTS contact (id INTEGER PRIMARY KEY, contact TEXT, group_to_use TEXT, group_reference TEXT DEFAULT -1 REFERENCES groups(id));
CREATE TRIGGER IF NOT EXISTS add_group_if_not_exists
AFTER INSERT ON contact
BEGIN
INSERT OR IGNORE INTO groups (group_name) VALUES(new.group_to_use);
UPDATE contact SET group_reference = (SELECT id FROM groups WHERE group_name = new.group_to_use), group_to_use = NULL WHERE id = new.id;
END;
SQL that would be used on an ongoing basis :-
INSERT INTO contact (contact,group_to_use) VALUES
('Fred','Friends'),
('Mary','Family'),
('Ivan','Enemies'),
('Sue','Work colleagues'),
('Arthur','Fellow Rulers'),
('Amy','Work colleagues'),
('Henry','Fellow Rulers'),
('Canute','Fellow Ruler')
;
The number of values and the actual values would vary.
SQL Just for demonstration of the result
SELECT * FROM groups;
SELECT contact,group_name FROM contact JOIN groups ON group_reference = groups.id;
Results
This results in :-
1) The groups (noting that the group "NOTASSIGNED", is intrinsic to the working of the above and hence added initially) :-
have to be careful regard mistakes like (Fellow Ruler instead of Fellow Rulers)
-1 used because it would not be a normal value automatically generated.
2) The contacts with the respective group :-
Efficient insertion
That could likely be debated from here to eternity so I leave it for the fence sitters/destroyers to decide :). However, some considerations:-
It works and appears to do what is wanted.
It's a little wasteful due to the additional wasted column.
It tries to minimise the waste by changing the column to an empty string (NULL may be even more efficient, but for some can be confusing)
There will obviously be an overhead BUT in comparison to the alternatives probably negligible (perhaps important if you were extracting every Facebook user) but if it's user input driven likely irrelevant.
What is the best practice here?
Fences again. :)
Note Hopefully obvious, but the DROP statements are purely for convenience and that all other SQL up until the INSERT is run once
to setup the tables and triggers in preparation for the single INSERT
that adds a group if necessary.
I am creating a table inside the DB.I am a successfully created a table and able to insert data on table my issue is that in my table I have two "ID" columns but I only created one .why two ID column generate in table.
Follow this step to generate issue :-
Click the bottom right button (enter the text in pop up screen ).Press "add" button .It generate the row in a "cases" table but when you inspect it show two column of "ID" why ?
Here is my code
http://codepen.io/anon/pen/OVPgoP
db.transaction(function(tx) {
tx.executeSql('CREATE TABLE "Cases" ("ID" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL , "CaseName" VARCHAR NOT NULL )');
})
It appears that this is the expected behaviour for SQLite, the technology underpinning Web SQL (which is a deprecated technology, btw).
From the docs:
If a table contains a column of type INTEGER PRIMARY KEY, then that column becomes an alias for the ROWID. You can then access the ROWID using any of four different names, the original three names described above or the name given to the INTEGER PRIMARY KEY column. All these names are aliases for one another and work equally well in any context.
Therefore, my guess is that the Chrome devtools are showing the rowid column using its alias (ID), in addition to the alias column explicitly added (ID).
It seems like you don't actually need to explicitly add an ID column with Web SQL/SQLite. It will add one for you, which you can refer to using rowid, _rowid_ or oid in any statement.
EDIT: Here is a fork of your CodePen, with updates and deletes all correctly working.
Totally new to webSQL, so this answer is very useful to me as well. You can't expect primary keys to work properly with webSQL because internally it tracks something called "rowid" as the primary key. Use unique instead, as seen in a useful example here. You can also use this error code from the spec to figure out if a non-unique column value was inserted.
I am using Visual Web Developer and Microsoft SQL server. I have a tag table "Entry_Tag" which is as follows:
entry_id
tag_id
I want to make the entry_id and tag_id pairing unique. A particular tag can only be applied to an entry once in the table. I made the two columns a primary key. They are also both foreign keys referencing the ids in their respective tables. When I dragged the tables into the Object Relationship Designer it only showed a relationship line between either "Entry_Tag" and "Entry" or when I tried again between "Entry_tag" and "Tag".
The "Entry_tag" table should have a relationship with both "Tag" and "Entry".
How do I go about doing this?
In general, you can add a unique constraint on the table that includes both columns. In this case, including both of the columns in the primary key should have already done this. If you have relationships set up for each field to other tables, then I believe those relationships should be displayed in the query designer... I see no cause for this given the information you've provided - perhaps you need to post more information.
Create an UNIQUE INDEX to for entry_id and tag_id.
CREATE UNIQUE INDEX index_name ON table (entry_id, tag_id)