I have a Xamarin.Forms app that uses a SQLite database locally on the device. Here's some sample data structure:
Table x: id, name
Table y: id, name
Table x_y: id, x_id, y_id
Since SQLite doesn't support altering columns, one of the schema updates we sent down in a patch did the following:
Rename table x to x_temp
Create new/updated table x
Insert all data from table x_temp into table x
Drop table if exists x
That seems to work just fine. However, when I'm attempting to run an insert statement on table x_y, I am getting a SQLite exception: "no such table: main.x_temp".
When I look at the SQLite query string while debugging there is no mention of table x_temp whatsoever. So, if I delete the entire database and re-create everything the insert works just fine.
I'm from a MSSQL background, am I not understanding something about SQLite in general? Is the foreign key constraint from table x_y trying to reference x_temp because I renamed the original table (I may have just answered my own question)? If that's the case, surely there is a way around this without having to cascade and re-create every table?
Any input would be appreciated. Thanks!
I believe that your issue may be related to the SQlite version in conjunction with whether or not Foreign Key Support has been turned on.
That is the likliehood is that :-
Is the foreign key constraint from table x_y trying to reference
x_temp because I renamed the original table (I may have just answered
my own question)?
Would be the issue, as you likely have Foreign Key Support turned on as per :-
Prior to version 3.26.0 (2018-12-01), FOREIGN KEY references to a table that is renamed were only edited if the PRAGMA foreign_keys=ON, or in other words if foreign key constraints were begin enforced.
With PRAGMA foreign_keys=OFF, FOREIGN KEY constraints would not be changed when the table that the foreign key referred to (the "parent table") was renamed.
Beginning with version 3.26.0, FOREIGN KEY constraints are always converted when a table is renamed, unless the PRAGMA legacy_alter_table=ON setting is engaged. The following table summaries the difference:
SQL As Understood By SQLite - ALTER TABLE
If that's the case, surely there is a way around this without having
to cascade and re-create every table?
Yes, as the latest version of SQlite on Android is 3.19.0 (I believe), then you can turn Foreign Key support off using the foreign_keys pragma when renaming the table.
Note Foreign Keys cannot be turned off within a transaction.
See SQL As Understood By SQLite - ALTER TABLE and PRAGMA foreign_keys = boolean;
Related
For example, let say DB has foreign key A.b_id -> B.id with SET NULL on delete.
If record with some B.id get deleted, all b_id references will be set to NULL.
But if A already contains record where A.b_id has value that is not in B.id (it was inserted without foreign keys support), is there a way to force SQLite DB check foreign keys and set to NULL such data?
In fact, in first place I'm solving an DB upgrading task.
On start app checks if internal DB (resource) has higher version than user DB.
If so it backups user DB, copies internal empty DB to user storage. Than turns off foreign keys support and fills new DB with data from backup, inserting automatically in loop table by table for all columns with same name. Turns on foreign keys support back.
Everything works fine, but if in some table in old DB there is no foreign key constrain previously, while new DB has one, the data will be inserted as is and link can point nowhere (possibly wrong links is unavoidable and not related to question).
Yes, I understand a way to insert without turning off foreign keys support, but it would need knowledge of tables dependencies order that I would like to avoid.
Thanks for any help in advance!
Although I don't know of a way that automatically will set to NULL all orphaned values of a column in a table that (should) reference another column in another table, there is a way to get a report of all these cases and then act accordingly.
This is the PRAGMA statement foreign_key_check:
PRAGMA schema.foreign_key_check;
or for a single table check:
PRAGMA schema.foreign_key_check(table-name);
From the documenation:
The foreign_key_check pragma checks the database, or the table called
"table-name", for foreign key constraints that are violated. The
foreign_key_check pragma returns one row output for each foreign key
violation. There are four columns in each result row. The first column
is the name of the table that contains the REFERENCES clause. The
second column is the rowid of the row that contains the invalid
REFERENCES clause, or NULL if the child table is a WITHOUT ROWID
table. The third column is the name of the table that is referred to.
The fourth column is the index of the specific foreign key constraint
that failed. The fourth column in the output of the foreign_key_check
pragma is the same integer as the first column in the output of the
foreign_key_list pragma. When a "table-name" is specified, the only
foreign key constraints checked are those created by REFERENCES
clauses in the CREATE TABLE statement for table-name.
Check a simplified demo of the way to use this PRAGMA statement, or its function counterpart pragma_foreign_key_check().
You can get a list of the rowids of all the problematic rows of each table.
In your case, you can execute an UPDATE statement that will set to NULL all the orphaned b_ids:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check() WHERE "table" = 'A')
This also works in later versions of SQLite:
UPDATE A
SET b_id = NULL
WHERE rowid IN (SELECT rowid FROM pragma_foreign_key_check('A'))
but it does not seem to work up to SQLite 3.27.0
In the PostgreSQL 11 Release Notes I found the following improvements to partitioning functionality:
Add support for PRIMARY KEY, FOREIGN KEY, indexes, and triggers on partitioned tables
I need this feature and tested it.
Create table:
CREATE TABLE public.tbl_test
(
uuid character varying(32) NOT null,
registration_date timestamp without time zone NOT NULL
)
PARTITION BY RANGE (registration_date);
Try to create Primary key:
ALTER TABLE public.tbl_test ADD CONSTRAINT pk_test PRIMARY KEY (uuid);
I get an error SQL Error [0A000]. If use composite PK (uuid, registration_date) then it's work. Because PK contains partitioning column
Conclusion: create PK in partitioning tables work with restrictions (PK need contains partitioning column).
Try to create Foreign key
CREATE TABLE public.tbl_test2
(
uuid character varying(32) NOT null,
test_uuid character varying(32) NOT null
);
ALTER TABLE tbl_test2
ADD CONSTRAINT fk_test FOREIGN KEY (test_uuid)
REFERENCES tbl_test (uuid);
I get an error SQL Error [42809]. It means FOREIGN KEY on partitioning tables not work.
Maybe i'm doing something wrong. Maybe somebody tried this functionality and know how this work.
Maybe somebody know workaround except implement constraint in the application.
PostgreSQL v12.0 will probably support foreign keys that reference partitioned tables. But this is still not guaranteed as v12.0 is still in development.
For v11 and lower versions, you may use triggers as described by depesz in these posts: part1, part2, and part3.
Update: PostgreSQL v12.0 was released on Oct 3, 2019, with this feature included
Postgres 11 only supports foreign keys from a partitioned table to a (non-partitioned) table.
Previously not even that was possible, and that's what the release notes are about.
This limitation is documented in the chapter about partitioning in the manual
While primary keys are supported on partitioned tables, foreign keys referencing partitioned tables are not supported. (Foreign key references from a partitioned table to some other table are supported.
(emphasis mine)
I am currently trying to create an sqlite database where I can import a table from another sqlite database (can't attach) and add some extra data to each column.
Since there is no INSERT OR UPDATE I came up with this:
I was thinking about splitting the data into two tables and join them afterwards so I can just dump the whole import into one table replacing everything that changed and manage the extra data separately since that does not change on import.
The first table (let's call it base_data) would look like
local_id | remote_id | base_data1 | base_data2 | ...
---------+-----------+------------+------------+----
besides the local_id everything would just be a mirror of the remote database (I'll probably add a sync timestamp but that does not matter now).
The second table would look similar but has remote_id set as foreign key
remote_id | extra_data1 | extra_data2 | ...
----------+-------------+-------------+----
CREATE TABLE extra_data (
remote_id INTEGER
REFERENCES base_data(remote_id)
ON DELETE CASCADE ON UPDATE CASCADE
DEFERRABLE INITIALLY DEFERRED,
extra_data1 TEXT,
extra_data2 TEXT,
/* etc */
)
Now my idea was to simply INSERT OR REPLACE INTO base_data ... values because the database I import from has no sync timestamp or whatsoever and I would have to compare everything to find out what row I have to UPDATE / what to INSERT.
But here lies the problem: INSERT OR REPLACE is actually a DELETE followed by an INSERT and the delete part triggers the foreign key ON DELETE which I thought I could prevent by making the constraint DEFERRED. It does not work if I wrap INSERT OR REPLACE in a transaction either. It's always deleting my extra data although the same foreign key exists after the statement.
Is it possible to stop ON DELETE to trigger until the INSERT OR REPLACE is finished? Maybe some special transaction mode / pragma ?
It seems work if I replace the ON DELETE CASCADE part by a trigger like:
CREATE TRIGGER on_delete_trigger
AFTER DELETE ON base_data
BEGIN
DELETE FROM extra_data WHERE extra_data.remote_id=OLD.remote_id;
END;
That trigger is only triggered by a DELETE statement and should solve my problem so far.
(Answer provided by the OP in the question)
Additional info by jmathew, citing the documentation:
When the REPLACE conflict resolution strategy deletes rows in order to satisfy a constraint, delete triggers fire if and only if recursive triggers are enabled.
Assuming you only have a single foreign key relationship on a primary key in your referenced table (as you do in your example), this proved to be a fairly painless solution for me.
Simply disable foreign key checks, run the replace query, then enable foreign keys again.
If the replace query is the only query that runs while foreign keys are disabled, you can be assured that no foreign keys will be fouled up. If you are inserting a new row, nothing will have had a chance to be linked to it yet and if you are replacing a row, the existing row will not be removed or have its primary key changed by the query so the constraint will still hold once foreign keys are re-enabled.
SQLlite code looks something like this:
PRAGMA foreign_keys=OFF;
INSERT OR REPLACE ...;
PRAGMA foreign_keys=ON;
What about the reasons of such behavior, there's an explanation from PostgreSQL team:
Yeah, this is per SQL spec as far as we can tell. Constraint checks can
be deferred till end of transaction, but "referential actions" are not
deferrable. They always happen during the triggering statement. For
instance SQL99 describes the result of a cascade deletion as being that
the referencing row is "marked for deletion" immediately, and then
All rows that are marked for deletion are effectively deleted
at the end of the SQL-statement, prior to the checking of any
integrity constraints.
I have a bunch of SQLite db files, and I need to merge them into one big db files.
How can I do that?
Added
Based on this, I guess those three commands should merge two db into one.
attach './abc2.db' as toMerge;
insert into test select * from toMerge.test
detach database toMerge
The problem is the db has PRIMARY KEY field, and I got this message - "Error: PRIMARY KEY must be unique".
This is the test table for the db.
CREATE TABLE test (id integer PRIMARY KEY AUTOINCREMENT,value text,goody text)
I'm just thinking off my head here... (and probably after everybody else has moved on, too).
Mapping the primary key to "NULL" should yield the wanted result (no good if you use it as foreign key somewhere else, since the key probably exists, but has different contents)
attach './abc2.db' as toMerge;
insert into test select NULL, value, goody from toMerge.test;
detach database toMerge;
actual test:
sqlite> insert into test select * from toMerge.test;
Error: PRIMARY KEY must be unique
sqlite> insert into test select NULL, value, goody from toMerge.test;
sqlite> detach database toMerge;
I'm not 100% sure, but it seems that I should read all the elements and insert the element (except the PRIMARY KEY) one by one into the new data base.
SQLite3 gives you a default primary key called rowid for each table if you don't specify a primary key. However, it looks like there are some disadvantages to relying on this:
The VACUUM command may change the ROWIDs of entries in tables that do not have an explicit INTEGER PRIMARY KEY.
http://www.sqlite.org/lang_vacuum.html
I want to alter an existing SQLite3 database to use explicit primary keys rather than implicit rowid's so I have the ability to run vacuum when necessary. Can I do this without rebuilding the whole database?
You don't need to rebuild the whole database. However since SQLite doesn't support ALTER TABLE statements you need to:
create a temporary table with the correct schema
copy all data from the original table to the temp table
delete the original table
rename the temp table
I suggest you use a app such as SQLiteman to do this for you.