SQLite3 gives you a default primary key called rowid for each table if you don't specify a primary key. However, it looks like there are some disadvantages to relying on this:
The VACUUM command may change the ROWIDs of entries in tables that do not have an explicit INTEGER PRIMARY KEY.
http://www.sqlite.org/lang_vacuum.html
I want to alter an existing SQLite3 database to use explicit primary keys rather than implicit rowid's so I have the ability to run vacuum when necessary. Can I do this without rebuilding the whole database?
You don't need to rebuild the whole database. However since SQLite doesn't support ALTER TABLE statements you need to:
create a temporary table with the correct schema
copy all data from the original table to the temp table
delete the original table
rename the temp table
I suggest you use a app such as SQLiteman to do this for you.
Related
suppose I have a database like:
CREATE TABLE top_table (
test_id PRIMARY KEY,
cmd TEXT);
CREATE TABLE job_table (
id PRIMARY KEY,
rid integer references top_table(rowid));
If I do a vacuum, would it preserve the relationship "rid integer references top_table(rowid)"? Meaning, would it either leave the top_table rowids unchanged, or would it change them and make corresponding changes to rid in job_table? I want to do the vacuum because I'm merging databases, so I read in a second table, insert its data into an existing table, and drop the second table. If the vacuum won't work properly, my next thought was to create the combined database, drop the table, do a sump and read the sql into a new database. Or is there an easier/cleaner method that I'm unaware of?
Using :-
CREATE TABLE top_table ( test_id PRIMARY KEY, cmd TEXT);
CREATE TABLE job_table ( id PRIMARY KEY, rid integer references top_table(rowid));
Could be an issue as you are referencing the rowid directly rather than referencing an alias of the rowid. Typically an alias of the rowid would be used as per :-
CREATE TABLE top_table ( test_id INTEGER PRIMARY KEY, cmd TEXT);
CREATE TABLE job_table ( id INTEGER PRIMARY KEY, rid INTEGER REFERENCES top_table(test_id));
That is specifying INTEGER PRIMARY KEY, instead of PRIMARY KEY (column affinity NUMERIC) is a special case which makes the column being defined an alias of the rowid, the rowid will then not be subject to change by the VACUUM command as per :-
The VACUUM command may change the ROWIDs of entries in any tables that do not have an explicit INTEGER PRIMARY KEY.
SQLITE -CREATE TABLE - ROWID's and the INTEGER PRIMARY KEY
SQLite - VACUUM
I'm somewhat new at this, but strictly speaking, am I using foreign keys? I'm not declaring it to be a foreign key, and "pragma foreign_keys" returns 0
No BUT only because Foreign KEY enforcement is off (hasn't been turned on), thus the REFERENCES top_table(test_id) has no effect, other than being parsed.
Assuming that you programmatically maintain the referential integrity this should not be an issue.
It can be advantageous to have FOREIGN KEY support on as not only will referential integrity be enforced, the ON UPDATE and ON DELETE actions can then be utilised which can simplify handling updates and deletions by using the CASCADE option (e.g. deleting a parent will a) work without a conflict and b) cascade the deletion so that the children rows will be deleted automatically (if they can be))
SQLite - Enabling Foreign Key Support
SQLite - ON DELETE and ON UPDATE Actions
regarding the comment
test_id is a string, which can be relatively long (60-80 characters or so). the original version of the schema had test_id as a member of both tables. the version that I've shown above (with a bunch of fields removed from both tables for this question) was a check on how much smaller the database got if I switched matching long text strings to matching the rowid which seemed to make more sense than adding a field that serves the same purpose)
In the case where a string is used for referencing will not only save space (in the longer term) BUT the overheads will also be greater than using an alias of the rowid.
In the longer term, because SQLite save chunks (pages). A table will (by default) takes up at least 4k and then 8k ....
First, space wise, not only will the extra 52-80 bytes waste space the parent's column should also be unique. If UNIQUE were specified then there is the additional overhead of an index.
However, with an alias of the rowid, then 8 bytes max for the integer, which is used anyway unless the table is defined using WITHOUT ROWID. If I recall there is an extra byte for the flag that the column is an alias of the rowid. The rowid and therefore alias has to be unique and the index (as such) exists. Furthermore due to it being central to SQLite accessing by rowid can be twice as fast.
In short an alias of the rowid is probably the best option in many (probably most) cases for relationships.
I have a Xamarin.Forms app that uses a SQLite database locally on the device. Here's some sample data structure:
Table x: id, name
Table y: id, name
Table x_y: id, x_id, y_id
Since SQLite doesn't support altering columns, one of the schema updates we sent down in a patch did the following:
Rename table x to x_temp
Create new/updated table x
Insert all data from table x_temp into table x
Drop table if exists x
That seems to work just fine. However, when I'm attempting to run an insert statement on table x_y, I am getting a SQLite exception: "no such table: main.x_temp".
When I look at the SQLite query string while debugging there is no mention of table x_temp whatsoever. So, if I delete the entire database and re-create everything the insert works just fine.
I'm from a MSSQL background, am I not understanding something about SQLite in general? Is the foreign key constraint from table x_y trying to reference x_temp because I renamed the original table (I may have just answered my own question)? If that's the case, surely there is a way around this without having to cascade and re-create every table?
Any input would be appreciated. Thanks!
I believe that your issue may be related to the SQlite version in conjunction with whether or not Foreign Key Support has been turned on.
That is the likliehood is that :-
Is the foreign key constraint from table x_y trying to reference
x_temp because I renamed the original table (I may have just answered
my own question)?
Would be the issue, as you likely have Foreign Key Support turned on as per :-
Prior to version 3.26.0 (2018-12-01), FOREIGN KEY references to a table that is renamed were only edited if the PRAGMA foreign_keys=ON, or in other words if foreign key constraints were begin enforced.
With PRAGMA foreign_keys=OFF, FOREIGN KEY constraints would not be changed when the table that the foreign key referred to (the "parent table") was renamed.
Beginning with version 3.26.0, FOREIGN KEY constraints are always converted when a table is renamed, unless the PRAGMA legacy_alter_table=ON setting is engaged. The following table summaries the difference:
SQL As Understood By SQLite - ALTER TABLE
If that's the case, surely there is a way around this without having
to cascade and re-create every table?
Yes, as the latest version of SQlite on Android is 3.19.0 (I believe), then you can turn Foreign Key support off using the foreign_keys pragma when renaming the table.
Note Foreign Keys cannot be turned off within a transaction.
See SQL As Understood By SQLite - ALTER TABLE and PRAGMA foreign_keys = boolean;
I create a table, lets name it CUSTOMERS in SQLite:
CREATE TABLE "CUSTOMERS" (
"tel" INTEGER NOT NULL,
"customer" VARCHAR ,
);
When I see the table from a GUI (I use SQLite Manager from Firefox), I noticed that there is an extra column rowid which is working like auto-increment. My question is, in tables where I don't use a primary key should I specify a column like:
ROWID INTEGER PRIMARY KEY AUTOINCREMENT
If I execute this query PRAGMA table_info(CUSTOMERS); I get only the two columns tel,customer.
Sqlite usually adds a rowid automatically as #laato linked in the comments SqLite : ROWID
That can be removed, but does not need to be specified. So there is no need to add it to the Create Table.
The hidden rowid allows delete's to be targetted at a single row, bu
t if you are using the ROWID as a specific foreign key, it would be better to name a column explicitly. That will then become a synonym with the rowid.
You can see here, as it was commented in your question.
However, if you are in a dilemma what to choose between these options, SQLite recommends that you should not use AUTOINCREMENT attribute because:
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and
disk I/O overhead and should be avoided if not strictly needed. It is
usually not needed.
More info you can read here.
I wanted to add a constraint to an existing column in my SQLite database. However, I read that it is not possible to do so.
I tried the solution from How do I rename a column in a SQLite database table?, but there seems to be missing the copying of all the metadata.
I pretty much want an exact copy of a given table, except for the new constraints.
How does the INSERT command look like to copy all the metadata, thus the indexes will increase correctly, for example.
I'm not a heavy user of sqlite3, but you can use the command line to get the data and "create table" and "create index" commands. I am using the 'History' DB from the Google chrome browser which has a table called "visits". The 'mode insert' command says to provide output in a format that can be used to input this data. The '.schema visits' command says to show the 'create table' and 'create index' statements. The 'select..' statement gives you the data. The database I used doesn't seem to have any foreign key constraints, but they could very well be part of the 'create table' information if your DB has any.
sqlite3 History
.mode insert
.schema visits
select * from visits;
I have a bunch of SQLite db files, and I need to merge them into one big db files.
How can I do that?
Added
Based on this, I guess those three commands should merge two db into one.
attach './abc2.db' as toMerge;
insert into test select * from toMerge.test
detach database toMerge
The problem is the db has PRIMARY KEY field, and I got this message - "Error: PRIMARY KEY must be unique".
This is the test table for the db.
CREATE TABLE test (id integer PRIMARY KEY AUTOINCREMENT,value text,goody text)
I'm just thinking off my head here... (and probably after everybody else has moved on, too).
Mapping the primary key to "NULL" should yield the wanted result (no good if you use it as foreign key somewhere else, since the key probably exists, but has different contents)
attach './abc2.db' as toMerge;
insert into test select NULL, value, goody from toMerge.test;
detach database toMerge;
actual test:
sqlite> insert into test select * from toMerge.test;
Error: PRIMARY KEY must be unique
sqlite> insert into test select NULL, value, goody from toMerge.test;
sqlite> detach database toMerge;
I'm not 100% sure, but it seems that I should read all the elements and insert the element (except the PRIMARY KEY) one by one into the new data base.