Im working on my second project with sqlite3 and have a question.
What is the difference between working with rowid and/or working with own AUTOINCREMENT INTEGER value?
Is one of both better/faster than the other?
According to the Sqlite documentation the difference is that the rowid can be reassigned, while an AUTOINCREMENT INTEGER can not.
To quote the important part:
(T)he use of AUTOINCREMENT requires additional work to be done as each row is inserted and thus causes INSERTs to run a little slower.
Related
I create a table, lets name it CUSTOMERS in SQLite:
CREATE TABLE "CUSTOMERS" (
"tel" INTEGER NOT NULL,
"customer" VARCHAR ,
);
When I see the table from a GUI (I use SQLite Manager from Firefox), I noticed that there is an extra column rowid which is working like auto-increment. My question is, in tables where I don't use a primary key should I specify a column like:
ROWID INTEGER PRIMARY KEY AUTOINCREMENT
If I execute this query PRAGMA table_info(CUSTOMERS); I get only the two columns tel,customer.
Sqlite usually adds a rowid automatically as #laato linked in the comments SqLite : ROWID
That can be removed, but does not need to be specified. So there is no need to add it to the Create Table.
The hidden rowid allows delete's to be targetted at a single row, bu
t if you are using the ROWID as a specific foreign key, it would be better to name a column explicitly. That will then become a synonym with the rowid.
You can see here, as it was commented in your question.
However, if you are in a dilemma what to choose between these options, SQLite recommends that you should not use AUTOINCREMENT attribute because:
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and
disk I/O overhead and should be avoided if not strictly needed. It is
usually not needed.
More info you can read here.
In Sqlite, there are two ways to create monotonically increasing primary key values generated by the database engine, through the default ROWID mechanism or through the AUTOINCREMENT mechanism.
sqlite> -- Through the default ROWID mechanism
sqlite> CREATE TABLE foo(id INTEGER NOT NULL PRIMARY KEY, foo);
sqlite> INSERT INTO foo (foo) VALUES ('foo');
sqlite> INSERT INTO foo (foo) VALUES ('bar');
sqlite> SELECT * FROM foo;
1|foo
2|bar
sqlite>
sqlite> -- Through the AUTOINCREMENT mechanism
sqlite> CREATE TABLE bar(id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, bar);
sqlite> INSERT INTO bar (bar) VALUES ('foo');
sqlite> INSERT INTO bar (bar) VALUES ('bar');
sqlite> SELECT * FROM bar;
1|foo
2|bar
sqlite> -- Only the AUTOINCREMENT mechanism uses the sqlite_sequence table
sqlite> SELECT * FROM sqlite_sequence WHERE name in ('foo', 'bar');
bar|2
The documentation seems to suggest that using AUTOINCREMENT is bad:
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.
If the AUTOINCREMENT keyword appears after INTEGER PRIMARY KEY, that changes the automatic ROWID assignment algorithm to prevent the reuse of ROWIDs over the lifetime of the database. In other words, the purpose of AUTOINCREMENT is to prevent the reuse of ROWIDs from previously deleted rows.
[With AUTOINCREMENT,] SQLite keeps track of the largest ROWID that a table has ever held using an internal table named "sqlite_sequence".The sqlite_sequence table is created and initialized automatically whenever a normal table that contains an AUTOINCREMENT column is created. The content of the sqlite_sequence table can be modified using ordinary UPDATE, INSERT, and DELETE statements. But making modifications to this table will likely perturb the AUTOINCREMENT key generation algorithm. Make sure you know what you are doing before you undertake such changes.
In what cases is it appropriate to use the AUTOINCREMENT keyword?
The documentation says that
the purpose of AUTOINCREMENT is to prevent the reuse of ROWIDs from previously deleted rows.
So it is appropriate to use the AUTOINCREMENT keyword when you need to prevent the reuse of ROWIDs from previously deleted rows. This might be needed if you have external references to those deleted rows, and must not confuse them with new rows.
"the purpose of AUTOINCREMENT is to prevent the reuse of ROWIDs from
previously deleted rows."
Out of context, this makes it sound like that's the only purpose. I would think one of the most obvious decision points for AUTOINCREMENT is whether or not you need the guarantee of monotonically increasing integers. In turn, surely that need most commonly arises because you need to know what order insertions happened in. One might prefer integers over timestamps for this because of: insufficient clock resolution, fear of wonky clocks, difficulty synchronizing if more than one clock source is involved, etc.
Of course, one can always construct off by one situations where a transaction finishes first but gets a higher number than a competing transaction. However, these seem to pretty much correspond to cases where two humans might disagree on which transaction "happened" first, so the guarantee of monotonicity is about is good as it gets, IMO. That's a guarantee SQLite offers if you use AUTOINCREMENT.
"do you have a use case for when it is a good idea to keep an external
reference to deleted rows?"
Not so much a matter of whether it's a good idea: if humans see/use IDs in any form, it can be confusing if a number they thought was unique gets reassigned. Here's an example seen in the wild. This is a subset of the range of needs for monotonically increasing IDs, but more common than you might think. Humans are good at subconsciously absorbing the fact that numbers "are supposed to" always get bigger. For example, in ye olde slashdot days, people could glance at your user ID number to gauge how long you had been a member. On slashdot, I guess a quick hover over the user name accomplishes the same thing and saves having to go look at their profile.
I believe there's a way in InnoDB to stop the auto-increment value from increasing if a record insert attempt failed/was ignored:
innodb_autoinc_lock_mode
However, each time someone connects to my server, I use this query on SQLite3:
"INSERT OR IGNORE INTO IPList (IP) VALUES (" + string(ip) + "); "
Unfortunately, if the IP is already in the table, the auto increment value increases anyways. If lots of people connect to my server that value will be incredibly high.
How do I stop it from doing this in SQLite3?
The documentation says:
Note that "monotonically increasing" does not imply that the ROWID always increases by exactly one. One is the usual increment. However, if an insert fails due to (for example) a uniqueness constraint, the ROWID of the failed insertion attempt might not be reused on subsequent inserts, resulting in gaps in the ROWID sequence. AUTOINCREMENT guarantees that automatically chosen ROWIDs will be increasing but not that they will be sequential.
To ensure that autoincrement values are sequential, drop the AUTOINCREMENT keyword from the table definition and use a plain INTEGER PRIMARY KEY:
The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.
On an INSERT, if the ROWID or INTEGER PRIMARY KEY column is not explicitly given a value, then it will be filled automatically with an unused integer, usually one more than the largest ROWID currently in use. This is true regardless of whether or not the AUTOINCREMENT keyword is used.
Well I don't think there's a setting to do this, I simply fixed it by checking if the row exists before attempting to insert it.
Apparently if insert fails anyways the auto-increment value goes up no matter what. So I just don't call INSERT if it already exists.
i am working with sqlite and i need to reset the auto increment values,
I found on StackOverflow:
SQLite Reset Primary Key Field
but when i do
delete from sqlite_sequence where name='my_table';
all I got is :
Error: no such table: sqlite_sequence
Did someone know the problem ?
I am on an iMac with sqlite3.
Thanks for help !
Have a nice day
There are two types of autoincrementing columns, ones declared as INTEGER PRIMARY KEY, and ones declared as INTEGER PRIMARY KEY AUTOINCREMENT.
Columns with AUTOINCREMENT have a record in the sqlite_sequence table, and can be reset with the above DELETE statement.
Plain INTEGER PRIMARY KEY columns are still autoincrementing, but derive the next value from the largest actual value in the table.
These can be simply reset by deleting all records from the data table itself.
I tested it on my own sqlite DB and it works fine.
See my Fiddle Demo.
Maybe in your case it could simply be a mistake of your spelling of your table name?
Or a misspelling on keyword sqlite_sequence maybe.
I tried INSERT OR REPLACE INTO, but it doesn't preserve the row id when it replaces the record to update it. Another option is do it in two steps: INSERT OR IGNORE INTO then UPDATE, but I would prefer a one step solution. So I am wondering if SQLite has something like the MERGE keyword or other simple solutions?
No, SQLite doesn't support MERGE or upsert.
You can use your two-step solution, but what you probably really want is for the ROWID to be a first-class column in your table. If you declare a column as INTEGER PRIMARY KEY, it will be an alias for the ROWID. Then INSERT OR REPLACE will work fine.