I'm using an sqlite database, and i want to reset rowid field after deleting rows. my table doesn't have a primary key or a specified Id, i'm only using the default rowid given by sqlite.
NOM TYPE CL VAL SP TP
"test1" "test1" "1" "1" "test1" "test1"
"test2" "test2" "2" "2" "test2" "test2"
"test3" "test3" "3" "3" "test3" "test3"
When i delete one or multiple rows, i want the default rowid to reset automatically, so i have read some answers and found that we can do that by using sql_sequence table .
delete from sqlite_sequence where name='table_name'
the problem is that i can't find that table in my sqlite
[ no such table: sqlite_sequence ]
Is there any solution to my problem
A table by default will have a rowid that does not use sqlite_sequence table.
It is only when an alias for the rowid is specified (defined using column_name INTEGER PRIMARY KEY or column_name INTEGER PRIMARY KEY AUTOINCREMENT) AND that the definition of the alias includes the optional AUTOINCREMENT keyword/phrase, that the algorithm for determining the rowid utilises the sqlite_sequence table to ensure that the rowid is greater than the last used rowid (typically it will be greater anyway).
Even if you used AUTOINCREMENT and thus the sqlite_sequence table existed and you deleted the row for the table from the sqlite_sequence table. This would not reset the sequence as I believe the algorithm used uses the higher of the highest existing rowid in the table and the value obtained from the sqlite_sequence table. here's a post I made in realtion to this
There is likely very little, or no, reason to utilise the rowid as if it's something other than the means to uniquely identify a row.
However, if you really want to manage the rowid then you can specifically set this value by specifying the rowid column (noting that there is an implied UNIQUE constraint).
So you could for example use :-
INSERT INTO yourtable (rowid, NOM, TYPE, CL, VAL, SP, TP) VALUES(123,'MyName','Testx', '123', '123', 'Testx', 'Testx');
rowid will be set to 123
You could also use update a rowid e.g. :-
UPDATE yourtable SET rowid = 5 WHERE rowid = 123;
- rowid that was 123 is changed to 5 unique constraint permitting
You could obtain the highest rowid using:-
SELECT max(rowid) from yourtable;
You could even use :-
INSERT INTO yourtable (rowid, NOM, TYPE, CL, VAL, SP, TP) VALUES((SELECT max(rowid) from yourtable)+1,'MyName','maxx', '???', '???', 'Maxx', 'Maxx');
insert with the next rowid sequence.
Note the above examples are without an alias of the rowid, obviously they not rely upon the non-existant sqlite_sequence table.
Of course you could always have your own equivalent of the sqlite_sequence table, just that it CANNOT be named this nor could it's name start with sqlite. Note! this would be the only persistent way using SQLite of resetting the id, but could be a nightmare
All the above assumes that :-
and i want to reset rowid field after deleting rows
is not ....after deleting ALL rows, as if this were the case the rowid would be effectively reset (basically without AUTOINCREMENT rowid is determined according to the highes existing rowid, so if none exist then the rowid will be 1).
See example 2 below.
NOTE!!
This answer is not condoning or recommending any of the above.
Example 1
using the following :-
CREATE TABLE IF NOT EXISTS rowidtable (NOM TEXT, TYPE TEXT, CL TEXT, VAL TEXT, SP TEXT, TP TEXT);
INSERT INTO rowidtable (rowid) VALUES(10);
INSERT INTO rowidtable (rowid) VALUES(12);
INSERT INTO rowidtable (rowid, NOM, TYPE, CL, VAL, SP, TP) VALUES(123,'MyName','Testx', '123', '123', 'Testx', 'Testx');
INSERT INTO rowidtable (NOM, TYPE, CL, VAL, SP, TP) VALUES('MyName','Testx', '123', '123', 'Testx', 'Testx');
UPDATE rowidtable SET rowid = 5 WHERE rowid = 123;
SELECT max(rowid) from rowidtable;
--// Following repteated 3 times (so rowid's 125,126 and 127 used respectively)
INSERT INTO rowidtable (rowid, NOM, TYPE, CL, VAL, SP, TP) VALUES((SELECT max(rowid) from rowidtable)+1,'MyName','maxx', '???', '???', 'Maxx', 'Maxx');
select rowid,* from rowidtable;
The resultant table is :-
Rows with id's 10 and 12 were inserted first (nulls (red) for other values)
Row with id 5 was originally row with id 123
Rows with id's 125, 126 and 127 were inserted using the rowid as obtained via subquery that adds 1 to the maximum current rowid (i.e. last insert run 3 times).
Example 2
If the following were then used :-
DELETE from rowidtable
INSERT INTO rowidtable (NOM) VALUES('after mass delete');
Then the resultant table would be :-
The sqlite_sequence table is only used for tables that have a INTEGER PRIMARY KEY AUTOINCREMENT column defined.
If you are relying on the value of the internal defaultrowid for a given row in a table, then you should define a named autoincrement primary key for that table, then you can reset the counter using the sqlite_sequence table.
Thanks for all your answers i found what i was looking for and it quite simple
DELETE FROM 'table_name' WHERE col='value';
REINDEX 'table_name';
and this will rebuild the specified table index, so if all the rows are deleted the index is reset to 0, and if is still row in the table, the index will be reordred correctly.
Related
I need to understand how to add a statement that will delete the results of the following query:
I understand that a DELETE statement with a WHERE clause would normally be used, but because I'm SELECTING two different columns, the where clause doesn't accept the comma. I've not yet been able to figure out how to turn this into a CTE (but maybe that's overkill?) and then call it in a DELETE statement later (assuming that's even an option). Examples always have the DELETE FROM..., or WHICH statements, neither of which seem implementable under this code. Do I have to rewrite my code so it includes a WHICH statement?
SELECT field1, field2
FROM table
GROUP BY field1, field2
HAVING SUM(field3) IS NULL
Expecting to be able to institute a DELETE statement to delete the results of the query.
I believe that you could use (assuming that the table is not a WITHOUT ROWID table) :-
DELETE FROM mytable WHERE rowid IN (SELECT rowid FROM mytable GROUP BY field1, field2 HAVING SUM(field3) IS NULL);
An alternative using a CTE (where the CTE has been given the name deletions) would be :-
WITH deletions(rowid) AS (SELECT rowid
FROM mytable
GROUP BY field1, field2
HAVING SUM(field3) IS NULL
)
DELETE FROM mytable WHERE rowid IN (SELECT rowid FROM deletions);
note that mytable has been used as the table name instead of table.
Considering the comment
Primary key is field1
then :-
DELETE FROM mytable WHERE field1 IN (SELECT field1 FROM mytable GROUP BY field1, field2 HAVING SUM(field3) IS NULL);
could be used, this would then work whether or not the table is defined as a WITHOUT ROWID table, a similar change could be applied to the CTE version.
Notes
Using GROUP BY on a PRIMARY KEY, as it is UNIQUE, will result in as many groups and therefore rows, as there are rows. Effectively the query could be SELECT field1 FROM mytable WHERE field3 IS NULL and therefore the deletion could simply be DELETE FROM mytable WHERE field3 IS NULL.
If this were not the case and field1 was not the PRIMARY KEY, then the complication is that values per group that are not aggregated values are values from an arbitrarily selected row. In short you would delete 1 from a number of the rows that where grouped.
If I have a non-integer primary-key the rowid is an auto-increment starting at 1.
sqlite> create table t1 (name text, documentid integer, primary key (name));
sqlite> insert into t1 (name, documentid) values ('max', 123);
sqlite> insert into t1 (name, documentid) values ('duf', 321);
sqlite> select rowid,* from t1;
1|max|123
2|duf|321
But if I have a INTEGER primary-key it seems the rowid is equal to it.
sqlite> create table t2 (name text, xid integer, primary key (xid));
sqlite> insert into t2 (name, xid) values ('max', 123);
sqlite> insert into t2 (name, xid) values ('duf', 321);
sqlite> select rowid,* from t2;
123|max|123
321|duf|321
Thats unexpected for me. I would expect rowid to behave like in the 1st sample.
Is that normal behaviour? Can I make it work like expected?
I am using SqlLite3 3.27
The problem is not the value as long it is uniqua (must be by definition of primary). But in JDBC I can not address ResultSet.getInt ("rowid") anymore - need to use getInt ("xid") instead" to make it work. Thats abnormal to a table with a non-integer primar-key.
An INTEGER PRIMARY KEY column is just an alias for the rowid. It acts the same (Having a value automatically assigned if left out when inserting a row), and doesn't even take up any extra space in the database. You can reference the column via its name, rowid, or any of the other standard aliases for rowid like oid.
From the documentation:
With one exception noted below, if a rowid table has a primary key that consists of a single column and the declared type of that column is "INTEGER" in any mixture of upper and lower case, then the column becomes an alias for the rowid. Such a column is usually referred to as an "integer primary key". A PRIMARY KEY column only becomes an integer primary key if the declared type name is exactly "INTEGER". Other integer type names like "INT" or "BIGINT" or "SHORT INTEGER" or "UNSIGNED INTEGER" causes the primary key column to behave as an ordinary table column with integer affinity and a unique index, not as an alias for the rowid.
If you just do
INSERT INTO t2(name) VALUES ('max');
a value will be automatically generated for xid instead of explicitly using the one provided in the insert like in your example.
Yes it's the normal behavior.
When you define an integer column xid as primary key, then xid is just an alias of rowid.
What you can do is define xid as UNIQUE and not PRIMARY KEY:
create table t2 (name text, xid integer unique)
Then you will have the functionality that you want, because the rowid will be a different auto increment column.
Or define xid as TEXT:
create table t2 (name text, xid text, primary key (xid));
In this case also rowid is a different column and don't worry about the data you store in xid.
You can treat this column just like an integer column so you can perform any arithmetic calculation and aggregation.
You can find more here: https://www.sqlite.org/rowidtable.html
Consider the following table definition:
CREATE TABLE names (
id INTEGER,
name TEXT NOT NULL,
PRIMARY KEY (id)
)
Does it guarantee that the id will be auto-incremented for every new insert AND that the values for deleted rows will not be reused?
I looked up in the documentation for Sqlite3, but couldn't find the answer.
id INTEGER PRIMARY KEY on it's own guarantees (requires) a unique integer value and will if no value is specifically assigned provide one until the highest value has reached the highest allowed value for a 64 bit signed integer (9223372036854775807) after which an unused value may be found and applied.
With AUTOINCREMENT there is a guarantee (if not circumvented) of always providing a higher value BUT if 9223372036854775807 is reached instead of allocating an unused number an SQLITE_FULL error will result. That is the only difference from the point of view of what number will be assigned.
Neither guarantees a monotonically increasing value.
Without AUTOINCREMENT the calculation/algorithm is equivalent to
1 + max(rowid) and if the value is greater than 9223372036854775807 an attempt is made to find an unused and therefore lower value.
I've not seen that anyone has come across the situation where a random unused value has not been assigned.
With AUTOINCREMENT the calculation/algorithim is
the greater of 1 + max(rowid) or SELECT seq FROM sqlite_sequence WHERE name = 'the_table_name_the_rowid_is_being_assigned_to' and if the value is greater than 9223372036854775807 then SQLITE_FULL ERROR.
noting that either way there is the possibility that the max rowid is for a row that eventually doesn't get inserted and therefore the potential for gaps.
The answer is perhaps best put as: it's best/recommended to use the id column solely for it's intended purpose, that of efficiently identifying a row and not as a means of handling other data requirements, and if done so, there there is no need for AUTOINCREMENT (which has overheads)
In short
Does it guarantee that the id will be auto-incremented
NO
values for deleted rows will not be reused?
NO for the given code
for :-
CREATE TABLE names (id INTEGER PRIMARY KEY AUTOINCREMENT name TEXT NOT NULL)
again NO as if 9223372036854775807 is reached then an SQLITE_FULL error will result, otherwise YES.
So really AUTOINCREMENT is only really relevant (if the id used as expected/intended) when the 9223372036854775807'th row has been inserted.
Perhaps consider the following :-
DROP TABLE IF EXISTS table1;
DROP TABLE IF EXISTS table2;
CREATE TABLE IF NOT EXISTS table1 (id INTEGER PRIMARY KEY, somecolumn TEXT);
CREATE TABLE IF NOT EXISTS table2 (id INTEGER PRIMARY KEY AUTOINCREMENT, somecolumn TEXT);
INSERT INTO table1 VALUES (9223372036854775807,'blah');
INSERT INTO table2 VALUES (9223372036854775807,'blah');
INSERT INTO table1 (somecolumn) VALUES(1),(2),(3);
SELECT * FROM table1;
INSERT INTO table2 (somecolumn) VALUES(1),(2),(3);
This creates the two similar tables, the only difference being the use of AUTOINCREMENT. Each has a row inserted with the highest allowable value for the id column.
An attempt is then made to insert 3 rows where the id will be assigned by SQLite.
3 rows are inserted into the table without AUTOINCREMENT but no rows are inserted when AUTOINCREMENT is used. as per :-
CREATE TABLE IF NOT EXISTS table1 (id INTEGER PRIMARY KEY, somecolumn TEXT)
> OK
> Time: 0.098s
CREATE TABLE IF NOT EXISTS table2 (id INTEGER PRIMARY KEY AUTOINCREMENT, somecolumn TEXT)
> OK
> Time: 0.098s
INSERT INTO table1 VALUES (9223372036854775807,'blah')
> Affected rows: 1
> Time: 0.094s
INSERT INTO table2 VALUES (9223372036854775807,'blah')
> Affected rows: 1
> Time: 0.09s
INSERT INTO table1 (somecolumn) VALUES(1),(2),(3)
> Affected rows: 3
> Time: 0.087s
SELECT * FROM table1
> OK
> Time: 0s
INSERT INTO table2 (somecolumn) VALUES(1),(2),(3)
> database or disk is full
> Time: 0s
The result of the SELECT for table1 (which may differ due to randomness) was :-
I have an SQLite table that uses an autogenerated id as primary key. I want to do bulk inserts using UNION SELECT, but it doesn't seem to be happy unless I specify an id for each row:
sqlite> create table CelestialObject (id INTEGER PRIMARY KEY, name VARCHAR(25), distance REAL);
sqlite> insert into CelestialObject select 'Betelguese' as name, 200 as distance UNION SELECT 'Procyon', 500;
Error: table CelestialObject has 3 columns but 2 values were supplied
If I specify AUTOINCREMENT for the id (i.e., "id INTEGER PRIMARY KEY AUTOINCREMENT") when I create the table, the error is the same.
Can anyone tell me whether there's a way to use a bulk insert without specifying an id for each row?
Either provide three values per row:
INSERT INTO CelestialObject SELECT NULL, 'x', 42 ...
or explicitly specify the columns you want to fill:
INSERT INTO CelestialObject(name, distance) SELECT 'x', 42 ...
If you have SQLite 3.7.11 or later, you can do bulk inserts easier:
INSERT INTO CelestialObject(name, distance)
VALUES ('Betelgeose', 200),
('Procyon', 500),
... ;
I've found a few "would be" solutions for the classic "How do I insert a new record or update one if it already exists" but I cannot get any of them to work in SQLite.
I have a table defined as follows:
CREATE TABLE Book
ID INTEGER PRIMARY KEY AUTOINCREMENT,
Name VARCHAR(60) UNIQUE,
TypeID INTEGER,
Level INTEGER,
Seen INTEGER
What I want to do is add a record with a unique Name. If the Name already exists, I want to modify the fields.
Can somebody tell me how to do this please?
Have a look at http://sqlite.org/lang_conflict.html.
You want something like:
insert or replace into Book (ID, Name, TypeID, Level, Seen) values
((select ID from Book where Name = "SearchName"), "SearchName", ...);
Note that any field not in the insert list will be set to NULL if the row already exists in the table. This is why there's a subselect for the ID column: In the replacement case the statement would set it to NULL and then a fresh ID would be allocated.
This approach can also be used if you want to leave particular field values alone if the row in the replacement case but set the field to NULL in the insert case.
For example, assuming you want to leave Seen alone:
insert or replace into Book (ID, Name, TypeID, Level, Seen) values (
(select ID from Book where Name = "SearchName"),
"SearchName",
5,
6,
(select Seen from Book where Name = "SearchName"));
You should use the INSERT OR IGNORE command followed by an UPDATE command:
In the following example name is a primary key:
INSERT OR IGNORE INTO my_table (name, age) VALUES ('Karen', 34)
UPDATE my_table SET age = 34 WHERE name='Karen'
The first command will insert the record. If the record exists, it will ignore the error caused by the conflict with an existing primary key.
The second command will update the record (which now definitely exists)
You need to set a constraint on the table to trigger a "conflict" which you then resolve by doing a replace:
CREATE TABLE data (id INTEGER PRIMARY KEY, event_id INTEGER, track_id INTEGER, value REAL);
CREATE UNIQUE INDEX data_idx ON data(event_id, track_id);
Then you can issue:
INSERT OR REPLACE INTO data VALUES (NULL, 1, 2, 3);
INSERT OR REPLACE INTO data VALUES (NULL, 2, 2, 3);
INSERT OR REPLACE INTO data VALUES (NULL, 1, 2, 5);
The "SELECT * FROM data" will give you:
2|2|2|3.0
3|1|2|5.0
Note that the data.id is "3" and not "1" because REPLACE does a DELETE and INSERT, not an UPDATE. This also means that you must ensure that you define all necessary columns or you will get unexpected NULL values.
INSERT OR REPLACE will replace the other fields to default value.
sqlite> CREATE TABLE Book (
ID INTEGER PRIMARY KEY AUTOINCREMENT,
Name TEXT,
TypeID INTEGER,
Level INTEGER,
Seen INTEGER
);
sqlite> INSERT INTO Book VALUES (1001, 'C++', 10, 10, 0);
sqlite> SELECT * FROM Book;
1001|C++|10|10|0
sqlite> INSERT OR REPLACE INTO Book(ID, Name) VALUES(1001, 'SQLite');
sqlite> SELECT * FROM Book;
1001|SQLite|||
If you want to preserve the other field
Method 1
sqlite> SELECT * FROM Book;
1001|C++|10|10|0
sqlite> INSERT OR IGNORE INTO Book(ID) VALUES(1001);
sqlite> UPDATE Book SET Name='SQLite' WHERE ID=1001;
sqlite> SELECT * FROM Book;
1001|SQLite|10|10|0
Method 2
Using UPSERT (syntax was added to SQLite with version 3.24.0 (2018-06-04))
INSERT INTO Book (ID, Name)
VALUES (1001, 'SQLite')
ON CONFLICT (ID) DO
UPDATE SET Name=excluded.Name;
The excluded. prefix equal to the value in VALUES ('SQLite').
Firstly update it. If affected row count = 0 then insert it. Its the easiest and suitable for all RDBMS.
Upsert is what you want. UPSERT syntax was added to SQLite with version 3.24.0 (2018-06-04).
CREATE TABLE phonebook2(
name TEXT PRIMARY KEY,
phonenumber TEXT,
validDate DATE
);
INSERT INTO phonebook2(name,phonenumber,validDate)
VALUES('Alice','704-555-1212','2018-05-08')
ON CONFLICT(name) DO UPDATE SET
phonenumber=excluded.phonenumber,
validDate=excluded.validDate
WHERE excluded.validDate>phonebook2.validDate;
Be warned that at this point the actual word "UPSERT" is not part of the upsert syntax.
The correct syntax is
INSERT INTO ... ON CONFLICT(...) DO UPDATE SET...
and if you are doing INSERT INTO SELECT ... your select needs at least WHERE true to solve parser ambiguity about the token ON with the join syntax.
Be warned that INSERT OR REPLACE... will delete the record before inserting a new one if it has to replace, which could be bad if you have foreign key cascades or other delete triggers.
If you have no primary key, You can insert if not exist, then do an update. The table must contain at least one entry before using this.
INSERT INTO Test
(id, name)
SELECT
101 as id,
'Bob' as name
FROM Test
WHERE NOT EXISTS(SELECT * FROM Test WHERE id = 101 and name = 'Bob') LIMIT 1;
Update Test SET id='101' WHERE name='Bob';
I believe you want UPSERT.
"INSERT OR REPLACE" without the additional trickery in that answer will reset any fields you don't specify to NULL or other default value. (This behavior of INSERT OR REPLACE is unlike UPDATE; it's exactly like INSERT, because it actually is INSERT; however if what you wanted is UPDATE-if-exists you probably want the UPDATE semantics and will be unpleasantly surprised by the actual result.)
The trickery from the suggested UPSERT implementation is basically to use INSERT OR REPLACE, but specify all fields, using embedded SELECT clauses to retrieve the current value for fields you don't want to change.
I think it's worth pointing out that there can be some unexpected behaviour here if you don't thoroughly understand how PRIMARY KEY and UNIQUE interact.
As an example, if you want to insert a record only if the NAME field isn't currently taken, and if it is, you want a constraint exception to fire to tell you, then INSERT OR REPLACE will not throw and exception and instead will resolve the UNIQUE constraint itself by replacing the conflicting record (the existing record with the same NAME). Gaspard's demonstrates this really well in his answer above.
If you want a constraint exception to fire, you have to use an INSERT statement, and rely on a separate UPDATE command to update the record once you know the name isn't taken.