I need to update a table row IF EXISTS, otherwise INSERT a new row.
I tried:
INSERT OR REPLACE INTO table VALUES ...
but if the row row exist this statement changes the row's ROWID, and that's what I'm trying to avoid (I need the rowid :D)
I also tried to find a way to get some sort of return value from the update, in the case where an update has taken place, but I still don't understand how... If I could get the return value from the update statement, I could choose wether to proceed with an insert or not.
Do you have any suggestion or solution to this problem? Or do I need to make a copy of the ROWID and use that instead of the "pure" table ROWID?
Thanks in advance, best regards
ps: I was looking HERE and I was wondering if sqlite has the OUTPUT special word too, but google didn't help me..
---- EDIT after reading comments:
table schema example
CREATE TABLE test (
table_id TEXT NOT NULL,
some_field TEXT NOT NULL,
PRIMARY KEY(table_id)
)
INSERT or REPLACE INTO test (table_id, some_field) VALUES ("foo","bar")
I tested Chris suggestion but the rowid still gets changed. I think the best alternative is to do a SELECT to see if a row with that key already exist. If so, UPDATE, otherwise, INSERT... good old fashion but guaranteed to work.
Combine it with select, like this
INSERT or REPLACE INTO test (ROWID, table_id, some_field)
VALUES ((SELECT ROWID from test WHERE table_id = 'foo' UNION SELECT max(ROWID) + 1 from test limit 1), 'foo','bar')
You need to specify that your table_id is unique in addition to being the primary key:
sqlite> CREATE TABLE test (
table_id TEXT NOT NULL,
some_field TEXT NOT NULL,
PRIMARY KEY(table_id),
UNIQUE(table_id)
);
sqlite> insert or replace into test values("xyz", "other");
sqlite> select * FROM test;
xyz|other
sqlite> insert or replace into test values("abc", "something");
sqlite> insert or replace into test values("xyz", "whatever");
sqlite> select * FROM test;
abc|something
xyz|whatever
From version 3.24.0 (2018-06-04), SQLite now supports an UPSERT clause that will do exactly what the OP needed: https://www.sqlite.org/lang_UPSERT.html
The insert would now look like this:
INSERT INTO test (table_id, some_field) VALUES ("foo","baz")
ON CONFLICT(table_id) DO UPDATE SET some_field=excluded.some_field;
Related
I want to create a table and then initialize it with some values, in as concise manner as possible.
However, this script gets executed every time my app starts, so the insert should happen only on items that were not already added previously.
I do not want to use IGNORE directive in 'INSERT IGNORE INTO', because I do not want to ignore unexpected errors.
For some reason, INSERT INTO fails with "SQL error (1136): Column count doesn't match value count at row 1", even though the select that follows gives the values that need to be added.
Here's the failing code:
START TRANSACTION;
CREATE TABLE IF NOT EXISTS `privileges` (
`id` TINYINT NOT NULL AUTO_INCREMENT,
`label` VARCHAR(25) UNIQUE,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `privileges` (`label`)
SELECT `label` FROM (
SELECT NULL AS `label`
UNION VALUES
('item1'),
('item2')
) X
WHERE `label` IS NOT NULL
AND `label` NOT IN (SELECT `label` FROM `privileges`)
COMMIT;
Currently I am solving this by first inserting the values into a temporary table, and then performing a select on that. But why isn't the above working and is there a more concise way to do what I'm trying to do?
I'm using MariaDB 10.3.9, added missing UNIQUE constraint
Edit 2: Thanks to LukStorms for figuring out the error was related to AUTO_INCREMENT, it seems passing NULL for AUTO_INCREMENT column solves the problem like so:
INSERT INTO `privileges` (id, label)
WITH ITEMS(label) AS (VALUES
('users:read'),('users:create'),
('clients:read'),('clients:write'),
('catalog:read'),('catalog:write'),
('cart:read'),('cart:write'),
('orders:read'),('orders:write'), ('test1')
) SELECT NULL, label FROM ITEMS i
WHERE label NOT IN (SELECT label FROM `privileges`);
In MariaDb 10.3+, using a CTE with a the VALUES expression can let you assign a column name to it.
with ITEMS(label) as
(VALUES
('item1')
,('item2'))
select i.label
from ITEMS i
where not exists (select 1 from privileges p where p.label = i.label)
But somehow it gives an error when inserting into a table that has a field with an AUTO_INCREMENT. Seems like a bug to me.
However, when you insert a NULL into a an AUTO_INCREMENT field then the NULL gets ignored. But you discovered that behaviour yourself.
So this works:
INSERT INTO privileges (id, label)
WITH ITEMS(label) as (
VALUES ('item1'), ('item2')
)
SELECT null, i.label
FROM ITEMS i
WHERE NOT EXISTS (SELECT 1 FROM privileges p WHERE p.label = i.label);
Test on db<>fiddle here
Using unioned selects also works though.
INSERT INTO privileges (label)
SELECT label
FROM (
SELECT 'item1' as label UNION ALL
SELECT 'item2'
) i
WHERE NOT EXISTS (SELECT 1 FROM privileges p WHERE p.label = i.label);
db<>fiddle here
Maybe another way is to use a temporary table (that will vanish when the session expires)
CREATE TEMPORARY TABLE tmp_items (label VARCHAR(25) NOT NULL PRIMARY KEY);
INSERT INTO tmp_items (label) VALUES
('item1')
,('item2');
INSERT INTO privileges (label)
SELECT label
FROM tmp_items i
WHERE label NOT IN (SELECT DISTINCT label FROM privileges);
Test on db<>fiddle here
First, your application is trying to double-insert values. It probably shouldn't be doing that (though I can think of a few valid use cases). Consider making it so that it does not try to add data that it's already added before. If you don't have easy access to inter-instance state, pull the current list out of the database on startup before deciding what to insert.
Second, if you want labels to be unique, why is there not a unique key on the label field? At the moment, INSERT IGNORE wouldn't even work because there is nothing in your schema preventing duplicate label values. I would ask yourself why you need an auto-incrementing ID: why not just have the label, and make it the primary key?
Then, if you still need to do this duplicate-elision at the SQL layer, you may use ON DUPLICATE KEY to suck up redundant inserts of an existing primary key:
INSERT INTO `privileges` (`label`)
VALUES
('item1'),
('item2')
)
ON DUPLICATE KEY UPDATE `label` = `label`
This solution is difficult to implement with your auto-increment ID key, because your application probably doesn't know what the ID is going to be. Another reason to consider dropping it.
Unfortunately, there's no ON DUPLICATE KEY IGNORE.
If you want to keep the ID key, and you don't want your application to do a read step on startup (perhaps for scalability reasons), then INSERT IGNORE to be quite honest is your best bet, though you're still going to need at least a unique key on label to make that work.
I would like to start different tables off at different values for their primary keys during testing to verify I don't have any bugs in my code. Is this possible in Sqlite?
As documented, the last value of an AUTOINCREMENT column is stored in the internal sqlite_sequence table, where it can be changed.
Yes, you can do that.
The simplest way is probably just to insert a row, and specify the the number. If I wanted to start with 1000, I might do something like this.
sqlite> create table test (test_id integer primary key, s char(1));
sqlite> insert into test values (999, 'a');
sqlite> insert into test (s) values ('b');
sqlite> select * from test;
999|a
1000|b
After you've inserted the "first" row (test_id is 1000), you can delete the "seed" row (test_id is 999).
I use a database in my project and when i insert values into a table i need to check if the field already has a value that does not produce an insert.
for exemple:
INSERT INTO myTable (column1) values ('some_value1')
if some_value1 alredy exists in column1 do not insert the value.
Put a unique constraint on myTable.column1. Then, whenever you try to insert a duplicate value, it won't let you as it violates the constraint. You can either catch and handle this error, or just let the DB engine do it's thing automatically.
EDIT: Note that SQLite doesn't allow you to do many alterations to your table, once it's created; so you may have to recreate your table with the constraint in place.
I believe this can be handled using the conflict resolution IGNORE method on SQLite. The code below should do the trick. The column1 here should be set to unique for this.
INSERT OR IGNORE INTO myTable (column1) values ('some_value1')
I'm using the following links for reference;
http://www.sqlite.org/lang_insert.html
http://www.sqlite.org/lang_conflict.html
The questions says it all really.
I have a table and I want to insert a row if it doesn't already exist.
or should I just do an insert and if the key constraint is violated then ignore it?
Use INSERT OR IGNORE: http://www.sqlite.org/lang_insert.html
Use a trigger that fires before INSERTs and discards the duplicate row, something along the lines of...
CREATE TRIGGER trigger_name
BEFORE INSERT on your_table
FOR EACH ROW WHEN EXISTS (SELECT * FROM your_table WHERE id = NEW.id)
BEGIN
SELECT RAISE(IGNORE);
END;
I have a list of strings to insert into a db. They MUST be unique. When i insert i would like their ID (to use as a foreign key in another table) so i use last_insert_rowid. I get 2 problems.
If i use replace, their id
(INTEGER PRIMARY KEY) updates which
breaks my db (entries point to
nonexistent IDs)
If i use ignore, rowid is not updated so i do not get the correct ID
How do i get their Ids? if i dont need to i wouldnt want to use a select statement to check and insert the string if it doesnt exist . How should i do this?
When a UNIQUE constraint violation occurs, the REPLACE algorithm deletes pre-existing rows that are causing the constraint violation prior to inserting or updating the current row and the command continues executing normally. This causes the rowid to change and creates the following problem
Y:> **sqlite3 test**
SQLite version 3.7.4
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> **create table b (c1 integer primary key, c2 text UNIQUE);**
sqlite> **insert or replace into b values (null,'test-1');**
sqlite> **select last_insert_rowid();**
1
sqlite> **insert or replace into b values (null,'test-2');**
sqlite> **select last_insert_rowid();**
2
sqlite> **insert or replace into b values (null,'test-1');**
sqlite> **select last_insert_rowid();**
3
sqlite> **select * from b;**
2|test-2
3|test-1
The work around is to change the definition of the c2 column as follows
create table b (c1 integer primary key, c2 text UNIQUE ON CONFLICT IGNORE);
and to remove the "or replace" clause from your inserts;
then when test after your insert, you will need to execute the following sql: select last_insert_rowid(), changes();
sqlite> **create table b (c1 integer primary key, c2 text UNIQUE ON CONFLICT IGNORE);**
sqlite> **insert into b values (null,'test-1');**
sqlite> **select last_insert_rowid(), changes();**
1|1
sqlite> **insert into b values (null,'test-2');**
sqlite> **select last_insert_rowid(), changes();**
2|1
sqlite> **insert into b values (null,'test-1');**
sqlite> **select last_insert_rowid(), changes();**
2|0
The return value of changes after the 3rd insert will be a notification to your application that you will need to lookup the rowid of "test-1", since it was already on file. Of course if this is a multi-user system, you will need to wrap all this in a transaction as well.
I use the below currently
insert into tbl(c_name) select 'val' where not exists(select id from tbl where c_name ='val');
select id from tbl where c_name ='val';
By "they MUST be unique", do they mean you are sure that they are, or that you want an error as a result if they aren't? If you just make the string itself a key in its table, then I don't understand how either 1 or 2 could be a problem -- you'll get an error as desired in case of unwanted duplication, otherwise the correct ID. Maybe you can clarify your question with a small example of SQL code you're using, the table in question, what behavior you are observing, and what behavior you'd want instead...?
Edited: thanks for the edit but it's still unclear to me what SQL is giving you what problems! If your table comes from, e.g.:
CREATE TABLE Foo(
theid INTEGER PRIMARY KEY AUTOINCREMENT,
aword TEXT UNIQUE ABORT
)
then any attempt to INSERT a duplicated word will fail (the ABORT keyword is optional, as it's the default for UNIQUE) -- isn't that what you want given that you say the words "MUST be unique", i.e., it's an error if they aren't?
The correct answer to your question is: This cannot be done in sqlite. You have to make an additional select query. Quoting the docs for last_insert_rowid:
An INSERT that fails due to a constraint violation is not a successful INSERT and does not change the value returned by this routine. Thus INSERT OR FAIL, INSERT OR IGNORE, INSERT OR ROLLBACK, and INSERT OR ABORT make no changes to the return value of this routine when their insertion fails
Having the same problem in 2022, but since SQLite3 version 3.35.0 (2021-03-12), we have RETURNING.
Combined with UPSERT, it is now possible to achieve this
sqlite> create table test (id INTEGER PRIMARY KEY, text TEXT UNIQUE);
sqlite> insert into test(text) values("a") on conflict do update set id = id returning id;
1
sqlite> insert into test(text) values("a") on conflict do update set id = id returning id;
1
sqlite> insert into test(text) values("b") on conflict do update set id = id returning id;
2
sqlite> insert into test(text) values("b") on conflict do update set id = id returning id;
2
sqlite> select * from test;
1|a
2|b
sqlite> insert into test(text) values("b") on conflict do nothing returning id;
sqlite>
Sadly, this is still a workaround rather than an elegant solution...
On conflict, the insert becomes an update. This means that your update triggers will fire, so you may want to stay away from this!
When the insert is converted into an update, it needs to do something (cf link). However, we don't want to do anything, so we do a no-op by updating id with itself.
Then, returning id gives us the what we want.
Notes:
Our no-op actually does an update, so it costs time, and the trigger on update will fire. But without triggers, it has no effect on the data
Using on conflict do nothing returning id does not fail, but does not return the id.
If usable (again, check your triggers), and if all your tables use the primary key id, then this technique does not need any specialization: just copy/paste on conflict do update set id = id returning id;