Insert VALUES not already in table - mariadb

I want to create a table and then initialize it with some values, in as concise manner as possible.
However, this script gets executed every time my app starts, so the insert should happen only on items that were not already added previously.
I do not want to use IGNORE directive in 'INSERT IGNORE INTO', because I do not want to ignore unexpected errors.
For some reason, INSERT INTO fails with "SQL error (1136): Column count doesn't match value count at row 1", even though the select that follows gives the values that need to be added.
Here's the failing code:
START TRANSACTION;
CREATE TABLE IF NOT EXISTS `privileges` (
`id` TINYINT NOT NULL AUTO_INCREMENT,
`label` VARCHAR(25) UNIQUE,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `privileges` (`label`)
SELECT `label` FROM (
SELECT NULL AS `label`
UNION VALUES
('item1'),
('item2')
) X
WHERE `label` IS NOT NULL
AND `label` NOT IN (SELECT `label` FROM `privileges`)
COMMIT;
Currently I am solving this by first inserting the values into a temporary table, and then performing a select on that. But why isn't the above working and is there a more concise way to do what I'm trying to do?
I'm using MariaDB 10.3.9, added missing UNIQUE constraint
Edit 2: Thanks to LukStorms for figuring out the error was related to AUTO_INCREMENT, it seems passing NULL for AUTO_INCREMENT column solves the problem like so:
INSERT INTO `privileges` (id, label)
WITH ITEMS(label) AS (VALUES
('users:read'),('users:create'),
('clients:read'),('clients:write'),
('catalog:read'),('catalog:write'),
('cart:read'),('cart:write'),
('orders:read'),('orders:write'), ('test1')
) SELECT NULL, label FROM ITEMS i
WHERE label NOT IN (SELECT label FROM `privileges`);

In MariaDb 10.3+, using a CTE with a the VALUES expression can let you assign a column name to it.
with ITEMS(label) as
(VALUES
('item1')
,('item2'))
select i.label
from ITEMS i
where not exists (select 1 from privileges p where p.label = i.label)
But somehow it gives an error when inserting into a table that has a field with an AUTO_INCREMENT. Seems like a bug to me.
However, when you insert a NULL into a an AUTO_INCREMENT field then the NULL gets ignored. But you discovered that behaviour yourself.
So this works:
INSERT INTO privileges (id, label)
WITH ITEMS(label) as (
VALUES ('item1'), ('item2')
)
SELECT null, i.label
FROM ITEMS i
WHERE NOT EXISTS (SELECT 1 FROM privileges p WHERE p.label = i.label);
Test on db<>fiddle here
Using unioned selects also works though.
INSERT INTO privileges (label)
SELECT label
FROM (
SELECT 'item1' as label UNION ALL
SELECT 'item2'
) i
WHERE NOT EXISTS (SELECT 1 FROM privileges p WHERE p.label = i.label);
db<>fiddle here
Maybe another way is to use a temporary table (that will vanish when the session expires)
CREATE TEMPORARY TABLE tmp_items (label VARCHAR(25) NOT NULL PRIMARY KEY);
INSERT INTO tmp_items (label) VALUES
('item1')
,('item2');
INSERT INTO privileges (label)
SELECT label
FROM tmp_items i
WHERE label NOT IN (SELECT DISTINCT label FROM privileges);
Test on db<>fiddle here

First, your application is trying to double-insert values. It probably shouldn't be doing that (though I can think of a few valid use cases). Consider making it so that it does not try to add data that it's already added before. If you don't have easy access to inter-instance state, pull the current list out of the database on startup before deciding what to insert.
Second, if you want labels to be unique, why is there not a unique key on the label field? At the moment, INSERT IGNORE wouldn't even work because there is nothing in your schema preventing duplicate label values. I would ask yourself why you need an auto-incrementing ID: why not just have the label, and make it the primary key?
Then, if you still need to do this duplicate-elision at the SQL layer, you may use ON DUPLICATE KEY to suck up redundant inserts of an existing primary key:
INSERT INTO `privileges` (`label`)
VALUES
('item1'),
('item2')
)
ON DUPLICATE KEY UPDATE `label` = `label`
This solution is difficult to implement with your auto-increment ID key, because your application probably doesn't know what the ID is going to be. Another reason to consider dropping it.
Unfortunately, there's no ON DUPLICATE KEY IGNORE.
If you want to keep the ID key, and you don't want your application to do a read step on startup (perhaps for scalability reasons), then INSERT IGNORE to be quite honest is your best bet, though you're still going to need at least a unique key on label to make that work.

Related

SQLite Copy Table Content from one Table to another Table and Update

I have two SQLite files, each of them has one table and the same table design. One Column is set as Primary Key. I want to copy all data from ItemsB into ItemsA. All data should be updated. The ItemsB Table is the newer one.
I've tried:
ATTACH DATABASE ItemsB AS ItemsB;
INSERT INTO ItemsA.PMItem (ItemID,VarID,Name1) SELECT ItemID,VarID,Name1 FROM ItemsB.PMItem;
Obviously this can't work due the Primary Key (which is the column VarID).
Then I tried it with ON CONFLICT:
ON CONFLICT (VarID) DO UPDATE SET Name1=excluded.Name1
But this won't work either.
Example Table:
CREATE TABLE PMItem (
ItemID INTEGER,
VarID INTEGER PRIMARY KEY,
Name1 TEXT
);
You need a WHERE clause with an always true condition, to overcome the ambiguity that is raised when ON CONFLICT is used after a SELECT statement:
INSERT INTO PMItem (ItemID,VarID,Name1)
SELECT ItemID,VarID,Name1
FROM ItemsB.PMItem
WHERE 1
ON CONFLICT(VarID) DO UPDATE
SET Name1 = EXCLUDED.Name1;

Insert or ignore a record

I have a table
CREATE TABLE "myTable" (
"id" INTEGER NOT NULL,
"name" VARCHAR,
PRIMARY KEY ("id")
)
and let's say it has 1 record
1 - James
I want to insert a new record. If it doesn't exist, insert it. If it does, do nothing.
I am not sure if the below query, is the right way to do this:
INSERT or IGNORE INTO myTable(id, name) VALUES(1, "Tom");
I tried it and I didn't get any error..
Your statement is fine for what you want to do. However, you don't need to supply a value for id in SQLite. You can just do:
INSERT INTO myTable(name)
VALUES('Tom');
This will auto-increment the id, so you don't have to worry about duplicates.
You used INSERT IGNORE, and the row won't actually be inserted because it results in a duplicate key. Your id column is a PRIMARY KEY, and you already have the value 1 stored in the database.
The statement won't generate an error but it will generate a warning.

SQLite Insert or Ignore/Replace with lastInsertID in one statement

I feel like this should be easy...
I have a table like this:
CREATE TABLE table_name (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
UNIQUE (name COLLATE NOCASE)
)
No two names are the same, case insensitive. Right now I have users adding names and it does this:
INSERT INTO table_name (name) VALUES ("my name");
And I need to get the id of the row, which is easy with PHP PDO's lastInsertID(). But I also want, if the user is adding a name that's already in the database, for nothing to be added to the database, but still get that id without having to do another database call. I was hoping for something like
INSERT OR REPLACE INTO table_name (name) VALUES ("my name");
And have it just overwrite the same data into the cell and return the lastInsertID (even though it wasn't inserted?). But that doesn't work. What are my other options? Will I have to do a separate database query to see if the name field already exists?
With the OR REPLACE clause, the statement always deletes any old row.
Just use two statements. (There is no technical reason for doing this in a single statement.)

SQLlite: strange "Abort due to constraint violation"

I'm trying to rename a column of a table. I have a lot of tables with the word "couleur" and I renamed "manually" to "bulle".
I've successfully renamed main_groupecouleurs to main_groupebulles. Now i'm working on main_groupe. I'm trying to rename groupe_couleurs_id to groupe_bulles_id
The SQL is quite self-explaining:
BEGIN TRANSACTION;
DROP INDEX main_groupe_fc5cee5b;
CREATE TABLE main_groupe7e12
(
id INTEGER PRIMARY KEY NOT NULL,
description TEXT NOT NULL,
exemple TEXT,
groupe_bulles_id INTEGER DEFAULT NULL,
reference TEXT,
FOREIGN KEY (groupe_bulles_id) REFERENCES main_groupebulles(id)
DEFERRABLE INITIALLY DEFERRED
);
CREATE UNIQUE INDEX main_groupe_fc5cee5b ON main_groupe7e12 (groupe_bulles_id);
INSERT INTO main_groupe7e12(id, description, exemple, groupe_bulles_id, reference)
SELECT id, description, exemple, groupe_couleurs_id, reference
FROM main_groupe;
DROP TABLE main_groupe;
ALTER TABLE main_groupe7e12 RENAME TO main_groupe;
COMMIT;
When I run it, I get:
[SQLITE_CONSTRAINT] Abort due to constraint violation
(UNIQUE constraint failed: main_groupe7e12.groupe_bulles_id)
This means (I think I'm wrong here but I dont know what I'm missing) that it tries to insert some groupe_couleurs_id that are not in the referring table (= main_groupebulles). Thus I tried to see in the original table the problem:
SELECT * FROM main_groupe WHERE groupe_couleurs_id NOT IN (
SELECT id FROM main_groupebulles
);
I got no rows! What am I missing?
You have an UNIQUE index on your groupe_bulles_id column but based on the comments, there are a lot of valid duplicate values for that column coming from main_groupe.groupe_couleus_id and that causes the constraint violation.
Since having duplicate values is what you want, remove the UNIQUE from the CREATE UNIQUE INDEX ....

SQLITE - INSERT or UPDATE without changing ROWID value

I need to update a table row IF EXISTS, otherwise INSERT a new row.
I tried:
INSERT OR REPLACE INTO table VALUES ...
but if the row row exist this statement changes the row's ROWID, and that's what I'm trying to avoid (I need the rowid :D)
I also tried to find a way to get some sort of return value from the update, in the case where an update has taken place, but I still don't understand how... If I could get the return value from the update statement, I could choose wether to proceed with an insert or not.
Do you have any suggestion or solution to this problem? Or do I need to make a copy of the ROWID and use that instead of the "pure" table ROWID?
Thanks in advance, best regards
ps: I was looking HERE and I was wondering if sqlite has the OUTPUT special word too, but google didn't help me..
---- EDIT after reading comments:
table schema example
CREATE TABLE test (
table_id TEXT NOT NULL,
some_field TEXT NOT NULL,
PRIMARY KEY(table_id)
)
INSERT or REPLACE INTO test (table_id, some_field) VALUES ("foo","bar")
I tested Chris suggestion but the rowid still gets changed. I think the best alternative is to do a SELECT to see if a row with that key already exist. If so, UPDATE, otherwise, INSERT... good old fashion but guaranteed to work.
Combine it with select, like this
INSERT or REPLACE INTO test (ROWID, table_id, some_field)
VALUES ((SELECT ROWID from test WHERE table_id = 'foo' UNION SELECT max(ROWID) + 1 from test limit 1), 'foo','bar')
You need to specify that your table_id is unique in addition to being the primary key:
sqlite> CREATE TABLE test (
table_id TEXT NOT NULL,
some_field TEXT NOT NULL,
PRIMARY KEY(table_id),
UNIQUE(table_id)
);
sqlite> insert or replace into test values("xyz", "other");
sqlite> select * FROM test;
xyz|other
sqlite> insert or replace into test values("abc", "something");
sqlite> insert or replace into test values("xyz", "whatever");
sqlite> select * FROM test;
abc|something
xyz|whatever
From version 3.24.0 (2018-06-04), SQLite now supports an UPSERT clause that will do exactly what the OP needed: https://www.sqlite.org/lang_UPSERT.html
The insert would now look like this:
INSERT INTO test (table_id, some_field) VALUES ("foo","baz")
ON CONFLICT(table_id) DO UPDATE SET some_field=excluded.some_field;

Resources