I have a user table, for which UPDATE queries are extremely slow to complete. SELECT queries seem fine.
The DB is MariaDB 10.1.48.
CREATE TABLE `user` (
id int(10) UNSIGNED NOT NULL,
active tinyint(1) UNSIGNED NOT NULL DEFAULT '0',
email varchar(100) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
mobile varchar(50) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
isMobile_verified enum('0','1') NOT NULL DEFAULT '0',
device_type enum('0','1','none') NOT NULL DEFAULT 'none' COMMENT '1-Ios,0-Android',
device_token text,
password varchar(255) DEFAULT NULL,
updated datetime NOT NULL,
updated_by int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER TABLE user
ADD PRIMARY KEY (id),
ADD UNIQUE KEY email (email),
ADD UNIQUE KEY mobile (mobile),
ADD KEY active (active);
ALTER TABLE user
MODIFY id int(10) UNSIGNED NOT NULL AUTO_INCREMENT;
Even via PHPMyAdmin or console, a very basic UPDATE query on an indexed column will take 4 to 5 seconds to run. As this is a core table for my application the cumulative effect causes some transactions to run for up to 40 seconds.
There are only c.190K records in the table. Other tables in the DB, including those of similar or larger size, do not seem to be affected.
Is there some issue with the DB version I should upgrade away from, or perhaps a cache problem? Proplem surfaced in the past few months and seems to be getting progressively worse with time. Any help greatly appreciated.
EDIT: Sample UPDATE query, with EXPLAIN output:
EXPLAIN UPDATE user SET isMobile_verified = 0 WHERE id = 1;
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
| select | table | type | keys | key | key_len | ref | rows | Extra |
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
| SIMPLE | user | range | PRIMARY | PRIMARY | 4 | NULL | 1 | Using where |
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
Foreign key constraints, specifically from other DBs, targeting the ID column had built up over time. Refactoring to remove those constraints (and allow for their removal!) resolves issue.
I have prepared a huge list of sql statements (INSERT calls)s in a file which I run from sqlite3 terminal with
.read tmp/commands.sql
When I run the command one or more times in session #1 all goes well with no error and result as expected the ignore prevents against error messages for the duplicates).
Now if I quit sqlite3 with .quit then immediately come back and run the same command .read tmp/commands.sql I get for each sql call in my commands.sql an error message "Error: near line xxx: FOREIGN KEY constraint failed"
My database is made of two tables with a relation
a foreign key is defined in the second table to link to the fist one on its unique ID (FolderID)
in the beginning of the sql script, I added PRAGMA foreign_keys = true; to I thought allow relations
Can you please explain why this is happening in the second session?
Thanks
The first call in the commands.sql file will look like this
PRAGMA foreign_keys = true;
/*
------------------------------------------------------------------
adding data in Folders for '0003_Runs/HiSeq2500/190921_7001450_0495_AH5JJ7BCX3
------------------------------------------------------------------
*/
INSERT OR IGNORE INTO Folders
(
Creator,
CreatorVersion,
DBAddDate,
FolderPath,
FolderName,
FolderSize,
Protection,
DeviceModel,
StartDate,
DeviceID,
RunNr,
FlowCellID,
ProjectNR,
Status,
DeliveryDate,
Comment
)
VALUES (
"AddIlluminaFolders2",
" 1.0; 2020-09-11",
"1600864879",
"0003_Runs/HiSeq2500",
"190921_7001450_0495_AH5JJ7BCX3",
"183173644288",
"0",
"HiSeq2500",
"190921",
"7001450",
"0495",
"AH5JJ7BCX3",
"",
"done",
"",
""
);
/*
------------------------------------------------------------------
adding data in Actions for 0003_Runs/HiSeq2500/190921_7001450_0495_AH5JJ7BCX3
------------------------------------------------------------------
*/
INSERT OR IGNORE INTO Actions
(
FolderID,
Creator,
CreatorVersion,
ActionDate,
ActionName,
Comment
)
VALUES (
last_insert_rowid(),
"AddIlluminaFolders2",
" 1.0; 2020-09-11",
"1600864879",
"AddIlluminaFolders2",
""
);
-- ----------------------------------------------------------------
followed by many other blocks of the same kind until end of file with PRAGMA foreign_keys = true;
Here is my full table definition
/*
Database structure for NCDataMngr and BigData_viewer
Author: Stéphane Plaisance - VIB-Nucleomics Core
database version: 1.0 - 2020-09-11
File Encoding : utf-8
REM: edit below if you change the database schema
=> INSERT INTO "version" (vnum, vdate) VALUES ("1.0", "2020-09-11");
*/
PRAGMA foreign_keys = false;
-- ----------------------------
-- Table structure for Folders
-- ----------------------------
DROP TABLE IF EXISTS "Folders";
CREATE TABLE "Folders" (
"FolderID" INTEGER NOT NULL PRIMARY KEY,
"Creator" TEXT(255,0),
"CreatorVersion" TEXT(255,0),
"DBAddDate" TEXT(255,0),
"FolderPath" TEXT(255,0) NOT NULL,
"FolderName" TEXT(255,0) NOT NULL,
"FolderSize" INTEGER(20,0),
"Protection" INTEGER(1,0),
"DeviceModel" TEXT(255,0),
"StartDate" TEXT(255,0),
"DeviceID" TEXT(255,0),
"RunNr" TEXT(255,0),
"FlowCellID" TEXT(255,0),
"ProjectNR" TEXT(255,0),
"Status" TEXT(255,0),
"DeliveryDate" TEXT(255,0),
"Comment" TEXT(255,0)
);
INSERT INTO "main".sqlite_sequence (name, seq) VALUES ("Folders", '0');
-- ----------------------------
-- Table structure for Actions
-- ----------------------------
DROP TABLE IF EXISTS "Actions";
CREATE TABLE "Actions" (
"FolderID" INTEGER NOT NULL,
"ActionID" INTEGER NOT NULL PRIMARY KEY,
"Creator" TEXT(255,0),
"CreatorVersion" TEXT(255,0),
"ActionDate" TEXT,
"ActionName" TEXT(255,0),
"Comment" TEXT(255,0),
CONSTRAINT "Folders2Actions" FOREIGN KEY ("FolderID") REFERENCES "Folders" ("FolderID") ON DELETE CASCADE ON UPDATE CASCADE
);
INSERT INTO "main".sqlite_sequence (name, seq) VALUES ("Actions", '0');
-- ----------------------------
-- Table structure for version
-- ----------------------------
DROP TABLE IF EXISTS "version";
CREATE TABLE "version" (
"vnum" text,
"vdate" text
);
INSERT INTO "main".sqlite_sequence (name, seq) VALUES ("version", '0');
INSERT INTO "version" (vnum, vdate) VALUES ("1.1", "2020-09-23");
-- ----------------------------
-- View structure for ActionView
-- ----------------------------
DROP VIEW IF EXISTS "ActionView";
CREATE VIEW "ActionView" AS SELECT
Actions.*,
Folders.FolderName
FROM
Actions
INNER JOIN Folders ON Folders.FolderID = Actions.FolderID
ORDER BY
Actions.FolderID ASC,
Actions.ActionID ASC;
-- ----------------------------
-- View structure for FolderView
-- ----------------------------
DROP VIEW IF EXISTS "FolderView";
CREATE VIEW "FolderView" AS SELECT
Folders.*
FROM
Folders
ORDER BY
Folders.FolderID ASC;
-- ----------------------------
-- Indexes structure for table Folders
-- ----------------------------
CREATE UNIQUE INDEX "idx_Folders_FolderPath_FolderName" on Folders ( "FolderPath", "FolderName" );
-- ----------------------------
-- Indexes structure for table Actions
-- ----------------------------
CREATE INDEX "idx_Actions_FolderID" ON Actions ("FolderID" ASC);
CREATE UNIQUE INDEX "idx_Actions_FolderID_Creator" ON Actions ("FolderID" ASC, "Creator");
PRAGMA foreign_keys = true;
last_insert_rowid(). When commands.sql is run mutliple times in the same session, last_insert_rowid() has the value of the last row inserted. Which will never give a FOREIGN KEY failure. When a new session is invoked, no rows are inserted (because of the "dupes"), last_insert_rowid() has value 0, thus FOREIGN KEY violation on Actions insert.
For some reason, I cannot get FOREIGN KEY to work.
Any INSERT into primary_specs will go through, despite the 'ad_ids' table being empty.
EDIT: for some reason the INSERT will NOT go through when using DB Browser, but my python script waltzes right pass that constraint and is able to save the data...
Main table:
CREATE TABLE "ad_ids" (
"ad_id" INTEGER,
"ad_url" TEXT,
PRIMARY KEY("ad_id")
)
Secondary table:
CREATE TABLE "primary_specs" (
"ad_id" INTEGER,
"version" TEXT,
"year" INTEGER,
PRIMARY KEY("ad_id"),
FOREIGN KEY("ad_id")
REFERENCES ad_ids("ad_id")
)
PRAGMA foreign_key_list(primary_specs);
returns:
id seq table from to on_update on_delete match
0 0 ad_ids ad_id ad_id NO ACTION NO ACTION NONE
PRAGMA foreign_keys
returns 1
Apparently PRAGMA foreign_keys = ON; is applied on connection and not a database.
I am trying to delete a record from table users.
Tried deleteing it with a DELETE-statement (DELETE FROM users WHERE user_id=10" as well as in my DB browser, but I get the above error, specifically it says: "Error deleting record: foreign key mismatch - "games" referencing "groups" (DELETE FROM "main"."users" WHERE rowid IN ('10');)".
Below my schema:
CREATE TABLE users (
user_id INTEGER PRIMARY KEY,
name VARCHAR(255) NOT NULL,
hash VARCHAR(255) NOT NULL,
UNIQUE(name));
CREATE TABLE 'groups' (
'group_name' VARCHAR(255) NOT NULL,
'turn' INTEGER NOT NULL,
'user_id'INTEGER,
FOREIGN KEY ('user_id') REFERENCES 'users'('user_id')
ON UPDATE CASCADE
ON DELETE CASCADE
);
CREATE TABLE games (
game_id INTEGER PRIMARY KEY,
active INTEGER,
turn INTEGER,
group_name VARCHAR(255) NOT NULL,
FOREIGN KEY (turn) REFERENCES groups(turn_id)
ON UPDATE CASCADE
ON DELETE CASCADE,
FOREIGN KEY (group_name) REFERENCES groups(group_name)
ON UPDATE CASCADE
ON DELETE NO ACTION
);
Why is this a problem? user_id is not even a foreign key in 'games'?? Thank you!
The problem in the end was my DB Browser for SQLite (3.11.2) whose GUI allows deletions but they don't actually work. When I tied again in the bash and closed and restarted the DB Browser, the rows were gone. Uninsightful but figured I'd post nonetheless in case anyone else comes across this.
I read all the previous issues with the foreign keys. I did all the possible configurations but I can not enforce it to work.
I can insert to the nodes table even if I don't have any record in the types table. But as I know the foreign key constraint should not allow this to happen.
CREATE TABLE nodes(
id int NOT NULL PRIMARY KEY ,
ver int NOT NULL,
lock int NOT NULL,
title varchar(50) NOT NULL,
typeid int NOT NULL REFERENCES types(id),
desc text NOT NULL,
CHECK(trim(id) <> '' AND trim(ver) <> '' AND trim(lock) <>'' AND trim(title)
<> '' AND trim(typeid) <> '' AND trim(desc) <> '' )
)
and
CREATE TABLE "types"
("id" INTEGER PRIMARY KEY NOT NULL ,
"name" TEXT NOT NULL )
SQLite 3.19.3
sqlite> PRAGMA foreign_keys;
foreign_keys
------------
1
sqlite> PRAGMA foreign_keys_list;
sqlite>
I don't know why PRAGMA foreign_keys_list answer is empty.
It is solved because I used s in my command
Correct : PRAGMA foreign_key_list(nodes)
sqlite> PRAGMA foreign_key_list(nodes);
id seq table from to on_update on_delete match
---------- ---------- ---------- ---------- ---------- ---------- -------- ------------
0 0 types typeid id NO ACTION NO ACTION NONE
The syntax is PRAGMA foreign_key_list(table-name);
Remove that s after key — SQlite won't warn you if you made a typo and simply returns nothing.