As posted on another forum, when upgrading XWiki from v7.0.1 to v13.10.9, a non-critical database table xwikistatsvisit, for the user visiting statistics, was preventing the after-upgrade migrations. It contained over seven million records and sized 3GB in total. As a workaround, we had to delete all records in the table, but the SQL command delete from table xwikistatsvisit took over two hours.
I have verified from ER diagram that the table is stand-alone without any foreign key referring to or from other tables. And the database is MariaDB v10.9.2 installed on the same host.
The host under test is a medium virtual machine with SSD, 4 CPUs of Intel i9 and 8GB of RAM, running MariaDB v10.9.2. Also, the hypervisor needs to enable “PAE/NX” and “Nested VT-x/ADM-V” for higher performance; otherwise, the computing task will get stuck forever.
My Questions:
Why the SQL command took so long? Is there any way to proceed faster? E.g. disabling the keys and restrictions, etc.; but I am unfamiliar with this area.
I will highly appreciate any hints or suggestions.
The definition of the table xwikistatsvisit:
--
-- Table structure for table `xwikistatsvisit`
--
DROP TABLE IF EXISTS `xwikistatsvisit`;
/*!40101 SET #saved_cs_client = ##character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `xwikistatsvisit` (
`XWV_ID` bigint(20) NOT NULL,
`XWV_NUMBER` int(11) DEFAULT NULL,
`XWV_NAME` varchar(255) NOT NULL,
`XWV_CLASSNAME` varchar(255) DEFAULT NULL,
`XWV_IP` varchar(255) NOT NULL,
`XWV_USER_AGENT` longtext NOT NULL,
`XWV_COOKIE` longtext NOT NULL,
`XWV_UNIQUE_ID` varchar(255) NOT NULL,
`XWV_PAGE_VIEWS` int(11) DEFAULT NULL,
`XWV_PAGE_SAVES` int(11) DEFAULT NULL,
`XWV_DOWNLOADS` int(11) DEFAULT NULL,
`XWV_START_DATE` datetime DEFAULT NULL,
`XWV_END_DATE` datetime DEFAULT NULL,
PRIMARY KEY (`XWV_ID`),
KEY `XWVS_END_DATE` (`XWV_END_DATE`),
KEY `XWVS_UNIQUE_ID` (`XWV_UNIQUE_ID`),
KEY `XWVS_PAGE_VIEWS` (`XWV_PAGE_VIEWS`),
KEY `XWVS_START_DATE` (`XWV_START_DATE`),
KEY `XWVS_NAME` (`XWV_NAME`),
KEY `XWVS_PAGE_SAVES` (`XWV_PAGE_SAVES`),
KEY `XWVS_DOWNLOADS` (`XWV_DOWNLOADS`),
KEY `XWVS_IP` (`XWV_IP`),
KEY `xwv_user_agent` (`XWV_USER_AGENT`(255)),
KEY `xwv_classname` (`XWV_CLASSNAME`),
KEY `xwv_number` (`XWV_NUMBER`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = #saved_cs_client */;
The DELETE command deletes all the matching rows one at a time. If you want to delete all the records from such a big table, TRUNCATE should be a lot faster, since it empties the full table at once.
TRUNCATE TABLE xwikistatsvisit;
Difference between DELETE and TRUNCATE.
Related
I have a user table, for which UPDATE queries are extremely slow to complete. SELECT queries seem fine.
The DB is MariaDB 10.1.48.
CREATE TABLE `user` (
id int(10) UNSIGNED NOT NULL,
active tinyint(1) UNSIGNED NOT NULL DEFAULT '0',
email varchar(100) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
mobile varchar(50) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
isMobile_verified enum('0','1') NOT NULL DEFAULT '0',
device_type enum('0','1','none') NOT NULL DEFAULT 'none' COMMENT '1-Ios,0-Android',
device_token text,
password varchar(255) DEFAULT NULL,
updated datetime NOT NULL,
updated_by int(10) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
ALTER TABLE user
ADD PRIMARY KEY (id),
ADD UNIQUE KEY email (email),
ADD UNIQUE KEY mobile (mobile),
ADD KEY active (active);
ALTER TABLE user
MODIFY id int(10) UNSIGNED NOT NULL AUTO_INCREMENT;
Even via PHPMyAdmin or console, a very basic UPDATE query on an indexed column will take 4 to 5 seconds to run. As this is a core table for my application the cumulative effect causes some transactions to run for up to 40 seconds.
There are only c.190K records in the table. Other tables in the DB, including those of similar or larger size, do not seem to be affected.
Is there some issue with the DB version I should upgrade away from, or perhaps a cache problem? Proplem surfaced in the past few months and seems to be getting progressively worse with time. Any help greatly appreciated.
EDIT: Sample UPDATE query, with EXPLAIN output:
EXPLAIN UPDATE user SET isMobile_verified = 0 WHERE id = 1;
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
| select | table | type | keys | key | key_len | ref | rows | Extra |
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
| SIMPLE | user | range | PRIMARY | PRIMARY | 4 | NULL | 1 | Using where |
+---------+--------+--------+----------+----------+----------+-------+-------+-------------+
Foreign key constraints, specifically from other DBs, targeting the ID column had built up over time. Refactoring to remove those constraints (and allow for their removal!) resolves issue.
Mariadb 10.3
MariaDB [demo]> show create table people;
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| people | CREATE TABLE `people` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`address_id` int(11) DEFAULT NULL,
`civility_id` int(11) DEFAULT NULL,
`marital_status_id` int(11) DEFAULT NULL,
`language_id` int(11) DEFAULT NULL,
`country_id` int(11) DEFAULT NULL,
`import_timestamp` int(11) DEFAULT NULL WITHOUT SYSTEM VERSIONING,
PRIMARY KEY (`id`),
FULLTEXT KEY `indexPeopleMultiWords` (`lastname`,`lastname_of_birth`,`firstname`)
) ENGINE=InnoDB AUTO_INCREMENT=5818 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci WITH SYSTEM VERSIONING
PARTITION BY SYSTEM_TIME
(PARTITION `p_hist` HISTORY ENGINE = InnoDB,
PARTITION `p_cur` CURRENT ENGINE = InnoDB) |
+--------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.001 sec)
But when I try to drop SYSTEM VERSIONING, I got this message error :
MariaDB [nemo]> ALTER TABLE people DROP SYSTEM VERSIONING;
ERROR 4131 (HY000): Can not DROP SYSTEM VERSIONING for table `people` partitioned BY SYSTEM_TIME
Question:
How to drop system versioning in my case ? Thank you
ref: dbfiddle
PARITITION BY SYSTEM TIME is a system versioned construct.
Per the error message you received, you need to remove the partitioning in order to removing the system versioning:
ALTER TABLE people REMOVE PARTITIONING
ALTER TABLE people DROP SYSTEM VERSIONING
I have table my_transitions and I am trying to delete field called myStage.
I am using this command:
alter table `my_transitions` drop `myStage`
I am seeing this error:
Key column 'myStage' doesn't exist in table
But when I list all fields using this command:
describe my_transitions
I can see
id bigint(20) unsigned NO PRI NULL auto_increment
myStage varchar(255) NO MUL NULL
updated_at timestamp YES NULL
Anyone can see if I am doing something wrong?
EDIT:
If I run show create table my_transitions;, I am getting:
CREATE TABLE `my_transitions` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`myStage` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`myStage1` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `unique_stage_combination` (`myStage`,`myStage1`),
KEY `my_transitions_myStage` (`myStage`),
KEY `my_transitions_myStage1` (`myStage1`)
) ENGINE=InnoDB AUTO_INCREMENT=26 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
I have solved it by first deleting the unique key
ALTER TABLE my_transitions DROP INDEX unique_stage_combination;
It seems like it is not possible to delete a column if it is a part of index key in Maria DB 10.5.8.
This is a pecular bug in MariaDB. It affects MariaDB 10.5.
Demo: https://dbfiddle.uk/?rdbms=mariadb_10.5&fiddle=867204670347fa29e40bd5eb510c6956
The workaround is to drop the UNIQUE KEY that column mystage is part of first, then drop the column.
alter table my_transitions drop key unique_stage_combination, drop column mystage;
P.S.: I tested this on MySQL 8.0 and it does not require the workaround. It does drop the column, but it leaves the table with a UNIQUE KEY column on just one column mystage1, which might not be what you want.
I am trying to delete a record from table users.
Tried deleteing it with a DELETE-statement (DELETE FROM users WHERE user_id=10" as well as in my DB browser, but I get the above error, specifically it says: "Error deleting record: foreign key mismatch - "games" referencing "groups" (DELETE FROM "main"."users" WHERE rowid IN ('10');)".
Below my schema:
CREATE TABLE users (
user_id INTEGER PRIMARY KEY,
name VARCHAR(255) NOT NULL,
hash VARCHAR(255) NOT NULL,
UNIQUE(name));
CREATE TABLE 'groups' (
'group_name' VARCHAR(255) NOT NULL,
'turn' INTEGER NOT NULL,
'user_id'INTEGER,
FOREIGN KEY ('user_id') REFERENCES 'users'('user_id')
ON UPDATE CASCADE
ON DELETE CASCADE
);
CREATE TABLE games (
game_id INTEGER PRIMARY KEY,
active INTEGER,
turn INTEGER,
group_name VARCHAR(255) NOT NULL,
FOREIGN KEY (turn) REFERENCES groups(turn_id)
ON UPDATE CASCADE
ON DELETE CASCADE,
FOREIGN KEY (group_name) REFERENCES groups(group_name)
ON UPDATE CASCADE
ON DELETE NO ACTION
);
Why is this a problem? user_id is not even a foreign key in 'games'?? Thank you!
The problem in the end was my DB Browser for SQLite (3.11.2) whose GUI allows deletions but they don't actually work. When I tied again in the bash and closed and restarted the DB Browser, the rows were gone. Uninsightful but figured I'd post nonetheless in case anyone else comes across this.
I am having two tables tbluserlogindetail and tblRoles.
tbluserlogindetail is as follows
CREATE TABLE `tbluserlogindetail` (
`LoginID` varchar(45) NOT NULL,
`Name` varchar(45) DEFAULT NULL,
`Password` varchar(45) DEFAULT NULL,
PRIMARY KEY (`LoginID`),
UNIQUE KEY `LoginID_UNIQUE` (`LoginID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1$$
EDIT doratesting.tbluserlogindetail;
and my second table tblRoles is as follows
CREATE TABLE `tblroles` (
`RoleID` int(11) NOT NULL,
`LoginID` varchar(45) NOT NULL,
PRIMARY KEY (`RoleID`,`LoginID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1$$
I tried the the following to assign a primary key for the column LoginID in tblroles table but i don't know where i went wrong can any one help me.
I have gone through the documentation but unable to rectify the error so please help me
ALTER TABLE tblroles
ADD FOREIGN KEY (loginid)
REFERENCES tbluserlogindetail(loginid)
The referenced columns must be unique in the referenced table. Try one of these options:
The primary key on tbluserlogindetail is (ID, loginid) so you could use that as your foreign key instead of (loginid). This will require you to add a column tbluserlogindetail_ID to tblroles.
Try adding a unique index to the LoginID column of the tbluserlogindetail table. This is only possible if it is actually unique.
Also, why is your primary key on tbluserlogindetail defined as PRIMARY KEY (ID, LoginID)? The ID field is an auto-increment field and is already unique. So why do you also include the LoginID as part of the primary key? I think you need to go back to your table design and rethink which columns to choose as your primary keys.
this is just guess/ assumption ,i like to share here,
PRIMARY KEY (ID,LoginID)
in tbluserlogindetail are considered as surrogate key,
when we execute the child table,That is tblroles,
child table expected primary key in parent table,
but actually we created the surrogate key, due to this reason, i alter query failed,
ALTER TABLE tblroles ADD FOREIGN KEY (LoginID) REFERENCES tbluserlogindetail(LoginID)
This is my assumption, give feedback for my answer,
I did following changes to execute the Alter table command:
1.tblroles create with given Create query command, after created i manually deleted the
LoginID primary key in tblroles table,
Changed varchar to int in LoginID,
in tbluserlogindetail, deleted the ID AUTO_INCREMENT,Pk.
Check i updated ALTER query