MariaDB 10.6.11
I am having trouble with a query, its behavior depends on the select, in the first case the query takes 1min 30s to complete, in the second case it only takes 3s to complete.
Tested on MySql without this behavior the query takes 3s to complete.
The only difference between these requests is the selection of one field.
Selecting the field test_town.id in the first case in the my_opened_tickets with clause make the use of a LATERAL DERIVED in the explain plan and the query takes 1min 30s to complete.
Selecting the field test_opened_ticket.test_town_id in the second case in the my_opened_tickets with clause make the use of a DERIVED in the explain plan and the query takes 3s to complete.
I can disabled the LATERAL DERIVED using this :
set optimizer_switch='split_materialized=off'
But I don't think this is a good way to get rid of this problem. I just want to know if this is a normal behavior or a bug, maybe my request is not good I don't know.
The table test_country has 300 entries
The table test_town has 20 000 entries
The table test_ticket has 30 857 690 entries
The table test_opened_ticket has 6 171 538 entries
Here is the first query 1min 30s :
with my_opened_tickets as(
select
test_town.id as id,
test_opened_ticket.nb2,
test_opened_ticket.nb1
from
test_town ,
test_ticket ,
test_opened_ticket
where
test_opened_ticket.id = test_ticket.id
and test_opened_ticket.test_country_id = 186
and test_opened_ticket.test_town_id = test_town.id
),
max_nb2 as(
select
id,
max(nb2) nb2
from
my_opened_tickets
group by id
),
max_by_id_nb2 as (
select
max_nb2.id,
max_nb2.nb2,
max(my_opened_tickets.nb1)
from
my_opened_tickets ,
max_nb2
where
my_opened_tickets.id = max_nb2.id
and max_nb2.nb2 = my_opened_tickets.nb2
group by max_nb2.id,max_nb2.nb2
)
select * from max_by_id_nb2;
Here is the explain plan :
id|select_type |table |type |possible_keys |key |key_len|ref |rows |Extra |
--+---------------+------------------+------+----------------------------------------------------------------------------------------------------------+-----------------------------------+-------+----------------------------------------+-----+---------------------------------------------------------+
1|PRIMARY |<derived5> |ALL | | | | |40802| |
5|DERIVED |test_town |index |PRIMARY,test_town_id_IDX |PRIMARY |4 | |20401|Using index; Using temporary; Using filesort |
5|DERIVED |test_opened_ticket|ref |PRIMARY,fk_test_opened_ticket_town1,fk_test_opened_ticket_country1_idx,test_opened_ticket_test_town_id_IDX|test_opened_ticket_test_town_id_IDX|10 |test_mlp.test_town.id,const |1 | |
5|DERIVED |test_ticket |eq_ref|PRIMARY,test_ticket_id_IDX |PRIMARY |4 |test_mlp.test_opened_ticket.id |1 |Using index |
5|DERIVED |<derived4> |ref |key0 |key0 |4 |test_mlp.test_town.id |2 |Using where |
4|LATERAL DERIVED|test_town |eq_ref|PRIMARY,test_town_id_IDX |PRIMARY |4 |test_mlp.test_opened_ticket.test_town_id|1 |Using where; Using index; Using temporary; Using filesort|
4|LATERAL DERIVED|test_opened_ticket|ref |PRIMARY,fk_test_opened_ticket_town1,fk_test_opened_ticket_country1_idx,test_opened_ticket_test_town_id_IDX|test_opened_ticket_test_town_id_IDX|10 |test_mlp.test_town.id,const |1 | |
4|LATERAL DERIVED|test_ticket |eq_ref|PRIMARY,test_ticket_id_IDX |PRIMARY |4 |test_mlp.test_opened_ticket.id |1 |Using index |
Here is the second query 3/4s :
with my_opened_tickets as(
select
test_opened_ticket.test_town_id as id,
test_opened_ticket.nb2,
test_opened_ticket.nb1
from
test_town ,
test_ticket ,
test_opened_ticket
where
test_opened_ticket.id = test_ticket.id
and test_opened_ticket.test_country_id = 186
and test_opened_ticket.test_town_id = test_town.id
),
max_nb2 as(
select
id,
max(nb2) nb2
from
my_opened_tickets
group by id
),
max_by_id_nb2 as (
select
max_nb2.id,
max_nb2.nb2,
max(my_opened_tickets.nb1)
from
my_opened_tickets ,
max_nb2
where
my_opened_tickets.id = max_nb2.id
and max_nb2.nb2 = my_opened_tickets.nb2
group by max_nb2.id,max_nb2.nb2
)
select * from max_by_id_nb2;
And here is the execution plan :
id|select_type|table |type |possible_keys |key |key_len|ref |rows |Extra |
--+-----------+------------------+------+----------------------------------------------------------------------------------------------------------+-----------------------------------+-------+------------------------------+-----+--------------------------------------------+
1|PRIMARY |<derived5> |ALL | | | | |20401| |
5|DERIVED |<derived4> |ALL | | | | |20401|Using where; Using temporary; Using filesort|
5|DERIVED |test_town |eq_ref|PRIMARY,test_town_id_IDX |PRIMARY |4 |max_nb2.id |1 |Using index |
5|DERIVED |test_opened_ticket|ref |PRIMARY,fk_test_opened_ticket_town1,fk_test_opened_ticket_country1_idx,test_opened_ticket_test_town_id_IDX|test_opened_ticket_test_town_id_IDX|10 |max_nb2.id,const |1 |Using where |
5|DERIVED |test_ticket |eq_ref|PRIMARY,test_ticket_id_IDX |PRIMARY |4 |test_mlp.test_opened_ticket.id|1 |Using index |
4|DERIVED |test_town |index |PRIMARY,test_town_id_IDX |PRIMARY |4 | |20401|Using index |
4|DERIVED |test_opened_ticket|ref |PRIMARY,fk_test_opened_ticket_town1,fk_test_opened_ticket_country1_idx,test_opened_ticket_test_town_id_IDX|test_opened_ticket_test_town_id_IDX|10 |test_mlp.test_town.id,const |1 | |
4|DERIVED |test_ticket |eq_ref|PRIMARY,test_ticket_id_IDX |PRIMARY |4 |test_mlp.test_opened_ticket.id|1 |Using index |
Here is a script which creates tables indexes and fill tables to reproduce
https://filetransfer.io/data-package/9kWfCYo6#link
Or here is the script
DROP TABLE IF EXISTS test_opened_ticket;
DROP TABLE IF EXISTS test_ticket;
DROP TABLE IF EXISTS test_town;
DROP TABLE IF EXISTS test_country;
CREATE TABLE `test_country` (
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`)
);
CREATE TABLE `test_town` (
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`)
);
CREATE TABLE `test_ticket` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`test_town_id` int(11) DEFAULT NULL,
`test_country_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `fk_test_ticket_town1` (`test_town_id`),
KEY `fk_test_ticket_country1_idx` (`test_country_id`),
CONSTRAINT `fk_test_ticket_country1_idx` FOREIGN KEY (`test_country_id`) REFERENCES `test_country` (`id`),
CONSTRAINT `fk_test_ticket_town1` FOREIGN KEY (`test_town_id`) REFERENCES `test_town` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
);
CREATE TABLE `test_opened_ticket` (
`id` int(11) NOT NULL,
`test_town_id` int(11) DEFAULT NULL,
`test_country_id` int(11) DEFAULT NULL,
`nb1` int(11) DEFAULT NULL,
`nb2` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `fk_test_opened_ticket_town1` (`test_town_id`),
KEY `fk_test_opened_ticket_country1_idx` (`test_country_id`),
CONSTRAINT `fk_test_opened_ticket_ticket1_idx` FOREIGN KEY (`id`) REFERENCES `test_ticket` (`id`),
CONSTRAINT `fk_test_opened_ticket_country1_idx` FOREIGN KEY (`test_country_id`) REFERENCES `test_country` (`id`),
CONSTRAINT `fk_test_opened_ticket_town1` FOREIGN KEY (`test_town_id`) REFERENCES `test_town` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
);
insert into test_country(id)
SELECT seq FROM seq_1_to_300;
insert into test_town(id)
SELECT seq FROM seq_1_to_20000;
insert into test_ticket(id,test_country_id,test_town_id)
SELECT seq,RAND()*258+1,RAND()*19999+1 FROM seq_1_to_27000000;
insert into test_ticket(test_country_id,test_town_id)
SELECT 186,RAND()*19999+1 FROM seq_1_to_3857690;
insert into test_opened_ticket()
select id, test_town_id,test_country_id ,RAND()*300,RAND()*300 from test_ticket where id % 5 = 0 and test_country_id != 186;
insert into test_opened_ticket()
select id, test_town_id,test_country_id ,RAND()*300,RAND()*300 from test_ticket where id % 5 = 0 and test_country_id = 186;
CREATE INDEX test_opened_ticket_test_town_id_IDX USING BTREE ON test_opened_ticket (test_town_id,test_country_id);
CREATE INDEX test_ticket_id_IDX USING BTREE ON test_ticket (id);
CREATE INDEX test_town_id_IDX USING BTREE ON test_town (id);
CREATE INDEX test_country_id_IDX USING BTREE ON test_country (id);
#slower
with my_opened_tickets as(
select
test_town.id as id,
test_opened_ticket.nb2,
test_opened_ticket.nb1
from
test_town ,
test_ticket ,
test_opened_ticket
where
test_opened_ticket.id = test_ticket.id
and test_opened_ticket.test_country_id = 186
and test_opened_ticket.test_town_id = test_town.id
),
max_nb2 as(
select
id,
max(nb2) nb2
from
my_opened_tickets
group by
id ),
max_by_id_nb2 as (
select
max_nb2.id,
max_nb2.nb2,
max(my_opened_tickets.nb1)
from
my_opened_tickets ,
max_nb2
where
my_opened_tickets.id = max_nb2.id
and max_nb2.nb2 = my_opened_tickets.nb2
group by
max_nb2.id,max_nb2.nb2)
select * from max_by_id_nb2;
#faster
with my_opened_tickets as(
select
test_opened_ticket.test_town_id as id,
test_opened_ticket.nb2,
test_opened_ticket.nb1
from
test_town ,
test_ticket ,
test_opened_ticket
where
test_opened_ticket.id = test_ticket.id
and test_opened_ticket.test_country_id = 186
and test_opened_ticket.test_town_id = test_town.id
),
max_nb2 as(
select
id,
max(nb2) nb2
from
my_opened_tickets
group by
id ),
max_by_id_nb2 as (
select
max_nb2.id,
max_nb2.nb2,
max(my_opened_tickets.nb1)
from
my_opened_tickets ,
max_nb2
where
my_opened_tickets.id = max_nb2.id
and max_nb2.nb2 = my_opened_tickets.nb2
group by
max_nb2.id,max_nb2.nb2)
select * from max_by_id_nb2;
If anyone read this
https://mariadb.org/mariadb-30x-faster/
ANALYZE TABLE tbl PERSISTENT FOR ALL;
solved my problem
Related
I have this table in mariadb
create or replace table test_table (
col_key int primary key,
col_a int not null,
col_b int not null check(col_b in (0, 1))
);
There is an additional constraint that the pair (col_a, col_b) must be unique only if col_b = 1
For example, you are allowed to have
| col_key | col_a | col_b |
| ------- | ----- | ----- |
| 1 | 1 | 0 |
| 2 | 1 | 0 |
But not
| col_key | col_a | col_b |
| ------- | ----- | ----- |
| 1 | 1 | 1 |
| 2 | 1 | 1 |
I think of 2 approaches:
Since col_b only has 2 values, I can take advantage of the fact that in mariadb null can by pass unique check. So instead of 0 and 1, I change the definition of the table to this and treat null as 0 in application code.
create or replace table test_table (
col_key int primary key,
col_a int not null,
col_b int check(col_b is null or col_b = 1),
unique (col_a, col_b)
);
The upside is I have the database handle the check for me. The downside is the application code become a bit complex and the code tightly couple to mariadb's implementation.
When update or insert, write query like this
update test_table t
set t.col_b = 1
where t.col_key = 2
and not exists (select 1 from test_table t2 where t2.col_a = 1 and t2.col_b = 1)
The upside is the application code actually works with 0 and 1 values. The downside is... I unsure if this work or not. Does the database lock the table when it run the update query? Is there any chance that some other process inserts a row with col_b = 1 after the subquery returns the result?
According to your question, the combination of cola and colb is unique, if colb has the value 1. This condiition can be already defined in the table definition:
CREATE TABLE test_table (
col_key int primary key,
col_a int not null,
col_b int check(col_b is null or col_b = 1),
unique (col_a, col_b)
)
Since the unique index allows multiple NULL values, multiple same combinations of col_a and col_b are allowed, as long the value of col_b is NULL.
For updating you don't need a subquery, since the server checks this condition itself. For not raising an error, just use UPDATE IGNORE
UPDATE IGNORE test_table SET col_b=1 WHERE col_key=2
With the first solution the logic is defined in the database (and doesn't need to be defined in your application) and it is much faster since it can use an index, while the subquery performs a table scan.
MariaDB [test]> insert into test_table select seq,1,NULL from seq_1_to_1000;
Query OK, 1000 rows affected (0,081 sec)
Records: 1000 Duplicates: 0 Warnings: 0
MariaDB [test]> explain update test_table set col_b=1 where col_key=18 and not exists (select 1 from test_table t2 where t2.col_a=1 and t2.col_b=1);
+------+-------------+------------+-------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+------------+-------+---------------+---------+---------+-------+------+-------------+
| 1 | PRIMARY | test_table | const | PRIMARY | PRIMARY | 4 | const | 1 | |
| 2 | SUBQUERY | t2 | ALL | NULL | NULL | NULL | NULL | 1000 | Using where |
+------+-------------+------------+-------+---------------+---------+---------+-------+------+-------------+
2 rows in set (0,003 sec)
MariaDB [test]> alter table test_table add unique (col_a, col_b);
Query OK, 0 rows affected (0,079 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [test]> explain update test_table set col_b=1 where col_key=18;
+------+-------------+------------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+------------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | test_table | range | PRIMARY | PRIMARY | 4 | NULL | 1 | Using where |
+------+-------------+------------+-------+---------------+---------+---------+------+------+-------------+
I'm trying to get ancestors of a child (dog) upto Level 5. For Example in attached picture I'll be sending "Spencer di Casa Massarelli" and in result want to have associated parents (both father and mother). In my DB structure I've used father_id and mother_id.
DB & version: 10.4.11-MariaDB
Table Script:
CREATE TABLE `dogs` (
`dog_id` int(11) NOT NULL,
`name` varchar(255) DEFAULT NULL,
`father_id` int(11) DEFAULT NULL,
`moter_id` int(11) DEFAULT NULL,
PRIMARY KEY (`dog_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
INSERT INTO `dogs` VALUES ('0', null, null, null);
INSERT INTO `dogs` VALUES ('1', 'Father', null, null);
INSERT INTO `dogs` VALUES ('2', 'Mother', null, null);
INSERT INTO `dogs` VALUES ('3', 'Father1', null, null);
INSERT INTO `dogs` VALUES ('4', 'Mother2', null, null);
INSERT INTO `dogs` VALUES ('5', 'Son', '1', '2');
INSERT INTO `dogs` VALUES ('6', 'Daughter', '3', '4');
INSERT INTO `dogs` VALUES ('7', 'GrandSon', '5', '6');
I've tried following self join query but the problem is I'm unable to get right parents i.e., parents(both father and mother) of first parent.
SELECT t1.name AS lev1,
t2.name AS lev2Father,
t3.name AS lev2Mother,
t4.name AS level3Father,
t5.name AS level3Mother,
t6.name AS level4Father,
t7.name AS level4Mother,
t8.name AS level5Father,
t9.name AS level5Mother,
t10.name AS level6Father,
t11.name AS level6Mother
FROM dogs AS t1
LEFT JOIN dogs AS t2 ON t2.dog_id = t1.father_id
LEFT JOIN dogs AS t3 ON t3.dog_id = t1.mother_id
LEFT JOIN dogs AS t4 ON t4.dog_id = t2.father_id
LEFT JOIN dogs AS t5 ON t5.dog_id = t2.mother_id
LEFT JOIN dogs AS t6 ON t6.dog_id = t4.father_id
LEFT JOIN dogs AS t7 ON t7.dog_id = t4.mother_id
LEFT JOIN dogs AS t8 ON t8.dog_id = t6.father_id
LEFT JOIN dogs AS t9 ON t9.dog_id = t6.mother_id
LEFT JOIN dogs AS t10 ON t10.dog_id = t8.father_id
LEFT JOIN dogs AS t11 ON t11.dog_id = t8.mother_id
WHERE t1.dog_id = 7
WITH RECURSIVE
cte AS (
SELECT *, 0 level, ' ' relation
FROM dogs
WHERE dog_id = 7
UNION ALL
SELECT dogs.*, level + 1, 'father'
FROM dogs
JOIN cte ON cte.father_id = dogs.dog_id
WHERE level < 5
UNION ALL
SELECT dogs.*, level + 1, 'mother'
FROM dogs
JOIN cte ON cte.mother_id = dogs.dog_id
WHERE level < 5
)
SELECT *
FROM cte
ORDER BY level, relation;
fiddle
Result
dog_id | name | father_id | mother_id | level | relation
-----: | :------- | --------: | --------: | ----: | :-------
7 | GrandSon | 5 | 6 | 0 |
5 | Son | 1 | 2 | 1 | father
6 | Daughter | 3 | 4 | 1 | mother
1 | Father | null | null | 2 | father
3 | Father1 | null | null | 2 | father
2 | Mother | null | null | 2 | mother
4 | Mother2 | null | null | 2 | mother
I'm trying to perform a very simple query for Firebase events stored in Google BigQuery but I´m not able to find a way to do it.
In the Android app, I´m logging an event like this:
Bundle params = new Bundle();
params.putInt("productID", productId);
params.putInt(FirebaseAnalytics.Param.VALUE, value);
firebaseAnalytics.logEvent("productEvent", params);
So, in BigQuery I have something like this:
___________________ _______________________ ____________________________
| event_dim.name | event_dim.params.key | event_dim.params.int_value |
|___________________|_______________________|____________________________|
| productEvent | productID | 25 |
| |_______________________|____________________________|
| | value | 1253 |
|___________________|_______________________|____________________________|
When I get the data from this table I get two rows:
___________________ _______________________ ____________________________
|event_dim.name | event_dim.params.key | event_dim.params.int_value |
|___________________|_______________________|____________________________|
| productEvent | productID | 25 |
| productEvent | value | 12353 |
But what I really need is a SELECT clause from this table to get the data as below:
___________________ _____________ _________
| name | productID | value |
|___________________|_____________|_________|
| productEvent | 25 | 12353 |
Any idea or suggestion?
You can pivot the values into columns like this
SELECT
event_dim.name as name,
MAX(IF(event_dim.params.key = "productID", event_dim.params.int_value, NULL)) WITHIN RECORD productID,
MAX(IF(event_dim.params.key = "value", event_dim.params.int_value, NULL)) WITHIN RECORD value,
FROM [events]
In case you want to generate this command using SQL, see this solution: Pivot Repeated fields in BigQuery
Using standard SQL (uncheck "Use Legacy SQL" under "Show Options" in the UI), you can express the query as:
SELECT
event_dim.name as name,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "productID") AS productID,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "value") AS value
FROM `dataset.mytable` AS t,
t.event_dim AS event_dim;
Edit: updated example to include int_value as part of value based on the comment below. Here is a self-contained example that demonstrates the approach as well:
WITH T AS (
SELECT ARRAY_AGG(event_dim) AS event_dim
FROM (
SELECT STRUCT(
"foo" AS name,
ARRAY<STRUCT<key STRING, value STRUCT<int_value INT64, string_value STRING>>>[
("productID", (10, NULL)), ("value", (5, NULL))
] AS params) AS event_dim
UNION ALL
SELECT STRUCT(
"bar" AS name,
ARRAY<STRUCT<key STRING, value STRUCT<int_value INT64, string_value STRING>>>[
("productID", (13, NULL)), ("value", (42, NULL))
] AS params) AS event_dim
)
)
SELECT
event_dim.name as name,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "productID") AS productID,
(SELECT value.int_value FROM UNNEST(event_dim.params)
WHERE key = "value") AS value
FROM T AS t,
t.event_dim AS event_dim;
I have two slave mysql db (5.5.27) (both are running on different machine with same OS).
There is three table (CATEGORY_TREE, SEO_METADATA, TAGS).
There schema definition are as follow:
| CATEGORY_TREE | CREATE TABLE `CATEGORY_TREE` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`child_cid` int(11) DEFAULT NULL,
`parent_cid` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `child_cid` (`child_cid`,`parent_cid`)
) ENGINE=InnoDB AUTO_INCREMENT=9528 DEFAULT CHARSET=latin1 |
| TAGS | CREATE TABLE `TAGS` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`name` varchar(64) NOT NULL,
`owner` varchar(20) DEFAULT NULL,
`type` varchar(20) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`,`name`,`owner`,`type`)
) ENGINE=InnoDB AUTO_INCREMENT=165498 DEFAULT CHARSET=latin1 |
| SEO_METADATA | CREATE TABLE `SEO_METADATA` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`canonicalString` varchar(128) DEFAULT NULL,
`dataId` varchar(20) DEFAULT NULL,
`oldId` int(30) DEFAULT NULL,
`type` varchar(1) DEFAULT NULL,
`canonicalString` varchar(128) DEFAULT NULL,
`searchString` text,
PRIMARY KEY (`id`),
KEY `id_type` (`dataId`,`type`),
KEY `oldid_type` (`oldId`,`type`),
KEY `canonicalstringidx` (`canonicalString`)
) ENGINE=InnoDB AUTO_INCREMENT=3863159 DEFAULT CHARSET=latin1 |
On running "EXPLAIN" for Select db query, I am getting different output
explain SELECT ct.child_cid AS 'tid', ct.parent_cid AS 'ptid', t.name, sm.canonicalString FROM CATEGORY_TREE ct, TAGS t, SEO_METADATA sm WHERE ct.child_cid = t.id AND sm.dataId = cast(ct.child_cid as char) AND sm.type = 't' AND (sm.searchString IS NULL OR sm.searchString = '') GROUP BY ct.child_cid LIMIT 10000;
**Machine 1**
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | ct | index | child_cid | child_cid | 10 | NULL | 2264 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | t | eq_ref | PRIMARY,id | PRIMARY | 8 | cms.ct.child_cid | 1 | Using where |
| 1 | SIMPLE | sm | ALL | NULL | NULL | NULL | NULL | 1588507 | Using where; Using join buffer |
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
**Machine 2**
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
| 1 | SIMPLE | sm | ALL | NULL | NULL | NULL | NULL | 1524208 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | ct | index | child_cid | child_cid | 10 | NULL | 2500 | Using where; Using index; Using join buffer |
| 1 | SIMPLE | t | eq_ref | PRIMARY,id | PRIMARY | 8 | cms.ct.child_cid | 1 | Using where |
+----+-------------+-------+--------+---------------+-----------+---------+------------------------+---------+----------------------------------------------+
In one it is using "index" first, but in other it is not.
I have gone through "Why does the same exact query produce 2 different MySQL explain results?", and checked all the parameter mentioned by #spencer7593.
Apart from this, I have run "optimize" table command in Machine 2 DB, but there no change in output.
I know that mysql can be "forced" to use index, but want to know the root cause for same.
Few post have mentioned that Innodb buffer-size can also be one reason for different output. In my case, both are almost same. Below is output of "SHOW ENGINE INNODB STATUS" from both db.
**Machine 1**
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 10989076480; in additional pool allocated 0
Dictionary memory allocated 5959814
Buffer pool size 655360
Free buffers 0
Database pages 645758
**Machine 2**
----------------------
BUFFER POOL AND MEMORY
----------------------
Total memory allocated 10989076480; in additional pool allocated 0
Dictionary memory allocated 5659318
Buffer pool size 655359
Free buffers 0
Database pages 645245
Thanks
An instance of WordPress that I manage goes down about once a day due to this monster MySQL query taking far too long:
SELECT SQL_CALC_FOUND_ROWS distinct wp_posts.*
FROM wp_posts
LEFT JOIN wp_term_relationships ON (wp_posts.ID = wp_term_relationships.object_id) LEFT JOIN wp_term_taxonomy ON wp_term_taxonomy.term_taxonomy_id = wp_term_relationships.term_taxonomy_id LEFT JOIN wp_ec3_schedule ec3_sch ON ec3_sch.post_id=id
WHERE 1=1 AND wp_posts.ID NOT IN ( SELECT tr.object_id
FROM wp_term_relationships AS tr INNER JOIN wp_term_taxonomy AS tt ON tr.term_taxonomy_id = tt.term_taxonomy_id
WHERE tt.taxonomy = 'category' AND tt.term_id IN ('1050') ) AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish') AND NOT EXISTS (SELECT *
FROM wp_term_relationships JOIN wp_term_taxonomy ON wp_term_taxonomy.term_taxonomy_id = wp_term_relationships.term_taxonomy_id
WHERE wp_term_relationships.object_id = wp_posts.ID AND wp_term_taxonomy.taxonomy = 'category' AND wp_term_taxonomy.term_id IN (533,3567) ) AND ec3_sch.post_id IS NULL GROUP BY wp_posts.ID ORDER BY wp_posts.post_date DESC LIMIT 0, 10;
What do I have to do to get rid of the very slow filesort? I would think that the multicolumn type_status_date index would be fast enough.
The EXPLAIN EXTENDED output is below.
+----+--------------------+-----------------------+--------+-----------------------------------+------------------+---------+---------------------------------------------------------------------------------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-----------------------+--------+-----------------------------------+------------------+---------+---------------------------------------------------------------------------------+------+----------------------------------------------+
| 1 | PRIMARY | wp_posts | ref | type_status_date | type_status_date | 124 | const,const | 7034 | Using where; Using temporary; Using filesort |
| 1 | PRIMARY | wp_term_relationships | ref | PRIMARY | PRIMARY | 8 | bwog_wordpress_w.wp_posts.ID | 373 | Using index |
| 1 | PRIMARY | wp_term_taxonomy | eq_ref | PRIMARY | PRIMARY | 8 | bwog_wordpress_w.wp_term_relationships.term_taxonomy_id | 1 | Using index |
| 1 | PRIMARY | ec3_sch | ref | post_id_index | post_id_index | 9 | bwog_wordpress_w.wp_posts.ID | 1 | Using where; Using index |
| 3 | DEPENDENT SUBQUERY | wp_term_taxonomy | range | PRIMARY,term_id_taxonomy,taxonomy | term_id_taxonomy | 106 | NULL | 2 | Using where |
| 3 | DEPENDENT SUBQUERY | wp_term_relationships | eq_ref | PRIMARY,term_taxonomy_id | PRIMARY | 16 | bwog_wordpress_w.wp_posts.ID,bwog_wordpress_w.wp_term_taxonomy.term_taxonomy_id | 1 | Using index |
| 2 | DEPENDENT SUBQUERY | tt | const | PRIMARY,term_id_taxonomy,taxonomy | term_id_taxonomy | 106 | const,const | 1 | |
| 2 | DEPENDENT SUBQUERY | tr | eq_ref | PRIMARY,term_taxonomy_id | PRIMARY | 16 | func,const | 1 | Using index |
+----+--------------------+-----------------------+--------+-----------------------------------+------------------+---------+---------------------------------------------------------------------------------+------+----------------------------------------------+
8 rows in set, 2 warnings (0.05 sec)
And CREATE TABLE:
CREATE TABLE `wp_posts` (
`ID` bigint(20) unsigned NOT NULL auto_increment,
`post_author` bigint(20) unsigned NOT NULL default '0',
`post_date` datetime NOT NULL default '0000-00-00 00:00:00',
`post_date_gmt` datetime NOT NULL default '0000-00-00 00:00:00',
`post_content` longtext NOT NULL,
`post_title` text NOT NULL,
`post_excerpt` text NOT NULL,
`post_status` varchar(20) NOT NULL default 'publish',
`comment_status` varchar(20) NOT NULL default 'open',
`ping_status` varchar(20) NOT NULL default 'open',
`post_password` varchar(20) NOT NULL default '',
`post_name` varchar(200) NOT NULL default '',
`to_ping` text NOT NULL,
`pinged` text NOT NULL,
`post_modified` datetime NOT NULL default '0000-00-00 00:00:00',
`post_modified_gmt` datetime NOT NULL default '0000-00-00 00:00:00',
`post_content_filtered` text NOT NULL,
`post_parent` bigint(20) unsigned NOT NULL default '0',
`guid` varchar(255) NOT NULL default '',
`menu_order` int(11) NOT NULL default '0',
`post_type` varchar(20) NOT NULL default 'post',
`post_mime_type` varchar(100) NOT NULL default '',
`comment_count` bigint(20) NOT NULL default '0',
`robotsmeta` varchar(64) default NULL,
PRIMARY KEY (`ID`),
KEY `post_name` (`post_name`),
KEY `type_status_date` (`post_type`,`post_status`,`post_date`,`ID`),
KEY `post_parent` (`post_parent`),
KEY `post_date` (`post_date`),
FULLTEXT KEY `post_related` (`post_title`,`post_content`)
)
Warnings:
mysql> SHOW warnings \G
*************************** 1. row ***************************
Level: Note
Code: 1276
Message: Field or reference 'bwog_wordpress_w.wp_posts.ID' of SELECT #3 was resolved in SELECT #1
*************************** 2. row ***************************
Level: Note
Code: 1003
Message: select distinct sql_calc_found_rows `bwog_wordpress_w`.`wp_posts`.`ID` AS `ID`,`bwog_wordpress_w`.`wp_posts`.`post_author` AS `post_author`,`bwog_wordpress_w`.`wp_posts`.`post_date` AS `post_date`,`bwog_wordpress_w`.`wp_posts`.`post_date_gmt` AS `post_date_gmt`,`bwog_wordpress_w`.`wp_posts`.`post_content` AS `post_content`,`bwog_wordpress_w`.`wp_posts`.`post_title` AS `post_title`,`bwog_wordpress_w`.`wp_posts`.`post_excerpt` AS `post_excerpt`,`bwog_wordpress_w`.`wp_posts`.`post_status` AS `post_status`,`bwog_wordpress_w`.`wp_posts`.`comment_status` AS `comment_status`,`bwog_wordpress_w`.`wp_posts`.`ping_status` AS `ping_status`,`bwog_wordpress_w`.`wp_posts`.`post_password` AS `post_password`,`bwog_wordpress_w`.`wp_posts`.`post_name` AS `post_name`,`bwog_wordpress_w`.`wp_posts`.`to_ping` AS `to_ping`,`bwog_wordpress_w`.`wp_posts`.`pinged` AS `pinged`,`bwog_wordpress_w`.`wp_posts`.`post_modified` AS `post_modified`,`bwog_wordpress_w`.`wp_posts`.`post_modified_gmt` AS `post_modified_gmt`,`bwog_wordpress_w`.`wp_posts`.`post_content_filtered` AS `post_content_filtered`,`bwog_wordpress_w`.`wp_posts`.`post_parent` AS `post_parent`,`bwog_wordpress_w`.`wp_posts`.`guid` AS `guid`,`bwog_wordpress_w`.`wp_posts`.`menu_order` AS `menu_order`,`bwog_wordpress_w`.`wp_posts`.`post_type` AS `post_type`,`bwog_wordpress_w`.`wp_posts`.`post_mime_type` AS `post_mime_type`,`bwog_wordpress_w`.`wp_posts`.`comment_count` AS `comment_count`,`bwog_wordpress_w`.`wp_posts`.`robotsmeta` AS `robotsmeta` from `bwog_wordpress_w`.`wp_posts` left join `bwog_wordpress_w`.`wp_term_relationships` on((`bwog_wordpress_w`.`wp_term_relationships`.`object_id` = `bwog_wordpress_w`.`wp_posts`.`ID`)) left join `bwog_wordpress_w`.`wp_term_taxonomy` on((`bwog_wordpress_w`.`wp_term_taxonomy`.`term_taxonomy_id` = `bwog_wordpress_w`.`wp_term_relationships`.`term_taxonomy_id`)) left join `bwog_wordpress_w`.`wp_ec3_schedule` `ec3_sch` on((`bwog_wordpress_w`.`ec3_sch`.`post_id` = `bwog_wordpress_w`.`wp_posts`.`ID`)) where ((not(<in_optimizer>(`bwog_wordpress_w`.`wp_posts`.`ID`,<exists>(select 1 AS `Not_used` from `bwog_wordpress_w`.`wp_term_relationships` `tr` join `bwog_wordpress_w`.`wp_term_taxonomy` `tt` where ((`bwog_wordpress_w`.`tr`.`term_taxonomy_id` = '3572') and ('category' = _utf8'category') and (<cache>(`bwog_wordpress_w`.`wp_posts`.`ID`) = `bwog_wordpress_w`.`tr`.`object_id`)))))) and (`bwog_wordpress_w`.`wp_posts`.`post_type` = _utf8'post') and (`bwog_wordpress_w`.`wp_posts`.`post_status` = _utf8'publish') and (not(exists(select 1 AS `Not_used` from `bwog_wordpress_w`.`wp_term_relationships` join `bwog_wordpress_w`.`wp_term_taxonomy` where ((`bwog_wordpress_w`.`wp_term_relationships`.`term_taxonomy_id` = `bwog_wordpress_w`.`wp_term_taxonomy`.`term_taxonomy_id`) and (`bwog_wordpress_w`.`wp_term_relationships`.`object_id` = `bwog_wordpress_w`.`wp_posts`.`ID`) and (`bwog_wordpress_w`.`wp_term_taxonomy`.`taxonomy` = _utf8'category') and (`bwog_wordpress_w`.`wp_term_taxonomy`.`term_id` in (533,3567)))))) and isnull(`bwog_wordpress_w`.`ec3_sch`.`post_id`)) group by `bwog_wordpress_w`.`wp_posts`.`ID` order by `bwog_wordpress_w`.`wp_posts`.`post_date` desc limit 10,10
2 rows in set (0.00 sec)
The filesort is unavoidable due to the GROUP BY. However:
IN(SUBQUERY) gets run on every row of
the joined tables. Execute that subquery
before-hand as an independent query and then substitute the (now
static) results into the IN() function.
Do something similar with the EXISTS(SUBQUERY) function.
Consider using WordPress's super-cache plugin.
What are the warnings showing up in the EXPLAIN?
I know this old, but it's not a WordPress issue - it's a plugin issue (as usual).
You must be using a plugin called EventCalendar - that's what is causing this problem.
http://wpcal-archive.firetree.net/2008-January/002892.html