wordpress generating slow mysql queries - is it index problem? - wordpress

I've got very slow Mysql queries coming up from my wordpress site. It's making everything slow and I think this is eating up CPU usage. I've pasted the Explain results for the two most frequently problematic queries below. This is a typical result - although very occasionally teh queries do seem to be performed at a more normal speed.
I have the usual wordpress indexes on the database tables. You will see that one of the queries is generated from wordpress core code, and not from anything specific - like the theme - for my site.
I have a vague feeling that the database is not always using the indexes/is not using them properly...
Is this right? Does anyone know how to fix it? Or is it a different problem entirely?
Many thanks in advance for any help anyone can offer - it is hugely appreciated
Query: [wp-blog-header.php(14): wp()]
SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private') ORDER BY wp_posts.post_date DESC LIMIT 0, 6
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
wp_posts
ref
type_status_date
type_status_date
63
const
427
Using where; Using filesort
Query time: 34.2829 (ms)
9) Query: [wp-content/themes/LMHR/index.php(40): query_posts()]
SELECT SQL_CALC_FOUND_ROWS wp_posts.* FROM wp_posts WHERE 1=1 AND wp_posts.ID NOT IN ( SELECT tr.object_id FROM wp_term_relationships AS tr INNER JOIN wp_term_taxonomy AS tt ON tr.term_taxonomy_id = tt.term_taxonomy_id WHERE tt.taxonomy = 'category' AND tt.term_id IN ('217', '218', '223', '224') ) AND wp_posts.post_type = 'post' AND (wp_posts.post_status = 'publish' OR wp_posts.post_status = 'private') ORDER BY wp_posts.post_date DESC LIMIT 0, 6
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
PRIMARY
wp_posts
ref
type_status_date
type_status_date
63
const
427
Using where; Using filesort
2
DEPENDENT SUBQUERY
tr
ref
PRIMARY,term_taxonomy_id
PRIMARY
8
func
1
Using index
2
DEPENDENT SUBQUERY
tt
eq_ref
PRIMARY,term_id_taxonomy,taxonomy
PRIMARY
8
antin1_lovemusic2010.tr.term_taxonomy_id
1
Using where
Query time: 70.3900 (ms)


Check this http://core.trac.wordpress.org/ticket/10964
The problem is SQL_CALC_NUM_ROWS, WordPress set this param automatically when you execute get_posts query, this make a slow query.


You can try wp-cache plugin.
Also you can read this article it explains that SQL_CALC_FOUND_ROWS not the best solution at queries.

Related

Optimizing a wordpress + woocomm store with 100k+ products

So i'm in the middle of an issue right now where the page loading speed is not up to par to what we're looking for and i'm honestly out of ideas for now and i hoped maybe someone smarter can lead me in the right direction.
First of all, the environment:
The site is hosted on a cloud nginx server with 32GB of RAM.
The site uses a Porto theme with WP Bakery builder. I know i know, page builders aren't famous for speed but testing the same site with storefront and without bakery shows no difference.
The site is multilingual and for translation we use WPML. We've thought about using seperate language files instead but it is quite an hassle to get those strings out of wpml.
Now the situation:
Without any caching plugins / optimizers we get a 3-5s load speed. With WP Rocket + Redis we get the load speed less than a second. Now when the user visits a page and it is isnt yet cached by Redis (recently flushed) we'd like to prevent the user from waiting 3-5s to initally load the pages.
What i've tried (without caching):
Using QM to monitor and find plugins/scripts affecting the
performance.
Database indexing and cleanup tips found online.
Image optimization.
Query monitor shows me that on the shop page ( slowest ), wp posts main query takes more than 1 second to complete then adding additional 2-4s to display the data.
Using Scalability PRO plugin i tried limiting the time it takes to execute the query but honestly it didn't help much. None of the sql changes really helped alot nor did any other settings inside the plugin. Maybe a 1s difference.
Only real difference is seen when wp rocket and redis are active but thats not an ideal solution. Is there a way to make the site run any faster or to make it seem like it does?
Here's all the data i can get for /shop page
https://prnt.sc/be5qfPkzGim0
https://prnt.sc/lSmmvrEZR2X0
https://prnt.sc/1QasUlurvFgI
https://prnt.sc/TtYXI0MXaY5k
https://prnt.sc/dOib1Z1uGD4u
https://prnt.sc/vydXlQfq8x9W
Query
Caller: WP_Query->get_posts()
Rows: 12
Time (s): 1.4356
SELECT SQL_CALC_FOUND_ROWS wp_posts.ID
FROM wp_posts
INNER JOIN wp_postmeta ON (wp_posts.ID = wp_postmeta.post_id)
JOIN wp_icl_translations wpml_translations ON wp_posts.ID = wpml_translations.element_id
AND wpml_translations.element_type = CONCAT('post_', wp_posts.post_type)
WHERE 1=1
AND (wp_posts.ID NOT IN
(SELECT object_id
FROM wp_term_relationships
WHERE term_taxonomy_id IN (7,
9) ))
AND ((wp_postmeta.meta_key = '_price'
AND wp_postmeta.meta_value > '0'))
AND ((wp_posts.post_type = 'product'
AND (wp_posts.post_status = 'publish'
OR wp_posts.post_status = 'private')))
AND (((wpml_translations.language_code = 'en'
OR (wpml_translations.language_code = 'en'
AND wp_posts.post_type IN ('post',
'page',
'custom_css',
'customize_changeset',
'oembed_cache',
'user_request',
'product',
'product_variation',
'shop_order_refund')
AND ((
(SELECT COUNT(element_id)
FROM wp_icl_translations
WHERE trid = wpml_translations.trid
AND language_code = 'en' ) = 0)
OR (
(SELECT COUNT(element_id)
FROM wp_icl_translations t2
JOIN wp_posts p ON p.id = t2.element_id
WHERE t2.trid = wpml_translations.trid
AND t2.language_code = 'en'
AND (p.post_status = 'publish'
OR p.post_type='attachment'
AND p.post_status = 'inherit') ) = 0))))
AND wp_posts.post_type IN ('post',
'page',
'attachment',
'custom_css',
'customize_changeset',
'oembed_cache',
'user_request',
'wp_block',
'wp_template',
'wp_template_part',
'wp_navigation',
'product',
'product_variation',
'shop_order_refund',
'faq',
'porto_builder'))
OR wp_posts.post_type NOT IN ('post',
'page',
'attachment',
'custom_css',
'customize_changeset',
'oembed_cache',
'user_request',
'wp_block',
'wp_template',
'wp_template_part',
'wp_navigation',
'product',
'product_variation',
'shop_order_refund',
'faq',
'porto_builder'))
GROUP BY wp_posts.ID
ORDER BY wp_posts.menu_order ASC,
wp_posts.post_title ASC
LIMIT 0,
12
Second slow query
Caller: _pad_term_counts()
Rows: 247746
Time (s): 1.0853
SELECT object_id, term_taxonomy_id
FROM wp_term_relationships
INNER JOIN wp_posts
ON object_id = ID
WHERE term_taxonomy_id IN (180862,181291,181328,181362,181541,182258,182825,180902,181307,181341,181373,181552,182698,181281,181318,181352,181384,181563,182752,180863,181292,181329,181363,181542,182259,182826,180927,181308,181342,181374,181553,182699,180853,181282,181319,181353,181385,181564,182753,180864,181293,181330,181364,181543,182260,182829,180928,181309,181343,181375,181554,182700,180854,181283,181320,181354,181386,181565,182754,180865,181294,181331,181365,181544,182261,183389,180929,181310,181344,181376,181555,182701,180855,181284,181321,181355,181387,181566,182778,180866,181295,181332,181366,181545,182645,183390,180930,181311,181345,181377,181556,182743,180856,181285,181322,181356,181388,181784,182779,180867,181301,181335,181367,181546,182646,183637,181243,181312,181346,181378,181557,182746,180857,181286,181323,181357,181389,181807,182780,180868,181302,181336,181368,181547,182647,184049,181275,181313,181347,181379,181558,182747,180858,181287,181324,181358,181390,182190,182781,180872,181303,181337,181369,181548,182648,208562,181276,181314,181348,181380,181559,182748,180859,181288,181325,181359,181391,182208,182782,180889,181304,181338,181370,181549,182649,208563,181277,181315,181349,181381,181560,182749,180860,181289,181326,181360,181539,182248,182783,180890,181305,181339,181371,181550,182696,181278,181316,181350,181382,181561,182750,180861,181290,181327,181361,181540,182249,182784,180896,181306,181340,181372,181551,182697,181279,181317,181351,181383,181562,182751)
AND post_type IN ('product')
AND post_status = 'publish'

Wordpress, WooCommerce - save product is very slow

Some products are extremely slow to save (30 sec)
here is a screen shot from the profiler:
I turn on Slow Query log in mysql, and found this query:
SELECT COUNT( DISTINCT ID )
FROM wpmf_posts p
LEFT JOIN (
SELECT object_id
FROM wpmf_term_relationships
WHERE term_taxonomy_id
IN ( 7 )
) AS exclude_join ON exclude_join.object_id = p.ID
INNER JOIN (
SELECT object_id
FROM wpmf_term_relationships
INNER JOIN wpmf_term_taxonomy
USING ( term_taxonomy_id )
WHERE term_id
IN ( 4246, 4247, 4254, 4257, 4263, 4280, 4290, 4335, 4375, 4397, 4486, 5114 )
) AS include_join ON include_join.object_id = p.ID
WHERE 1 =1
AND p.post_status = 'publish'
AND p.post_type = 'product'
AND exclude_join.object_id IS NULL
LIMIT 0 , 30
which returns 4799, and takes 27 sec:
# Time: 190127 8:38:29
# User#Host: lfytcoil_uprdb[lfytcoil_uprdb] # localhost []
# Query_time: 27.287704 Lock_time: 0.000076 Rows_sent: 1 Rows_examined: 483196982
Any idea what is the purpose of this query?
This is wpmf_term_taxonomy with id
product_visibility
The count shows
31234
This is a WooCommerce site with 10717 products,
I suspect that there too many products for WooCommerce to handle.
NOTE: after running delete FROMwpmf_term_relationshipswhereterm_taxonomy_id= 7 which deleted 31234, the save take only 2 sec
My question:
Is WooCommerce not able to handle 10717 products?
Is there a way to somehow improve the save?
e.g. by disabling this query? or by caching the result? or improve the DB structure? or install some smart modules?
I notice the call to wp_update_term_count, it seems that if I disable this call, then the problem will be solved, what could be the side effects of this?

The solution is explain in this issue:
https://github.com/woocommerce/woocommerce/issues/14900
you should replace the file includes/wc-term-functions.php
with this:
https://raw.githubusercontent.com/woocommerce/woocommerce/ec489ce1a0c9030a73080ada28f25162f441dfdb/includes/wc-term-functions.php

NOTE: My previous answer is wrong
I found you can defer term count by adding
wp_defer_term_counting(true);
before the save operation (i.e. in the beginning of the script)
Than do
wp_defer_term_counting(false);
at the end of the script,
which also run the term count

Wordpress and author__in

I manipulate a query with pre_get_posts
But i want to show posts of multiple authors.
These for i use "author__in".
$query->set('author__in', [53, 285]);
But my query still runs with 2 post_author parameters. How can i exclude the second one?
SELECT SQL_CALC_FOUND_ROWS wp_posts.ID
FROM wp_posts
WHERE 1=1
AND wp_posts.ID IN (349956,327745,177707)
AND wp_posts.post_author IN (53,285)
AND (wp_posts.post_author = 53)
AND wp_posts.post_type = 'post'
ORDER BY wp_posts.post_date DESC
LIMIT 0, 10

There are various author filters available: https://codex.wordpress.org/Class_Reference/WP_Query#Author_Parameters
Adding a new one won't remove others - so you'll need to remove the original author filter. You can do this using the following code:
unset( $query->query_vars['author'] );

Slow Query on Wordpress Search

Please I have a very slow query on my wordpress search, checking the processes it takes upto 40sec to execute using SHOW FULL PROCESSLIST; below is a sample of query
SELECT SQL_CALC_FOUND_ROWS wp_posts.ID FROM wp_posts WHERE 1=1 AND (((wp_posts.post_title LIKE '%Industreet%') OR (wp_posts.post_excerpt LIKE '%Industreet%') OR (wp_posts.post_content LIKE '%Industreet%'))) AND (wp_posts.post_password = '') AND wp_posts.post_type IN ('post', 'page', 'attachment') AND (wp_posts.post_status = 'publish') ORDER BY wp_posts.post_title LIKE '%Industreet%' DESC, wp_posts.post_date DESC LIMIT 0, 25
I have tried my best to fix this, but nothing is working out. though I have a total post of 120k on my site. I really need help.
Thanks in advance

Drupal 6: "Facebook" style comment ordering (threaded, descending+ascending)

Working with Drupal 6, my goal is to order a set of threaded comments similar to how Facebook outputs comments: with the 'anchor' or first comment in each thread sorted DESC, and any 'internal' thread replies sorted ASC so the newest comments are at the bottom.
Here's the SQL query from comment_render, with ordering by COMMENT_ORDER_NEWEST_FIRST:
SELECT c.cid as cid, c.pid, c.nid,
c.subject, c.comment, c.format,
c.timestamp, c.name, c.mail,
c.homepage, u.uid, u.name AS
registered_name, u.signature,
u.picture, u.data, c.thread, c.status
FROM {comments} c INNER JOIN {users} u
ON c.uid = u.uid WHERE c.nid = 141737
AND c.status = 0 ORDER BY c.thread
DESC
This returns all comments, ordered by the newest thread first:
03/
03.00/
02/
02.06/
02.05/
02.04/
02.03/
02.01/
02.00/
01/
The desired ordering in my case is this:
03/
03.00/
02/
02.00/
02.01/
02.02/
02.03/
02.04/
02.05/
02.06/
01/
Again just think of the Facebook wall and you get the idea.
Can anyone assist in enhancing the SQL query appropriately? In my case (but perhaps not in all cases) the thread depth is forcibly limited to 1 via a custom module.
One other note - in my case it only has to work under MySQL.

Well this query works, but again I'm a SQL noob so this probably isn't fully correct.
SELECT c.cid as cid, c.pid, c.nid,
c.subject, c.comment, c.format,
c.timestamp, c.name, c.mail,
c.homepage, u.uid, u.name AS
registered_name, u.signature,
u.picture, u.data, c.thread, c.status
FROM (SELECT c.cid as cid, c.pid,
c.nid, c.subject, c.comment, c.format,
c.timestamp, c.name, c.mail,
c.homepage, u.uid, u.name AS
registered_name, u.signature,
u.picture, u.data, c.thread, c.status
FROM comments c INNER JOIN users u
ON c.uid = u.uid WHERE c.nid = 141737
AND c.status = 0 ORDER BY
SUBSTRING(thread, 1, (LENGTH(thread) -
1))) c INNER JOIN users u ON c.uid =
u.uid WHERE c.nid = 141737 AND
c.status = 0 ORDER BY
SUBSTRING(thread, 1, 2) DESC
The idea is to first query with an ordering based on the full thread (02.05, etc) and then do another SELECT on just the first two characters of the thread field. Can I get a bit of help optimizing this or is it otherwise "correct"?

Resources