Mysql sub query analysis one

First of all, I will introduce several commonly used optimization analysis command tools. I am not used to writing detailed descriptions at the beginning of blogging.

• EXPLAIN or DESC (describe abbreviation) SQL execution analysis plan tool
• The EXTENDED option of EXPLAIN can provide "query optimization SQL" based on the original EXPLAIN, execute EXPLAIN EXTENDED SELECT... and then use the SHOW WARNINGS command to view MySQL's optimization of the original SQL
• Configure profiling to analyze the execution process of query SQL
  wrote
  show variables like "%profil%";
  set profiling = 1;
  select @@profiling;
• show profiles; //View operation records
• show profile for query 1; //View the specific operation analysis process

Next, let’s get to the topic. A report sql written before contains sub-queries. At first, I didn’t find any problems. Recently, it was found that the data volume was only 3,000 or 4,000, and the query speed reached about 2S.

Simplified SQL:
SELECT
count(1)
FROM
tb_mpsqd AS sqd1
LEFT JOIN tb_customer AS cust1 ON sqd1.customer_id = cust1.customer_id
LEFT JOIN t_s_depart AS depart1 ON sqd1.sys_org_code = depart1.org_code
LEFT JOIN tb_mpzqsqd AS zqsqd1 ON sqd1.mpsqd_id = zqsqd1.mpsqd_id AND sqd1.delayed_times = zqsqd1.delayed_times
LEFT JOIN tb_mppgd AS pgd1 ON sqd1.mpsqd_id = pgd1.mpsqd_id
LEFT JOIN (
select group_concat(distinct(a.classify_name)) as classify_name, group_concat(b.pawn_name) as pawn_name, group_concat(b.stand_by separator ' ') as wp_stand_by, b.mppgd_id from tb_classify as a inner join tb_mppgd_base as b on a.classify_id = b.classify_id GROUP BY b.mppgd_id
) AS pgdbase1 ON pgdbase1.mppgd_id = pgd1.mppgd_id
The execution plan is as follows:
+----+-------------+------------+--------+-----------------+--------------+---------+-------------------------+------+----------------+
| id | select_type | table      | type   | possible_keys   | key          | key_len | ref                     | rows | Extra          |
+----+-------------+------------+--------+-----------------+--------------+---------+-------------------------+------+----------------+
|  1 | PRIMARY     | sqd1       | ALL    | NULL            | NULL         | NULL    | NULL                    | 1329 |                |
|  1 | PRIMARY     | cust1      | eq_ref | PRIMARY         | PRIMARY      | 4       | tdperp.sqd1.customer_id |    1 | Using index    |
|  1 | PRIMARY     | depart1    | ALL    | NULL            | NULL         | NULL    | NULL                    |  104 |                |
|  1 | PRIMARY     | zqsqd1     | ref    | mpsqd_id        | mpsqd_id     | 4       | tdperp.sqd1.mpsqd_id    |    1 |                |
|  1 | PRIMARY     | pgd1       | ref    | mppgd_id_idx    | mppgd_id_idx | 4       | tdperp.sqd1.mpsqd_id    |    1 | Using index    |
|  1 | PRIMARY     | <derived2> | ALL    | NULL            | NULL         | NULL    | NULL                    | 1116 |                |
|  2 | DERIVED     | b          | ALL    | idx_classify_id | NULL         | NULL    | NULL                    | 1446 | Using filesort |
|  2 | DERIVED     | a          | eq_ref | PRIMARY         | PRIMARY      | 4       | tdperp.b.classify_id    |    1 | Using where    |
+----+-------------+------------+--------+-----------------+--------------+---------+-------------------------+------+----------------+

analyze:

1. First of all, this report business is associated with 7 tables, involving the attributes group_concat and group by operations of some sub-tables
2. There are subqueries in sql, and the query type with DERIVED can be seen through the execution plan , which is stored in a temporary table. There is also a Using filesort file sorting problem (caused by group by)
3. By splitting the query, I didn't find any issues affecting the speed. Only slows down when subquery is integrated with main query

After a simple test, the positioning problem mainly lies in the subquery here:

1. When using subqueries, mysql needs to create temporary tables to store and process the data of the subquery ( filesort ), and in this case the result set of the subquery is a large string content

Subquery optimization - solve the problem of filesort

SELECT group_concat(DISTINCT(select classify_name from tb_classify a where b.classify_id = a.classify_id)) AS classify_name,
group_concat(b.pawn_name) AS pawn_name,
group_concat(b.stand_by SEPARATOR ' ') AS wp_stand_by,
b.mppgd_id
FROM tb_mppgd_base AS b GROUP BY b.mppgd_id;
+----+--------------------+-------+--------+---------------+--------------+---------+----------------------+------+-------------+
| id | select_type        | table | type   | possible_keys | key          | key_len | ref                  | rows | Extra       |
+----+--------------------+-------+--------+---------------+--------------+---------+----------------------+------+-------------+
|  1 | PRIMARY            | b     | index  | NULL          | mppgd_id_idx | 4       | NULL                 | 1446 |             |
|  2 | DEPENDENT SUBQUERY | a     | eq_ref | PRIMARY       | PRIMARY      | 4       | tdperp.b.classify_id |    1 | Using where |
+----+--------------------+-------+--------+---------------+--------------+---------+----------------------+------+-------------+

Subquery optimization - solving speed problems

SELECT
count(1)
FROM
tb_mpsqd AS sqd1
LEFT JOIN tb_customer AS cust1 ON sqd1.customer_id = cust1.customer_id
LEFT JOIN t_s_depart AS depart1 ON sqd1.sys_org_code = depart1.org_code
LEFT JOIN tb_mpzqsqd AS zqsqd1 ON sqd1.mpsqd_id = zqsqd1.mpsqd_id AND sqd1.delayed_times = zqsqd1.delayed_times
LEFT JOIN tb_mppgd AS pgd1 ON sqd1.mpsqd_id = pgd1.mpsqd_id
LEFT JOIN tb_mppgd_base AS b ON b.mppgd_id = pgd1.mppgd_id
LEFT JOIN tb_classify AS c ON b.classify_id = c.classify_id
GROUP BY sqd1.mpsqd_id;
+----+-------------+---------+--------+-----------------+-----------------+---------+--------------------------+------+-------------+
| id | select_type | table   | type   | possible_keys   | key             | key_len | ref                      | rows | Extra       |
+----+-------------+---------+--------+-----------------+-----------------+---------+--------------------------+------+-------------+
|  1 | SIMPLE      | sqd1    | index  | NULL            | PRIMARY         | 4       | NULL                     | 1329 |             |
|  1 | SIMPLE      | cust1   | eq_ref | PRIMARY         | PRIMARY         | 4       | tdperp.sqd1.customer_id  |    1 | Using index |
|  1 | SIMPLE      | depart1 | ref    | unique_org_code | unique_org_code | 195     | tdperp.sqd1.sys_org_code |    1 | Using index |
|  1 | SIMPLE      | zqsqd1  | ref    | mpsqd_id        | mpsqd_id        | 4       | tdperp.sqd1.mpsqd_id     |    1 |             |
|  1 | SIMPLE      | pgd1    | ref    | mppgd_id_idx    | mppgd_id_idx    | 4       | tdperp.sqd1.mpsqd_id     |    1 | Using index |
|  1 | SIMPLE      | b       | ref    | mppgd_id_idx    | mppgd_id_idx    | 4       | tdperp.pgd1.mppgd_id     |    1 |             |
|  1 | SIMPLE      | c       | eq_ref | PRIMARY         | PRIMARY         | 4       | tdperp.b.classify_id     |    1 | Using index |
+----+-------------+---------+--------+-----------------+-----------------+---------+--------------------------+------+-------------+

After optimization, the speed is increased from 2000ms to below 300ms.

Summarize:

1. The use of subqueries needs to be used with caution. The versions after Mysql 5.6 have been optimized and can be ignored. For versions before Mysql 5.6, it is recommended to replace subqueries with Join associated queries as much as possible.
2. Try to build an index on the attributes associated with the join
3. Use ORDER BY null to avoid filesort