MySQL in query optimization

https://blog.csdn.net/gua___gua/article/details/47401621

 

 

MySQL in query optimization <1>

Original  2015-08-10 17:57:56

• 5137

The developer said that he wrote a SQL that was very slow, let's take a look.

[html] view plain  copy

 

1. select * from t_channel where id_ in(select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00');  
2. ......  
3. 30min+  

 

Then I query the internal SQL and it only takes 3s+

 

[html] view plain  copy

 

1. mysql> select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00';  
2. .....  
3. 1755 rows in set (3.30 sec)  
4.   
5. mysql> select count(*) from t_channel;  
6. ....  
7. 12062 rows in set (0.70 sec)  

Why is the SQL written by development so slow? Check out the execution plan

 

 

[html] view plain  copy

 

1. explain extended select * from t_channel where id_ in(select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00');  
2. +----+--------------------+-----------+-------+---------------+----------+---------+------+--------+----------+------------------------------+  
3. | id | select_type        | table     | type  | possible_keys | key      | key_len | ref  | rows   | filtered | Extra                        |  
4. +----+--------------------+-----------+-------+---------------+----------+---------+------+--------+----------+------------------------------+  
5. |  1 | PRIMARY            | t_channel | ALL   | NULL          | NULL     | NULL    | NULL |  12062 |   100.00 | Using where                  |  
6. |  2 | DEPENDENT SUBQUERY | sjkk_gcjl | range | idx_jgsj      | idx_jgsj | 8       | NULL | 731868 |   100.00 | Using where; Using temporary |  
7. +----+--------------------+-----------+-------+---------------+----------+---------+------+--------+----------+------------------------------  

Look at the statement after the database conversion

[html] view plain  copy

 

1. mysql> show warnings;  
2. SELECT  
3.     `shanghai_full`.`t_channel`.`ID_` AS `ID_`,  
4.     `shanghai_full`.`t_channel`.`Code_` AS `Code_`,  
5.      ...... -- all fields will be listed here  
6. FROM  
7.     `shanghai_full`.`t_channel`  
8. WHERE  
9.     < in_optimizer > (  
10.         `shanghai_full`.`t_channel`.`ID_` ,< EXISTS > (  
11.             SELECT DISTINCT  
12.                 1  
13.             FROM  
14.                 `shanghai_full`.`sjkk_gcjl`  
15.             WHERE  
16.                 (  
17.                     (  
18.                         `shanghai_full`.`sjkk_gcjl`.`jgsj` > '2015-01-02 08:00:00'  
19.                     )  
20.                     AND (  
21.                         `shanghai_full`.`sjkk_gcjl`.`jgsj` < '2015-01-02 12:00:00'  
22.                     )  
23.                     AND (  
24.                         < CACHE > (  
25.                             `shanghai_full`.`t_channel`.`ID_`  
26.                         ) = `shanghai_full`.`sjkk_gcjl`.`cdbh`  
27.                     )  
28.                 )  
29.         )  
30.     );  

It can be seen that after the mysql optimizer, in is converted into exists (mysql thinks it is faster to change to exists, huh, huh).

The reason for the slowness: after exists, the query uses the external t_channel as the main table to perform a full table scan, scans a row of data each time, and then queries the subquery to see if the data meets the conditions.

Special note: The mysql version is 5.5. In 5.6, mysql has been improved to convert the query of in into join.

 

Optimization method 1 (convert in query to join query)

 

[html] view plain  copy

 

1. select t1.* from t_channel t1,(select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00') t2 where t1.id_=t2.cdbh;  
2. ......  
3. 1264 rows in set (3.30 sec)  
4.   
5. mysql> explain extended select t1.* from t_channel t1,(select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00') t2 where t1.id_www.255055.cn/ =t2.cdbh;  
6. +----+-------------+------------+--------+---------------+----------+---------+---------+--------+----------+------------------------------+  
7. | id | select_type | table      | type   | possible_keys | key      | key_len | ref     | rows   | filtered | Extra                        |  
8. +----+-------------+------------+--------+---------------+----------+---------+---------+--------+----------+------------------------------+  
9. |  1 | PRIMARY     | <derived2> | ALL    | NULL          | NULL     | NULL    | NULL    |   1755 |   100.00 |                              |  
10. |  1 | PRIMARY     | t1    www.boshenyl.cn      | eq_ref | PRIMARY,ID_   | PRIMARY  | 74      | t2.cdbh |      1 |   100.00 | Using where                  |  
11. |  2 | DERIVED     | sjkk_gcjl  | range  | idx_jgsj      | idx_jgsj | 8       | NULL    | 731868 |   100.00 | Using where; Using temporary |  
12. +----+-------------+------------+--------+---------------+----------+---------+---------+--------+----------+------------------------------+  
13.   
14. mysql> show warnings;  
15.   
16. SELECT  
17.     `shanghai_full`.`t1`.`ID_` AS `ID_`,  
18.     ........ --- here will list all fields FROM  
19.     `shanghai_full`.`t_channel` `t1`  
20. JOIN (  
21.     SELECT DISTINCT  
22.         `shanghai_full`.`sjkk_gcjl`.`cdbh` AS `www.cnzhaotai.com cdbh`  
23.     FROM  
24.         `shanghai_full`.www.fengshen157.com` sjkk_gcjl`  
25.     WHERE  
26.         (  
27.             (  
28.                 `shanghai_full`.`sjkk_gcjl`.`jgsj` > '2015-01-02 08:00:00'  
29.             )  
30.             AND (  
31.                 `shanghai_full`.`sjkk_gcjl`.`jgsj` < '2015-01-02 12:00:00'  
32.             )  
33.         )  
34. ) `t2`  
35. WHERE  
36.     (  
37.         `shanghai_full`.`t1`.`ID_` = www.taohuayuan178.com `t2`.`cdbh`  
38.     );  

 

Optimization method 2 (using the temporary table of the memory engine)

 

[html] view plain  copy

 

1. mysql> create temporary table tmp_www.yibaoyule1.com channel  engine=memory (select distinct cdbh from sjkk_gcjl where jgsj>'2015-01-02 08:00:00' and jgsj<'2015-01-02 12:00:00');  
2. Query OK, 1755 rows affected (9.00 sec)  
3. Records: 1755  Duplicates: 0  Warnings: 0  
4. mysql> select * from t_channel where id_ in(select * from tmp_channel);  
5. .....  
6. 1264 rows in set (0.26 sec)  
7.   
8. mysql> explain extended select * from t_channel where id_ in(select * from tmp_channel);  
9. +----+--------------------+-------------+------+---------------+------+---------+------+------+----------+-------------+  
10. | id | select_type        | table       | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |  
11. +----+--------------------+-------------+------+---------------+------+---------+------+------+----------+-------------+  
12. |  1 | PRIMARY            | t_channel   | ALL  | NULL          | NULL | NULL    | NULL | 3224 |   100.00 | Using where |  
13. |  2 | DEPENDENT SUBQUERY | tmp_channel | ALL  | NULL          | NULL | NULL    | NULL |   20 |   100.00 | Using where |  
14. +----+--------------------+-------------+------+---------------+------+---------+------+------+----------+-------------+  
15.   
16. mysql> show warnings;  
17. | Note  | 1003 | select `vmc_jiaqi`.`t_channel`.`ID_` AS `ID_`,`vmc_jiaqi`.`t_channel`.`Code_` AS `Code_`,`vmc_jiaqi`.`t_channel`.`HostId_` AS `HostId_`,`vmc_jiaqi`.`t_channel`.`RoadMonitorStationId_` AS `RoadMonitorStationId_`,`vmc_jiaqi`.`t_channel`.`ChannelNo_` AS `ChannelNo_`,`vmc_jiaqi`.`t_channel`.`Name_` AS `Name_`,`vmc_jiaqi`.`t_channel`.`ChannelType_` AS `ChannelType_`,`vmc_jiaqi`.`t_channel`.`Ext_` AS `Ext_`,`vmc_jiaqi`.`t_channel`.`DeviceAddress_` AS `DeviceAddress_`,`vmc_jiaqi`.`t_channel`.`MinSpeed_`www.mhylpt.comS `MinSpeed_`,`vmc_jiaqi`.`t_channel`.`MaxSpeed_` AS `MaxSpeed_`,`vmc_jiaqi`.`t_channel`.`LimitMinRatio_` AS `LimitMinRatio_`,`vmc_jiaqi`.`t_channel`.`LimitMaxRatio_` AS `LimitMaxRatio_`,`vmc_jiaqi`.`t_channel`.`Direction_` AS `Direction_`,`vmc_jiaqi`.`t_channel`.`JcDirection_` AS `JcDirection_`,`vmc_jiaqi`.`t_channel`.`LaneNum_` AS `LaneNum_`,`vmc_jiaqi`.`t_channel`.`FrameDropNum_` AS `FrameDropNum_`,`vmc_jiaqi`.`t_channel`.`RecogizeRectLeft_` AS `RecogizeRectLeft_`,`vmc_jiaqi`.`t_channel`.`RecogizeRectTop_` AS `RecogizeRectTop_`,`vmc_jiaqi`.`t_channel`.`RecogizeRectWidth_` AS `RecogizeRectWidth_`,`vmc_jiaqi`.`t_channel`.`RecogizeRectHeight_` AS `RecogizeRectHeight_`,`vmc_jiaqi`.`t_channel`.`RmpServerIp_` AS `RmpServerIp_`,`vmc_jiaqi`.`t_channel`.`RmpServerPort_` AS `RmpServerPort_`,`vmc_jiaqi`.`t_channel`.`VirtualGateServerId_` AS `VirtualGateServerId_`,`vmc_jiaqi`.`t_channel`.`LaneType_` AS `LaneType_`,`vmc_jiaqi`.`t_channel`.`DeviceType_` AS `DeviceType_`,`vmc_jiaqi`.`t_channel`.`ManufacturerId_` AS `ManufacturerId_`,`vmc_jiaqi`.`t_channel`.`IsLocalSavePicture_` AS `IsLocalSavePicture_`,`vmc_jiaqi`.`t_channel`.`OrderNO_` AS `OrderNO_`,`vmc_jiaqi`.`t_channel`.`channelStatus` AS `channelStatus` from `vmc_jiaqi`.`t_channel` where <in_optimizer>(`vmc_jiaqi`.`t_channel`.`ID_`,<exists>(select 1 from `vmc_jiaqi`.`tmp_channel` where (<cache>(`vmc_jiaqi`.`t_channel`.`ID_`) = `vmc_jiaqi`.`tmp_channel`.`cdbh`)))   

Note: The second method still uses the execution method of exists, so this method is not as good as the first method, and may be useful under certain conditions.