Mysql advanced learning summary 11: Locate the slow execution of sql method, analyze the use of query statement EXPLAIN
1. Locate the SQL that executes slowly
Mysql's slow query log, used to record in mysqlResponse time exceeds thresholdstatement, specifically refers to the run time exceedslong_query_timeThe sql of the value will be recorded in the slow query.
By default, the mysql database does not enable the slow query log, you need to manually set this parameter.If it is not required for tuning, it is generally not recommended to enable this parameter, because turning on the slow query log will more or less have a certain performance impact.
1.1 Enable slow query log parameters
1. Enable slow_query_log
You can use the following command to check whether the slow query log is enabled:
mysql> show variables like '%slow_query_log';
+----------------+-------+
| Variable_name | Value |
+----------------+-------+
| slow_query_log | OFF |
+----------------+-------+
1 row in set (0.63 sec)
This parameter can be enabled:
mysql> set global slow_query_log = on;
Query OK, 0 rows affected (0.59 sec)
After enabling this parameter, check the variables related to this parameter:
you can see that slow_query_log has been enabled at this time, and the corresponding location of the file on the disk is also displayed.
mysql> show variables like '%slow_query_log%';
+---------------------+-----------------------------------+
| Variable_name | Value |
+---------------------+-----------------------------------+
| slow_query_log | ON |
| slow_query_log_file | /var/lib/mysql/koping-HP-slow.log |
+---------------------+-----------------------------------+
2 rows in set (0.00 sec)
2. Modify the long_query_time threshold
Next, check the time threshold setting of the slow query:
mysql> show variables like '%long_query_time%';
+-----------------+-----------+
| Variable_name | Value |
+-----------------+-----------+
| long_query_time | 10.000000 |
+-----------------+-----------+
1 row in set (0.00 sec)
The default is more than 10 seconds is slow query, here for the convenience of demonstration, shorten the time to 1 second:
global modification: set global long_query_time=1;
session modification: set long_query_time=1;
-- 全局修改
mysql> set global long_query_time=1;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%long_query_time%';
+-----------------+-----------+
| Variable_name | Value |
+-----------------+-----------+
| long_query_time | 10.000000 |
+-----------------+-----------+
1 row in set (0.00 sec)
-- 会话修改
mysql> set long_query_time=1;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%long_query_time%';
+-----------------+----------+
| Variable_name | Value |
+-----------------+----------+
| long_query_time | 1.000000 |
+-----------------+----------+
1 row in set (0.01 sec)
1.2 View the number of slow queries
Query how many slow query records are in the current system:
mysql> SHOW GLOBAL STATUS LIKE '%Slow_queries%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Slow_queries | 0 |
+---------------+-------+
1 row in set (0.00 sec)
1.3 Case demonstration
Here is a demonstration with the help of the student_info table created in 1.1 in the previous blog post ( Mysql advanced learning summary 10: 11 cases suitable for index creation and 7 cases not suitable for index creation ), which has 1 million pieces of data .
1.4 Testing and Analysis
Run a time-consuming sql statement:
mysql> SELECT student_id, COUNT(*) AS num FROM student_info GROUP BY student_id ORDER BY create_time DESC LIMIT 100;
100 rows in set (1.92 sec)
It can be seen that the sql statement exceeds 1 second, and the number of slow queries is checked again at this time, and it becomes 1 at this time.
mysql> SHOW GLOBAL STATUS LIKE '%Slow_queries%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| Slow_queries | 1 |
+---------------+-------+
1 row in set (0.00 sec)
1.5 Slow query log analysis tool: mysqldumpslow
After posting the slow query, you can analyze it through the slow query log analysis tool. First look at the parameters of the modification tool:
koping@koping-HP:~$ mysqldumpslow -h
Option h requires an argument
ERROR: bad option
Usage: mysqldumpslow [ OPTS... ] [ LOGS... ]
Parse and summarize the MySQL slow query log. Options are
--verbose verbose
--debug debug
--help write this text to standard output
-v verbose
-d debug
-s ORDER what to sort by (al, at, ar, c, l, r, t), 'at' is default
al: average lock time
ar: average rows sent
at: average query time
c: count
l: lock time
r: rows sent
t: query time
-r reverse the sort order (largest last instead of first)
-t NUM just show the top n queries
-a don't abstract all numbers to N and strings to 'S'
-n NUM abstract numbers with at least n digits within names
-g PATTERN grep: only consider stmts that include this string
-h HOSTNAME hostname of db server for *-slow.log filename (can be wildcard),
default is '*', i.e. match all
-i NAME name of server instance (if using mysql.server startup script)
-l don't subtract lock time from total time
Next, let’s check the top 5 slow query statements currently recorded in the slow query log. As you can see, we did see the slow query statement we just used in the test.
mysqldumpslow -a -s t -t 5 /var/lib/mysql/koping-HP-slow.log
root@koping-HP:~# mysqldumpslow -a -s t -t 5 /var/lib/mysql/koping-HP-slow.log
Reading mysql slow query log from /var/lib/mysql/koping-HP-slow.log
Count: 1 Time=1.93s (1s) Lock=0.00s (0s) Rows=100.0 (100), root[root]@localhost
SELECT student_id, COUNT(*) AS num FROM student_info GROUP BY student_id ORDER BY create_time DESC LIMIT 100
Died at /usr/bin/mysqldumpslow line 167, <> chunk 1.
1.6 Turn off the slow query log
1) Stop the slow query log function:
set global slow_query_log=off;
2. Analyze the query statement: EXPLAIN
2.1 Overview
After locating the sql with slow query, you can useEXPALINOr the DESCRIBE tool makes targeted analysis query statements.
2.2 Basic syntax
The syntax of the EXPALIN or DESCRIBE statement is as follows:
EXPALIN SELECT select_options
或者
DESCRIBE SELECT select_options
NOTE: The output of EXPALIN isImplementation plan, not actually executing the statement.
The functions of each column output by the EXPALIN statement are as follows:
2.3 Test data preparation
1) Create 2 tables
CREATE TABLE s1 (
id INT AUTO_INCREMENT,
key1 VARCHAR(100),
key2 INT,
key3 VARCHAR(100),
key_part1 VARCHAR(100),
key_part2 VARCHAR(100),
key_part3 VARCHAR(100),
common_field VARCHAR(100),
PRIMARY KEY (id),
INDEX idx_key1 (key1),
UNIQUE INDEX idx_key2 (key2),
INDEX idx_key3 (key3),
INDEX idx_key_part(key_part1, key_part2, key_part3)
) ENGINE=INNODB CHARSET=utf8;
CREATE TABLE s2 (
id INT AUTO_INCREMENT,
key1 VARCHAR(100),
key2 INT,
key3 VARCHAR(100),
key_part1 VARCHAR(100),
key_part2 VARCHAR(100),
key_part3 VARCHAR(100),
common_field VARCHAR(100),
PRIMARY KEY (id),
INDEX idx_key1 (key1),
UNIQUE INDEX idx_key2 (key2),
INDEX idx_key3 (key3),
INDEX idx_key_part(key_part1, key_part2, key_part3)
) ENGINE=INNODB CHARSET=utf8;
2) Create a stored function
DELIMITER //
CREATE FUNCTION rand_string1(n INT)
RETURNS VARCHAR(255) #该函数会返回一个字符串
BEGIN
DECLARE chars_str VARCHAR(100) DEFAULT 'abcdefghijklmnopqrstuvwxyzABCDEFJHIJKLMNOPQRSTUVWXYZ';
DECLARE return_str VARCHAR(255) DEFAULT '';
DECLARE i INT DEFAULT 0;
WHILE i < n DO
SET return_str =CONCAT(return_str,SUBSTRING(chars_str,FLOOR(1+RAND()*52),1));
SET i = i + 1;
END WHILE;
RETURN return_str;
END //
DELIMITER ;
3) Create 2 stored procedures
DELIMITER //
CREATE PROCEDURE insert_s1 (IN min_num INT (10),IN max_num INT (10))
BEGIN
DECLARE i INT DEFAULT 0;
SET autocommit = 0;
REPEAT
SET i = i + 1;
INSERT INTO s1 VALUES(
(min_num + i),
rand_string1(6),
(min_num + 30 * i + 5),
rand_string1(6),
rand_string1(10),
rand_string1(5),
rand_string1(10),
rand_string1(10));
UNTIL i = max_num
END REPEAT;
COMMIT;
END //
DELIMITER ;
DELIMITER //
CREATE PROCEDURE insert_s2 (IN min_num INT (10),IN max_num INT (10))
BEGIN
DECLARE i INT DEFAULT 0;
SET autocommit = 0;
REPEAT
SET i = i + 1;
INSERT INTO s2 VALUES(
(min_num + i),
rand_string1(6),
(min_num + 30 * i + 5),
rand_string1(6),
rand_string1(10),
rand_string1(5),
rand_string1(10),
rand_string1(10));
UNTIL i = max_num
END REPEAT;
COMMIT;
END //
DELIMITER ;
4) #Call the stored procedure and insert 10,000 pieces of data into the two tables respectively
mysql> CALL insert_s1(10001,10000);
Query OK, 0 rows affected (9.64 sec)
mysql> CALL insert_s2(10001,10000);
Query OK, 0 rows affected (11.64 sec)
5) Check whether the test data is successfully viewed
mysql> SELECT COUNT(*) FROM s1;
+----------+
| COUNT(*) |
+----------+
| 10000 |
+----------+
1 row in set (0.00 sec)
mysql> SELECT COUNT(*) FROM s2;
+----------+
| COUNT(*) |
+----------+
| 10000 |
+----------+
1 row in set (0.00 sec)
2.4 Function of each column of EXPLAIN
2.4.1 table
Each row of the query corresponds to a single table
mysql> EXPLAIN SELECT * FROM s1;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
| 1 | SIMPLE | s1 | NULL | ALL | NULL | NULL | NULL | NULL | 9895 | 100.00 | NULL |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------+
1 row in set, 1 warning (0.00 sec)
mysql> EXPLAIN SELECT * FROM s1 INNER JOIN s2;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
| 1 | SIMPLE | s1 | NULL | ALL | NULL | NULL | NULL | NULL | 9895 | 100.00 | NULL |
| 1 | SIMPLE | s2 | NULL | ALL | NULL | NULL | NULL | NULL | 9895 | 100.00 | Using join buffer (Block Nested Loop) |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+---------------------------------------+
2 rows in set, 1 warning (0.00 sec)
2.4.2 id
In a large query statement, each SELECT keyword corresponds to a unique id
For example, there are two select query statements below, then there are two ids.
mysql> EXPLAIN SELECT * FROM s1 WHERE key1 IN (SELECT key1 FROM s2) OR key3='a';
+----+-------------+-------+------------+-------+---------------+----------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+----------+---------+------+------+----------+-------------+
| 1 | PRIMARY | s1 | NULL | ALL | idx_key3 | NULL | NULL | NULL | 9895 | 100.00 | Using where |
| 2 | SUBQUERY | s2 | NULL | index | idx_key1 | idx_key1 | 303 | NULL | 9895 | 100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+----------+---------+------+------+----------+-------------+
2 rows in set, 1 warning (0.00 sec)
However, it should be noted that the query optimizer may rewrite query statements involving subqueries .
For example, in the following statement, although there are two selects, the query optimizer has optimized it, so the result is still only one id:
mysql> EXPLAIN SELECT * FROM s1 WHERE key1 IN (SELECT key2 FROM s2 WHERE common_field='a');
+----+-------------+-------+------------+------+---------------+----------+---------+-----------------+------+----------+------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+----------+---------+-----------------+------+----------+------------------------------------+
| 1 | SIMPLE | s1 | NULL | ALL | idx_key1 | NULL | NULL | NULL | 9895 | 100.00 | Using where |
| 1 | SIMPLE | s2 | NULL | ref | idx_key2 | idx_key2 | 5 | dbtest1.s1.key1 | 1 | 10.00 | Using index condition; Using where |
+----+-------------+-------+------------+------+---------------+----------+---------+-----------------+------+----------+------------------------------------+
2 rows in set, 2 warnings (0.00 sec)
Summary id:
- If the ids are the same, they can be considered as a group and executed sequentially from top to bottom
- In all groups, the larger the id value, the higher the priority, and the earlier the execution
- Concern: Each number of the id number represents an independent query , and the fewer queries of a sql, the better
2.4.3 select_type
Mysql defines an attribute called select_type for each small query represented by the select keyword , which means that as long as we know the select_type attribute of a small query , we know that this small query plays a role in the entire large query. what role . Let's see what values select_type can take:
2.4.4 partitions
Represents the hit situation in the partition table, and the non-partition table, this item is NULL. In general, the value of the partitions column of the execution plan of the query statement is NULL
2.4.5 type
A record of the execution plan represents the access method of mysql when executing a query on a certain table, and the type column indicates what the access method is.
The resulting values from best to worst are:
system > const > eq_ref > ref > fulltext > ref_or_null > index_merge > unique_subquery > index_subquery > range > index > All
The goal of SQL performance optimization: at least reach the range level, the requirement is the ref level, preferably the consts level. (Required by Alibaba Development Manual)
2.4.6 possible_keys和key
The possible_keys column indicates which indexes may be used when performing a single-table query on a certain table in a certain query statement.
The key column indicates which indexes are actually used. If it is NULL, no index is used.
2.4.7 key_len
The actual index length (that is, the number of bytes) used
2.4.8 ref
When using the index column equivalence query, the object information for equivalence matching with the index column.
This field is viewed together with the type field for corresponding matching.
2.4.9 rows
The estimated record tree that needs to be read, the smaller the value, the better.
2.4.10 filtered
The percentage of the number of remaining records after a table is filtered by the search condition.
For single-table queries, this column has no meaning. The filtered value of the execution plan record corresponding to the driving table in the connection query determines the number of times the driven table will be executed: rows * filtered
2.4.11 Extra
Extra is used to illustrate some additional information, which can provide how mysql executes a given query statement.