MySQL combat: explain in detail (on)

How to analyze slow sql?

At work, the most common thing we use to capture performance problems is to open the slow query log and locate the SQL with poor execution efficiency. Then when we locate an SQL, it is not finished, we also need to know the execution plan of the SQL, such as Whether it is a full table scan or an index scan, these need to be done through EXPLAIN. The EXPLAIN command is the primary way to see how the optimizer decides to execute a query.

It is important to note that the resulting QEP is not deterministic and may vary based on many factors. MySQL does not bind a QEP to a given query, the QEP will be determined by the actual conditions of each execution of the SQL statement, even when using stored procedures. Although the SQL statements are pre-parsed in the stored procedure, the QEP is still determined each time the stored procedure is called. (QEP: sql generates an execution plan query Execution plan)

Before formally introducing the use of explain, we need to know what are the access methods of a single table?

single table access method

We first build a single_table table to facilitate the demonstration of the following results

CREATE TABLE single_table (
    id INT NOT NULL AUTO_INCREMENT,
    key1 VARCHAR(100),
    key2 INT,
    key3 VARCHAR(100),
    key_part1 VARCHAR(100),
    key_part2 VARCHAR(100),
    key_part3 VARCHAR(100),
    common_field VARCHAR(100),
    PRIMARY KEY (id),
    KEY idx_key1 (key1),
    UNIQUE KEY idx_key2 (key2),
    KEY idx_key3 (key3),
    KEY idx_key_part(key_part1, key_part2, key_part3)
) Engine=InnoDB CHARSET=utf8;

const

Locate a record by equivalence comparison of the primary key or unique secondary index with a constant value

For example the following statement

-- 通过主键和常数值进行比较
select * from single_table where id = 400;
-- 通过唯一二级索引和常数值进行比较
select * from single_table where key2 = 100;

Note : If the primary key or the index column of a unique secondary index consists of multiple columns, this const access method will only work if each item in the index column is compared for equality with a constant (because that is the only way to guarantee At most one record is eligible)

select * from single_table where key2 is null

When the above statement is executed, the access method is not const, because the unique secondary index does not limit the number of null values, so the above statement may access multiple records. So what is the access method? Then look down

ref

Equally compares a common secondary index to a constant

select * from single_table where key1 = 'abc'

For ordinary secondary indexes, multiple consecutive secondary index records may be matched after equal value comparison through index columns, unlike the primary key or unique secondary index, which can only match at most one record, so ref Access methods are worse than const.

In addition, the following two situations should be noted

1. Whether it is an ordinary secondary index or a unique secondary index, the number of null values ​​contained in the index column is not limited, so the search condition in the form of key is null can only use the ref access method at most, not the const access method. method
2. Equivalent queries that satisfy the leftmost prefix principle may use the ref access method

For example the following sentences

select * from single_table where key_part1 = 'a';

select * from single_table where key_part1 = 'a' and key_part2 = 'b';

select * from single_table where key_part1 = 'a' and key_part2 = 'b' AND key_part3 = 'c'; 

If the index columns are not all equivalent queries, the access method is not ref, but range

select * from single_table where key_part1 = 'a' AND key_part2 > 'b';

ref_or_null

At the same time, find the records whose value of a secondary index column is equal to a certain constant value, and also find the records whose value is null in this column.

select * from single_table where key1 = 'abc' or key1 is null

range

When using an index to perform a query, the corresponding scan interval is several single-point scan intervals or range scans

select * from single_table where key2 in (11, 12) or (key2 >= 30)

The scan interval of the above sql is [11, 11], [12, 12], and [30,+∞)

An access method whose scan interval is (-∞, +∞) cannot be called a range

index

select key_part1, key_part2, key_part3 from single_table where key_part2 = 'abc'

It can be seen that key_part2 is not the leftmost column of the joint index, so the ref access method cannot be used to execute this statement. But it has the following two characteristics

1. The columns of the query are key_part1, key_part2, key_part3. And the index idx_key_part contains the column values ​​of these 3 columns
2. The search condition has only key_part2 column, and this column is also included in idx_key_part

At this point, we can directly traverse all the records in the idx_key_part index, determine the value of key_part2, and return the values ​​of key_part1, key_part2, and key_part3. At this time, the scanning interval is (-∞, +∞)

The cost of scanning all secondary index records is much less than directly scanning all clustered index records (because the leaf nodes of the clustered index need to store all columns and hidden columns, and the secondary only needs to store the column value and primary key value of the index column , so the tree height may be lower), this method is index

In addition, when the statement adds the order by primary key, the access method is also index

So when the query meets the following conditions, the access method is index

1. Scan all secondary index records
2. Added order by primary key statement

all

Full table scan, that is, directly scan all clustered index records

select * from single_table

Use of explain

The usage of explain is very simple, just add explain before the executed select statement

explain select * from t1

Types of	describe
id	In a large query statement, each select keyword corresponds to a unique id
select_type	The query type corresponding to the select keyword
table	surface
type	Access method for single table
possible_keys	What are the potential indexes that can be used when querying against a table
key	actually used index
key_len	The actual length of the index used
ref	references between tables
rows	Estimated number of result records
filtered
Extra	additional information

The following is a detailed analysis of the meaning of each column value

table

We first construct two tables that are exactly the same as the single_table table, named s1 table and s2 table. There are 10,000 records in each of these two tables, and random values ​​are inserted in all columns except the id column.

No matter how complex our query is, and how many tables it contains, in the end it will be a single table access. Each row in the output of the explain statement corresponds to the access method of a single table, and the table column is the table name of the table

explain select * from t1 inner join t2

id

Create the following 3 tables

course table

CREATE TABLE `course` (
  `cid` int(3) NOT NULL,
  `cname` varchar(20) NOT NULL,
  `tid` int(3) NOT NULL,
  PRIMARY KEY (`cid`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

teacher table

CREATE TABLE `teacher` (
  `tid` int(3) NOT NULL,
  `tname` varchar(20) NOT NULL,
  `tcid` int(3) NOT NULL,
  PRIMARY KEY (`tid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

teacher_card表

CREATE TABLE `teacher_card` (
  `tcid` int(3) NOT NULL,
  `tcdesc` varchar(20) NOT NULL,
  PRIMARY KEY (`tcid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Each select keyword in a query is assigned a unique id value. For join queries, the from clause after a select keyword can be followed by multiple tables. Therefore, in the execution plan of the join query, each table corresponds to a record, but the id values ​​of these records are the same

Let's query the information of the teacher whose course number is 2 or whose teacher's certificate number is 3

SELECT
	t.* 
FROM
	course c,
	teacher t,
	teacher_card tc 
WHERE
	c.tid = t.tid 
	AND t.tcid = tc.tcid 
	AND ( c.cid = 2 OR t.tcid = 3 )

After explaining the above SQL as shown below

The id values ​​are the same and are executed in order from top to bottom . The one that appears in the front is the driving table, and the one that appears in the back is the driven table

Query the description of a teacher who teaches a SQL course

SELECT
	tc.tcdesc 
FROM
	teacher_card tc 
WHERE
	tc.tcid = (
SELECT
	t.tcid 
FROM
	teacher t 
WHERE
	t.tid = ( SELECT c.tid FROM course c WHERE c.cname = "sql" ) 
	);

This SQL is to first query the c table, then query the t table, and finally query the tc table, execute explain to see

The id value is different, the larger the id value, the higher the priority . Change the above SQL to the following form

SELECT
	tc.tcdesc 
FROM
	teacher t,
	teacher_card tc 
WHERE
	tc.tcid = t.tcid 
	AND t.tid = ( SELECT c.tid FROM course c WHERE c.cname = "sql" )

The id values ​​are the same and different. The larger the id value, the higher the priority. The id values ​​are the same, and are executed in order from top to bottom.

select_type

name	describe
simple	The query does not contain unions or subqueries
primary	For large queries containing union union all or subqueries, it consists of several subqueries, and the select type value of the leftmost query is primary
union	For a large query containing union or union all, it consists of several small queries. Except for the leftmost small query, the select type value of the remaining small queries is union
union result	MySQL uses a temporary table to complete the deduplication of the union. The select type of the query against the temporary table is union result
subquery	The query statement containing the subquery cannot be converted into the corresponding semi-join form, and the query is not a correlated subquery. When the query optimizer decides to use the materialized scheme of the subquery to execute the subquery, the first select of the subquery The select_type corresponding to the keyword is subquery
dependent subquery	A query statement containing a subquery cannot be converted to the corresponding semi-join form, and the query is a correlated subquery, and the select_type corresponding to the first select keyword of the subquery is dependent subquery
dependent union	In a large query containing union or union all, if each small query depends on the outer query, except for the leftmost small query, the rest of the select types are dependent union
derived	Execute a query containing a derived table in a materialized manner, and the select_type of the self-query corresponding to the derived table is derived
materialized	When the query optimizer executes a statement containing a subquery, and chooses to materialize the subquery to join the query with the outer query, the select_type corresponding to the subquery is materialized

SIMPLE : The query does not contain unions or subqueries

explain select * from t1

explain select * from t1 inner join t2

PRIMARY : For large queries containing union , union all or sub-queries , it consists of several sub-queries, and the select type value of the leftmost query is primary

explain select * from t1 union select * from t2

UNION : For a large query containing union or union all, it consists of several small queries. Except for the leftmost small query, the select type value of the remaining small queries is union

UNION RESULT : MySQL uses a temporary table to complete the deduplication of union, and the select type of the query against the temporary table is union result

SUBQUERY : The query statement containing the subquery cannot be converted into the corresponding semi-join form, and the query is not a correlated subquery. When the query optimizer decides to execute the subquery using the materialized scheme of the subquery, the The select_type corresponding to each select keyword is subquery.

One thing to note is that since the subquery whose select_type is SUBQUERY will be materialized, it only needs to be executed once

explain select * from t1 where key1 in (select key1 from t2)

DEPENDENT SUBQUERY : The query statement containing the subquery cannot be converted into the corresponding semi-join form, and the query is a correlated subquery, and the select_type corresponding to the first select keyword of the subquery is dependent subquery

DEPENDENT UNION : In a large query containing union or union all, if each small query depends on the outer query, except the leftmost small query, the rest of the select types are dependent union

DERIVED : In a query containing a derived table (cr is a derived table in the following example), if the query is executed in the form of a materialized derived table, the select_type of the subquery corresponding to the derived table is DERIVED

SELECT
	cr.cname 
FROM
	( SELECT * FROM course WHERE tid IN ( 1, 2 ) ) cr

The table name of the table with id 1 is <derived2>, indicating that the query is for the table after materialization of the derived table, and the derived table is obtained from the execution process of the id 2.

MATERIALIZED : When the query optimizer executes a statement containing a subquery and chooses to materialize the subquery to join the query with the outer query, the select_type corresponding to the subquery is materialized

type

A record of the execution plan represents the access method when mysql executes a query on a table, and the type indicates the access method to the table ?

We have only introduced some methods of table access in the InnoDB engine. The complete access method is as follows

name	describe
system	There is only one record in the table and the statistics of the storage engine used by the table are accurate, the derived table is a subquery with only one data
const	When equal matching with constants based on primary key or unique secondary index
eq_ref	During join query, the driven table is accessed by means of equal value matching through the primary key or unique secondary index column
ref	Ordinary secondary index and constant value for equal value matching
ref_or_null	Ordinary secondary index performs equal value matching, when the index column value can be null
fulltext	full text index, skip
index_merge	Query the table using index merge
unique_subquery	In a statement containing an in self-query, if the query optimizer decides to convert the in sub-query into an exists self-query, and the sub-query can use the primary key for equivalent matching
index_subquery	index_subquery is similar to unique_subquery, except that ordinary indexes are used to access the tables in the subquery
range	Use index column to get records of range interval
index	Perform a full index scan on the secondary index
all	Full table scan of clustered index

The commonly used execution efficiency is as follows

const，system>eq_ref>ref>range>index>all

system：

1. There is only one record in the table and the statistics of the storage engine used by the table are accurate, such as MyISAM, Memory (all when there is only one data in the table and the InnoDB engine)
2. The derived table is a subquery with only one item of data

SELECT
	a.tname 
FROM
	( SELECT * FROM teacher t WHERE t.tid = 1 ) a

const : when equal-value matching is performed with a constant based on the primary key or unique secondary index

explain select * from t1 where id = 5

eq_ref：

explain select * from t1 inner join t2 on t1.id = t2.id

ref : non-unique index, for each index key query, returns matching all rows (0, many)

Modify the table to be as follows (it is only temporarily changed for this example), a teacher with the same name Zhang San appears, and a common index is added to the name column of the teacher table to demonstrate the situation where there are multiple matching rows

teacher表

teacher_card表

SELECT
	* 
FROM
	teacher 
WHERE
	tname = "张三"

range : Retrieve the rows of the specified range, where is followed by a range query (between, >, <, >=, in sometimes fails, thus turning into no index ALL)

explain select * from t1 where key1 in ('a', 'b', 'c')

or

explain select * from t1 where key1 > 'a' and key1 < 'b'

index : Index coverage can be used, but all index records need to be scanned

explain select key_part2 from t1 where key_part3 = 'a'

all : full table scan

explain select * from t1

Reference blog

[1]https://www.cnblogs.com/gomysql/p/3720123.html
[2]https://blog.csdn.net/lijn_huo/article/details/52442675
[3]http://blog.jobbole .com/100349/
Slow query log
[4]https://mp.weixin.qq.com/s/_SWewX-8nFam20Wcg6No1Q