navigation:
Table of contents
1. Optimization steps of the database server
2. View system performance parameters
2.1 SHOW STATUS LIKE 'parameters'
2.2 Check the query cost of SQL
3. Locate the SQL that executes slowly: slow query log
3.1 Enable slow query log parameters
3.2 View the number of slow queries
3.5 Slow query log analysis tool: mysqldumpslow
3.6 Turn off the slow query log
4. Locate slow query statements and view SQL execution costs: show profile
5.3 Introduction to Execution Schedule
5.4 EXPLAIN four output formats
6. Analyze the optimizer execution plan: trace
7. MySQL monitoring analysis view-sys schema
1. Optimization steps of the database server
In database tuning, our goal is faster response time and greater throughput . Using macro monitoring tools and micro log analysis can help us quickly find ideas and methods for tuning.
Tuning process:
- SHOW STATUS observes the server status, whether there are periodic fluctuations; if there is, optimize the cache ;
- If there is still irregular delay or lag, start slow query and explain query statement ;
- If you find that the sql wait time is long, tune the server parameters ; if you find that the sql execution time is long, optimize the index and table;
- If there are still irregular delays or freezes, observe whether the SQL query has reached the bottleneck; if so, separate reading and writing, and divide databases and tables.
Three analysis tools (three steps for SQL tuning): slow query, EXPLAN, SHOW PROFLING
The whole process is divided into two parts: observation (Show status) and action (Action). The part with the letter S stands for observation (the corresponding analysis tool will be used), and the part with the letter A is action (the action that can be taken according to the analysis).
2. View system performance parameters
2.1 SHOW STATUS LIKE 'parameters'
In MySQL, you can use the SHOW STATUS statement to query the performance parameters and execution frequency of some MySQL database servers .
The syntax of the SHOW STATUS statement is as follows:
SHOW [GLOBAL|SESSION] STATUS LIKE '参数';For example, check the number of database connections and running time:
Brackets mean that it can be omitted.
Some commonly used performance parameters are as follows:
• Connections: The number of connections to the MySQL server.
• Uptime: The online time of the MySQL server. It will reset after restarting the server.
• Slow_queries: The number of slow queries . The query duration exceeds the specified time, and the fewer times the better.
• Innodb_rows_read: the number of rows returned by the Select query
• Innodb_rows_inserted: the number of rows inserted by the INSERT operation
• Innodb_rows_updated: the number of rows updated by the UPDATE operation
• Innodb_rows_deleted: the number of rows deleted by the DELETE operation
• Com_select: The number of query operations.
• Com_insert: The number of insert operations. For batch insert INSERT operations, only accumulate once.
• Com_update: The number of update operations.
• Com_delete: The number of delete operations.
• last_query_cost: The cost of a query on the query optimizer , and the number of data pages used in the latest deletion.
2.2 Check the query cost of SQL
SHOW STATUS LIKE 'last_query_cost';
SQL query is a dynamic process, from the point of view of page load:
1. Buffer pool query efficiency is better than disk query
If the page is in the database buffer pool , then the efficiency is the highest , otherwise it needs to be read from memory or disk. Of course, for a single page read, if the page exists in memory, it will be faster than reading from disk. The extraction efficiency is much higher.
MySQL's buffer pool is divided into several different buffer pools, including:
- Query cache: used to cache query results.
- InnoDB cache pool: used to cache hot tables and index data pages.
- MyISAM cache pool: used to cache table data blocks.
When a lot of data has been stored in the buffer pool, MySQL will use a method called the buffer pool replacement algorithm to replace part of the cached data to make room for new data to be cached.
MySQL's buffer pool uses the LRU (least recently used) algorithm , which caches the most recently used data first. When the space in the buffer pool is insufficient, MySQL will replace the least commonly used data from the buffer pool to make room for new data.
2. Batch sequential query is higher on average per page query
If we randomly read a single page from the disk, the efficiency is very low (about 10ms), but if we use sequential reading to read pages in batches , the average reading efficiency of one page will be greatly improved. Even faster than a random read of a single page in memory.
So, don’t worry about encountering IO. If you find the right method, the efficiency is still very high. We must first consider the location of data storage. If the data is frequently used, we should put it in the buffer pool . Secondly, we can make full use of the throughput capacity of the disk to read data in batches at one time , so that the reading efficiency of a single page is also low. Got a boost.
The test buffer pool caches the used tables and indexes in the memory, which is highly efficient: the query cost of querying 900001 and 900001~9000100 is much different, and the query speed is similar
Example student information form:
CREATE TABLE `student_info` ( `id` INT(11) NOT NULL AUTO_INCREMENT, `student_id` INT NOT NULL , `name` VARCHAR(20) DEFAULT NULL, `course_id` INT NOT NULL , `class_id` INT(11) DEFAULT NULL, `create_time` DATETIME DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`id`) ) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;If we want to query the record with id=900001 , and then look at the query cost, we can search directly on the clustered index:
SELECT student_id, class_id, NAME, create_time FROM student_info WHERE id = 900001;Running result (1 record, running time is 0.042s)
Then look at the cost of the query optimizer , in fact we only need to retrieve one page :
SHOW STATUS LIKE 'last_query_cost';+-----------------+----------+ | Variable_name | Value | +-----------------+----------+ | Last_query_cost | 1.000000 | +-----------------+----------+What if we want to query the student records with id between 900001 and 9000100 ?
SELECT student_id, class_id, NAME, create_time FROM student_info WHERE id BETWEEN 900001 AND 900100;Running result (100 records, running time is 0.046s):
Then look at the cost of the query optimizer . At this time, we need to query about 20 pages.
mysql> SHOW STATUS LIKE 'last_query_cost'; +-----------------+-----------+ | Variable_name | Value | +-----------------+-----------+ | Last_query_cost | 21.134453 | +-----------------+-----------+You can see that the number of pages is 20 times that of just now, but the efficiency of the query has not changed significantly . In fact, the time of the two SQL queries is basically the same, because the pages are loaded at one time by sequential reading. to the buffer pool before doing the lookup. Although the number of pages (last_query_cost) has increased a lot , the query time has not been increased much through the mechanism of the buffer pool.
Why is it checked directly from the buffer pool for the second time?
Because the mysql cache elimination strategy lru is the least recently used, the most recent query data will be cached first, and the least recently used data will be eliminated first.
Usage scenario: It is very useful for comparing costs, especially when we have several query methods to choose from.
3. Locate the SQL that executes slowly: slow query log
3.0 Introduction
MySQL's slow query log is used to record the statements whose response time exceeds the threshold in MySQL , specifically refers to the SQL whose running time exceeds the long-query_time value , and will be recorded in the slow query log. The default value of long_query_time is 10, which means that statements that run for more than 10 seconds (excluding 10 seconds) are considered to exceed our maximum tolerance time value.
Its main role is to help us find those SOL queries that take a long time to execute, and optimize them in a targeted manner, thereby improving the overall efficiency of the system. When our database server is blocked and running slowly, check the slow query log and find those slow queries, which is very helpful to solve the problem. For example, if a sqq executes for more than 5 seconds, we will consider it as slow SQL. We hope to collect the SQL for more than 5 seconds and combine it with explain for comprehensive analysis.
By default, the MySQL database does not enable the slow query log, and we need to manually set this parameter. If it is not necessary for tuning, it is generally not recommended to enable this parameter , because enabling the slow query log will more or less bring about a certain performance impact
. The slow query log supports writing log records to files
3.1 Enable slow query log parameters
0. Whether slow query is enabled
show variables like '%slow_query_log';
Slow query log location:
show variables like '%slow_query_log%';
Slow query threshold:
1. Enable slow query log slow_query_log
set global slow_query_log='ON';Then let's check whether the slow query log is enabled and the location of the slow query log file:
You can see that the slow query analysis has been enabled at this time, and the file is saved in the /var/lib/mysql/atguigu02-slow.log file
middle.
2. Modify the slow query threshold long_query_time
View the time threshold for slow queries:
show variables like '%long_query_time%';Check the global slow query time threshold:
show global variables like '%long_query_time%';
Temporarily modify the time threshold for slow queries:
current session:
set long_query_time = 1; global:
set global long_query_time = 1; For the "global" option, a configuration parameter at the global level. It can be set in the MySQL configuration file at MySQL server startup or MySQL installation, or changed at runtime with the SET GLOBAL command. Global level configuration parameters are valid for all MySQL connections.
Permanent modification (still valid after restarting the database, permanent modification is not recommended , only opened during optimization, slow query performance):
Modify my.cnf
[mysqld]
slow_query_log=ON # 开启慢查询日志的开关
slow_query_log_file=/var/lib/mysql/atguigu-slow.log #慢查询日志的目录和文件名信息
long_query_time=3 #设置慢查询的闽值为3秒，超出此设定值的SQL即被记录到慢查询日志
log_output=FILENotice:
#测试发现：设置global的方式对当前session的long_query_time失效。对新连接的客户端有效。所以可以一并 执行下述语句 set global long_query_time = 1; #设置全局慢查询阈值1s show global variables like '%long_query_time%'; #全局1s set long_query_time=1; show variables like '%long_query_time%'; #当前会话10s
3.2 View the number of slow queries
Query how many slow query records are in the current system
SHOW GLOBAL STATUS LIKE '%Slow_queries%';3.5 Slow query log analysis tool: mysqldumpslow
`mysqldumpslow` is a command-line tool for analyzing MySQL slow query logs. By analyzing the slow query log, you can understand the performance problems of the database and optimize it.
View the help information of mysqldumpslow
mysqldumpslow --help
The specific parameters of the mysqldumpslow command are as follows:
-a: Do not abstract numbers into N, abstract strings into S
-s: indicates which way to sort:
- c: number of visits
- l: lock time
- r: return record
- t: query time
- al: average lock time
- ar: the average number of records returned
- at: average query time (default mode)
- ac: average number of queries
-t: is to return the previous number of data;
-g: followed by a regular matching pattern, case insensitive;
case:
Example: sort by query time, check the first five slow query SQL statements, just write like this:
mysqldumpslow -s t -t 5 /var/lib/mysql/atguigu01-slow.log
For example, suppose you want to view the slowest 10 queries, you can use the following command:
mysqldumpslow -s t -t 10 /var/log/mysql/mysql-slow.logThis command means sorting by time and displaying the 10 slowest queries. /var/log/mysql/mysql-slow.log is the path of the MySQL slow query log.
In addition, if you need to filter specific queries, you can use the `-g` parameter, such as:
mysqldumpslow -s t -t 10 -g "SELECT * FROM user" /var/log/mysql/mysql-slow.logThis command is sorted by time and displays the slowest 10 queries, where the keyword is "SELECT * FROM user".
In general, the `mysqldumpslow` command provides a simple and quick way to help developers and DBAs analyze MySQL slow query problems and optimize database performance.
Other common use cases:
#得到返回记录集最多的10个SQL
mysqldumpslow -s r -t 10 /var/lib/mysql/atguigu-slow.log
#得到访问次数最多的10个SQL
mysqldumpslow -s c -t 10 /var/lib/mysql/atguigu-slow.log
#得到按照时间排序的前10条里面含有左连接的查询语句
mysqldumpslow -s t -t 10 -g "left join" /var/lib/mysql/atguigu-slow.log
#另外建议在使用这些命令时结合 | 和more 使用 ，否则有可能出现爆屏情况
mysqldumpslow -s r -t 10 /var/lib/mysql/atguigu-slow.log | more
3.6 Turn off the slow query log
There are two ways to stop the slow query log function of the MySQL server:
Way 1: Permanent way
[mysqld]
slow_query_log=OFF
Mysql closes the slow query log by default , or comment out or delete the slow_query_log item
[mysqld]
#slow_query_log =OFF
Restart the MySQL service, and execute the following statement to query the slow log function.
SHOW VARIABLES LIKE '%slow%'; #查询慢查询日志所在目录
SHOW VARIABLES LIKE '%long_query_time%'; #查询超时时长
Method 2: Temporary method
Use the SET statement to set.
Stop the MySQL slow query log function, the specific SQL statement is as follows.
SET GLOBAL slow_query_log=off;
golbal is valid globally.
Restart the MySQL service , use the SHOW statement to query the slow query log function information, and find that the slow query log has been closed successfully.
SHOW VARIABLES LIKE '%slow%'; #发现关闭成功 #慢查询阈值 SHOW VARIABLES LIKE '%long_query_time%'; #10s。前面改的时候没有加global，所以重启服务器后阈值恢复10s。
3.7 Delete slow query logs
delete manually
Use the SHOW statement to display slow query log information. The specific SQL statement is as follows.
SHOW VARIABLES LIKE `slow_query_log%`;
It can be seen from the execution results that the slow query log directory defaults to the MySQL data directory, and you can manually delete the slow query log files in this directory.
automatically delete
Use the command mysqladmin flush-logs to regenerate the query log file. After execution, the slow query log file will be regenerated in the data directory.
Regenerate slow query log files (delete old ones directly)
mysqladmin -uroot -p flush-logs slow
hint
Slow query logs are deleted and rebuilt using the mysqladmin flush-logs command . When using it, it must be noted that once this command is executed, the slow query log will only exist in the new log file. If the old query log is needed, it must be backed up in advance.
4. Locate slow query statements and view SQL execution costs: show profile
show profile is a resource consumption tool provided by MySQL that can be used to analyze what SQL has done in the current session and execute it. It can be used for the measurement of SQL tuning. It is off by default and saves the results of the last 15 runs.
SHOW PROFILE is a MySQL command for viewing profiling information for queries executed by a session. It can help developers and DBAs analyze bottlenecks in query execution and find out which parts need to be optimized.
Check whether the profile is enabled in the configuration:
mysql > show variables like 'profiling';
- SHOW VARIABLES shows the current configuration variables of the MySQL server, including global configuration variables and session configuration variables, and their values. SHOW VARIABLES is used to view the detailed information of MySQL configuration system parameters and modify the system parameters.
- SHOW STATUS displays server performance parameters, including status information on connections, threads, queries, etc., and their values.
Open show profile:
mysql > set profiling = 'ON';
Then execute the relevant query statement:
select * from employeesshow profiles; Query the duration of all query statements in the current session
mysql > show profiles;
show profile; Query the execution cost of the latest SQL statement in the current session:
You can see that the current session has a total of 2 queries. If we want to see the cost of the most recent query, we can use:
mysql > show profile;
show profile cpu for 2; query the cpu information of the specified QueryID:
You can view the overhead of the specified QueryID, such as show profile for query 2. In SHOW PROFILE, you can view the overhead of different parts, such as cpu, block.io, etc.:
mysql> show profile cpu,block io for query 2
Common query parameters for show profile:
① ALL: Display all overhead information.
② BLOCK IO: Display block IO overhead.
③ CONTEXT SWITCHES: Context switching overhead.
④ CPU: Display CPU overhead information.
⑤ IPC: Display sending and receiving overhead information.
⑥ MEMORY: Display memory overhead information.
⑦ PAGE FAULTS: Display page fault overhead information.
⑧ SOURCE: Display overhead information related to Source_function, Source_file, Source_line.
⑨ SWAPS: Display swap times overhead information.
Pay attention to daily development:
① converting HEAP to MyISAM: The query result is too large, the memory is not enough, and the data is moved to the disk.
② Creating tmp table: Create a temporary table. Copy the data to the temporary table first, and then delete the temporary table after use.
③ Copying to tmp table on disk: Copy the temporary table in the memory to the disk, be careful!
④ locked。
If any of the above four results appear in the show profile diagnosis result, the SQL statement needs to be optimized.
Notice:
However, the SHOW PROFILE command will be deprecated , and we can view it from the profiling data table in information_schema .
5. Execution plan: EXPLAIN
5.1 Introduction
MySQL's EXPLAIN is a tool for analyzing the query performance of SQL statements . When we execute a SELECT statement in MySQL, EXPLAIN can help us see how MySQL executes the query, that is, the execution plan , including which indexes to use, which tables to select, and how to read data . By analyzing the output of EXPLAIN, we can better optimize query statements and improve query efficiency.
The use of EXPLAIN is very simple, just add the EXPLAIN keyword in front of the SELECT statement , for example:
EXPLAIN SELECT * FROM my_table WHERE my_column = 'my_value';After executing the above command, MySQL will return a query execution plan table, which contains the detailed information of MySQL executing this query. We can understand query performance bottlenecks by analyzing query execution plan tables , and how to optimize query statements to improve query performance.
Notice:
-
EXPLAIN does not consider various Cache
-
EXPLAIN cannot display the optimization work done by MySQL when executing queries
-
EXPLAIN will not tell you about triggers, stored procedures, or how user-defined functions affect queries
-
Some statistics are estimated and not exact
After using the show profile to locate the slow query SQL in the previous step , we can use the EXPLAIN or DESCRIBE tool to analyze the query statement . The DESCRIBE statement is used in the same way as the EXPLAIN statement, and the analysis results are the same.
There is an optimizer module in MySQL that is responsible for optimizing the SELECT statement . Its main function is to provide the Query requested by the client with the optimal execution plan (he considers the optimal data retrieval method) by calculating and analyzing the statistical information collected in the system. , but not necessarily the DBA (database administrator) thinks it is optimal, this part is the most time-consuming).
This execution plan shows how to execute the query next , such as the order of multi-table joins, what access method is used for each table to execute the query, and so on. MySOL provides us with the EXPLAIN statement to help us view the specific execution plan of a certain query statement. If you understand the output items of the EXPLAIN statement, you can improve the performance of our query statement in a targeted manner.
1. What can be done?
- table read order
- The operation type of the data read operation
- Which indexes can be used
- which indexes are actually used
- References between tables
- How many rows per table are queried by the optimizer
2. Introduction to the official website
MySQL :: MySQL 5.7 Reference Manual :: 8.8.2 EXPLAIN Output Format
MySQL :: MySQL 8.0 Reference Manual :: 8.8.2 EXPLAIN Output Format
3. Version status
- Before MySQL 5.6.3, only EXPLAIN SELECT can be used ; after MYSQL 5.6.3, EXPLAIN SELECT, UPDATE, DELETE can be used
- In versions before 5.7, you need to use the explain partitions command to display partitions; you need to use the explain extended command to display filtered. After version 5.7, the default explain directly displays the information in partitions and filtered.
5.2 Basic syntax
The syntax of the EXPLAIN or DESCRIBE statement is as follows:
EXPLAIN SELECT select_options
#或者
DESCRIBE SELECT select_options
If we want to see the execution plan of a certain query, we can add an EXPLAIN before the specific query statement, like this:
EXPLAIN SELECT * FROM course_base;
The output of the above information is the so-called execution plan. With the help of this execution plan, we need to know how to improve our query statement to make the query execute more efficiently. In fact, in addition to the query statement starting with SELECT, the rest of the DELETE, INSERT, REPLACE and UPDATE statements can be added with EXPLAIN to view the execution plan of these statements, but usually we are more interested in the SELECT statement
5.3 Introduction to Execution Schedule
The role of each column in the execution plan
| id | Each SELECT clause or join operation will be assigned a unique number. The smaller the number, the higher the priority. Statements with the same id can be considered as a group. If the id is NULL, it means an independent subquery, and the priority of the subquery is higher than that of the main query. |
| select_type | The type of query. See below for details. |
| table | Table Name. Display which table the data of the current row belongs to. |
| partitions | Matching partition information. NULL if the table is not partitioned. |
| type | Access type, the query optimization strategy is executed according to methods such as indexing and full table scanning. |
| possible_keys | Indexes that may be used. Lists which indexes MySQL can use for queries. If there is only one possible_keys for the column, it usually means that the query is efficient. If this column has multiple possible_keys, and MySQL only uses one of them, you need to consider whether you need to add a joint index on this column. |
| key | The index actually used. If no KEY is explicitly specified, MySQL will automatically select the optimal index according to the query conditions. |
| key_len | The length of the index actually used. The shorter the index, the faster it is, and generally the smaller the index field, the better. |
| ref | When using the index column equivalent query, the object information for equivalent matching with the index column |
| rows | Estimated number of records that need to be read. The smaller the value, the better, which means the smaller the result set, the more efficient the query. |
| filtered | The percentage of the number of remaining records after a table is filtered by the search condition. The smaller the value, the better, indicating that the data can be returned directly through the index. |
| Extra | Some additional information. Such as USING WHERE, USING INDEX, etc. |
- select_type: The type of query, with the following values:
- SIMPLE: A simple SELECT query that does not use subqueries or UNION, and does not contain UNION ALL.
- PRIMARY: The outermost SELECT query.
- DERIVED: A SELECT statement that appears as a subquery in the FROM clause.
- UNION: The second or subsequent SELECT query in a UNION.
- UNION RESULT: SELECT query to get data from the result set of UNION.
- SUBQUERY: A subquery that does not appear in the FROM clause, usually used in the SELECT statement.
- DEPENDENT SUBQUERY: The subquery depends on the result set of the outer query.
- key: The index actually used. The INDEX keyword is used when creating an index in MySQL, but in the EXPLAIN execution plan table, KEY is displayed, because MySQL allows specifying statistical information, such as minimum value, maximum value, etc., when creating an index . These statistical information It is regarded as an index key (Index key) in the index , so it is displayed as KEY in the execution plan table.
- type: access type, the query optimization strategy is executed according to methods such as indexing and full table scanning. When the value of the type column is not const, we need to focus on performance tuning of indexes and caches, optimize SQL statements, and properly fix possible data design problems.
- system: Only one row of data will be queried. This is the fastest type of query and typically occurs in queries on system tables.
- const: Used when a primary key or unique index is used to find a single row. At this time, the query can only return one row of data. This is a very fast type of query.
- eq_ref: Used when the connection uses a unique index to find data that meets the query conditions. Each connection type needs to use a unique index for access, which is faster than ref.
- ref: Used when searching for data using a non-unique index, the query result is larger than eq_ref, but still very fast.
- range: It is used when using the index range to find data, and may find data within a certain range, such as queries when using BETWEEN or > or > < operations.
- index: Full table scan is used when no good index is applicable, which is faster than full table scan.
- all: table scan, scan the entire table to obtain the required data, the speed is the slowest, and must be avoided as much as possible.
- unique_subquery: When filtering the query results or using the IN operation, the optimizer will choose to use this type of query. The subquery using the In operator depends on the unique index of the outer query.
- index_subquery: The In operator is used but the subquery uses a normal index instead of a unique index.
- range_check: Used when using indexes to check foreign key references.
5.4 EXPLAIN four output formats
Here to talk about the output format of EXPLAIN. EXPLAIN can output four formats: 传统格式, JSON格式, TREE格式and 可视化输出. Users can choose the format suitable for them according to their needs.
1. Traditional format
The traditional format is straightforward, and the output is a table summarizing the query plan.
mysql> EXPLAIN SELECT s1.key1, s2.key1 FROM s1 LEFT JOIN s2 ON s1.key1 = s2.key1 WHERE s2.common_field IS NOT NULL;
2. JSON format
The statement output introduced in the first format EXPLAINlacks an important attribute to measure the execution quality - 成本. The JSON format is the output format among the four formats 信息最详尽, which contains the execution cost information.
- JSON format: add FORMAT=JSON between the EXPLAIN word and the real query statement.
EXPLAIN FORMAT=JSON SELECT ....
- id: the unique identifier (id) corresponding to the query
- select_type: query type (type)
- table: the table being accessed (table_name)
- partitions: the partition being accessed (partition_name)
- type: the access method used (access_type)
- possible_keys: Possible indexes to use (possible_keys)
- key: the index actually used (key)
- key_len: index length (key_length)
- ref: the column or constant (ref) to match the index
- rows: estimated number of retrieved rows (rows)
- filtered: After using WHERE to filter, the percentage of the remaining rows (filtered)
- Extra: other information (extra)
3. TREE format
The TREE format is a new format introduced after version 8.0.16. It 各个部分之间的关系mainly 各部分的执行顺序describes how to query based on the query and .
mysql> EXPLAIN FORMAT=tree SELECT * FROM s1 INNER JOIN s2 ON s1.key1 = s2.key2 WHERE
s1.common_field = 'a'\G
*************************** 1. row ***************************
EXPLAIN: -> Nested loop inner join (cost=1360.08 rows=990)
-> Filter: ((s1.common_field = 'a') and (s1.key1 is not null)) (cost=1013.75
rows=990)
-> Table scan on s1 (cost=1013.75 rows=9895)
-> Single-row index lookup on s2 using idx_key2 (key2=s1.key1), with index
condition: (cast(s1.key1 as double) = cast(s2.key2 as double)) (cost=0.25 rows=1)
1 row in set, 1 warning (0.00 sec)
4. Visual output
Visual output, you can visualize the execution plan of MySQL through MySQL Workbench. By clicking the magnifying glass icon of Workbench, a visualized query plan can be generated.
3. TREE format
The TREE format is a new format introduced after version 8.0.16. It 各个部分之间的关系mainly 各部分的执行顺序describes how to query based on the query and .
mysql> EXPLAIN FORMAT=tree SELECT * FROM s1 INNER JOIN s2 ON s1.key1 = s2.key2 WHERE
s1.common_field = 'a'\G
*************************** 1. row ***************************
EXPLAIN: -> Nested loop inner join (cost=1360.08 rows=990)
-> Filter: ((s1.common_field = 'a') and (s1.key1 is not null)) (cost=1013.75
rows=990)
-> Table scan on s1 (cost=1013.75 rows=9895)
-> Single-row index lookup on s2 using idx_key2 (key2=s1.key1), with index
condition: (cast(s1.key1 as double) = cast(s2.key2 as double)) (cost=0.25 rows=1)
1 row in set, 1 warning (0.00 sec)
4. Visual output
Visual output, you can visualize the execution plan of MySQL through MySQL Workbench. By clicking the magnifying glass icon of Workbench, a visualized query plan can be generated.
The figure above shows the tables in join order from left to right. Red boxes indicate `full table scan`, while green boxes indicate using `index lookup`. For each table, the indexes used are shown. Also note that above the box for each table is an estimate of the number of rows found by each table access and the cost of accessing that table.
5.5 Use of SHOW WARNINGS
In MySQL, SHOW WARNINGS is a command that can view the warning information generated in the most recently executed statement. When MySQL executes a statement, if it finds something that does not meet expectations, it will generate some warning messages. These warning messages can include non-fatal errors, such as certain types of data cannot be implicitly converted or some data is truncated.
When we execute the SHOW WARNINGS command, MySQL will return a detailed list of warning information, including:
- Warning: the type of warning
- Level: The level of the warning, usually Note, Warning or Error
- Code: The return code for the warning
- Message: the content of the warning message
You can use SELECT to view the warning information of the last operation:
SHOW WARNINGS;
You can also use INSERT, UPDATE, DELETE, ALTER TABLE and other commands to check the warning information generated by a specific operation:
INSERT INTO my_table (name, age) VALUES ('John Doe', 150);
SHOW WARNINGS;
In the process of development and debugging, SHOW WARNINGS is very useful for locating and solving some problems, such as data truncation, type conversion and other problems.
6. Analyze the optimizer execution plan: trace
In MySQL, you can use the trace command to track and analyze the optimizer execution plan. The trace command can display the decisions taken by the MySQL optimizer when generating an execution plan, including which tables are processed, and which indexes, algorithms, etc. are used.
To use the trace command, you need to enable the two system variables general_log and performance_schema first, and then you need to use the SET statement to set some parameters, such as trace-unique-check, trace-max-protocol, trace-protocol, trace-feature, trace-feature-check wait. After the setting is complete, the format of the output result can be selected through the SET global trace_format='json' statement.
Here is a simple example of using the trace command:
First, set the parameters:
SET @trace_feature = 'qa';
SET @max_execution_time=50000;
SET @trace_level = '+ddl,+engine';
SET @trace_feature_check = 1;
SET @trace_unique_check = 1;
SET @trace_protocol = 1;
SET @trace_max_protocol = 6;
Then, enable general_log and performance_schema:
SET global general_log = on;
SET global performance_schema = on;
Next, execute the query and view the results:
SELECT *
FROM my_table
WHERE my_column = 'some_value';
SHOW SESSION STATUS LIKE 'Last_Query_Plan';
Finally, close general_log and performance_schema:
SET global general_log = off;
SET global performance_schema = off;
In the trace output, we can see details such as the index used by the optimizer in the execution plan, the execution algorithm, and the number of rows estimated. By analyzing the trace results, we can find the root cause of some performance problems and make corresponding adjustments and optimizations. However, it should be noted that the trace command may bring additional performance consumption and IO overhead, and should not be enabled in a production environment for a long time.
7. MySQL monitoring analysis view-sys schema
7.1 Introduction
MySQL introduced the sys schema in version 8.0, which contains views and functions for monitoring and analyzing MySQL server performance. The sys schema provides a set of easy-to-use views and functions that can help us better understand and analyze the behavior and performance of MySQL databases.
The following are some commonly used monitoring and analysis views in sys schema:
- sys.statements_with_sorting: Shows which statements use sorting operations, including which sorting operations are used, the number of sorts per statement, and the resource consumption of the sorting operations.
- sys.statements_with_runtimes_in_95th_percentile: Shows the statement with the longest execution time.
- sys.io_global_by_file_by_bytes: Displays the number of disk IO bytes per file, which can be used to detect IO bottlenecks.
- sys.memory_by_host_by_current_bytes: Displays the current memory usage of each client, which can be used to detect memory leaks or high memory usage.
- sys.waits_global_by_latency: Shows which waiting operations take the most time, which can help us find the bottleneck of performance problems.
- sys.processlist: Displays information about currently running threads and processes, including executed statements, query ID, user, host, thread ID, and status.
In general, the views and functions contained in the sys schema provide us with deeper MySQL performance analysis and monitoring capabilities, which can help us better understand the behavior and performance bottlenecks of the MySQL database.
- Host-related : Start with host_summary, which mainly summarizes the information of IO delay.
- Innodb related : starts with innodb, summarizes innodb buffer information and transaction waiting for innodb lock information.
- I/o related : starts with io, and summarizes waiting for I/O and I/O usage.
- Memory usage : starts with memory, displays memory usage from the perspective of host, thread, event, etc.
- Connection and session information : processlist and session-related views summarize session-related information.
- Table-related : The view starting with schema_table shows the statistical information of the table.
- Index information : counts the usage of indexes, including redundant indexes and unused indexes.
- Statement related : It starts with statement and contains statement information for performing full table scans, using temporary tables, sorting, etc.
- User-related : The view starting with user collects statistics on file I/O and statement execution used by users.
- Related information about waiting events : starts with wait, showing the delay of waiting events.
7.2 Usage Scenarios
Index situation
#1. 查询冗余索引
select * from sys.schema_redundant_indexes;
#2. 查询未使用过的索引
select * from sys.schema_unused_indexes;
#3. 查询索引的使用情况
select index_name,rows_selected,rows_inserted,rows_updated,rows_deleted
from sys.schema_index_statistics where table_schema='dbname';
table related
# 1. 查询表的访问量
select table_schema,table_name,sum(io_read_requests+io_write_requests) as io from
sys.schema_table_statistics group by table_schema,table_name order by io desc;
# 2. 查询占用bufferpool较多的表
select object_schema,object_name,allocated,data
from sys.innodb_buffer_stats_by_table order by allocated limit 10;
# 3. 查看表的全表扫描情况
select * from sys.statements_with_full_table_scans where db='dbname';
sentence related
#1. 监控SQL执行的频率
select db,exec_count,query from sys.statement_analysis
order by exec_count desc;
#2. 监控使用了排序的SQL
select db,exec_count,first_seen,last_seen,query
from sys.statements_with_sorting limit 1;
#3. 监控使用了临时表或者磁盘临时表的SQL
select db,exec_count,tmp_tables,tmp_disk_tables,query
from sys.statement_analysis where tmp_tables>0 or tmp_disk_tables >0
order by (tmp_tables+tmp_disk_tables) desc;
IO-related
#1. 查看消耗磁盘IO的文件
select file,avg_read,avg_write,avg_read+avg_write as avg_io
from sys.io_global_by_file_by_bytes order by avg_read limit 10;
Innodb related
#1. 行锁阻塞情况
select * from sys.innodb_lock_waits;