MySQL tuning series (1) - performance monitoring

Starting from this article, we start the road of MySQL tuning.
This article mainly explains the performance monitoring of mysql from show profile, performance_schema and show processlist.

1. MySql architecture system

At the beginning of the tuning road, we first understand the MySql architecture system:

this is the official system diagram, from which we summarize as follows:
(1) Connectors: Interaction with MySQL in different languages.
(2) Connection pool: store and manage connection information.
(3) System management and control tools: provide management and control functions of the database system, such as backing up and restoring data in the database, ensuring the security of the entire database, providing security management, and coordinating and managing the cluster of the entire database.
(4) SQL interface: Receive customer commands and return execution results.
(5) Parser: When the SQL command is passed to the parser, it will be verified and parsed by the parser. Classify/Verify Semantic Validity.
(6) Query optimizer: optimize the query requested by the client, analyze it based on a series of algorithms based on the query statement requested by the client and some statistical information in the database, obtain an optimal strategy, and tell the subsequent program how to obtain the result of the query statement. (The optimizer converts it into an execution plan, and then interacts with the storage engine.)
(7) Cache: When executing a query statement, the cache will be queried first (removed after MySQL 8.0, because this function is not very practical). (The cache mechanism is composed of a series of small caches. For example, table cache, record cache, permission cache, engine cache, etc. If the query cache has a hit query result, the query statement can directly go to the query cache to fetch data.) (8) Pluggable storage engine: It is mainly responsible for data storage and reading, and interacts with the underlying files. It uses a plug-in architecture and supports InnoDB, MyISAM, Memory and other storage engines
. A separate article will be published later to analyze the storage engine.
For mysql version 5.7, the default storage engine is as follows: show engines; the

default storage engine is InnoDB, and only InnoDB supports transactions.
(9) System file layer: This layer is responsible for storing database data and logs on the file system, and completing the interaction with the storage engine, which is the physical storage layer of files. It mainly includes log files, data files, configuration files, pid files, socket files, etc.
Summarized as follows:

Two, show profile

show profile can find out the running status of all the statements executed on the server, including what operations were executed during the running process, and how much time each took.
Profile is no longer recommended and will be eliminated soon. It is disabled by default.

select @@profiling;//查询当前状态
set profiling=1;//设置打开

as the picture shows:

1. show profile syntax

SHOW PROFILE [type [, type] ... ]
    [FOR QUERY n]
    [LIMIT row_count [OFFSET offset]]

type: {
    
    
    ALL                显示所有信息
  | BLOCK IO           显示块输入和输出操作的数量
  | CONTEXT SWITCHES   显示自愿上下文切换和非自愿上下文切换的数量
  | CPU                显示用户和系统的CPU使用时间
  | IPC                显示已发送和已接收消息（messages）的数量
  | MEMORY             -- 尚未生效
  | PAGE FAULTS        显示主要和次要页面错误的数量
  | SOURCE             显示源代码中函数名称以及该函数所在文件的名称和行号
  | SWAPS              显示SWAP数量
}

2、show profiles

The execution time of each statement will be displayed.

3、show profile

It is used to separately analyze the detailed resource occupation information and working status of the latest executed sql statement, and the status and their duration are displayed by default.

You can specify a for query id to show profile to view the statement with the specified id, and you can also add a new column to the output, and the new column looks at the previous type.

show profile all for query id;//id 为 show profiles 查出来的某条记录的Query_ID。

三、performance_schema

​ MySQL's performance_schema is used to monitor the resource consumption and resource waiting of the MySQL server during a lower-level operation.

performance_schema is enabled by default.
Regarding the configuration and parameters of performance_schema, there are many explanations elsewhere. I think it is stinky and long, so I won’t summarize it.
For the actual query operation, let’s record it here:

--1、哪类的SQL执行最多？
SELECT DIGEST_TEXT,COUNT_STAR,FIRST_SEEN,LAST_SEEN FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--2、哪类SQL的平均响应时间最多？
SELECT DIGEST_TEXT,AVG_TIMER_WAIT FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--3、哪类SQL排序记录数最多？
SELECT DIGEST_TEXT,SUM_SORT_ROWS FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--4、哪类SQL扫描记录数最多？
SELECT DIGEST_TEXT,SUM_ROWS_EXAMINED FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--5、哪类SQL使用临时表最多？
SELECT DIGEST_TEXT,SUM_CREATED_TMP_TABLES,SUM_CREATED_TMP_DISK_TABLES FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--6、哪类SQL返回结果集最多？
SELECT DIGEST_TEXT,SUM_ROWS_SENT FROM events_statements_summary_by_digest ORDER BY COUNT_STAR DESC
--7、哪个表物理IO最多？
SELECT file_name,event_name,SUM_NUMBER_OF_BYTES_READ,SUM_NUMBER_OF_BYTES_WRITE FROM file_summary_by_instance ORDER BY SUM_NUMBER_OF_BYTES_READ + SUM_NUMBER_OF_BYTES_WRITE DESC
--8、哪个表逻辑IO最多？
SELECT object_name,COUNT_READ,COUNT_WRITE,COUNT_FETCH,SUM_TIMER_WAIT FROM table_io_waits_summary_by_table ORDER BY sum_timer_wait DESC
--9、哪个索引访问最多？
SELECT OBJECT_NAME,INDEX_NAME,COUNT_FETCH,COUNT_INSERT,COUNT_UPDATE,COUNT_DELETE FROM table_io_waits_summary_by_index_usage ORDER BY SUM_TIMER_WAIT DESC
--10、哪个索引从来没有用过？
SELECT OBJECT_SCHEMA,OBJECT_NAME,INDEX_NAME FROM table_io_waits_summary_by_index_usage WHERE INDEX_NAME IS NOT NULL AND COUNT_STAR = 0 AND OBJECT_SCHEMA <> 'mysql' ORDER BY OBJECT_SCHEMA,OBJECT_NAME;
--11、哪个等待事件消耗时间最多？
SELECT EVENT_NAME,COUNT_STAR,SUM_TIMER_WAIT,AVG_TIMER_WAIT FROM events_waits_summary_global_by_event_name WHERE event_name != 'idle' ORDER BY SUM_TIMER_WAIT DESC
--12-1、剖析某条SQL的执行情况，包括statement信息，stege信息，wait信息
SELECT EVENT_ID,sql_text FROM events_statements_history WHERE sql_text LIKE '%count(*)%';
--12-2、查看每个阶段的时间消耗
SELECT event_id,EVENT_NAME,SOURCE,TIMER_END - TIMER_START FROM events_stages_history_long WHERE NESTING_EVENT_ID = 1553;
--12-3、查看每个阶段的锁等待情况
SELECT event_id,event_name,source,timer_wait,object_name,index_name,operation,nesting_event_id FROM events_waits_history_longWHERE nesting_event_id = 1553;

Fourth, show processlist

show processlist is to display the user's running threads and observe the status of the threads. Except the root user can see all the running threads, other users can only see their own running threads.
The information displayed by show processlist comes from the processlist table in the MySQL system library information_schema.

The attributes are as follows:

Reference: show processlist The most complete parameter explanation and solution in history