[Day 21] SQL Advanced - Query Optimization - performance_schema Series 3: Event Recording (SQL Xiao Xuzhu)

Teleportation back to the city – "32 Days SQL Foundation Building"

Zero. Foreword

Today is the 21st day of learning SQL check-in , and every day I will provide an article for group members to read ( no subscription payment required ).

I hope that everyone will think for themselves first, if you really have no idea, then read the following problem-solving ideas, and implement it again by yourself. In the Xiaoxuzhu JAVA community , the corresponding [check-in stickers] check-in, today's task is completed, and develop a good habit of learning to check-in every day.

​ Brother Xuzhu will organize everyone to study the same article together, so if you have any questions, you can ask them in the group. Friends in the group can help you quickly. One person can go fast, and a group of people can go very fast. How lucky it is to have comrades-in-arms who study and communicate together.

​ My learning strategy is very simple, question sea strategy + Feynman learning method. If you can implement all these questions seriously, it means that SQL has successfully established its foundation. For the advanced learning later, you can continue to follow me and walk towards the road of architect together.

Today's learning content is: Advanced SQL - Query Optimization - performance_schema Series 3: Event Recording

The content of this article is more than 10,000 words, and there is a lot of content. Suggested reading method: remember the title and the introduction of the small modules. The detailed explanation of the content can help you understand. It’s okay if you don’t understand it.
This article is suitable for collection. When you need to consult the specified configuration, you can quickly find the detailed explanation of the configuration.

1. Exercise topics

topic link	difficulty
-	-

Two, SQL ideas

Advanced SQL - Query Optimization - performance_schema Series 3: Event Recording

Today I will explain the original record table of events in performance_schema.

wait event table

During performance optimization, the hardware load is not high, and SQL optimization and database table structure optimization can’t be done. The bottleneck of performance optimization has been reached. It is necessary to analyze by means of waiting events to find out whether the database response is slow inside MySQL Server. where.
The waiting event record table contains three tables, which record the current and recent waiting events that have occurred in the MySQL instance and the time consumption.

• events_waits_current table: records the currently executing waiting events, each thread only records 1 row

• events_waits_history table: records the history of the latest waiting events that have been executed. By default, only 10 rows of records are recorded for each thread

• events_waits_history_long table: records the history of the latest waiting events that have been executed, and the default total number of records for all threads is 10,000

Note: In the wait event related configuration, most of the wait event instruments in the setup_instruments table are not enabled (most of the IO related wait event instruments are enabled by default), and the waits related consumers configuration in the setup_consumers table is not enabled by default.

events_waits_current 表

The events_waits_current table contains the current wait event information, and each thread only displays the current status of the most recently monitored wait event.
Of all the tables containing wait event rows, the events_waits_current table is the most fundamental source of data. Other tables containing wait event data are logically derived from the current event information in the events_waits_current table (except for the summary table). For example, the data in the events_waits_history and events_waits_history_long table is a small collection of data in the events_waits_current table (specifically how many rows of data sets are stored has its own variable control).

Try it in practice:

Enable listening: The setup_consumers table lists the available consumer types and enabled consumer types:

SELECT * FROM performance_schema.setup_consumers;

The default events_waits_current is off, modifying the state will immediately affect the monitoring.

UPDATE performance_schema.setup_consumers
SET ENABLED = 'YES'
WHERE NAME = 'events_waits_current';

sleep 100 seconds

select sleep(100);

select * from events_waits_current;

The TIMER_WAIT field indicates the time cost of the event, in picoseconds. In actual application scenarios: we can use the information in this field to sort in reverse order to find out the waiting events with the largest time cost .

Field description:

• THREAD_ID, EVENT_ID: The thread ID associated with the event and the current event ID. The THREAD_ID and EVENT_ID values ​​constitute the unique identifier of the event information row (there will be no duplicate THREAD_ID+EVENT_ID values);

• END_EVENT_ID: When an event is being executed, the value of this column is NULL. When an event is executed, the ID of the event is updated to this column;

• EVENT_NAME: The name of the instruments that generated the event. The name comes from the NAME field value of the setup_instruments table;

• SOURCE: The name of the source file where the instruments that generated the event are located and the code line number that detected the point where the event occurred. You can view the source code to determine which code is involved. For example, if a mutex, lock is blocked, you can check the context in which this happens;

• TIMER_START, TIMER_END, TIMER_WAIT: Time information of the event. Units are picoseconds (trillionths of a second). The TIMER_START and TIMER_END values ​​represent the event start and end times. TIMER_WAIT is the elapsed time of the event (that is, how long the event has been executed);

  • If the event is not completed, TIMER_END is the current timer time value (current time), and TIMER_WAIT is the time elapsed so far (TIMER_END - TIMER_START)

  • If the instruments configuration item TIMED = NO to collect the event, the time information of the event will not be collected, and TIMER_START, TIMER_END and TIMER_WAIT are all recorded as NULL in this case;

• SPINS: For mutexes and spin counts. If the column value is NULL, it means that the spin is not used in the code or the spin is not monitored;

• OBJECT_SCHEMA, OBJECT_NAME, OBJECT_TYPE, OBJECT_INSTANCE_BEGIN: These columns identify an object being executed, and the meaning of the information recorded in these columns depends on the type of the object.

• INDEX_NAME: Indicates the name of the index used. PRIMARY indicates that the primary key is used. NULL means no index is used

• NESTING_EVENT_ID: Indicates which event the EVENT_ID event in the row information is nested in, that is, the EVENT_ID of the parent event;

• NESTING_EVENT_TYPE: Indicates the nested event type of the EVENT_ID event in the row information. Valid values ​​are: TRANSACTION, STATEMENT, STAGE or WAIT, which is the event type of the parent event. If it is TRANSACTION, you need to find the event corresponding to the value of NESTING_EVENT_ID in the transaction event table. The same is true for other types

• OPERATION: the type of operation performed, such as: lock, read, write, timed_wait;

• NUMBER_OF_BYTES: The number of bytes or rows read or written by the operation. For file IO waiting, the column value indicates the number of bytes; for table I/O waiting (events of wait/io/table/sql/handler instruments), the column value indicates the number of rows. If the value is greater than 1, it means that the event corresponds to a batch I/O operation.

• FLAGS: reserved for future use

• PS: The events_waits_current table allows the TRUNCATE TABLE statement

events_waits_history 表

The events_waits_history table contains the most recent N wait events for each thread. When the server starts, the value of N is automatically adjusted. If you want to explicitly set this N size, you can adjust the value of the system parameter performance_schema_events_waits_history_size before the server starts. Waiting events need to be added to the events_waits_history table when the execution ends (saved in the events_waits_current table when there is no end). When adding new events to the events_waits_history table, if the table is full, older events are discarded per thread

The events_waits_history is defined the same as the events_waits_current table

PS: TRUNCATE TABLE statement is allowed

events_waits_history_long 表

The events_waits_history_long table contains the most recent N wait events (events for all threads). When the server starts, the value of N is automatically adjusted. If you want to explicitly set this N size, you can adjust the system parameters before starting the server

The value of performance_schema_events_waits_history_long_size. Waiting events will be added to the events_waits_history_long table when the execution ends (saved in the events_waits_current table when there is no end), when adding new events to the events_waits_history_long table, if the table is full, the older events in the table will be discarded.

events_waits_history_long has the same structure as events_waits_current table

PS: The TRUNCATE TABLE statement is allowed

statement event table

The statement event record table is the same as the wait event record table, and it also has three tables. These tables record which statement events have occurred in the MySQL instance currently and recently, and how much time is consumed. Records statement event information generated by various statement executions.
Note: In the statement event-related configuration, all instrument configurations starting with statement/* in the setup_instruments table are enabled by default, and the consumers configuration related to statements in the setup_consumers table is enabled by default. events_statements_history_long is not enabled.

events_statements_current 表

The events_statements_current table contains the current statement events, and each thread only shows the current status of the most recently monitored statement events.

Of the tables containing statement event rows, the events_statements_current current events table is the underlying table. The data in the other include statement event tables is logically derived from the current event table (except the summary table). For example: the events_statements_history and events_statements_history_long tables are the collection of recent statement event history, each thread in the events_statements_history table retains 10 rows of event history information by default, and the events_statements_history_long table retains 10,000 rows of event history information by default for all threads

Try it in practice:

SELECT * FROM performance_schema.setup_consumers;

The default events_statements_current is enabled
sleep 100 seconds

select sleep(100);

select * from events_statements_current;

Field description:

• THREAD_ID, EVENT_ID: The thread number associated with the event and the event number when the event starts. You can use the THREAD_ID and EVENT_ID column values ​​to uniquely identify the row. The same data row will not appear when the values ​​of these two rows are used as a combination condition

• END_EVENT_ID: When an event starts to execute, the column value of the corresponding row record is set to NULL, and when an event execution ends, the column value of the corresponding row record is updated to the ID of the event

• EVENT_NAME: The name of the monitoring instrument that generated the event. The column value comes from the NAME value of the setup_instruments table. For SQL statements, the initial instrument of the EVENT_NAME value is statement/com/Query, until the statement is parsed, it will be changed to a more appropriate specific instrument name, such as: statement/sql/insert

• SOURCE: The name of the source file and the line number in the source file where the code used to detect this event is located, you can check the source code to determine the code involved

• TIMER_START, TIMER_END, TIMER_WAIT: Time information of the event. The units for these values ​​are picoseconds (trillionths of a second). The TIMER_START and TIMER_END values ​​represent the start time and end time of the event. TIMER_WAIT is the time consumed by event execution (duration)

  • If the event is not completed, TIMER_END is the current time, and TIMER_WAIT is the time elapsed so far (TIMER_END - TIMER_START).

  • If the corresponding monitor TIMED field in the monitoring instrument configuration table setup_instruments is set to NO, the time information of the monitor will not be collected, and the values ​​of the TIMER_START, TIMER_END and TIMER_WAIT fields in the information records collected for this event are all NULL

• LOCK_TIME: The time to wait for a table lock. The value is calculated in microseconds, but is finally converted to picoseconds for easier comparison with timers in other performance_schema

• SQL_TEXT: The text of the SQL statement. If the row event is a command event that has nothing to do with the SQL statement, the value of this column is NULL. By default, the maximum display length of a statement is 1024 bytes. If you want to modify, set the value of the system variable performance_schema_max_sql_text_length before the server starts

• DIGEST: The MD5 hash value of the statement digest, which is a 32-digit hexadecimal string. If the statement_digest configuration line in the setup_consumers table is not enabled, the value of this column in the statement event is NULL

• DIGEST_TEXT: Standardized converted statement digest text. If the statements_digest configuration line in the setup_consumers table is not enabled, the value of this column in the statement event is NULL. The performance_schema_max_digest_length system variable determines the byte length of the maximum summary statement text when it is stored in the table (1024 bytes by default). Note: the byte length of the original statement text used to calculate the summary statement text is controlled by the system variable max_digest_length. The byte length stored in the table is controlled by the system variable performance_schema_max_digest_length, so if performance_schema_max_digest_length is less than max_digest_length, the calculated summary statement text will be truncated if it exceeds the length defined by performance_schema_max_digest_length

• CURRENT_SCHEMA: The default database used by the statement (use the use db_name statement to specify the default database), if not, it is NULL

• OBJECT_SCHEMA, OBJECT_NAME, OBJECT_TYPE: For nested statements (stored procedures are ultimately called by statements, so if a statement is called by a stored procedure, although the statement event is nested in the stored procedure, in fact for the event type, still nested within the statement event), these columns contain information about the parent statement. If it is not a nested statement or an event generated by the parent statement itself, these column values ​​​​are NULL

• OBJECT_INSTANCE_BEGIN: The unique identifier of the statement, the value of this column is the address of the object in memory

• MYSQL_ERRNO: The error number of statement execution, this value comes from the statement diagnosis area of ​​the code area

• RETURNED_SQLSTATE: The SQLSTATE value of statement execution, this value comes from the statement diagnosis area of ​​​​the code area

• MESSAGE_TEXT: The specific error message of statement execution, this value comes from the statement diagnosis area of ​​the code area

• ERRORS: Whether an error occurred during statement execution. If the SQLSTATE value begins with 00 (done) or 01 (warning), the column value is 0. For any other SQLSTATE value, the value of this column is 1

• WARNINGS: The number of statement warnings, this value comes from the statement diagnostics area of ​​the code area

• ROWS_AFFECTED: The number of rows affected by the statement. For a description of what "affected" means, see the link:

• https://dev.mysql.com/doc/refman/5.7/en/mysql-affected-rows.html

  • The mysql_affected_rows() function may be called immediately after executing a statement using the mysql_query() or mysql_real_query() functions. If UPDATE, DELETE, or INSERT, returns the number of rows changed, deleted, or inserted by the last statement. For SELECT statements, mysql_affected_rows() works the same way as mysql_num_rows() (the effected statistics cannot be seen in the information returned at the end of the execution result)

  • For UPDATE statements, the Rows Affected value defaults to the actual number of rows changed. If the CLIENT_FOUND_ROWS flag was specified to the mysql_real_connect() function when connecting to mysqld, then the value of affected-rows is the number of "found" rows. That is, the number of rows matched by the WHERE clause

  • For the REPLACE statement, if a new and old row replacement operation occurs, the affected row value is 2, because in this case, the old value is actually deleted, and the new value is inserted after two row operations

  • For the INSERT ... ON DUPLICATE KEY UPDATE statement, if the row is inserted as a new row, the affected count for each row is 1, if the old row is updated to a new row, the affected count for each row is 2, if no insert or update occurs, then The affected count of each row is 0 (but if the CLIENT_FOUND_ROWS flag is specified, when no insert or update occurs, that is, when the set value is the current value, the affected row value count of each row is 1 instead of 0)

  • After the CALL statement of the stored procedure is called, the number of affected rows returned by mysql_affected_rows() is the number of affected rows executed by the last statement in the stored procedure. If the statement returns -1, the stored procedure finally returns 0 affected. Therefore, the number of affected rows returned when the stored program is executed is not reliable, but you can implement a counter variable in the stored program by yourself and use ROW_COUNT() at the SQL level to obtain the affected row values ​​​​of each statement and add them up, and finally pass The stored procedure returns the value of this variable.

• ROWS_SENT: The number of data rows returned by the statement to the client

• ROWS_EXAMINED: The number of rows of data read from the storage engine during the execution of the statement

• CREATED_TMP_DISK_TABLES: The same count value as the Created_tmp_disk_tables state variable, but here it is only used for statement statistics in this event and not for global and session levels

• CREATED_TMP_TABLES: The same count value as the Created_tmp_tables state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SELECT_FULL_JOIN: The same count value as the Select_full_join state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SELECT_FULL_RANGE_JOIN: The same count value as the Select_full_range_join state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SELECT_RANGE: A count value like the Select_range state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SELECT_RANGE_CHECK: A count value like the Select_range_check state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SELECT_SCAN: A count value like the Select_scan state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SORT_MERGE_PASSES: A count value like the Sort_merge_passes state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SORT_RANGE: The same count value as the Sort_range state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SORT_ROWS: The same count value as the Sort_rows state variable, but here it is only used for statement statistics in this event and not for global and session levels

• SORT_SCAN: The same count value as the Sort_scan state variable, but here it is only used for statement statistics in this event and not for global and session levels

• NO_INDEX_USED: If the statement performs a table scan without using an index, the value of this column is 1, otherwise it is 0

• NO_GOOD_INDEX_USED: If the server cannot find a suitable index for the statement, the column value is 1, otherwise it is 0

• NESTING_EVENT_ID, NESTING_EVENT_TYPE, NESTING_EVENT_LEVEL: These three columns are used in combination with other columns to provide the following event information for top-level (unknown abstract statement or parent statement) statements and nested statements (statements executed in stored programs)

  • 对于顶级语句：
    OBJECT_TYPE = NULL,OBJECT_SCHEMA = NULL,OBJECT_NAME = NULL,NESTING_EVENT_ID = NULL,NESTING_EVENT_TYPE = NULL,NESTING_LEVEL = 0

  • For nested statements: OBJECT_TYPE = parent statement object type, OBJECT_SCHEMA = parent statement database-level name, OBJECT_NAME = parent statement table-level object name, NESTING_EVENT_ID = parent statement EVENT_ID, NESTING_EVENT_TYPE = 'STATEMENT', NESTING_LEVEL = parent statement NESTING_LEVEL plus one, for example : 1, indicating the next level of nested statement of the parent statement

• PS: The TRUNCATE TABLE statement is allowed

events_statements_history 表

The events_statements_history table contains the latest N statement events per thread. When the server starts, the value of N is automatically adjusted. To explicitly set the size of N, set the value of the system variable performance_schema_events_statements_history_size before the server starts. Statement events are added to the table only when execution is complete. When adding a new event to the table, if the quota of the corresponding thread's events in the table is full, the corresponding thread's older events will be discarded

events_statements_history has the same structure as events_statements_current table

PS: The TRUNCATE TABLE statement is allowed

events_statements_history_long 表

The events_statements_history_long table contains the most recent N statement events. When the server starts, the value of N is automatically adjusted. To explicitly set the size of N, set the value of the system variable performance_schema_events_statements_history_long_size before the server starts. The statement event needs to be added to the table when the execution ends. When new events are added to the table, older events in the table are discarded if the table's global quota is full

events_statements_history_long has the same structure as events_statements_current table

PS: The TRUNCATE TABLE statement is allowed

transaction event table

The transaction event record table, like the wait event record table, also has three tables. These tables record which transaction events have occurred in the MySQL instance currently and recently, and how much time is consumed

Note: In the configuration related to transaction events, there is only one instrument named transaction in the setup_instruments table, which is closed by default, and the consumers configuration related to transactions in the setup_consumers table is closed by default

events_transactions_current 表

The events_transactions_current table contains current transaction event information, and each thread only retains one row of recent transaction transaction events

Of the tables containing transaction event information, events_transactions_current is the underlying table. Data in other tables containing transaction event information logically originates from the current event table. For example: the events_transactions_history and events_transactions_history_long tables contain the latest 10 rows of transaction event information for each thread and the last 10,000 rows of global transaction event information

Try it in practice:

SELECT * FROM performance_schema.setup_consumers;

select * from events_transactions_current;

Field description:

• THREAD_ID, EVENT_ID: The thread number associated with the event and the event number when the event starts. You can use the THREAD_ID and EVENT_ID column values ​​to uniquely identify the row. The same data row will not appear when the values ​​of these two rows are used as a combination condition

• END_EVENT_ID: When an event starts to execute, the column value of the corresponding row record is set to NULL, and when an event execution ends, the column value of the corresponding row record is updated to the ID of the event

• EVENT_NAME: The name of the instruments that collect events for this transaction. NAME column value from setup_instruments table

• STATE: current transaction state. Valid values ​​are: ACTIVE (after the START TRANSACTION or BEGIN statement is executed, but before the transaction is not committed or rolled back), COMMITTED (after the COMMIT is executed), ROLLED BACK (after the ROLLBACK statement is executed)

• TRX_ID: not used, the field value is always NULL

• GTID: Contains the value of the gtid_next system variable. Its value may be a GTID in the format: UUID:NUMBER, or it may be: ANONYMOUS, AUTOMATIC. For transaction events with AUTOMATIC column values, the GTID column will be changed when the transaction is committed and the GTID of the corresponding transaction is actually assigned (if the gtid_mode system variable is ON or ON_PERMISSIVE, the GTID column will be changed to the GTID of the transaction, if gtid_mode is OFF or OFF_PERMISSIVE , the GTID column will be changed to ANONYMOUS)

• XID_FORMAT_ID, XID_GTRID and XID_BQUAL: Components of the XA transaction identifier. For details about XA transaction syntax, see the link:

• https://dev.mysql.com/doc/refman/5.7/en/xa-statements.html

• XA_STATE: The state of the XA transaction. Valid values ​​are: ACTIVE (after XA START is executed, before other subsequent XA statements are executed), IDLE (after XA END statement is executed, before other subsequent XA statements are executed), PREPARED (after XA PREPARE statement is executed, not executed Before other subsequent XA statements), ROLLED BACK (after executing the XA ROLLBACK statement, but before executing other subsequent XA statements), COMMITTED (after executing the XA COMMIT statement)

• SOURCE: The name of the source file and the line number in the source file where the code used to detect this event is located, you can check the source code to determine the code involved

• TIMER_START, TIMER_END, TIMER_WAIT: Time information of the event. The units for these values ​​are picoseconds (trillionths of a second). The TIMER_START and TIMER_END values ​​represent the start time and end time of the event. TIMER_WAIT is the time (duration) consumed by event execution.
  If the event is not executed, TIMER_END is the current time, and TIMER_WAIT is the time elapsed so far (TIMER_END - TIMER_START)

• If the corresponding monitor TIMED field in the monitoring instrument configuration table setup_instruments is set to NO, the time information of the monitor will not be collected, and the values ​​of the TIMER_START, TIMER_END and TIMER_WAIT fields in the information records collected for this event are all NULL

• ACCESS_MODE: Transaction access mode. Valid values ​​are: READ ONLY or READ WRITE

• ISOLATION_LEVEL: Transaction isolation level. Valid values ​​are: REPEATABLE READ, READ COMMITTED, READ UNCOMMITTED, SERIALIZABLE

• AUTOCOMMIT: Whether the autocommit mode is enabled at the beginning of the transaction, if it is enabled, it is YES, if it is not enabled, it is NO

• NUMBER_OF_SAVEPOINTS, NUMBER_OF_ROLLBACK_TO_SAVEPOINT, NUMBER_OF_RELEASE_SAVEPOINT: Number of SAVEPOINT, ROLLBACK TO SAVEPOINT and RELEASE SAVEPOINT statements executed within the transaction

• OBJECT_INSTANCE_BEGIN: not used, the field value is always NULL

• NESTING_EVENT_ID: The parent event EVENT_ID value of the nested transaction event

• NESTING_EVENT_TYPE: Nesting event type. Valid values ​​are: TRANSACTION, STATEMENT, STAGE, WAIT (Transaction will not appear in this column value because transactions cannot be nested)

• PS: The TRUNCATE TABLE statement is allowed

events_transactions_history 表

• The events_transactions_history table contains the most recent N transaction events for each thread. When the server starts, the value of N is automatically adjusted. To explicitly set the size of N, you can set the system variable before the server starts

• The value of performance_schema_events_transactions_history_size. Transaction events are not added to this table until execution is complete. When a new transaction event is added to the table, if the table is full, the corresponding thread's older transaction event will be discarded

• events_transactions_history has the same structure as events_transactions_current table

PS: The TRUNCATE TABLE statement is allowed

events_transactions_history_long 表

The events_transactions_history_long table contains the most recent N transaction events globally. When the server starts, the value of N is automatically adjusted. To explicitly set the size of N, you can set the system variable before the server starts

The value of performance_schema_events_transactions_history_long_size. Transaction events are not added to this table until execution is complete. When new transactional events are added, older events are discarded if the table is full

events_transactions_history_long has the same structure as events_transactions_current table

PS: The TRUNCATE TABLE statement is allowed

3. Summary

This article explains three types of event original record tables in performance_schema: wait event table, statement event table and transaction event table. The actual combat demonstration and field description are carried out respectively.
The content of this article is more than 10,000 words, and there is a lot of content. Suggested reading method: remember the title and the introduction of the small modules. The detailed explanation of the content can help you understand. It’s okay if you don’t understand it.
This article is suitable for collection. When you need to consult the specified configuration, you can quickly find the detailed explanation of the configuration.

4. Reference

Event record | performance_schema comprehensive introduction
official mysql_affected_rows detailed explanation
official xa-statements transaction syntax detailed explanation

I'm Brother Xuzhu, see you tomorrow~