sys system library notes (c) - actual application case

First, take advantage of sql slow waiting for the event to find out where

Use sys.session view combined performance_schema waiting for the event.

First enable consumers to wait for events related instruments and

call sys.ps_setup_enable_instrument('wait');
call sys.ps_setup_enable_consumer('wait');

Then use sys.session view query, concern last_wait, last_wait_latency field

select * from session where command='query' and conn_id!=connection_id()\G;

 

Second, to see if there is a transaction lock wait

innodb_lock_wait view usually due to analyze the transactions lock (line lock) wait, attention must be matters awaiting the view will have a value. In fact, this view when it is used in applications previously described performance_schema too.

MySQL 5.7 version can also be used sys.innodb_lock_waits view of the query, but in 8.0, view the different tables join query (the information_schema.innodb_locks and information_schema.innodb_lock_waits table used to replace the previous version in order performance_schema.data_locks and performance_schema.data_lock_waits table ).

select * from sys.innodb_lock_waits\G;

*************************** 1. row ***************************
            wait_started: 2018-01-14 21:51:59
                wait_age: 00:00:11 <-- 等待时间
          wait_age_secs: 11
            locked_table: `xiaoboluo`.`t_luoxiaobo` <-- 表信息
    locked_table_schema: xiaoboluo
      locked_table_name: t_luoxiaobo
  locked_table_partition: NULL
locked_table_subpartition: NULL
            locked_index: PRIMARY <-- 主键字段
            locked_type: RECORD <-- 行锁
          waiting_trx_id: 55566
    waiting_trx_started: 2018-01-14 21:51:59
        waiting_trx_age: 00:00:11
waiting_trx_rows_locked: 1
waiting_trx_rows_modified: 0
            waiting_pid: 8 <-- 被阻塞的线程id
          waiting_query: update t_luoxiaobo set datet_time=now() where id=2  <-- 被阻塞者正执行的sql语句
        waiting_lock_id: 55566:2:4:2
      waiting_lock_mode: X <-- 锁模式
        blocking_trx_id: 55562
            blocking_pid: 7  <-- 阻塞的线程id
          blocking_query: NULL <-- 阻塞者正执行的sql语句（已经执行完了，但没提交）
        blocking_lock_id: 55562:2:4:2
      blocking_lock_mode: X
    blocking_trx_started: 2018-01-14 21:34:44
        blocking_trx_age: 00:17:26 <-- 持锁时间
blocking_trx_rows_locked: 1
blocking_trx_rows_modified: 1
sql_kill_blocking_query: KILL QUERY 7
sql_kill_blocking_connection: KILL 7  <-- kill阻塞者的语句
1 row in set (0.02 sec)

5.6 and previous versions is not sys default library, you can use the following statement instead:

SELECT r.trx_wait_started AS wait_started,
  TIMEDIFF(NOW(), r.trx_wait_started) AS wait_age,
  TIMESTAMPDIFF(SECOND, r.trx_wait_started, NOW()) AS wait_age_secs,
  rl.lock_table AS locked_table,
  rl.lock_index AS locked_index,
  rl.lock_type AS locked_type,
  r.trx_id AS waiting_trx_id,
  r.trx_started as waiting_trx_started,
  TIMEDIFF(NOW(), r.trx_started) AS waiting_trx_age,
  r.trx_rows_locked AS waiting_trx_rows_locked,
  r.trx_rows_modified AS waiting_trx_rows_modified,
  r.trx_mysql_thread_id AS waiting_pid,
  sys.format_statement(r.trx_query) AS waiting_query,
  rl.lock_id AS waiting_lock_id,
  rl.lock_mode AS waiting_lock_mode,
  b.trx_id AS blocking_trx_id,
  b.trx_mysql_thread_id AS blocking_pid,
  sys.format_statement(b.trx_query) AS blocking_query,
  bl.lock_id AS blocking_lock_id,
  bl.lock_mode AS blocking_lock_mode,
  b.trx_started AS blocking_trx_started,
  TIMEDIFF(NOW(), b.trx_started) AS blocking_trx_age,
  b.trx_rows_locked AS blocking_trx_rows_locked,
  b.trx_rows_modified AS blocking_trx_rows_modified,
  CONCAT('KILL QUERY ', b.trx_mysql_thread_id) AS sql_kill_blocking_query,
  CONCAT('KILL ', b.trx_mysql_thread_id) AS sql_kill_blocking_connection
FROM information_schema.innodb_lock_waits w
  INNER JOIN information_schema.innodb_trx b ON b.trx_id = w.blocking_trx_id
  INNER JOIN information_schema.innodb_trx r ON r.trx_id = w.requesting_trx_id
  INNER JOIN information_schema.innodb_locks bl ON bl.lock_id = w.blocking_lock_id
  INNER JOIN information_schema.innodb_locks rl ON rl.lock_id = w.requested_lock_id
ORDER BY r.trx_wait_started;

 

Third, to see if there is a lock wait MDL

MDL can view the current thread to wait for information schema_table_lock_wait view (new 5.8.9), which displays the session is blocked and blocked source. Data sources threads, metadata_locks, events_statements_current table under performance_schema, to obtain the corresponding view definition can view sql.

First of all relevant instruments should enable MDL wait

call sys.ps_setup_enable_instrument('wait/lock/metadata/sql/mdl');

schema_table_lock_wait view field below, obvious meaning, a detailed reference  https://dev.mysql.com/doc/refman/5.7/en/sys-schema-table-lock-waits.html

desc schema_table_lock_waits;
+------------------------------+---------------------+------+-----+---------+-------+
| Field                        | Type                | Null | Key | Default | Extra |
+------------------------------+---------------------+------+-----+---------+-------+
| object_schema                | varchar(64)         | YES  |     | NULL    |       |
| object_name                  | varchar(64)         | YES  |     | NULL    |       |
| waiting_thread_id            | bigint(20) unsigned | NO   |     | NULL    |       |
| waiting_pid                  | bigint(20) unsigned | YES  |     | NULL    |       |
| waiting_account              | text                | YES  |     | NULL    |       |
| waiting_lock_type            | varchar(32)         | NO   |     | NULL    |       |
| waiting_lock_duration        | varchar(32)         | NO   |     | NULL    |       |
| waiting_query                | longtext            | YES  |     | NULL    |       |
| waiting_query_secs           | bigint(20)          | YES  |     | NULL    |       |
| waiting_query_rows_affected  | bigint(20) unsigned | YES  |     | NULL    |       |
| waiting_query_rows_examined  | bigint(20) unsigned | YES  |     | NULL    |       |
| blocking_thread_id           | bigint(20) unsigned | NO   |     | NULL    |       |
| blocking_pid                 | bigint(20) unsigned | YES  |     | NULL    |       |
| blocking_account             | text                | YES  |     | NULL    |       |
| blocking_lock_type           | varchar(32)         | NO   |     | NULL    |       |
| blocking_lock_duration       | varchar(32)         | NO   |     | NULL    |       |
| sql_kill_blocking_query      | varchar(31)         | YES  |     | NULL    |       |
| sql_kill_blocking_connection | varchar(25)         | YES  |     | NULL    |       |
+------------------------------+---------------------+------+-----+---------+-------+
18 rows in set (0.00 sec)

 

Fourth, view the InnoDB buffer pool Hot library

Use innodb_buffer_stats_by_schema view statistics in accordance with schema grouping query InnoDB buffer pool

select * from innodb_buffer_stats_by_schema;
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| object_schema | allocated | data | pages | pages_hashed | pages_old | rows_cached |
+---------------+------------+-----------+-------+--------------+-----------+-------------+
| InnoDB System | 23.73 MiB | 21.76 MiB | 1519 | 0 | 24 | 21474 |
| mysql | 240.00 KiB | 14.57 KiB | 15 | 0 | 15 | 179 |
| xiaoboluo | 128.00 KiB | 38.93 KiB | 8 | 0 | 5 | 982 |
| sys | 16.00 KiB | 354 bytes | 1 | 0 | 1 | 6 |
| 小萝卜 | 16.00 KiB | 135 bytes | 1 | 0 | 1 | 3 |
+---------------+------------+-----------+-------+--------------+-----------+-------------+

View fields have the following meanings:

• object_schema: where the object schema, if the table belongs to Innodb storage engine, this field is displayed as InnoDB System, if the other engine, this field displays for each schema name (db name).
• The number of currently assigned to the schema of the total bytes of memory: allocated
• data: the amount of memory currently allocated to the byte data section using the schema
• pages: schema currently assigned to the total number of pages of memory
• The total number of adaptive hash index page currently assigned to the schema: pages_hashed
• pages_old: Total old page currently assigned to the schema (in the sub-list in the old block in the LRU list pages)
• rows_cached: buffer pool buffer for the total number of data schema rows

 

Fifth, view redundancy index

Use the new MySQL 5.7.9 sys.schema_redundant_indexes view, the data source for sys.x $ schema_flattened_keys.

Field Meaning Reference  https://www.docs4dev.com/docs/zh/mysql/5.7/reference/sys-schema-redundant-indexes.html

select * from schema_redundant_indexes;

 

Sixth, the index view unused

schema_unused_indexes view can see the unused index, the data source for performance_schema.table_io_waits_summary_by_index_usage. The view in the database to run long enough after the data have reference value, before deleting the index must keep up with the business side to confirm.

Field Meaning Reference  https://www.docs4dev.com/docs/zh/mysql/5.7/reference/sys-schema-unused-indexes.html

select * from schema_unused_indexes;

 

Seven, see Table IO and time-consuming statistics

schema_table_statistics_with_buffer view can see by the table, delete, change, check the amount of data, IO time-consuming, as well as statistical information such as buffer occupancy pool in InnoDB.

select * from schema_table_statistics_with_buffer;

Field Meaning Reference  https://dev.mysql.com/doc/refman/5.7/en/sys-schema-table-statistics-with-buffer.html

 

Eight, traffic and disk read and write disk files generated proportional view

io_global_by_file_by_bytes can view + name of the grouping (disk file name) to view the global number of bytes IO by file path, file read and write IO statistics for the number of events by default in descending order according to the total number of bytes read and write IO.

select * from io_global_by_file_by_bytes;

Field name	significance
File	File name being operated
Count_read	There are many times read
Total_read	A total number of bytes read
Avg_read	The average number of bytes read each
Count_write	How many times in total write
Total_written	Wrote a total of how many bytes
Avg_write	Average write bytes each
Total	A total of IO read and write
Write_pct	Write in the proportion accounted for IO

Field Meaning Reference  https://dev.mysql.com/doc/refman/5.7/en/sys-io-global-by-file-by-bytes.html

 

Nine, to see which statements use a full table scan

statements_with_full_table_scans see the full table scan or index is not used to an optimum statement (after normalized statement text conversion), the default scan number according to the average percentage of total delay time and statements (execution time) in descending order. Source: performance_schema.events_statements_summary_by_digest

select * from statements_with_full_table_scans limit 1\G;

*************************** 1. row ***************************
              query: SELECT `performance_schema` .  ... ance` . `SUM_TIMER_WAIT` DESC 
                  db: sys
          exec_count: 1
      total_latency: 938.45 us
no_index_used_count: 1
no_good_index_used_count: 0
  no_index_used_pct: 100
          rows_sent: 3
      rows_examined: 318
      rows_sent_avg: 3
  rows_examined_avg: 318
          first_seen: 2017-09-07 09:34:12
          last_seen: 2017-09-07 09:34:12
              digest: 5b5b4e15a8703769d9b9e23e9e92d499
1 row in set (0.01 sec)

View fields have the following meanings:

• query: standardized conversion of the statement string
• db: default corresponding database statement, if there is no default database, the field is NULL
• exec_count: total number of statements executed
• total_latency: The total delay time of execution of sentence (execution time)
• The total number of statements executed without the use of an index scan table (but the use of full table scan) of: no_index_used_count
• no_good_index_used_count: statement execution is not used to better the total number of index scan table
• Percentage statement is executed does not use an index scan table (but the use of full table scan) the number and total number of statement execution: no_index_used_pct
• rows_sent: statement execution returns from the table to the total number of rows of data the client
• rows_examined: the total number of rows of data from storage engine checks the statement is executed
• rows_sent_avg: Returns the number of clients from the table the average data row for each statement is executed
• rows_examined_avg: the average number of rows of data each statement read from the memory execution engine
• first_seen: time to first occurrence of the statement
• last_seen: last time the statement appeared
• digest: md5 hash value calculation of summary statement

Field Meaning Reference  https://dev.mysql.com/doc/refman/5.7/en/sys-statements-with-full-table-scans.html

 

Ten, to see which statements use the file sorting

statements_with_sorting view to see the document ordering the execution statement, in descending order according to the statement of the total delay time (execution time) By default, data sources: performance_schema.events_statements_summary_by_digest

select * from statements_with_sorting limit 1\G;

*************************** 1. row ***************************
        query: SELECT IF ( ( `locate` ( ? , ` ...  . `COMPRESSED_SIZE` ) ) DESC 
          db: sys
  exec_count: 4
total_latency: 46.53 s
sort_merge_passes: 48
avg_sort_merges: 12
sorts_using_scans: 16
sort_using_range: 0
  rows_sorted: 415391
avg_rows_sorted: 103848
  first_seen: 2017-09-07 12:36:58
    last_seen: 2017-09-07 12:38:37
      digest: 59abe341d11b5307fbd8419b0b9a7bc3
1 row in set (0.00 sec)

View fields have the following meanings:

• query: standardized conversion of the statement string
• db: default corresponding database statement, if there is no default database, the field is NULL
• exec_count: total number of statements executed
• total_latency: The total delay time of execution of sentence (execution time)
• sort_merge_passes: The total number of statements consolidated statement ordering execution occurs
• avg_sort_merges: Statement of the average number of sort-merge SUM_SORT_MERGE_PASSES / COUNT_STAR
• sorts_using_scans: statement ordering the implementation of the total number of full table scans
• sort_using_range: the total number of statements to perform the sort of range scan
• rows_sorted: number of statements that perform the sort occurred total data row
• avg_rows_sorted: Statement of the average number of rows to sort data SUM_SORT_ROWS / COUNT_STAR
• first_seen: time to first occurrence of the statement
• last_seen: last time the statement appeared
• digest: md5 hash value calculation of summary statement

Field Meaning Reference  https://dev.mysql.com/doc/refman/5.7/en/sys-statements-with-sorting.html

 

XI, see which statements use a temporary table

tatements_with_temp_tables view statement uses a temporary table, in descending order according to the number of disk and memory temporary table temporary table quantities by default. Source: performance_schema.events_statements_summary_by_digest

select * from statements_with_temp_tables limit 1\G;

*************************** 1. row ***************************
              query: SELECT `performance_schema` .  ... name` . `SUM_TIMER_WAIT` DESC 
                  db: sys
          exec_count: 2
      total_latency: 1.53 s
  memory_tmp_tables: 458
    disk_tmp_tables: 38
avg_tmp_tables_per_query: 229
tmp_tables_to_disk_pct: 8
          first_seen: 2017-09-07 11:18:31
          last_seen: 2017-09-07 11:19:43
              digest: 6f58edd9cee71845f592cf5347f8ecd7
1 row in set (0.00 sec)

View fields have the following meanings:

• query: standardized conversion of the statement string
• db: default corresponding database statement, if there is no default database, the field is NULL
• exec_count: total number of statements executed
• total_latency: The total delay time of execution of sentence (execution time)
• memory_tmp_tables: The total number of internal memory to create a temporary table when the statement is executed
• The total number of internal temporary disk tables created when you execute the statement: disk_tmp_tables
• avg_tmp_tables_per_query: statement using the average number of SUM_CREATED_TMP_TABLES memory temporary tables / COUNT_STAR
• tmp_tables_to_disk_pct: the total number of percentage of the total amount of memory and disk temporary tables tables indicate that the conversion rate SUM_CREATED_TMP_DISK_TABLES disk temporary tables / SUM_CREATED_TMP_TABLES
• first_seen: time to first occurrence of the statement
• last_seen: last time the statement appeared
• digest: md5 hash value calculation of summary statement

Field Meaning Reference  https://dev.mysql.com/doc/refman/5.7/en/sys-statements-with-temp-tables.html