In the last article "Conversation and Lock Information Query View | A Comprehensive Understanding of the sys System Library" , we introduced how to use the general view of the sys system library to query session status information and lock waiting information. The content of this issue will be introduced first. Quick view of statistics related to query tables and indexes. Please follow us to start the system learning journey of sys system library.
PS: Due to the particularity of the view functions mentioned in this article (DBA may need to query some statistical information for some data analysis in daily work), the text of select statements in some views will be listed below, so that everyone is more intuitive To learn them.
01
schema_auto_increment_columns
In all databases (excluding system dictionary libraries mysql, sys, INFORMATION_SCHEMA, performance_schema), search for base tables with auto-incrementing columns and related information. By default, they are sorted in descending order according to auto-increment usage rate and auto-increment column type maximum. Data source: INFORMATION_SCHEMA's COLUMNS, TABLES
This view was added in MySQL 5.7.9
View query text
SELECT TABLE_SCHEMA,
TABLE_NAME,
COLUMN_NAME,
DATA_TYPE,
COLUMN_TYPE,
(LOCATE('unsigned', COLUMN_TYPE) = 0) AS is_signed,
(LOCATE('unsigned', COLUMN_TYPE) > 0) AS is_unsigned,
(
CASE DATA_TYPE
WHEN 'tinyint' THEN 255
WHEN 'smallint' THEN 65535
WHEN 'mediumint' THEN 16777215
WHEN 'int' THEN 4294967295
WHEN 'bigint' THEN 18446744073709551615
END >> IF(LOCATE('unsigned', COLUMN_TYPE) > 0, 0, 1)
) AS max_value,
AUTO_INCREMENT,
AUTO_INCREMENT / (
CASE DATA_TYPE
WHEN 'tinyint' THEN 255
WHEN 'smallint' THEN 65535
WHEN 'mediumint' THEN 16777215
WHEN 'int' THEN 4294967295
WHEN 'bigint' THEN 18446744073709551615
END >> IF(LOCATE('unsigned', COLUMN_TYPE) > 0, 0, 1)
) AS auto_increment_ratio
FROM INFORMATION_SCHEMA.COLUMNS
INNER JOIN INFORMATION_SCHEMA.TABLES USING (TABLE_SCHEMA, TABLE_NAME)
WHERE TABLE_SCHEMA NOT IN ('mysql', 'sys', 'INFORMATION_SCHEMA', 'performance_schema')
AND TABLE_TYPE='BASE TABLE'
AND EXTRA='auto_increment'
ORDER BY auto_increment_ratio DESC, max_value;
Let's take a look at the results returned by the query using this view
admin@localhost : sys 11:11:58> select * from schema_auto_increment_columns limit 5;
+--------------+------------+-------------+-----------+------------------+-----------+-------------+------------+----------------+----------------------+
| table_schema | table_name | column_name | data_type | column_type | is_signed | is_unsigned | max_value | auto_increment | auto_increment_ratio |
+--------------+------------+-------------+-----------+------------------+-----------+-------------+------------+----------------+----------------------+
| sbtest | sbtest1 | id | int | int(10) unsigned | 0 | 1 | 4294967295 | 10713891 | 0.0025 |
| sbtest | sbtest2 | id | int | int(10) unsigned | 0 | 1 | 4294967295 | 10710865 | 0.0025 |
| sbtest | sbtest3 | id | int | int(10) unsigned | 0 | 1 | 4294967295 | 10714919 | 0.0025 |
| sbtest | sbtest4 | id | int | int(10) unsigned | 0 | 1 | 4294967295 | 10714039 | 0.0025 |
| sbtest | sbtest5 | id | int | int(10) unsigned | 0 | 1 | 4294967295 | 10713075 | 0.0025 |
+--------------+------------+-------------+-----------+------------------+-----------+-------------+------------+----------------+----------------------+
5 rows in set (1.50 sec)
The meanings of the view fields are as follows:
TABLE_SCHEMA: The schema name of the self-incremented table
TABLE_NAME: The name of the table containing the AUTO_INCREMENT value
column_name: column name of AUTO_INCREMENT value
data_type: the data type of the auto-increment column
COLUMN_TYPE: The column attribute type of the self-incrementing column, that is, some other information is added to the data type. For example: for bigint(20) unsigned, the entire information is called the column attribute type, and the data type just refers to bigint
is_signed: Whether the column type is signed
is_unsigned: Whether the column type is unsigned
MAX_VALUE: The maximum value of the auto-increment column
auto_increment: the current AUTO_INCREMENT attribute value of the auto-increment column
auto_increment_ratio: The ratio of the self-increment value currently used by the auto-increment column to the maximum self-increment value of the auto-increment column, indicating the current utilization rate of the auto-increment column
02
schema_index_statistics,x$schema_index_statistics
Index statistics are sorted by default in descending order of the total delay time (execution time) of adding, deleting, modifying and checking operations using the index, data source: performance_schema.table_io_waits_summary_by_index_usage
View query text
# 不带x$前缀的视图
SELECT OBJECT_SCHEMA AS table_schema,
OBJECT_NAME AS table_name,
INDEX_NAME as index_name,
COUNT_FETCH AS rows_selected,
sys.format_time(SUM_TIMER_FETCH) AS select_latency,
COUNT_INSERT AS rows_inserted,
sys.format_time(SUM_TIMER_INSERT) AS insert_latency,
COUNT_UPDATE AS rows_updated,
sys.format_time(SUM_TIMER_UPDATE) AS update_latency,
COUNT_DELETE AS rows_deleted,
sys.format_time(SUM_TIMER_INSERT) AS delete_latency
FROM performance_schema.table_io_waits_summary_by_index_usage
WHERE index_name IS NOT NULL
ORDER BY sum_timer_wait DESC;
# 带x$前缀的视图查询语句与不带x$前缀的视图查询语句相比，只是少了单位格式化函数
......
Let's take a look at the results returned by the query using this view
# 不带x$前缀的视图
admin@localhost : sys 11:19:43> select * from schema_index_statistics limit 5;
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
| table_schema | table_name | index_name | rows_selected | select_latency | rows_inserted | insert_latency | rows_updated | update_latency | rows_deleted | delete_latency |
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
| xiaoboluo | test | PRIMARY | 1159 | 3.57 s | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps |
| sys | sys_config | PRIMARY | 1 | 62.53 ms | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps |
| sbtest | sbtest1 | i_c | 20 | 31.43 ms | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps |
| xiaoboluo | test | i_test | 400 | 3.77 ms | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps |
| luoxiaobo | public_num | PRIMARY | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps | 0 | 0 ps |
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
5 rows in set (0.45 sec)
# 带x$前缀的视图
admin@localhost : sys 11:20:21> select * from x$schema_index_statistics limit 5;
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
| table_schema | table_name | index_name | rows_selected | select_latency | rows_inserted | insert_latency | rows_updated | update_latency | rows_deleted | delete_latency |
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
| xiaoboluo | test | PRIMARY | 1159 | 3573795058125 | 0 | 0 | 0 | 0 | 0 | 0 |
| sys | sys_config | PRIMARY | 1 | 62528964375 | 0 | 0 | 0 | 0 | 0 | 0 |
| sbtest | sbtest1 | i_c | 20 | 31429669125 | 0 | 0 | 0 | 0 | 0 | 0 |
| xiaoboluo | test | i_test | 400 | 3765146625 | 0 | 0 | 0 | 0 | 0 | 0 |
| luoxiaobo | public_num | PRIMARY | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+--------------+------------+------------+---------------+----------------+---------------+----------------+--------------+----------------+--------------+----------------+
5 rows in set (0.00 sec)
The meanings of the view fields are as follows:
TABLE_SCHEMA: The name of the schema containing the index table
TABLE_NAME: The name of the table containing the index
INDEX_NAME: The name of the index
rows_selected: The total number of rows of data read using the index
select_latency: the total latency of reading using the index (execution time)
rows_inserted: the total number of rows inserted into the index
insert_latency: the total latency of inserting index rows (execution time)
rows_updated: the total number of rows updated by the index
update_latency: the total delay time for index update rows (execution time)
rows_deleted: the total number of rows deleted from the index
delete_latency: the total latency of deleting rows from the index (execution time)
03
schema_object_overview
The statistical information of the tables, views, indexes and other objects contained in each schema is sorted by default according to the schema name and object type. Data source: routines, tables, statistics, triggers, events of information_schema
Note: For MySQL instances with a large number of database objects, this view may take a long time to complete
View query text
SELECT ROUTINE_SCHEMA AS db, ROUTINE_TYPE AS object_type, COUNT(*) AS count FROM information_schema.routines GROUP BY ROUTINE_SCHEMA, ROUTINE_TYPE
UNION
SELECT TABLE_SCHEMA, TABLE_TYPE, COUNT(*) FROM information_schema.tables GROUP BY TABLE_SCHEMA, TABLE_TYPE
UNION
SELECT TABLE_SCHEMA, CONCAT('INDEX (', INDEX_TYPE, ')'), COUNT(*) FROM information_schema.statistics GROUP BY TABLE_SCHEMA, INDEX_TYPE
UNION
SELECT TRIGGER_SCHEMA, 'TRIGGER', COUNT(*) FROM information_schema.triggers GROUP BY TRIGGER_SCHEMA
UNION
SELECT EVENT_SCHEMA, 'EVENT', COUNT(*) FROM information_schema.events GROUP BY EVENT_SCHEMA
ORDER BY DB, OBJECT_TYPE;
Let's take a look at the results returned by the query using this view
admin@localhost : sys 11:20:27> select * from schema_object_overview limit 10;
+--------------------+---------------+-------+
| db | object_type | count |
+--------------------+---------------+-------+
| information_schema | SYSTEM VIEW | 61 |
| luoxiaobo | BASE TABLE | 3 |
| luoxiaobo | INDEX (BTREE) | 3 |
| mysql | BASE TABLE | 31 |
| mysql | INDEX (BTREE) | 69 |
| performance_schema | BASE TABLE | 87 |
| qfsys | BASE TABLE | 1 |
| qfsys | INDEX (BTREE) | 1 |
| sbtest | BASE TABLE | 8 |
| sbtest | INDEX (BTREE) | 17 |
+--------------------+---------------+-------+
10 rows in set (0.27 sec)
The meanings of the view fields are as follows:
db: schema name
OBJECT_TYPE: database object type, valid values are: BASE TABLE, INDEX (index_type), EVENT, FUNCTION, PROCEDURE, TRIGGER, VIEW
count: the number of database objects in each schema
04
schema_redundant_indexes
Find duplicate or redundant indexes, data source: sys.x$schema_flattened_keys, the data source view is called the auxiliary view of the schema_redundant_indexes view
schema_redundant_indexes view is added in MySQL 5.7.9
Let's take a look at the results returned by the query using this view
admin@localhost : sys 11:21:13> select * from schema_redundant_indexes limit 1\G;
*************************** 1. row ***************************
table_schema: test
table_name: test
redundant_index_name: i_id
redundant_index_columns: id
redundant_index_non_unique: 1
dominant_index_name: i_id_id2
dominant_index_columns: id,id2
dominant_index_non_unique: 1
subpart_exists: 0
sql_drop_index: ALTER TABLE `test`.`test` DROP INDEX `i_id`
1 row in set (0.01 sec)
The meanings of the view fields are as follows:
TABLE_SCHEMA: The schema name corresponding to the table containing redundant or duplicate indexes
TABLE_NAME: The name of the table that contains redundant or duplicate indexes
redundant_index_name: redundant or duplicate index name
redundant_index_columns: redundant or duplicate index column names
redundant_index_non_unique: The number of non-unique columns in redundant or duplicate indexes
dominant_index_name: the dominant (best) index name compared to duplicate or redundant indexes
dominant_index_columns: the column name in the dominant (best) index
dominant_index_non_unique: the number of non-unique columns in the dominant (best) index
subpart_exists: whether the duplicate or redundant index is a prefix index
sql_drop_index: drop index statement generated for repeated or redundant indexes
05
schema_table_statistics,x$schema_table_statistics
View the statistics of the table. By default, the total table I/O latency (execution time, which can also be understood as the table with the most table I/O contention) is sorted in descending order according to the total table I/O delay time of the addition, deletion, modification, and query operation. Data source: performance_schema .table_io_waits_summary_by_table, sys.x$ps_schema_table_statistics_io
These views use an auxiliary view x$ps_schema_table_statistics_io
Let's take a look at the results returned by the query using this view
# 不带x$前缀的视图
admin@localhost : sys 11:52:25> select * from schema_table_statistics limit 1\G
*************************** 1. row ***************************
table_schema: xiaoboluo
table_name: test
total_latency: 2.10 m
rows_fetched: 1561
fetch_latency: 2.08 m
rows_inserted: 1159
insert_latency: 865.33 ms
rows_updated: 0
update_latency: 0 ps
rows_deleted: 0
delete_latency: 0 ps
io_read_requests: 43
io_read: 178.86 KiB
io_read_latency: 15.00 ms
io_write_requests: 10
io_write: 160.00 KiB
io_write_latency: 76.24 us
io_misc_requests: 42
io_misc_latency: 9.38 ms
1 row in set (0.03 sec)
# 带x$前缀的视图
admin@localhost : sys 11:52:28> select * from x$schema_table_statistics limit 1\G;
*************************** 1. row ***************************
table_schema: xiaoboluo
table_name: test
total_latency: 125711643303375
rows_fetched: 1561
fetch_latency: 124846318302750
rows_inserted: 1159
insert_latency: 865325000625
rows_updated: 0
update_latency: 0
rows_deleted: 0
delete_latency: 0
io_read_requests: 43
io_read: 183148
io_read_latency: 15001512375
io_write_requests: 10
io_write: 163840
io_write_latency: 76237125
io_misc_requests: 42
io_misc_latency: 9384933000
1 row in set (0.02 sec)
The meanings of the view fields are as follows:
TABLE_SCHEMA: The name of the schema where the table containing the TABLE_NAME field is located
TABLE_NAME: table name
total_latency: the total delay time (execution time) of I/O events of the table, for table additions, deletions, and changes
rows_fetched: The total number of data rows in table read operations, for table query operations
fetch_latency: The total delay time (execution time) of the I/O event of the table select operation, for table query operations
rows_inserted: The total number of data rows for table insert operations, for table insert operations
insert_latency: I/O event delay time (execution time) of table insert operation, for table insert operation
rows_updated: The total number of data rows in table update operations, for table update operations
update_latency: The total delay time (execution time) of the I/O event of the table update operation, for the table update operation
rows_deleted: the total number of rows of data in table delete operations, for table delete operations
delete_latency: The total delay time (execution time) of I/O events for table delete operations, for table delete operations
io_read_requests: the total number of requests for table read operations, read I/O operations for table .ibd and .frm files
io_read: The total number of bytes of all file read operations related to table read operations, read I/O operations for table .ibd and .frm files
io_read_latency: The total latency (execution time) of all file read operations related to table read operations, and read I/O operations for table .ibd and .frm files
io_write_requests: The total number of requests for table write operations, write I/O operations for table .ibd and .frm files
io_write: The total number of bytes of all file write operations related to table write operations, and write I/O operations for table .ibd and .frm files
io_write_latency: The total latency (execution time) of all file write operations related to table write operations, and write I/O operations for table .ibd and .frm files
io_misc_requests: The total number of I/O requests for all files related to various other miscellaneous operations, and other miscellaneous I/O operations for .ibd and .frm files
io_misc_latency: The total latency (execution time) of I/O requests for all files related to various other miscellaneous operations, and other miscellaneous I/O operations for table .ibd and .frm files
06
schema_table_statistics_with_buffer,x$schema_table_statistics_with_buffer
Query table statistics, which also includes InnoDB buffer pool statistics. By default, the total table I/O delay time (execution time, which can also be understood as the existence of the most table I/O contention Table) Sorting in descending order, data source: performance_schema.table_io_waits_summary_by_table, sys.x$ps_schema_table_statistics_io, sys.x$innodb_buffer_stats_by_table
These views use the auxiliary view sys.x$ps_schema_table_statistics_io
Let's take a look at the results returned by the query using this view
# 不带x$前缀的视图
admin@localhost : sys 12:36:57> select * from schema_table_statistics_with_buffer limit 1\G;
*************************** 1. row ***************************
table_schema: xiaoboluo
table_name: test
rows_fetched: 1561
fetch_latency: 2.08 m
rows_inserted: 1159
insert_latency: 865.33 ms
rows_updated: 0
update_latency: 0 ps
rows_deleted: 0
delete_latency: 0 ps
io_read_requests: 48
io_read: 179.29 KiB
io_read_latency: 15.02 ms
io_write_requests: 10
io_write: 160.00 KiB
io_write_latency: 76.24 us
io_misc_requests: 47
io_misc_latency: 9.47 ms
innodb_buffer_allocated: 112.00 KiB
innodb_buffer_data: 48.75 KiB
innodb_buffer_free: 63.25 KiB
innodb_buffer_pages: 7
innodb_buffer_pages_hashed: 0
innodb_buffer_pages_old: 0
innodb_buffer_rows_cached: 1162
1 row in set (2.21 sec)
# 带x$前缀的视图
admin@localhost : sys 12:37:35> select * from x$schema_table_statistics_with_buffer limit 1\G;
*************************** 1. row ***************************
table_schema: xiaoboluo
table_name: test
rows_fetched: 1561
fetch_latency: 124846318302750
rows_inserted: 1159
insert_latency: 865325000625
rows_updated: 0
update_latency: 0
rows_deleted: 0
delete_latency: 0
io_read_requests: 48
io_read: 183595
io_read_latency: 15019373250
io_write_requests: 10
io_write: 163840
io_write_latency: 76237125
io_misc_requests: 47
io_misc_latency: 9465938250
innodb_buffer_allocated: 114688
innodb_buffer_data: 49917
innodb_buffer_free: 64771
innodb_buffer_pages: 7
innodb_buffer_pages_hashed: 0
innodb_buffer_pages_old: 0
innodb_buffer_rows_cached: 1162
1 row in set (2.12 sec)
The meanings of the view fields are as follows:
The meaning of the statistical information fields related to the table is the same as that of the view schema_table_statistics, which is omitted here. For details, please refer to the schema_table_statistics, x$schema_table_statistics view explanation section
innodb_buffer_allocated: the total number of bytes in the buffer pool currently allocated to the table
innodb_buffer_data: The total number of bytes in the buffer pool used by the data portion currently allocated to the table
innodb_buffer_free: the total number of bytes in the buffer pool currently allocated to the non-data part of the table (that is, the number of bytes where the free page is located, calculation formula: innodb_buffer_allocated-innodb_buffer_data)
innodb_buffer_pages: the total number of pages of the buffer pool currently allocated to the table
innodb_buffer_pages_hashed: The total number of adaptive hash index pages currently allocated to the table
innodb_buffer_pages_old: The total number of old pages currently allocated to the table (the number of pages in the old block sublist in the LRU list)
innodb_buffer_rows_cached: the total number of rows of data buffered for the table in the buffer pool
07
schema_unused_indexes
Look at the inactive indexes (the index without any incident, which means that the index has never been used), sorted by schema name and table name by default. Data source: performance_schema.table_io_waits_summary_by_index_usage
After the view runs for a long time after the server is started, the queried data is more applicable, otherwise the data queried by the view may not be very reliable, because the statistical data may not be accurate, and some business query logic may be Too late to query
View query text
SELECT object_schema,
object_name,
index_name
FROM performance_schema.table_io_waits_summary_by_index_usage
WHERE index_name IS NOT NULL
AND count_star = 0
AND object_schema != 'mysql'
AND index_name != 'PRIMARY'
ORDER BY object_schema, object_name;
Let's take a look at the results returned by the query using this view
admin@localhost : sys 12:40:28> select * from schema_unused_indexes limit 3;
+---------------+-------------+-------------------+
| object_schema | object_name | index_name |
+---------------+-------------+-------------------+
| luoxiaobo | public_num | public_name_index |
| sbtest | sbtest1 | k_1 |
| sbtest | sbtest2 | k_2 |
+---------------+-------------+-------------------+
3 rows in set (0.00 sec)
The meanings of the view fields are as follows:
object_schema: schema name
OBJECT_NAME: table name
INDEX_NAME: unused index name
The content of this issue is introduced here, and the reference link for the content of this issue is as follows:
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-unused-indexes.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-auto-increment-columns.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-index-statistics.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-object-overview.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-redundant-indexes.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-table-statistics.html
https://dev.mysql.com/doc/refman/5.7/en/sys-schema-table-statistics-with-buffer.html
| About the author
Luo Xiaobo·Database Technology Expert
One of the authors of "A Thousand Gold Recipes-MySQL Performance Optimization Pyramid Rule", "Data Ecology: MySQL Replication Technology and Production Practice". Familiar with MySQL architecture, good at overall database tuning, like to specialize in open source technology, and keen on the promotion of open source technology, have done many public database topic sharing online and offline, and published nearly 100 database-related research articles.
The full text is over.
Enjoy MySQL :)
Teacher Ye's "MySQL Core Optimization" class has been upgraded to MySQL 8.0, scan the code to start the journey of MySQL 8.0 practice