There are many monitoring methods for the current database, which are divided into three categories: built-in database, commercial, and open source, each of which has its own characteristics;

As for the mysql database, due to its high community activity, there are various monitoring methods. The core of any monitoring method is the monitoring data. After obtaining comprehensive monitoring data, it is the flexible display part.

Then today we will introduce the collection and acquisition of monitoring data using mysql's own method, which can achieve the fastest, most convenient, and least loss in a single system.

This article completely uses the show command that comes with mysql to obtain the monitoring data from the aspects of connections, buffercache, lock, SQL, statement, Database throughputs, and serverconfig.

1. Number of connections (Connects)

1.1, the maximum number of used connections

show status like 'Max_used_connections';

1.2, the number of currently open connections

show status like 'Threads_connected';

2. Cache (bufferCache)

2.1, the number of times not read from the buffer pool

show status like 'Innodb_buffer_pool_reads';

2.2, the number of times read from the buffer pool

show status like 'Innodb_buffer_pool_read_requests';

2.3. The total number of pages in the buffer pool

show status like 'Innodb_buffer_pool_pages_total';

2.4. The number of free pages in the buffer pool

show status like 'Innodb_buffer_pool_pages_free';

2.5, cache hit rate calculation

（1-Innodb_buffer_pool_reads/Innodb_buffer_pool_read_requests）*100%

2.6. The cache pool usage rate is

((Innodb_buffer_pool_pages_total-Innodb_buffer_pool_pages_free）/Innodb_buffer_pool_pages_total）*100%

3. Lock (lock)

Remarks: The counted number of lock waiting statistics is cumulative data, which can be subtracted from the previous data each time it is acquired to obtain the current statistical data

3.1, the number of lock waiting

show status like 'Innodb_row_lock_waits';

3.2. Average waiting time for each lock

show status like 'Innodb_row_lock_time_avg';

3.3. Check whether there is a table lock. If there is data, it means that there is a lock table. If it is empty, it means no table lock.

show open TABLES where in_use>0

4. Slow SQL

Remarks: When the execution of the mysqldumpslow command fails, the slow log will be synchronized to the local for formatting.

4.1. Check whether the mysql slow sql switch is turned on

show variables like 'slow_query_log'; --ON 为开启状态，OFF 为关闭状态

set global slow_query_log=1 -- 可进行开启

4.2. View mysql slow sql threshold

show variables like 'long_query_time';

set global long_query_time=0.1 -- 根据页面传递阈值参数，修改阈值

4.3. View mysql slow sql directory

show variables like 'slow_query_log_file';

4.4, format slow sql log

Note: This statement cannot be executed through jdbc, it belongs to the command line execution.
It means: display the execution information of the 10 longest SQL statements, and 10 can be modified as the number of TOP. The displayed information is: execution times, average execution time, SQL statement

mysqldumpslow -s at -t 10 /export/data/mysql/log/slow.log

5、statement

5.1, the number of inserts

show status like 'Com_insert';

5.2, delete quantity

show status like 'Com_delete';

5.3. Number of updates

show status like 'Com_update';

5.4, select quantity

show status like 'Com_select';

6. Throughput (Database throughputs)

6.1. Sending throughput

show status like 'Bytes_sent';

6.2. Receive throughput

show status like 'Bytes_received';

6.3. Total throughput

Bytes_sent+Bytes_received

7. Database parameters (serverconfig)

7.1、show variables

8. Slow SQL troubleshooting steps

Slow SQL refers to MySQL slow queries, specifically SQL that takes longer than the long_query_time value to run.
We often hear that MySQL has binary log binlog, relay log relaylog, redo rollback log redolog, undolog, etc. For slow queries, there is also a slow query log slowlog, which is used to record statements whose response time exceeds the threshold in MySQL. The impact of slow SQL on actual production business is fatal, so it is particularly important for testers to monitor the execution of database SQL statements during performance testing and provide accurate performance optimization advice for development. Then how to use the slow query log provided by the Mysql database to monitor the execution of SQL statements and find the SQL statements with high consumption. The following details the steps of using the slow query log:

8.1. Make sure to turn on the slow SQL switch slow_query_log

8.2. Set the slow SQL domain value long_query_time

This long_query_time is used to define the "slow query" when it is slower than how many seconds. Note that the unit is seconds. I set the value of long_query_time to 1 by executing the sql command set long_query_time=1, that is, the execution time exceeds 1 second. Calculate the slow query, as follows:

8.3. Check the slow SQL log path

8.4. Format and analyze slow SQL logs through the slow SQL analysis tool mysqldumpslow

mysqldumpslow is a slow query analysis tool that comes with mysql after installation. You can view the parameter description through ./mysqldumpslow --help

8.4.1, common usage

Take out the 10 most used slow queries

./mysqldumpslow -s c -t 10 /export/data/mysql/log/slow.log

Take out the 3 slow queries with the slowest query time

./mysqldumpslow -s t -t 3 /export/data/mysql/log/slow.log

Note: The analysis result using mysqldumpslow will not display the specific and complete sql statement, but only the composition structure of sql;
if: SELECT FROM sms_send WHERE service_id=10 GROUP BY content LIMIT 0, 1000;
After the mysqldumpslow command is executed, it will display:
Count: 2 Time=1.5s (3s) Lock=0.00s (0s) Rows=1000.0 (2000), vgos_dba[vgos_dba]@[10.130.229.196]SELECT FROM sms_send WHERE service_id=N GROUP BY content LIMIT N, N

8.4.2 Detailed explanation of the analysis results of mysqldumpslow

Count: Indicates the number of execution times of this type of statement. In the figure above, the select statement is executed twice.
Time: Indicates the average execution time (total time) of this type of statement
Lock: The lock time is 0s.
Rows: 1000 records are returned for a single return, and a total of 2000 records are returned for 2 times.

Through this tool, you can find out which SQL statements are slow SQL, so as to feed back the research and development for optimization, such as adding indexes, the implementation method of the application, and so on.

8.5. Troubleshoot common slow SQL

8.5.1. Do not use subqueries

SELECT FROM t1 WHERE id (SELECT id FROM t2 WHERE name='hechunyang');

In MySQL 5.5, the internal execution planner executes the subquery in this way: first check the outer table and then match the inner table instead of first checking the inner table t2. When the data in the outer table is large, the query speed will be very slow.
In the MariaDB10/MySQL5.6 version, it is optimized by using the join association method, and this SQL will be automatically converted to

 SELECT t1. FROM t1 JOIN t2 ON t1.id = t2.id;

But please note that optimization is only valid for SELECT and not valid for UPDATE/DELETE subqueries. In the production environment, try to avoid using subqueries.

8.5.2. Avoid function indexes

SELECT FROM t WHERE YEAR(d) >= 2016;

Since MySQL does not support functional indexes like Oracle, even if the d field has an index, it will scan the entire table directly.
should be changed to:

 SELECT FROM t WHERE d >= ‘2016-01-01’;

8.5.3. Use IN to replace OR inefficient query

slow

SELECT FROM t WHERE LOC_ID = 10 OR LOC_ID = 20 OR LOC_ID = 30;

efficient query

SELECT FROM t WHERE LOC_IN IN (10,20,30);

8.5.4, LIKE double percent signs cannot use the index

SELECT FROM t WHERE name LIKE '%de%';

should be changed to:

SELECT FROM t WHERE name LIKE 'de%';

8.5.5. Group statistics can disable sorting

SELECT goods_id,count() FROM t GROUP BY goods_id;

By default, MySQL sorts all GROUP BY col1, col2... fields. If the query includes GROUP BY and you want to avoid consumption of sorted results, you can specify ORDER BY NULL to disable sorting.
should be changed to:

SELECT goods_id,count () FROM t GROUP BY goods_id ORDER BY NULL;

8.5.6. Prohibit unnecessary ORDER BY sorting

SELECT count(1) FROM user u LEFT JOIN user_info i ON u.id = i.user_id WHERE 1 = 1 ORDER BY u.create_time DESC;