MYSQL Advanced - Query Optimization - Slow Query Logs You Know and Slow Query Logs You Don't Know

Sending back to the city – " 100 days proficient in MYSQL from entry to employment"

1. Exercise topics

topic link	difficulty
Advanced SQL - Query Optimization - Slow Query Log	★★★☆☆

2. SQL Ideas: Advanced SQL - Query Optimization - Slow Query Log

Initialization data

drop table if exists student;
CREATE TABLE `student` (
`id` int NOT NULL AUTO_INCREMENT COMMENT '自增ID',
`order_num` long NOT NULL  COMMENT '序号',
`student_name` varchar(20) NOT NULL COMMENT '姓名',
`age` int COMMENT '年龄',
 PRIMARY KEY (`id`)
 )ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_general_ci;

Insert 1 million pieces of data. Here, a stored procedure is used to submit data in batches. Turn off automatic submission first, and then insert a certain number of pieces before submitting.

--创建存储过程

drop procedure if exists add_student;

CREATE  PROCEDURE `add_student`(in n int,in batchNum int) 
BEGIN
    DECLARE i INT DEFAULT 1;
		
	WHILE (i < n+1 ) DO
	
	set autocommit = 0;
		INSERT into student (order_num,student_name,age) VALUES (i,concat('student_name',i),20);
		set i=i+1;
		if i mod batchNum = 0 then
                commit;
     end if;
	END WHILE;
	 commit;
END

-- 调用
CALL add_student(1000000,100000) 

solution

Request processing:

• Choose one correct answer from four options

analyze:

What is the slow query log

mysql supports executing sql statements longer than a certain time long_query_time to be recorded in the log, which is convenient for developers or DBA personnel to optimize sql statements. More than a certain time: This can be configured by the parameter long_query_time .

How to enable the slow query log

the first way

The slow query log can be enabled in the mysql configuration file.
"my.ini" file in mysql:

• 1. In the Linux system, the default location of the file is "/etc/mysql/my.cnf";
• 2. In the windows system, the default location of the file is "C:\ProgramData\MySQL\MySQL Server **\Data\my.ini".

Among them, C:\ProgramData is a hidden directory, you need to display the hidden directory to find it.

[mysqld]
slow_query_log = 1
slow_query_log_file = /data/mysql/log/query_log/slow_statement.log
long_query_time = 10
log_output = FILE

Description of configuration items:

slow_query_log: 1 means to enable the slow query log. 0: means close the slow query log.
slow_query_log_file: The place where the slow query log is saved and the corresponding file name.
long_query_time: Specifies how many seconds the SQL statement execution time exceeds to record the slow query log.
log_output: File: means to store the log in a file, the default value is 'FILE'; Table: means to store the log in the database, so that the log information will be written into the mysql.slow_log table.
Experience: When recorded in the data table, the slow query time recorded in the data table can only be accurate to seconds; if it is recorded in the log file, the slow query time recorded in the log file can be accurate to microseconds.
It is recommended to record slow query logs to files in actual work.

Important: After the configuration is complete, restart the mysql server! The configuration will take effect!

windows combat

The default location of this file is "C:\ProgramData\MySQL\MySQL Server **\Data\my.ini".
Find the file:

modify the my.ini configuration:

# General and Slow logging.
slow_query_log=1
slow_query_log_file="d:/build/mysql/slow_statement.log"
long_query_time=10
log_output=FILE

After saving the configuration, restart the mysql service.

Open cmd with administrator privileges, enter the following command in cmd, and press Enter:

 net stop mysql57

Note: Different mysql versions may have different service names, you can go to the service to find the corresponding service name

and restart the mysql service.

net start mysql57

Check whether the configuration takes effect:

show variables like '%slow_query_log%'; 
show variables like '%long_query_time%'; 
show variables like '%log_output%'; 

slow_query_log is always off, no effect

Hey, the slow_query_log parameter does not take effect. I checked the parameter name and there is no typo. What is the reason?

Brother Xuzhu checked the information and found that under Linux: the directory where the slow_query_log_file is located requires the writable permission of the MySQL running account. If you need to change the log file directory, remember to grant 777 permissions to the new directory, otherwise slow_query_log will always be OFF

Expanding from this idea, will mysql or will not help create a directory that does not exist.

slow_query_log_file="d:/build/mysql/slow_statement.log"

Sure enough, it didn't help to create a directory. Try to make up the directory, and then restart the service to try:
it really works

The slow query log file is also generated to

draw the key point: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!
Key points: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!
Key points: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!

verify

Execute the sql statement to trigger the slow query mechanism:

SELECT BENCHMARK(99999999, MD5('mysql'));

View the log content, as shown in the figure, it has been recorded successfully:

linux actual combat

First create a storage directory for slow query logs:

mkdir /usr/local/mysql/data/query_log

Give the directory read and write permissions

chmod 777 query_log/

The default location of this file is "/etc/mysql/my.cnf".
Brother Xuzhu customizes the location of my.cnf: /etc/my.cnf
SQL quick start-install MYSQL environment (multi-environment enterprise-level guidance)
Find the file:

Modify my.cnf configuration:

slow_query_log = 1
slow_query_log_file = /usr/local/mysql/data/query_log/slow_statement.log
long_query_time = 10
log_output = FILE

After saving the configuration, restart the mysql service.
Check the status of the mysql service:

systemctl status mysqld

The service is turned on, first shut down, and then start.

systemctl stop mysqld

systemctl start mysqld

Then check the status of the mysql service:

Check whether the configuration is effective:

show variables like ‘%slow_query_log%’;
show variables like ‘%long_query_time%’;
show variables like ‘%log_output%’;

The slow query log file is also generated to

draw the key point: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!
Key points: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!
Key points: the file directory specified by slow_query_log_file must exist! If it does not exist, manually create the directory first, and then restart the service!

verify

Execute the sql statement to trigger the slow query mechanism:

SELECT BENCHMARK(99999999, MD5('mysql'));

View the log content, as shown in the figure, it has been recorded successfully:

the second way

Execute the command on the MySQL command line to enable the slow query log:

SET GLOBAL slow_query_log = 1;
SET GLOBAL slow_query_log_file = '/data/mysql/log/query_log/slow_statement.log';
SET GLOBAL long_query_time = 10;
SET GLOBAL log_output = 'FILE';

Key point: This method does not require restarting the mysql service. It will fail after restarting MySQL.

This method does not distinguish between operating systems

Practical

First check the relevant parameters of the slow query log

show variables like '%slow_query_log%'; 
show variables like '%long_query_time%'; 
show variables like '%log_output%'; 

We see that the slow_query_log parameter value is OFF, indicating that the slow query log is not enabled.
Execute the command on the MySQL command line to enable the slow query log, and other parameters will not be set. The first method is configured:

SET GLOBAL slow_query_log = 1;

Look at the relevant parameters of the slow query log, which has been enabled

show variables like '%slow_query_log%'; 

At the same time, the slow query log file is also generated:

verify

Execute the sql statement to trigger the slow query mechanism:

SELECT BENCHMARK(99999999, MD5('mysql'));

View the log content, as shown in the figure, it has been recorded successfully:

Turn off the slow query log

To turn off the slow query log, you only need to configure slow_query_log=0 in the my.cnf file or my.ini file or delete this option directly.

the first way

windows combat

Modify the my.ini configuration:

# General and Slow logging.
slow_query_log=0

After saving the configuration, restart the mysql service.

Open cmd with administrator privileges, enter the following command in cmd, and press Enter:

net stop mysql57

Note: Different mysql versions may have different service names, you can go to the service to find the corresponding service name

and restart the mysql service.

net start mysql57

Manually delete the previous slow query log first:

check whether the configuration takes effect:

show variables like '%slow_query_log%'; 

The configuration takes effect:

verify

Execute the sql statement to see if the slow query mechanism will be triggered:

SELECT BENCHMARK(99999999, MD5('mysql'));

Result Close the slow query log successfully:

linux actual combat

Modify my.cnf configuration:

slow_query_log=0

After saving the configuration, restart the mysql service.
Check the status of the mysql service:

systemctl status mysqld

The service is turned on, first shut down, and then start.

systemctl stop mysqld

systemctl start mysqld

Check the status of the mysql service again:

manually delete the previous slow query log first:

Check whether the configuration takes effect:

show variables like '%slow_query_log%'; 

The configuration takes effect:

verify

Execute the sql statement to see if the slow query mechanism will be triggered:

SELECT BENCHMARK(99999999, MD5('mysql'));

Result Close the slow query log successfully:

the second way

Execute the command on the MySQL command line to close the slow query log:

SET GLOBAL slow_query_log = 0;

The slow query threshold is 10s, changed to 1s.

set global long_query_time= 1;

After the modification, it is recommended to open a session query again (the value of the old session query may be the original 10s).
Key point: this method does not need to restart the mysql service. It will fail after restarting MySQL.

expand

Slow query log failure reason 1: Online dynamic setting of long_query_time will not take effect for the currently established connection

Check whether the slow query log is enabled

show variables like '%slow_query_log%'; 
show variables like '%long_query_time%'; 

is already enabled. If it is not enabled, please operate as described above, and will not repeat it here.
The slow query threshold is 10s, changed to 1s.

SET GLOBAL long_query_time = 1;

After modification, it is recommended to re-open a session query (the value of the old session query may be the original 10s)

Slow query log failure reason 2: SQL records that do not use indexes will not be written into the slow query log

Check the settings, the default is off.

show variables like 'log_queries_not_using_indexes';

Check whether the slow query log is enabled

show variables like '%slow_query_log%'; 
show variables like '%long_query_time%'; 

is already enabled. If it is not enabled, please operate as described above, and will not repeat it here.

Test sql
to check the index of the student table:

show index from student;

Execute sql without using indexes:

select *
from student
where student_name ='student_name446741'

It is not recorded in the slow query log.

Set SQL records that do not use indexes to be written to the slow query log:

set global log_queries_not_using_indexes=on;

Then execute the slow query sql

select *
from student
where student_name ='student_name446741'

Data without indexes is also added to the slow query log.

Note: The log_queries_not_using_indexes parameter should be used in conjunction with the parameter log_throttle_queries_not_using_indexes

log_throttle_queries_not_using_indexes: This parameter determines the upper limit of the number of unused index SQL records per minute, because there may be a lot of unused index SQL, causing the slow log space to grow rapidly.

Slow query log failure reason 3: there is a lock waiting in the slow sql

There are a lot of lock waiting in slow SQL, and the execution time of slow SQL does not include the lock waiting time

Slow query log failure reason 4: By default, the slow SQL of management commands is not recorded

log_slow_admin_statements=0, so alter, create index, analyze table and other operations will not be recorded in the slow log even if they exceed long_query_time.
log_slow_admin_statements This parameter determines whether to record management commands, including ALTER TABLE, ANALYZE TABLE, CHECK TABLE, CREATE INDEX, DROP INDEX, OPTIMIZE TABLE, REPAIR TABLE. By default, such statements are not recorded in the slow log.

Slow query log failure reason five: min_examined_row_limit is not 0

min_examined_row_limit This parameter defines the number of data rows read by a SQL;
if min_examined_row_limit is set to a non-zero value, the number of rows checked by SQL does not exceed this value, and will not be recorded.
The default value is 0

Slow query log failure reason six: The slow log file handle has changed

The drill handle of the slow log file has changed, such as opening the log with vim during operation, and saving and exiting at the end. At this time, the file handle has changed, and flush slow logs needs to be executed.

Slow query log failure reason seven: the copy statement from the database is not recorded by default

log_slow_slave_statements: This parameter is set on the slave library to determine whether to record SQL that exceeds long_query_time during the replication process. If the binlog format is row, even if this parameter is enabled, related SQL will not be recorded.
Unless the binlog format is statement and log_slow_slave_statements is enabled.

3. Summary

This article introduces how to enable and disable the slow query log in the windows environment and the Linux environment. There are two ways to operate the enable and disable respectively. One is to modify the configuration and restart the service to take effect; the other is to quickly take effect through commands, but it will fail after restarting MySQL.
You can choose according to the actual business scenario.
Seven failure causes of slow query logs are additionally added, all of which are valuable experience explored from actual combat.
So, um, the answer to this question is optional. . Tell Brother Xu Zhu loudly in the comment area.

4. Reference

MySQL advanced skill tree –> slow query log

I'm Brother Xu Zhu, see you tomorrow~