sql --- slow sql positioning and optimization

1. Locating slow SQL

1. First confirm whether slow query is enabled

mysql> show variables like "%slow%";

slow_query_log is OFF, indicating that the slow query is not enabled, directly set global slow_query_log=on;slow_query_log_file is the address for storing the slow query log

(set global only takes effect for the global session, and it will fail after restarting. If the above configuration needs to take effect permanently, it needs to be configured in mysql.ini (linux my.cnf))

 

2. Set the time limit for slow queries

mysql> show variables like "long_query_time";

The value value is the operation time longer than this value, it is considered to be slow SQL
 
test time value can be set smaller

 

3. Query the slow query log to locate the specific slow SQL

Time : the time the log was recorded

User@Host: Executed user and host

Query_time: Time spent on query Lock_time Lock table time Rows_sent Number of records sent to the requester Rows_examined Number of records scanned by the statement

The point in time when the SET timestamp statement was executed

executed statement

 

4. Related sql query

Query mysql operation information show status Display all mysql operation information

/* 获得mysql的插入次数; */
show status like "com_insert%"; 
/* 获得mysql的删除次数; */
show status like "com_delete%"; 
/* 获得mysql的查询次数; */
show status like "com_select%"; 
/* 获得mysql服务器运行时间; */
show status like "uptime"; 
/* 获得mysql连接次数; */
show status like 'connections'; 
/* 服务器启动以来执行时间最长的20条SQL语句; */

 

5. Use Explain to analyze specific SQL statements

id: selection identifier

select_type: Indicates the type of query.

table: The table that outputs the result set

partitions: matching partitions

type: Indicates the connection type of the table

possible_keys: Indicates the index that may be used when querying

key: Indicates the actual index used

key_len: the length of the index field

ref: comparison of column and index

rows: the number of rows scanned (the estimated number of rows)

filtered: Percentage of rows filtered by table criteria

Extra: Description and clarification of the implementation

 

2. Slow SQL optimization

1. Do not use subqueries

SELECT * FROM t1 WHERE id (SELECT id FROM t2 WHERE name='hechunyang');

( Optimization is only valid for SELECT, not for UPDATE/DELETE subqueries )

 

2. Read the appropriate record LIMIT M,N

can be changed to:

SELECT * FROM t WHERE 1 LIMIT 10;

 

3. Group statistics can prohibit sorting

SELECT goods_id,count(*) FROM t GROUP BY goods_id;

   By default, MySQL sorts all GROUP BY col1, col2... fields. If the query includes GROUP BY and you want to avoid consumption of sorted results, you can specify ORDER BY NULL to disable sorting.

can be changed to:

SELECT goods_id,count(*) FROM t GROUP BY goods_id ORDER BY NULL;

 

4. Prohibit unnecessary ORDER BY sorting

SELECT count(1) FROM user u LEFT JOIN user_info i ON u.id = i.user_id WHERE 1 = 1 ORDER BY u.create_time DESC;

can be changed to:

SELECT count(1) FROM user u LEFT JOIN user_info i ON u.id = i.user_id;

 

5. Try not to join more than three tables

  The data types of the fields that need to be joined must be absolutely consistent; when multi-table association queries are made, ensure that the associated fields need to have indexes

 

6. When creating an index on a varchar field, the index length must be specified

   There is no need to index all fields, and the index length is determined according to the actual text discrimination.

   The length and discrimination of the index are a pair of contradictions. Generally, for string type data, the index with a length of 20 will have a discrimination of more than 90%. You can use count(distinct left(column name, index length))/count( *) to determine the degree of discrimination

 

7. Do not use select *

   Return only the required fields

 

8. Please try to use ascending order for sorting

 

9. Try to use numeric fields

  If the field containing only numerical information should not be designed as a character type, this will reduce the performance of query and connection, and will increase storage overhead.

 

10. Avoid index invalidation

10.1 Field type conversion results in no index

  For example, quotes are not used for string types, quotes are used for numeric types, etc., which may not use indexes and cause full table scans;

 

10.2 According to the second and subsequent fields of the joint index, the index cannot be used for separate queries

 

10.3 Functions/addition and subtraction operations cannot be added in front of the field, otherwise the index will become invalid

The following statement will perform a full table scan:

select id from t where num/2=100 

SELECT * FROM t WHERE YEAR(d) >= 2016

can be changed to:

select id from t where num=100*2 

SELECT * FROM t WHERE d >= '2016-01-01';

 

10.4 Left blur or full blur is strictly prohibited in search

select name from t where name like %s 
select name from t where name like %s% 

  If necessary, please use the search engine to solve it, because the index file has the leftmost prefix matching feature of B-Tree, if the value on the left is not determined, then this index cannot be used.

 

10.5 Avoid using the != or <> operator in the where clause, otherwise the engine will give up using the index and perform a full table scan

select id from t where num != 2 

can be changed to:

select id from t where num > 2 and  num < 2

 

10.6 Avoid judging the null value of the field in the where clause, otherwise the engine will give up using the index and perform a full table scan

select id from t where num is null 

Can be changed to: set the default value of num to 0 to ensure that there is no null value

select id from t where num=0 

 

10.7 Replace OR Inefficient Queries with IN or UNION

SELECT * FROM t WHERE LOC_ID = 10 OR LOC_ID = 20 OR LOC_ID = 30;

can be changed to:

SELECT * FROM t WHERE LOC_IN IN (10,20,30);
或
SELECT * FROM t WHERE LOC_IN = 10 UNION ALL SELECT * FROM t WHERE LOC_IN = 20 UNION ALL SELECT * FROM t WHERE LOC_IN = 30

For continuous values, don't use in if you can use between

select id from t where num between 1 and 3 

 

10.8 Using parameters in the where clause will also cause a full table scan

  Because SQL resolves local variables only at runtime, the optimizer cannot defer the choice of an access plan until runtime; it must choose at compile time. However, if the access plan is established at compile time, the value of the variable is still unknown and thus cannot be used as an input for index selection. The following statement will perform a full table scan:

select id from t where num=@num 

You can instead force the query to use the index:

select id from t with(index(索引名)) where num=@num 

 

11. To delete all records in the table, please use truncate instead of delete

 

12. Stored procedure and trigger settings

  Set SET NOCOUNT ON at the beginning and SET NOCOUNT OFF at the end of all stored procedures and triggers. There is no need to send a DONE_IN_PROC message to the client after every statement in stored procedures and triggers is executed.

 

13. Use cursor-based methods or temporary tables

  Try to avoid using cursors, because cursors are less efficient

  Before using a cursor-based or temporary table approach, look for a set-based solution to your problem, which is usually more efficient.

 

14. Use temporary tables

   Temporary tables can be used when you need to repeatedly reference a dataset in a large table or frequently used table. However, for one-time events, it is better to use an export table.

  When creating a temporary table, if the amount of data inserted at one time is large, you can use select into instead of create table to avoid creating a large number of logs to improve the speed; if the amount of data is not large, in order to ease the resources of the system table, you should first create table , and then insert.

  If temporary tables are used, all temporary tables must be explicitly deleted at the end of the stored procedure, first truncate table, and then drop table, so as to avoid long-term locking of system tables.

  Avoid frequent creation and deletion of temporary tables to reduce the consumption of system table resources.

 
 
Reference link:
https://blog.miuyun.work
https://www.jianshu.com/p/600503b1c791
https://zhuanlan.zhihu.com/p/442169347
https://www.jianshu.com/p/ 3ab117c83d0b
 
If there is anything wrong, please point it out, thanks~