[MySQL] MySQL tuning skills

Why SQL Tuning?

First of all, we need to understand why SQL tuning is needed. In fact, the most important thing for enterprises to require SQL tuning is to help the company save money. Why do you say that? Let's look at this comparison

Cost optimization: hardware > system configuration > database table structure > SQL and indexes
Optimization effect: hardware<system configuration<database table structure<SQL and index

So we can see that if the SQL is tuned well, the availability of the entire system will be greatly improved.

five principles

We generally carry out five principles for SQL optimization:

1. Reduce data access: set reasonable field types, enable compression, reduce disk IO through index access, etc., and store unimportant data in Redis or MongoDB for sub-database storage.

2. Return less data: only return required fields or paging data to reduce disk IO and network IO

3. Reduce the number of connections: it is easy to perform batch operations on DML (commonly speaking, add, delete, modify and check) and function storage

4. Reduce server CPU overhead: minimize database sorting operations and full table queries, and reduce CPU memory usage

5. Utilize more resources: Using table partitions can increase parallel operations and maximize the use of CPU resources.

When we understand the principle of SQL optimization, we must first figure out the execution order of SQL:

You can jump to this link SELECT statement - grammatical order_Shizhu Baishao's Blog-CSDN Blog

About SQL Optimization Strategies

1. Avoid the situation of not taking the index

The following is the situation that causes the database engine to give up the index and then perform a full table scan, as well as its optimization strategy

1. Try to avoid fuzzy query at the beginning of the field

SELECT *
FROM user
WHERE username LIKE '%白%'
--->尽量在字段后面使用模糊查询。
SELECT *
FROM user
WHERE username LIKE '白%'

2. Try to avoid using in and not in

SELECT * 
FROM user 
WHERE id IN (2,3)
--->如果是连续数值，可以用between代替
SELECT * 
FROM user 
WHERE id BETWEEN 2 AND 3

If it is a subquery, you can use exists instead.

-- 不走索引
select * from A where A.id in (select id from B);
-- 走索引
select * from A where exists (select * from B where B.id = A.id);

3. Try to avoid using or

SELECT * 
FROM t 
WHERE id = 1 OR id = 3
--->可以使用union代替or
SELECT * FROM t WHERE id = 1
    UNION
SELECT * FROM t WHERE id = 3

4. Try to avoid judging NULL

SELECT * 
FROM t
WHERE score IS NULL
--->可以给默认字段添加默认值0，对0进行判断
SELECT *
FROM t
WHERE score = 0

5. Try to avoid performing expression or function operations on the left side of the equal sign in the where condition

--->全表扫描
SELECT * FROM T WHERE score/10 = 9
--->走索引
SELECT * FROM T WHERE score = 10*9

6. When the amount of data is large, avoid using the condition where 1=1. Usually, in order to facilitate the assembly of query conditions, we will use this condition by default.

SELECT 
    username, age, sex 
FROM T 
WHERE 1=1

Optimization method: judge when assembling SQL with code, remove where if there is no where condition, and add and if there is a where condition.

7. The query condition cannot use <> or !=

When using index columns as conditions for query, you need to avoid using <> or != and other judgment conditions. If there is a real business need and the not equal symbol is used, the index creation needs to be re-evaluated to avoid indexing this field and replace it with other index fields in the query condition.

8. The where condition only contains the non-leading columns of the composite index

As follows: The composite (joint) index contains three columns: key_part1, key_part2, and key_part3, but the SQL statement does not include the index front column "key_part1". According to the leftmost matching principle of MySQL's joint index, the joint index will not be used.

select col1 from table where key_part2=1 and key_part3=2

9. Implicit type conversion causes no index to be used

The following SQL statement cannot be indexed correctly because the column type of the index is varchar, but the given value is a numeric value, which involves implicit type conversion.

select col1 from table where col_varchar=123;

10. The order by condition must be consistent with the condition in where, otherwise the order by will not use the index to sort

---> 不走age索引
SELECT * FROM t order by age;
 
---> 走age索引
SELECT * FROM t where age > 0 order by age;

The correct processing sequence for this SQL is:

Generate an execution plan based on WHERE conditions and statistical information, and get data.
When executing order by, the database will first check the execution plan of the first step to see whether the fields of order by use indexes in the execution plan. If so, you can use the index order to directly get the sorted data. Otherwise reorder operations.
return sorted results

Only when the fields in order by (or group by, union, etc.) appear in the where condition, the index will be used instead of secondary sorting.

2. About the optimization of the SELECT statement

1. Avoid select*

Everyone is familiar with this, so why avoid select*? Here are three points:

Unnecessary columns increase data transfer time as well as network overhead
For useless large fields, such as varchar, text, will increase IO operations
Possibility of losing MySQL optimizer "covering index" strategy optimization
- SELECT* eliminates the possibility of covering indexes, and the "covering index" strategy based on our MySQL optimizer is extremely fast and efficient, and is recommended by the industry.

2. Avoid functions with indeterminate results

For example, it is easy for us to use functions with uncertain results such as now(), rand(), sysdate(), etc., resulting in inconsistent data between the master library and the slave library.

3. When multi-table association query, the small table comes first, and the large table follows.

Because in MySQL, table association query is from left to right, and a table will involve full table scanning, so we generally put small tables in front, so that the scanning efficiency will be higher.

4. Use table aliases

When there are multiple tables, it is recommended to put the alias of the table before each column name.

3. Optimize the DML statement

1. Insert data in large batches

insert into T values(1,2); 
 
insert into T values(1,3); 
 
insert into T values(1,4);

--->
Insert into T values(1,2),(1,3),(1,4);

When using the second type of insertion, it only needs to be parsed once. And the SQL is shorter, which can also reduce the IO of network transmission