Commonly used MySQL optimization methods

On the one hand, database optimization is to find out the bottleneck of the system and improve the overall performance of the MySQL database. On the other hand, it requires reasonable structural design and parameter adjustment to improve the corresponding speed of users. At the same time, it is necessary to save system resources as much as possible so that The system provides a greater load.
　
In this article, let’s talk about the MySQL optimization methods commonly used in projects, a total of 19, as follows:

1. EXPLAIN

To optimize MySQL, we need to make good use of EXPLAIN to view the SQL execution plan.

Let's take a simple example to mark (1, 2, 3, 4, 5) the data we want to focus on:

• type column, connection type. A good SQL statement must at least reach the range level. Avoid all levels.
• The key column, the name of the index used. If no index is selected, the value is NULL. Compulsory indexing can be adopted
• key_len column, index length.
• rows column, the number of rows to scan. This value is an estimate.
• extra column, detailed description. Note that common unfriendly values ​​are as follows: Using filesort, Using temporary.

2. The value contained in IN in the SQL statement should not be too much

MySQL has made corresponding optimizations for IN, that is, all the constants in IN are stored in an array, and this array is sorted. However, if the value is large, the consumption will be relatively large. Another example: select id from t where num in(1,2,3) For continuous values, don't use in if you can use between; or use connection to replace.

3. The SELECT statement must specify the field name

SELECT* adds a lot of unnecessary consumption (CPU, IO, memory, network bandwidth); increases the possibility of using covering indexes; when the table structure changes, the previous break also needs to be updated. Therefore, it is required to connect the field name directly after the select.

4. When only one piece of data is needed, use limit 1

This is to make the type column in EXPLAIN reach the const type

5. If the sorting field does not use an index, try to sort as little as possible

6. If there is no index for other fields in the restriction, try to use or as little as possible

If one of the fields on both sides of or is not an index field, and the other conditions are not index fields, the query will not use the index. In many cases, using union all or union (when necessary) instead of "or" will get better results.

Seven, try to use union all instead of union

The main difference between union and union all is that the former needs to merge the result sets and then perform unique filtering operations, which involves sorting, increases a lot of CPU operations, increases resource consumption and delay. Of course, the prerequisite for union all is that there is no duplicate data in the two result sets.

8. Do not use ORDER BY RAND()

select id from `dynamic` order by rand() limit 1000;

The above SQL statement can be optimized as:

select id from `dynamic` t1 join (select rand() * (select max(id) from `dynamic`) as nid) t2 on t1.id > t2.nidlimit 1000;

9. Distinguish between in and exists, not in and not exists

select * from 表A where id in (select id from 表B)

The above SQL statement is equivalent to

select * from 表A where exists(select * from 表B where 表B.id=表A.id)

The distinction between in and exists is mainly due to the change of the driving sequence (this is the key to performance change). If it is exists, then the outer table is the driving table and is accessed first. If it is IN, then the subquery is executed first. Therefore, IN is suitable for the case where the appearance is large and the inner table is small; EXISTS is suitable for the case where the appearance is small and the inner table is large.

Regarding not in and not exists, it is recommended to use not exists, not only for efficiency issues, but for not in there may be logic problems. How to efficiently write a SQL statement that replaces not exists?

Original SQL statement:

select colname … from A表 where a.id not in (select b.id from B表)

Efficient SQL statement:

select colname … from A表 Left join B表 on where a.id = b.id where b.id is null

The fetched result set is shown in the figure below, the data in table A is not in table B:

10. Use a reasonable paging method to improve the efficiency of paging

select id,name from product limit 866613, 20

When using the above SQL statement for paging, some people may find that as the amount of table data increases, directly using the limit paging query will become slower and slower.

The optimization method is as follows: You can take the id of the maximum number of rows on the previous page, and then limit the starting point of the next page according to the maximum id. For example, in this column, the largest id of the previous page is 866612. SQL can be written as follows:

select id,name from product where id> 866612 limit 20

Eleven, segment query

In some user selection pages, some users may select too large a time range, resulting in slow query. The main reason is that there are too many scan lines. At this time, the program can be used to query by segment, loop through, and combine the results for display.

For the SQL statement shown in the figure below, segmented query can be used when the number of scanned rows is more than one million:

12. Avoid judging the null value of the field in the where clause

The judgment of null will cause the engine to give up using the index and perform a full table scan.

13. It is not recommended to use % prefix fuzzy query

For example, LIKE "%name" or LIKE "%name%", this kind of query will cause the index to fail and perform a full table scan. But you can use LIKE "name%".

So how to query %name%?

As shown in the figure below, although an index is added to the secret field, it is not used in the explain result:

So how to solve this problem, the answer: use full-text indexing.

Select id, fnum, fdst from dynamic_201606 where user_name like '%zhangsan%'; is often used in our queries. For such a statement, ordinary indexes cannot meet the query requirements. Fortunately, in MySQL, there are full-text indexes to help us.

The SQL syntax for creating a full-text index is:

ALTER TABLE `dynamic_201606` ADD FULLTEXT INDEX `idx_user_name` (`user_name`);

The SQL statement using the full-text index is:

select id,fnum,fdst from dynamic_201606 where match(user_name) against('zhangsan' in boolean mode);

Note: Before you need to create a full-text index, please contact the DBA to determine whether it can be created. At the same time, it should be noted that the writing method of the query statement is different from that of ordinary indexes.

14. Avoid performing expression operations on fields in the where clause

for example:

select user_id,user_project from user_base where age*2=36;

Arithmetic operations are performed on the fields, which will cause the engine to give up using the index. It is recommended to change it to:

select user_id,user_project from user_base where age=36/2;

15. Avoid implicit type conversion

The type conversion occurs when the type of the column field in the where clause is inconsistent with the type of the parameter passed in. It is recommended to determine the parameter type in where first.

16. For joint indexes, the leftmost prefix rule should be followed

For example, the index contains the fields id, name, and school. You can use the id field directly, or you can use the order of id, name, but name; school cannot use this index. Therefore, when creating a joint index, you must pay attention to the order of the index fields, and the commonly used query fields are placed at the front.

Seventeen, if necessary, you can use force index to force the query to go to a certain index

Sometimes the MySQL optimizer uses the index it thinks is appropriate to retrieve the SQL statement, but the index it uses may not be what we want. At this time, forceindex can be used to force the optimizer to use the index we made.

18. Pay attention to range query statements

For a joint index, if there is a range query, such as between, >, < and other conditions, it will cause the subsequent index fields to become invalid.

The LEFT JOIN A table is the driving table, the INNER JOIN MySQL will automatically find the table with less data to act as the driving table, and the RIGHT JOIN B table is the driving table.

Notice:

1) There is no full join in MySQL, which can be solved in the following ways:

select * from A left join B on B.name = A.namewhere B.name is nullunion allselect * from B;

2) Try to use inner join and avoid left join:

There are at least two tables participating in the joint query, and generally there are different sizes. If the connection method is inner join, MySQL will automatically select the small table as the driving table if there are no other filter conditions, but the left join follows the principle of driving the left to the right in the selection of the driving table, that is, the table name on the left side of the left join for the drive table.

3) Reasonable use of indexes:

The index field of the driven table is used as the restriction field of on.

4) Use small tables to drive large tables:

It can be seen intuitively from the schematic diagram that if the driver table can be reduced, the number of loops in the nested loop can be reduced to reduce the total amount of IO and the number of CPU operations.

5) Use STRAIGHT_JOIN skillfully:

The inner join is selected by MySQL as the driving table, but in some special cases, another table needs to be selected as the driving table, such as when there are group by, order by, etc. "Using filesort" and "Using temporary". STRAIGHT_JOIN to force the connection order, the table name on the left side of STRAIGHT_JOIN is the driving table, and the right side is the driven table. A prerequisite for using STRAIGHT_JOIN is that the query is an inner join, that is, an inner join. It is not recommended to use STRAIGHT_JOIN for other links, otherwise it may cause inaccurate query results.

This method can sometimes reduce the time by 3 times.