The optimization content in this article is mainly aimed at MySQL, and some will be tested in actual operations. What is the principle of index design
with reference to
the summary of the most dry things in SQL optimization (2020 latest version) ?
How to avoid index failure?
Sum up three points
- Maximize the use of indexes
- Avoid full table scans as much as possible
- Reduce queries for invalid data
The example used in this article: the mainten table
SELECT count(*) FROM or_mainten
already has an index
1. Avoid select *
Using select * to take out all the columns will make the optimizer unable to complete the optimization of index coverage scan, which will affect the optimizer's choice of execution plan, increase network bandwidth consumption, and bring additional I/O, memory and CPU consume.
Suggestion: only query the fields required by the business
2. To optimize the query, you should try to avoid full table scanning. First, you should consider building indexes on the columns involved in where and order by
Note: Only order by alone cannot use the index
3. Try to avoid fuzzy query at the beginning of the field, which will cause the database engine to give up the index for full table scan. as follows:
| SELECT * FROM t WHERE username LIKE '%陈%' |
| SELECT * FROM t WHERE username LIKE '陈%' |
Practical test:
If the requirement is to use fuzzy query in front,
- Use the MySQL built-in function INSTR(str, substr) to match, the function is similar to indexOf() in java, query the position of the subscript where the string appears
- Use FullText full-text index, search with match against
- In the case of a large amount of data, it is recommended to use ElasticSearch and Solr, and the retrieval speed of billions of data is in seconds
- When the amount of table data is small (thousands of items), don't be fancy, just use like '%xx%' directly.
4. Try to avoid using the != or <> operator in the where clause, otherwise the engine will give up using the index and perform a full table scan
5. Try to avoid using or to connect the conditions in the where clause, otherwise the engine will give up using the index and perform a full table scan, such as:
In the figure below, both REAL_MAINTEN_PERSON and status are in the index INDEX_REAL_MAINTEN_PERSON, but the index fails in the case of or
Solution: Create an index for each column in the or condition, but it is contrary to the fact that the index cannot be created too much
6. Try to avoid performing expression operations on fields in the where clause, which will cause the engine to give up using indexes and perform full table scans, such as:
| select id from t where num/2=100 |
should be changed to:
| select id from t where num=100*2 |
Actual operation
select int field
add index
7. Do not perform functions, arithmetic operations or other expression operations on the left side of "=" in the where clause, otherwise the system may not be able to use the index correctly.
Actual operation
Add index
Index failure
Index normal application
8. Try to avoid performing function operations on fields in the where clause, which will cause the engine to give up using indexes and perform full table scans. like:
| select id from t where substring(name,1,3)='abc' |
should be changed to:
| select id from t where name like 'abc%' |
Actual operation
9. The leftmost prefix matching principle
When using an index field as a condition, if the index is a composite index, then the first field in the index must be used as the condition to ensure that the system uses the index, otherwise the index will not be used, and should be used as much as possible The order of the fields matches the order of the index.
Actual operation
10. Select a column with a high degree of discrimination as an index
Not all indexes are effective for queries. For example, if there is a field sex in a table, and men and women are almost half equally, then even if an index is built on sex, it will have no effect on query efficiency.
The reason is that when the index column has a large amount of data duplication, MySQL also has a query optimizer. When the query optimizer finds that a certain value appears in a high percentage of data rows in the table, it generally ignores the index and performs a full table scan. .
A customary percentage cut-off is "30%". (When the amount of matching data exceeds a certain limit, the queryer will give up using the index (this is also one of the scenarios where the index fails).
11. More indexes are not better
Indexes can improve the efficiency of the corresponding select, but at the same time reduce the efficiency of insert and update, because the index may be rebuilt during insert or update, so how to build the index needs to be carefully considered, depending on the specific situation.
It is best not to have more than 6 indexes for a table. If there are too many, you should consider whether it is necessary to build indexes on some columns that are not frequently used.
12. Try to use numeric fields
If the field containing only numerical information should not be designed as a character type, this will reduce the performance of query and connection, and will increase storage overhead.
This is because the engine will compare each character in the string one by one when processing queries and connections, but for numeric types, only one comparison is enough
13. Use varchar instead of char as much as possible
Because firstly, the storage space of the variable-length field is small, which can save storage space, and
secondly, for the query, the search efficiency in a relatively small field is obviously higher.
14. Create indexes for frequently queried fields
When we build an index, we need to build an index for those fields that are often used as query conditions, which can improve the query speed of the entire table.
However, the query condition is generally not a field, so more joint indexes are generally established.
In addition, there are generally fuzzy queries such as like in the query conditions. If it is a fuzzy query, it is best to follow the leftmost prefix query principle.
15. Avoid indexing "big fields"
This can be said in other words: try to use fields with a small amount of data as indexes.
For example, suppose there are two such fields, one is varchar(5) and the other is varchar(200). In this case, it is preferred to build an index for the field of varchar(5), because MySQL maintains the index Sometimes the field values are maintained together, which will inevitably cause the index to take up more space, and it will take more time to compare when sorting.
So what if you want to create an index for varchar(100)? Then take part of the data, for example, the address type is varchar(200), and you can write it like this when building an index:
| CREATE INDEX tbl_address ON dual(address(20)); |
16. Avoid implicit conversions
Assuming that the field age type is int, then we usually query like this
| SELECT * FROM student WHERE age=15 |
| SELECT * FROM student WHERE age='15' |
The actual operation found in MySQL will not fail, and it has not been verified in other databases
1. Try to avoid judging the null value of the field in the where clause, otherwise the engine will give up using the index and perform a full table scan, such as:
| select id from t where num is null |
You can set a default value of 0 on num, make sure that there is no null value in the num column in the table, and then query like this:
| select id from t where num=0 |
Hands-on discovery in MySQL doesn't give up using indexes
2. Try to avoid using in and not in, which will cause the engine to perform a full table scan. as follows:
| SELECT * FROM t WHERE id = 1 OR id = 3 |
Actual operation
