In fact, the return table we talked about before is to use two index trees at the same time. First, the corresponding primary key value is searched in the secondary index tree, and then the complete record is queried in the primary key index tree.
But my question today is, will two different secondary index trees take effect at the same time? In theory, it should be able to take effect at the same time, otherwise this MySQL is too stupid. However, according to Songge's daily development experience, it is best to avoid this kind of thing. If two index trees are searched at the same time, there is probably a problem with your index design. At this time, you should check whether the index design is reasonable.
Bold is practical experience, but we still need to understand the knowledge points that two indexes take effect at the same time. Let's take a look.
1. Index Merge
For example, I have the following table structure:
CREATE TABLE `user` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`username` varchar(32) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`address` varchar(32) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`password` varchar(32) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`email` varchar(16) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `username` (`username`),
KEY `address` (`address`)
) ENGINE=InnoDB AUTO_INCREMENT=100001 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
There are two indexes, username and address, in this table. Note that there are two indexes, and each index has a field. This is not a joint index.
Now my query SQL is as follows:
select * from user where username='1' or address='1';
There are two search conditions, username and address, which are two indexes that belong to two different index trees. So when it searches, will it search both index trees? Or just search one index tree and use another search criteria to filter the results of the first tree search?
Let's take a look at the database execution plan:
At a glance at the execution plan, you can guess that both indexes are used here, and there are several new faces in this execution plan:
- type is
index_merge. - Extra is
Using union(username,address); Using where.
in this type index_mergeis index merge.
2. Old version gameplay
Of course, this is index_mergenot the beginning, this is something introduced since MySQL 5.0. Although everyone will not use the version before MySQL 5.0 anymore, I will talk about it here to deepen everyone's understanding of MySQL. Before MySQL 5.0, for the query SQL we gave above, there would be no index, but a full table scan. In those days, if you wanted to implement the above query, but also wanted to use the index, your SQL had to be written like this:
select * from user where username='1' union all select * from user where address='1' and username!='1'
However, this writing is obviously a bit clumsy.
Therefore, starting from MySQL 5.0, multiple indexes can be automatically used for scanning in the query, and the results are merged, which is the index merge ( index_merge) we mentioned earlier.
3. Three cases
There are three variants of the index merge algorithm, which we will look at separately.
3.1 union
This is to find the union of two indices.
Let's look at the following SQL:
select * from user where username like '1%' or address like '1%';
This SQL will involve two indexes in the process of execution. You need to search in two index trees, and then combine the search results. Let's take a look at the execution plan of the SQL:
As you can see, an index merge has occurred in this execution plan (see type, key, Extra).
So is the index merge always sent as long as there are two index queries? Let's look at another chestnut:
select * from user where username>'a' or address='1';
Let's take a look, it's just that the search conditions have changed a bit. Here, there is no index merging, but a full table scan. Why? This leads to a condition for index merging, that is: the search conditions corresponding to each index, the searched primary keys must be in order, if the searched primary keys are out of order, sorry, index merging cannot be used. In the secondary index, the data is sorted in the order of the secondary index, and the structure is similar to the following:
| username | primary key |
|---|---|
| a | 20 |
| b | 30 |
| c | 9 |
| c | 10 |
| c | 18 |
| d | 1 |
| d | 5 |
When the usernames are the same, the primary keys are in order. When the usernames are different, the primary keys cannot be guaranteed to be in order. If the acquired primary keys are out of order, index merging cannot be achieved.
This brings up another question, why can index merging occur only when the obtained primary keys are ordered? Because only when the primary key is ordered, it will be more efficient to remove duplicates (union, sort-union) or find intersection (intersect) in the future.
Starting from MySQL 5.0, index merging is enabled by default. Of course, you can also choose to disable it. The way to disable union index merging is as follows:
SET optimizer_switch = 'index_merge_union=off';
After closing, look at the execution plan:
As you can see, index merging still occurs, but this time it is not union, but sort_union, so let's take a look at what sort_union is.
3.2 sort_union
sort_union is basically the same as union, but with one more sorting ability.
Because we said earlier that if an unordered primary key is obtained, index merging will not occur, and a full table scan may eventually be performed directly. Therefore, MySQL has another sort_union, which is to search both the username index tree and the address index tree at the same time. After getting the primary key value , sort it first, then remove the duplicates after sorting, and then return to the table to get the complete data.
Compared with union, it is mainly a bold step.
Then we continue and close sort_union, as follows:
SET optimizer_switch = 'index_merge_sort_union=off';
After closing, look at the execution plan, as follows:
At this point, there is no index merging, and a full table scan is performed directly.
3.3 intersect
This is to find the intersection of two indices.
For example the following SQL:
select * from user where username like '1%' and address like '1%';
In the process of executing this SQL, there may be a situation of seeking intersection. Of course, this is not absolute, and it depends on the situation after optimization by the optimizer.
Song Ge has been trying for a long time and can't reproduce an example, mainly because my simulation data is not quite right. If you have ready-made Using intersect examples, please leave a message to share (Using intersect will appear in the execution plan Extra).
But I will share this principle with you here, let's take a look at the following picture:
Suppose there is a secondary index S and a secondary index T, and now cross to obtain the primary key ( one thing to note here, if we are searching on S and T separately, and the search condition on S is username like '1%', the search condition on T is address like '1%', then in During the search process, the primary key ids obtained by each are in order, which is also the premise of intersect ):
- First, go to the secondary index S to search and find the first record that meets the conditions. Since the leaf nodes of the secondary index store the primary key value, don't rush back to the table after getting the primary key value.
- Next, go to the secondary index T to search, find the first record that meets the conditions, and get the corresponding primary key value.
- Compare the primary key values obtained from the first and second searches:
3.1 If the primary key values are not equal, discard the primary key with a smaller value and leave the larger primary key. The next time you search on S, take this The big primary key is compared with the primary key searched on S.
3.2 If the primary key values are equal, it means that the primary key satisfies the search conditions, then take this primary key back to the table. - Repeat the first three steps until there are no records that meet the conditions in the respective indexes.
This is called cross-fetching the primary key .
Well, these are the three cases of index merging.
4. Summary
Many small partners may say that since there is an index merge, can I just create an index? nonono! Index merging is a last resort. If an index merging occurs, there is a high probability that the index you designed is unreasonable, so we should think about how to optimize the index.
References:
- https://dev.mysql.com/doc/refman/8.0/en/index-merge-optimization.html
- "How MySQL Works"
- "High Performance MySQL"
- https://www.modb.pro/db/29619