Definition of index
MySQL's official definition of index is: Index is a data structure that assists MySQL in obtaining data efficiently.
Essentially, the purpose of indexing is to improve query efficiency, by constantly narrowing the scope of the data you want to obtain to filter out the final desired results, and at the same time turning random events into sequential events, that is to say, with this Index mechanism, we can always use the same search method to lock data.
It can be likened to a safe in a bank, for example, if you want to find a safe that belongs to you. If there is no index, you need to hold the key and try the safes one by one to find the safe that belongs to you. But if there is an index, and the safe can be physically partitioned in the corresponding area, and you can find the storage room A1003 where the safe is located according to the number on the key (A1003-10-17), and find the storage room insurance On the 10th row of the cabinet, find the 17th position and find your safe. This position is much faster. In the absence of an index, it is quite difficult to accomplish this.
Principle of Index
In addition to safes, many similar index examples can be cited in life, such as the catalog of dictionary dictionaries, the search records of libraries, and the seat schedule of trains.
Their principle is the same: continuously narrow the data range to filter the data, and turn random data into sequential data, so that we can lock the data faster.
This kind of index understanding also applies to our database query, but the database will have many more complicated situations. In addition to equivalent queries, there are range queries (>, <, between, in), fuzzy queries (like), and unions. Query (or), intersection query (and), etc. This requires the database to choose a more complex and mature way to deal with all problems.
According to the case of our safe above, the data can be split according to certain rules, so that the scope of matching is reduced, but this is far from enough to meet the complex query requirements of the database. Therefore, the designer of the database system optimizes it from the perspective of the query algorithm.
The most basic query algorithm is linear search. The complexity of this algorithm is O(n), which is not ideal when the amount of data is large, and the larger the amount of data, the more complicated the calculation.
But it doesn't matter, powerful computer science provides more excellent search algorithms, such as binary search, binary tree search, and so on.
However, these search algorithms are required to be applied to specific data structures. For example, binary search requires the retrieved data to be ordered, while binary tree search can only be operated on the binary search tree structure, and the organizational structure of the data itself cannot fully meet various requirements. Data structure, theoretically, it is impossible to require multiple columns to be organized in order at the same time.
Therefore, in addition to data, the database system also maintains data structures that meet specific search algorithms. These data structures reference (point to) data in a certain way, so that advanced search algorithms can be implemented on these data structures. This data structure is the index.
This echoes the official MySQL definition of index above.
Look at the picture below:
The figure illustrates an indexing method. On the right is a data table. Here, a total of two columns and seven rows of data are simulated. Field 1 is the physical address of the data record (in actual applications, logically adjacent records are not necessarily physically adjacent to each other on the disk. This side mainly To give an example). In order to speed up the search of field 2, you can maintain a binary search tree as shown on the left. Each node contains the index key value and a pointer to the physical address of the corresponding data record, so that you can use the binary search in O(log2n) Obtain the corresponding data within the complexity of O(log2n).
This is a manifestation of the index, but the actual database system is more commonly implemented by using B+ trees. The B in the B+ tree stands for balance, not binary. Because the B+ tree evolved from the earliest balanced binary tree, we can first understand the next binary search tree, the balanced binary tree (AVLTree) and the balanced multiple search tree (B-Tree), because the B+ tree is gradually developed from these trees. Evolved.
Binary search tree
The binary tree has the following properties: the key value of the left subtree is less than the key value of the root, and the key value of the right subtree is greater than the key value of the root. So the left, middle and right is a process of increasing sequentially.
As shown in the figure below, it is a binary search tree,
Observing the binary tree, we find that the number of searches for a node with a depth of 1 is 1, the number of searches for a node with a depth of 2 is 2, and the number of searches for a node with a depth of n is n, so the average number of searches is (1+2+ 2+3+3+3+3) / 7 = 2.4 times.
The binary search tree can also be of the following structure (also satisfying the characteristics of the left<middle<large of the binary tree), which is also the seven numbers of 7,21,35,42,51,77,89, which can also be used as shown in the figure below structure:
But the query efficiency of this binary tree is low, the average number of searches is (1+2+3+4+5+6+6)/7=3.8 times.
Therefore, if you want the query efficiency of the binary tree to be as high as possible, the binary tree needs to be balanced, which leads to a new definition: AVL tree (that is, balanced binary tree).
Balanced Binary Tree (AVL Tree)
A balanced binary tree (AVL tree) meets the condition of a binary search tree, and the maximum difference of the height of the two subtrees of any node is 1. In the two pictures below, the left is an AVL tree, and the height difference between the two subtrees of any node is <=1; the right is not an AVL tree, and the height of the left subtree of its root node is 3, and the height of the right subtree is 1. , Height difference>1;
In the same way, inserting or deleting nodes in a balanced binary tree may also cause the AVL tree to lose balance. This unbalanced binary tree can have four states: LL (left left), RR (right right), LR (left and right), RL (right left).
Look at the icon:
Let's take a look at these states one by one.
LL (LeftLeft), that is, left left. It means that after inserting or deleting a node, the left child of the left child of the root node (Left Child) has non-empty nodes, causing the left subtree of the root node to be higher than the right subtree> 1, and the AVL tree loses balance .
RR (RightRight), that is, right right. It means that after inserting or deleting a node, the right child of the root node's right child (Right Child) still has non-empty nodes, which causes the root node's right subtree to be higher than the left subtree> 1, and the AVL tree loses balance .
LR (LeftRight), that is, left and right. After inserting or deleting a node, the left child of the root node (Left Child) and the right child (Right Child) still have non-empty nodes, causing the left subtree of the root node to be higher than the right subtree by >1, and the AVL tree loses balance.
RL (RightLeft), that is, right left. After inserting or deleting a node, the left child of the right child of the root node still has non-empty nodes, causing the right subtree of the root node to be higher than the left subtree> 1, and the AVL tree loses balance.
An AVL tree that is out of balance can be repaired by rotation. The essence of rotation is to adjust the nodes of the tree to achieve the purpose of restoring balance. Let's look at them one by one below.
Rotation of LL: When LL is out of balance, the AVL tree can be rebalanced with one rotation. Proceed as follows:
1. Use the left child of the root node as the new root node.
2. Use the right child of the new root node as the left child of the original root node.
3. Make the original root node the right child of the new root node.
As shown below:
Rotation of RR: When RR is out of balance, the method of rotation is opposite to that of LL. The steps are as follows:
1. Use the right child of the root node as the new root node.
2. Take the left child of the new root node as the right child of the original root node.
3. Use the original root node as the left child of the new root node.
As shown below:
Rotation of LR: When LR is out of balance, two rotations are required. The steps are as follows:
1. Perform RR rotation around the left child of the root node.
2. Perform LL rotation around the root node.
As shown in the figure below, it was rotated twice and finally restored to an AVL tree:
Rotation of RL: Two rotations are required when RL loses balance. The rotation method is opposite to LR rotation. The steps are as follows:
1. Perform LL rotation around the right child of the root node.
2. Perform RR rotation around the root node.
As shown in the figure below, it was rotated twice and finally restored to an AVL tree:
Balanced multi-way search tree (B-Tree)
We know that disk storage devices are based on disk blocks (block), and B-tree is also a balanced search tree designed based on this storage method.
Therefore, when we read data from the system disk, we use the disk block as the basic unit to map it to the memory. The data in the same disk block will be read out at once instead of just fetching the required data. The InnoDB storage engine has the concept of Page, which is the smallest unit of its disk management. The default size of each page in the InnoDB storage engine is 16KB. The page size can be set to 4K, 8K, 16K through the parameter innodb_page_size. We can enter the following script in the command window to view:
1 mysql> show variables like 'innodb_page_size';
2 +------------------+-------+
3 | Variable_name | Value |
4 +------------------+-------+
5 | innodb_page_size | 16384 |
6 +------------------+-------+
7 1 row in set
However, the storage space of a disk block in the system is often not so large, so every time InnoDB applies for disk space, there will be a number of consecutive disk blocks to reach the page size of 16KB.
InnoDB uses pages as the basic unit when reading disk data to disk. When querying data, if each piece of data in a page can help locate the location of the data record,
This will reduce the number of disk I/Os and improve query efficiency.
The data in the B-Tree structure allows the system to efficiently find the disk block where the data is located. In order to describe B-Tree, first define a record as a two-tuple [key, data], key is the key value of the record, corresponding to the primary key value in the table, and data is the data except the primary key in a row of records. For different records, the key values are different from each other.
A B-Tree of order m has the following characteristics:
1. Each node has at most m children.
2. Except for the root node and leaf nodes, every other node has at least Ceil(m/2) children.
3. If the root node is not a leaf node, it has at least 2 children
4. All leaf nodes are on the same layer and do not contain other keyword information
5. Each non-terminal node contains n key information (P0, P1,...Pn, k1,...kn)
6. The number of keywords n satisfies: ceil(m/2)-1 <= n <= m-1
7. ki(i=1,...n) are keywords, and the keywords are sorted in ascending order.
8. Pi(i=1,...n) is a pointer to the root node of the subtree. All node keys of the subtree pointed to by P(i-1) are less than ki, but are greater than k(i-1)
Each node in the B-Tree can contain a large amount of keyword information and branches according to the actual situation. The following figure shows a 3-level B-Tree:
Each node occupies the disk space of one disk block. A node has two keywords in ascending order and three pointers to the root node of the subtree. The pointer stores the address of the disk block where the subnode is located. The three range fields divided into the two key-value data correspond to the range fields of the data of the subtree pointed to by the three pointers. Take the root node as an example. The two key value data are 33 and 66, the data range of the subtree pointed to by the P1 pointer is less than 33, the data range of the subtree pointed to by the P2 pointer is between 33 and 66, and the subtree pointed to by the P3 pointer is between 33 and 66. The data range of the tree is greater than 66.
Simulate the process of searching for keyword 55:
1. Find the disk block Disk1 according to the root node and read it into the memory. The first disk I/O operation.
2. Compare the key value 55 in the interval (33, 66) and find the pointer P2 of the disk block Disk1.
3. Find the disk block Disk3 according to the P2 pointer and read it into the memory. The second disk I/O operation.
4. Compare the key value 55 in the interval (39, 62) and find the pointer P2 of the disk block Disk3.
5. Find the disk block Disk8 according to the P2 pointer and read it into the memory. The third disk I/O operation.
6. Find the keyword 55 in the key value list in Disk8.
Through the above operation process, it is found that 3 disk I/O operations and 3 memory lookup operations are required. Since the key in the memory is an ordered list structure, the binary search can be used to improve efficiency. The three disk I/O operations are the decisive factor that affects the efficiency of the entire B-Tree search.
Compared with AVLTree, B-Tree reduces the number of nodes, so that the data fetched into memory each time by disk I/O plays a role, thereby improving query efficiency.
B+Tree
B+Tree is an optimization based on B-Tree, which makes it more suitable for implementing external storage index structure. InnoDB storage engine uses B+Tree to realize its index structure.
From the above B-Tree structure diagram, we can see that each node contains not only the key value of the data, but also the data value. The storage space of each page is limited. If the data data is large, the number of keys that each node (ie a page) can store will be small. When the amount of stored data is large, it will also cause B- The depth of the tree is large, which increases the number of disk I/Os during the query, which in turn affects the query efficiency. In B+Tree, all data record nodes are stored in the same layer of leaf nodes in the order of key value size, instead of storing only key value information on leaf nodes, this can greatly increase the number of key values stored in each node , Reduce the height of B+Tree and improve search efficiency.
The difference between B+Tree and B-Tree:
1. Non-leaf nodes only store key-value information.
2. There is a chain pointer between all leaf nodes.
3. Data records are stored in leaf nodes.
Optimize the above B-Tree. Since the non-leaf nodes of B+Tree only store key value information, assuming that each disk block can store 4 key values and pointer information, the structure after becoming B+Tree is as shown in the figure below :
There are usually two head pointers on B+Tree, one pointing to the root node, and the other pointing to the leaf node with the smallest key, and all leaf nodes (that is, data nodes) are in a chain ring structure. Therefore, two search operations can be performed on B+Tree: one is the range search and paging search for the primary key, and the other is random search starting from the root node.
Perhaps there are only 22 data records in the above example, and the advantages of B+Tree cannot be seen. Here is an extrapolation:
The page size in the InnoDB storage engine is 16KB, the primary key type of the general table is INT (occupies 4 bytes) or BIGINT (occupies 8 bytes), and the pointer type is generally 4 or 8 bytes, that is to say, one The page (a node in the B+Tree) stores approximately 16KB/(8B+8B)=1K key values (because it is an estimate, for the convenience of calculation, the value of K here is 〖10〗^3). In other words, a B+Tree index with a depth of 3 can maintain 10^3 * 10^3 * 10^3 = 1 billion records.
In the actual situation, each node may not be filled, so in the database, the height of the B+Tree is generally 2 to 4 layers. The InnoDB storage engine of mysql is designed to resident the root node in memory, that is to say, only 1 to 3 disk I/O operations are required to find the row record of a certain key value.
The B+Tree index in the database can be divided into a clustered index (clustered index) and a secondary index (secondary index). The implementation of the above B+Tree example graph in the database is a clustered index, and the leaf nodes in the B+Tree of the clustered index store the row record data of the entire table. The difference between the auxiliary index and the clustered index is that the leaf node of the auxiliary index does not contain all the data of the row record, but the clustered index key that stores the corresponding row data, that is, the primary key. When querying data through the auxiliary index, the InnoDB storage engine traverses the auxiliary index to find the primary key, and then finds the complete row record data in the clustered index through the primary key.
to sum up
According to the above, binary search trees, red-black trees and other data structures can also be used to implement indexes, but file systems and database systems generally use B+Tree as the index structure (currently MySQL's MYISAM and INNODB both use B+Tree as the index Structure), this is because the design of the B+Tree index is based on the theoretical basis of the computer disk storage structure.
The index is stored on the disk in the form of an index file. When a B+Tree search is used, the impact of disk I/O consumption on performance is much smaller than other methods (the most important indicator for evaluating the pros and cons of a data structure as an index is The incremental complexity of the number of disk I/O operations in the search process).
In other words, the structure of the index should minimize the number of disk I/O accesses in the search process, and B+Tree is undoubtedly the better algorithm.
Original link: http://www.cnblogs.com/wzh2010/p/14411428.html
If you think this article is helpful to you, you can pay attention to my official account and reply to the keyword [Interview] to get a compilation of Java core knowledge points and an interview gift package! There are more technical dry goods articles and related materials to share, let's learn and make progress together!
