HBASE-LSM tree
1.B + tree
About B tree, B + tree, B understand tree Reference: *
http://blog.csdn.net/v_july_v/article/details/6530142
advantage:
Liang Bin teacher author into the search engines for the B-tree, B + tree gives his opinion (for authenticity, special reference to its own words, without any change):
"B + tree also one of the biggest benefits of easy sweep library, B tree must sequentially scan method in order traversal library, and B + tree directly from the leaf nodes one by one sweep again would be finished, B + tree supports range-query very convenient , while the B-tree is not supported. this is the main reason for the choice of B + tree database. "
Why B + tree is more suitable database index file index and the practical application of the operating system than the B-tree ?
(. 1) B + tree lower the cost of disk reads and writes
B + internal nodes of the tree and no pointers to specific information keywords. So the internal node B is relatively smaller trees. If all of the same internal node key stored in the same disk block, the disk block number of keywords can accommodate the more. Disposable read into memory in the keyword you want to find the more. IO read and write times will be relatively reduced.For example, assuming a disk in the disk accommodating 16Bytes block, and a keyword 2Bytes, a specific keyword information pointer 2bytes. A 9-order B-tree (a maximum of eight node key) internal nodes requires two fast disks. And B + internal node tree requires only a disc fast. When it is desired to internal node into memory when, B tree Find time (time that the disk rotation in the disk) than a B + tree once more disk blocks.
(2) B + tree query efficiency is more stable
due to the non-leaf node is not the end point node contents of the file, but only the index leaf node in keywords. So look for any keywords must follow a path from the root to leaf node. All the same keyword query path length, resulting in a data query efficiency of each of pretty.(3) B tree does not address the inefficiencies in traversal of the increase in disk IO performance at the same time. Precisely in order to solve this problem, B + tree came into being. As long as B + tree leaf node traversal can be achieved traverse the entire tree. And in a database based on a range of queries it is very frequent, and the B-tree is not supported by this operation (or inefficient).
Disadvantages:
B + tree biggest performance problem is the IO will produce large amounts of random, as inserted, the leaf node of the new data will slowly split, leaf node logically successive discontinuities often physically separated even very far, but doing range query, it will have a lot to read random IO. For a large number of random write, too, for example a large span insertion key, e.g., 7> 1000-> 3-> 2000 ... newly inserted data stored on the disk very far apart, will generate a lot of random write IO.
As can be seen from the above, low speed disk seek seriously affect the performance (In recent years, the development of disk seek speed is almost at a standstill state)
2.LSM tree:
Storing the B-tree storage engine and the same engine, the same support for add, delete, read, modified, sequential scanning operation. And to avoid the problem through random write disk bulk storage technology
principle:
To split a tree into N little tree, it is first written to memory, with trees growing, the trees will flush memory to disk, the disk can be done on a regular basis in the tree merge operation, combined into a tree, to optimize read performance.
Read and write performance:
LSM tree compared to the B-tree, part of the expense of read performance, a substantial increase write performance.
LSM Tree, for the most simple two-story LSM Tree, the in-memory and disk data in your data merge operations, as shown below:
281219493293115.png
3.hbase and LSM tree
principle:
Data is first written to memory in order to prevent loss of data memory, write memory at the same time the need persisted to disk, and corresponds to the HBase MemStore HLog;
MemStore After the data reaches a certain threshold value, the data needs to be flushed to the disk, which generates HFile (also a small B + tree) file;
hbase the minor (a small amount HFile small file merge) major (all HFile file a region of merger) to perform compact operation, delete invalid data (data expired and deleted), and more young trees merge into a tree in this opportunity, to enhanced read performance.
For LSM tree hbase read performance optimization:
Bloom-filter: with a random probability is a bitmap, you can quickly tell you, a small ordered structure, there is no specified data. So you can not binary search, but only a few simple calculations can know whether a small set of data in it. Efficiency has improved, but the pay is the cost of space.
compact: trees merged into trees: trees because there are performance problems, so there must be a continuous process will be incorporated into the trees on the tree, so most of the old data queries can also be used log2N direct way to find, no then (N / m) * log2n the query.
4. Supplement: hbase architecture diagram:
Author: Nevin Moore ST
link: https: //www.jianshu.com/p/06f9f7f41fdb
Source: Jane books
are copyrighted by the author. Commercial reprint please contact the author authorized, non-commercial reprint please indicate the source.