Article directory
B+ tree
A B+ tree of order m has at most m - 1 keywords, and the number of child nodes is the order
The picture above is a 4-order B+ tree. Wikipedia defines that the number of keywords is 1 less than the number of child nodes
structure
The B+ number is a variant of the B tree. It is a multi-fork search tree. An n-order B+ tree has the following characteristics:
- Each node has at most n children
- Non-root node key value range: n / 2 <= k <= n - 1
- Leaf nodes are directly connected by
- Non-leaf nodes do not store data, but only serve as an index (more pointers can be stored)
- The key values of internal nodes and parent nodes cannot be repeated, and page nodes and their parent nodes can be repeated
Inquire
Compared with the key, look down layer by layer
insert process
-
Insert data into leaf nodes
-
Determine whether the key number of leaf nodes is less than n
less than
- Finish
more than the
- Split the node into two leaf nodes
- Copy the ( n / 2 )th key and add it to the parent node (internal node)
-
Determine whether the number of keys of the parent node is less than n
more than the
- split parent node
- Add the ( n / 2 )th key to the parent node
Split leaf nodes (specific content nodes)
The key value in the node is copied to the parent node (parent node and child node can be repeated)
- The extracted value is equivalent to an index, and the corresponding value is not stored
for example:
父节点 1
子节点 1 2 3 4 5 6
// 子节点此时候太多了需要分裂
父节点 1 4
子节点 1 2 3 4 5 6
Split internal nodes (aka index nodes)
That is to say, there are too many internal nodes and need to be squeezed to the parent node (parent node and child node cannot be repeated)
- Originally it is an index, so the index column is also extracted, which can be extracted from the child node
父节点 1
子节点 2 3 4 5 6
--------------------------------
父节点 1 4
子节点 2 3 5 6
delete process
Rules :
The number of child nodes is greater than (B+ tree order / 2 - 1) Math.ceil(m-1)/2 - 1 Round up
Process :
- After deleting the node, judge whether it complies with the rules
- If there is a spare key in the key of the sibling node, borrow a key from the sibling node, and at the same time, the borrowed key will replace the index of the parent node, otherwise step 3
- If there is no spare key in the key of the sibling node, the current node and the sibling node are merged into a new node, the key of the parent node is deleted, and the current node points to the parent node (must be an index node)
- If the number of keys of the index node is greater than or equal to the rule, end, otherwise step 5
- If there are surplus sibling nodes, the key of the parent node is moved down, the key of the sibling node is moved up, and the end is completed, otherwise step 6
- The current node, sibling nodes, and parent nodes are moved down and the key is merged into a new node. The current node points to the parent node, repeat step 4
PS : After the deletion operation of the B+ tree, the key existing in the index node does not necessarily have a corresponding record in the child node
Prerequisite Summary
Auxiliary class
BTreePageId: page identifier
Unique identifier for BTreeInternalPage, BTreeLeafPage, BTreeHeaderPage, and BTreeRootPtrPage objects.
- private final int tableId; // tableid
- private final int pgNo; // page id in the table
- private final int pgcateg; // page type
- public final static int ROOT_PTR = 0;
- public final static int INTERNAL = 1;
- public final static int LEAF = 2;
- public final static int HEADER = 3;
BTreeInternalPage: The internal node of the B+ tree
Each instance of BTreeInternalPage stores the data of one page of BTreeFile and implements the Page interface used by BufferPool.
- private final byte[] header; //occupancy of the slot
- private final Field[] keys; // record key array
- private final int[] children; // Store the serial number (PgNo) of the page, used to obtain the BTreePageId of the left child and the right child
- private final int numSlots; // number of slots
- private int childCategory; // type of child node: leaf or internal node
BTreeLeafPage: The leaf node of the B+ tree
Each instance of BTreeLeafPage stores the data of one page of BTreeFile and implements the Page interface used by BufferPool
- private final byte[] header; // Slot occupancy
- private final Tuple[] tuples; // array of tuples
- private final int numSlots; // number of slots
- private int leftSibling; // leaf node (pgNo) or 0
- private int rightSibling; // leaf node (pgNo) or 0
BTreeEntry：
-
private Field key; // the key in the internal node
-
private BTreePageId leftChild; // left child
-
private BTreePageId rightChild; // right child
-
private RecordId rid; // path stored in the page
Exercise 1
Experiment 1 focused on search
BTreeFile
parameter
- private final File f; // file
- private final TupleDesc td; // schema of the tuple
- private final int tableid ; // tableid
- private final int keyField; // the field where the index key is located
method
- findLeafPage: a recursive function that finds and locks the leaf page in the B+ tree corresponding to the leftmost page that may contain the key field f
Map<PageId, Page> dirtypages: When creating a new page or changing the data and pointers in the page, it needs to be added to dirtypages
private BTreeLeafPage findLeafPage(TransactionId tid, Map<PageId, Page> dirtypages, BTreePageId pid, Permissions perm,
Field f)
throws DbException, TransactionAbortedException {
// some code goes here
// 获页面类别
int type = pid.pgcateg();
// 叶子节点
if(type == BTreePageId.LEAF){
return (BTreeLeafPage) getPage(tid, dirtypages, pid, perm);
}
// 锁定路径上的所有内部节点，以READ_ONLY锁
BTreeInternalPage internalPage = (BTreeInternalPage) getPage(tid, dirtypages, pid, Permissions.READ_ONLY);
Iterator<BTreeEntry> it = internalPage.iterator();
BTreeEntry entry = null;
while(it.hasNext()){
entry = it.next();
// 如果要搜索的字段为空，找到最左节点(用于迭代器)
if(f == null){
return findLeafPage(tid, dirtypages, entry.getLeftChild(), perm, f);
}
// 如果找到的节点相等，返回
if(entry.getKey().compare(Op.GREATER_THAN_OR_EQ, f)){
return findLeafPage(tid, dirtypages, entry.getLeftChild(), perm, f);
}
}
return findLeafPage(tid, dirtypages, entry.getRightChild(), perm, f);
}
-
getPage()
A method that encapsulates the process of locking and fetching pages. First, the method checks the local cache ("dirtypages"), and if the requested page is not found there, it is fetched from the buffer pool. It also adds pages to the dirtypages cache if fetching them with read-write permissions, since presumably they will be dirty by this transaction soon.
test
BTreeFileReadTest.java and the system tests in BTreeScanTest.java.
Exercise 2
Experiment 2 is aimed at adding new nodes
BTreeFile
parameter
- private final File f; // file
- private final TupleDesc td; // schema of the tuple
- private final int tableid ; // tableid
- private final int keyField; // the field where the index key is located
method
splitLeafPage(): Split leaf nodes
public BTreeLeafPage splitLeafPage(TransactionId tid, Map<PageId, Page> dirtypages, BTreeLeafPage page, Field field)
throws DbException, IOException, TransactionAbortedException {
// some code goes here
//
// Split the leaf page by adding a new page on the right of the existing
// page and moving half of the tuples to the new page. Copy the middle key up
// into the parent page, and recursively split the parent as needed to accommodate
// the new entry. getParentWithEmtpySlots() will be useful here. Don't forget to update
// the sibling pointers of all the affected leaf pages. Return the page into which a
// tuple with the given key field should be inserted.
// 1. 获取空白的页面作为新的右页面 (叶子页面)
BTreeLeafPage newRightPage = (BTreeLeafPage)getEmptyPage(tid, dirtypages, BTreePageId.LEAF);
// 2. 插入当前的tuple，分割一半节点给右节点
// 获取反向迭代器
int tupleNum = page.getNumTuples();
Iterator<Tuple> it = page.reverseIterator();
for (int i = 0; i < tupleNum / 2; i++) {
Tuple tuple = it.next();
// 原页面删除
page.deleteTuple(tuple);
// 写入新页面
newRightPage.insertTuple(tuple);
}
// 3. 如果当前 page 有右兄弟，连接右兄弟
BTreePageId oldRightPageId = page.getRightSiblingId();
// 获取页面
BTreeLeafPage oldRightPage = oldRightPageId == null ? null : (BTreeLeafPage) getPage(tid, dirtypages, oldRightPageId, Permissions.READ_ONLY);
if(oldRightPage != null){
// 连接
oldRightPage.setLeftSiblingId(newRightPage.getId());
newRightPage.setRightSiblingId(oldRightPageId);
// 放入脏页缓存
dirtypages.put(oldRightPageId, oldRightPage);
}
// 4. 分裂节点连接
page.setRightSiblingId(newRightPage.getId());
newRightPage.setLeftSiblingId(page.getId());
// 放入脏页缓存
dirtypages.put(page.getId(), page);
dirtypages.put(newRightPage.getId(), newRightPage);
// 5. 获取原节点的内部节点
BTreeInternalPage parent = getParentWithEmptySlots(tid, dirtypages, page.getParentId(), field);
// 右节点的第一个节点作为要挤入父节点的新内部节点值
Field mid = newRightPage.iterator().next().getField(keyField);
// 创建新的内部节点
BTreeEntry entry = new BTreeEntry(mid, page.getId(), newRightPage.getId());
parent.insertEntry(entry);
// 放入脏页缓存
dirtypages.put(parent.getId(), parent);
// 6. 更新page 和 newRightPage的父指针
updateParentPointers(tid, dirtypages, parent);
// 7. 返回 field 所在的页
// 如果当前值大于等于中点，说明在右边节点
if(field.compare(Op.GREATER_THAN_OR_EQ, mid)){
return newRightPage;
}
// 否则在左边节点，也就是原节点
return page;
}
- getEmptyPage(): Get an idle leaf page
- getParentWithEmptySlots(): Get the parent node (internal node), if it is full, it will trigger splitting, that is, the following method, splitting the internal node
splitInternalPage(): split internal page
public BTreeInternalPage splitInternalPage(TransactionId tid, Map<PageId, Page> dirtypages,
BTreeInternalPage page, Field field)
throws DbException, IOException, TransactionAbortedException {
// some code goes here
//
// Split the internal page by adding a new page on the right of the existing
// page and moving half of the entries to the new page. Push the middle key up
// into the parent page, and recursively split the parent as needed to accommodate
// the new entry. getParentWithEmtpySlots() will be useful here. Don't forget to update
// the parent pointers of all the children moving to the new page. updateParentPointers()
// will be useful here. Return the page into which an entry with the given key field
// should be inserted.
// 1. 获取空白页面 (内部节点)
BTreeInternalPage newRightPage = (BTreeInternalPage) getEmptyPage(tid, dirtypages, BTreePageId.INTERNAL);
// 2. 拆分当前节点
// 获取反向迭代器
Iterator<BTreeEntry> iterator = page.reverseIterator();
int tupleNum = page.getNumEntries();
// 一半的节点移动到右节点
for (int i = 0; i < tupleNum / 2 ; i++) {
BTreeEntry entry = iterator.next();
page.deleteKeyAndRightChild(entry);
newRightPage.insertEntry(entry);
}
// 3. 抽出中间的内部节点
BTreeEntry mid = iterator.next();
// 左页面删除当前节点
page.deleteKeyAndRightChild(mid);
mid.setLeftChild(page.getId());
mid.setRightChild(newRightPage.getId());
BTreeInternalPage parent = getParentWithEmptySlots(tid, dirtypages, page.getParentId(), mid.getKey());
parent.insertEntry(mid);
// 4. 写入脏页缓存
dirtypages.put(page.getId(), page);
dirtypages.put(newRightPage.getId(), newRightPage);
dirtypages.put(parent.getId(), parent);
updateParentPointers(tid, dirtypages, parent);
updateParentPointers(tid, dirtypages, page);
updateParentPointers(tid, dirtypages, newRightPage);
// 5. 返回 field 所在的页
// 如果当前值大于等于中点，说明在右边节点
if(field.compare(Op.GREATER_THAN_OR_EQ, mid.getKey())){
return newRightPage;
}
// 否则在左边节点，也就是原节点
return page;
}
- deleteKeyAndRightChild(): delete the incoming key and right pointer
- The reason in the picture is to split, that is, to divide into two sections, and the right pointer should be disconnected
BufferPoll
Make up for previous mistakes
method
-
insertTuple(): HeapFile was originally written to death, changed to DbFile
public void insertTuple(TransactionId tid, int tableId, Tuple t) throws DbException, IOException, TransactionAbortedException { // some code goes here // not necessary for lab1 // 获取 数据库文件 DBfile DbFile dbFile = Database.getCatalog().getDatabaseFile(tableId); // 将页面刷新到缓存中 updateBufferPoll(dbFile.insertTuple(tid, t), tid); } -
deleteTuple(): Same reason as above
public void deleteTuple(TransactionId tid, Tuple t) throws DbException, IOException, TransactionAbortedException { // some code goes here // not necessary for lab1 // 查询所属表对应的文件 DbFile dbFile = Database.getCatalog().getDatabaseFile(t.getRecordId().getPageId().getTableId()); // 将页面刷新到缓存中 updateBufferPoll(dbFile.deleteTuple(tid, t), tid); } -
discord(): no cache exists on the incoming page
public synchronized void discardPage(PageId pid) {
// some code goes here
// not necessary for lab1
// 删除使用记录
if(pageStore.containsKey(pid)){
remove(pageStore.get(pid));
// 删除缓存
pageStore.remove(pid);
}
}
- updateBufferPoll()
private void updateBufferPoll(List<Page> pageList, TransactionId tid) throws DbException {
for (Page page : pageList){
page.markDirty(true, tid);
// 如果缓存池已满，执行淘汰策略
if(pageStore.size() > numPages){
evictPage();
}
// 如果缓存中有当前节点，更新
LinkedNode node;
if(pageStore.containsKey(page.getId())){
// 获取节点，此时的页一定已经在缓存了，因为刚刚被修改的时候就已经放入缓存了
node = pageStore.get(page.getId());
// 更新新的页内容
node.page = page;
}
// 如果没有当前节点，新建放入缓存
else{
// 是否超过大小
if(pageStore.size() >= numPages){
// 使用 LRU 算法进行淘汰最近最久未使用
evictPage();
}
node = new LinkedNode(page.getId(), page);
addToHead(node);
}
// 更新到缓存
pageStore.put(page.getId(), node);
}
}
It is recommended here to abstract this new node method, I am too lazy to do it, I forgot before
test
BTreeFileInsertTest and BTreeDeadlockTest
Exercise 3、4
The content of experiment three is to delete nodes
BTreeFile
method
stealFromLeafPage
public void stealFromLeafPage(BTreeLeafPage page, BTreeLeafPage sibling,
BTreeInternalPage parent, BTreeEntry entry, boolean isRightSibling) throws DbException {
// some code goes here
//
// Move some of the tuples from the sibling to the page so
// that the tuples are evenly distributed. Be sure to update
// the corresponding parent entry.
// 1. 判断是 左兄弟 还是 右兄弟
Iterator<Tuple> iterator = isRightSibling ? sibling.iterator() : sibling.reverseIterator();
// 2. 根据兄弟节点中的数量，确定窃取的数量
int curTupleNum = page.getNumTuples();
int siblingTupleNum = sibling.getNumTuples();
int targetTupleNum = (curTupleNum + siblingTupleNum) / 2;
// 窃取到target
while(curTupleNum < targetTupleNum){
Tuple tuple = iterator.next();
sibling.deleteTuple(tuple);
page.insertTuple(tuple);
curTupleNum++;
}
// 3. 提到中间节点到原内部节点
Tuple mid = iterator.next();
entry.setKey(mid.getField(keyField));
parent.updateEntry(entry);
}
stealFromLeftInternalPage
public void stealFromLeftInternalPage(TransactionId tid, Map<PageId, Page> dirtypages,
BTreeInternalPage page, BTreeInternalPage leftSibling, BTreeInternalPage parent,
BTreeEntry parentEntry) throws DbException, TransactionAbortedException {
// some code goes here
// Move some of the entries from the left sibling to the page so
// that the entries are evenly distributed. Be sure to update
// the corresponding parent entry. Be sure to update the parent
// pointers of all children in the entries that were moved.
// 1. 确定窃取几个key
Iterator<BTreeEntry> iterator = leftSibling.reverseIterator();
int curEntryNum = page.getNumEntries();
int siblingEntryNum = leftSibling.getNumEntries();
int targetNum = (curEntryNum + siblingEntryNum) / 2;
// 2. 窃取父节点(内部节点和父节点没有重复节点)
BTreeEntry entry = iterator.next();
BTreeEntry mid = new BTreeEntry(parentEntry.getKey(), entry.getRightChild(), page.iterator().next().getLeftChild());
page.insertEntry(mid);
curEntryNum++;
// 3. 窃取左兄弟节点
while(curEntryNum < targetNum){
leftSibling.deleteKeyAndRightChild(entry);
page.insertEntry(entry);
entry = iterator.next();
curEntryNum++;
}
// 4. 更新父节点
// 从左节点删除，拉到父节点
leftSibling.deleteKeyAndRightChild(entry);
parentEntry.setKey(entry.getKey());
parent.updateEntry(parentEntry);
// 5. 更新指标
dirtypages.put(page.getId(), page);
dirtypages.put(parent.getId(), parent);
dirtypages.put(leftSibling.getId(), leftSibling);
updateParentPointers(tid, dirtypages, page);
}
stealFromRightInternalPage
public void stealFromRightInternalPage(TransactionId tid, Map<PageId, Page> dirtypages,
BTreeInternalPage page, BTreeInternalPage rightSibling, BTreeInternalPage parent,
BTreeEntry parentEntry) throws DbException, TransactionAbortedException {
// some code goes here
// Move some of the entries from the right sibling to the page so
// that the entries are evenly distributed. Be sure to update
// the corresponding parent entry. Be sure to update the parent
// pointers of all children in the entries that were moved.
// 1. 确定窃取几个key
Iterator<BTreeEntry> iterator = rightSibling.iterator();
int curEntryNum = page.getNumEntries();
int siblingEntryNum = rightSibling.getNumEntries();
int targetNum = (curEntryNum + siblingEntryNum) / 2;
// 2. 窃取父节点(内部节点和父节点没有重复节点)
BTreeEntry entry = iterator.next();
BTreeEntry mid = new BTreeEntry(parentEntry.getKey(), page.reverseIterator().next().getRightChild(), entry.getLeftChild());
page.insertEntry(mid);
curEntryNum++;
// 3. 窃取左兄弟节点
while(curEntryNum < targetNum){
rightSibling.deleteKeyAndLeftChild(entry);
page.insertEntry(entry);
entry = iterator.next();
curEntryNum++;
}
// 4. 更新父节点
// 从左节点删除，拉到父节点
rightSibling.deleteKeyAndRightChild(entry);
parentEntry.setKey(entry.getKey());
parent.updateEntry(parentEntry);
// 5. 更新指标
dirtypages.put(page.getId(), page);
dirtypages.put(parent.getId(), parent);
dirtypages.put(rightSibling.getId(), rightSibling);
updateParentPointers(tid, dirtypages, page);
}
mergeLeafPages
public void mergeLeafPages(TransactionId tid, Map<PageId, Page> dirtypages,
BTreeLeafPage leftPage, BTreeLeafPage rightPage, BTreeInternalPage parent, BTreeEntry parentEntry)
throws DbException, IOException, TransactionAbortedException {
// some code goes here
//
// Move all the tuples from the right page to the left page, update
// the sibling pointers, and make the right page available for reuse.
// Delete the entry in the parent corresponding to the two pages that are merging -
// deleteParentEntry() will be useful here
// 1. 将右兄弟的所有节点添加到左节点
Iterator<Tuple> iterator = rightPage.iterator();
while(iterator.hasNext()){
Tuple tuple = iterator.next();
rightPage.deleteTuple(tuple);
leftPage.insertTuple(tuple);
}
// 2. 更新右兄弟
BTreePageId rightSiblingPageId = rightPage.getRightSiblingId();
if(rightSiblingPageId == null){
leftPage.setRightSiblingId(null);
}
else{
leftPage.setRightSiblingId(rightSiblingPageId);
// 右兄弟更新左兄弟
BTreeLeafPage rightSiblingPage = (BTreeLeafPage) getPage(tid, dirtypages, rightSiblingPageId, Permissions.READ_WRITE);
rightSiblingPage.setLeftSiblingId(leftPage.getId());
}
// 3. 将右兄弟在 header 中置空
setEmptyPage(tid, dirtypages, rightPage.pid.getPageNumber());
// 4. 删除父节点中的 entry
deleteParentEntry(tid, dirtypages, leftPage, parent, parentEntry);
// 5. 放到脏页缓存
dirtypages.put(leftPage.getId(), leftPage);
dirtypages.put(parent.getId(), parent);
}
mergeInternalPages
public void mergeInternalPages(TransactionId tid, Map<PageId, Page> dirtypages,
BTreeInternalPage leftPage, BTreeInternalPage rightPage, BTreeInternalPage parent, BTreeEntry parentEntry)
throws DbException, IOException, TransactionAbortedException {
// some code goes here
//
// Move all the entries from the right page to the left page, update
// the parent pointers of the children in the entries that were moved,
// and make the right page available for reuse
// Delete the entry in the parent corresponding to the two pages that are merging -
// deleteParentEntry() will be useful here
// 1. 获取中间节点插入
BTreeEntry mid = new BTreeEntry(parentEntry.getKey(), leftPage.reverseIterator().next().getRightChild(),
rightPage.iterator().next().getLeftChild());
leftPage.insertEntry(mid);
// 2. 将右兄弟连接到左兄弟
Iterator<BTreeEntry> iterator = rightPage.iterator();
while(iterator.hasNext()){
BTreeEntry entry = iterator.next();
rightPage.deleteKeyAndLeftChild(entry);
leftPage.insertEntry(entry);
}
// 3. 更新左兄弟的孩子指针
updateParentPointers(tid, dirtypages, leftPage);
// 4. 将 rightPage 在 header中标空
setEmptyPage(tid, dirtypages, rightPage.getId().getPageNumber());
// 5. 从父节点中删除左右孩子指针
deleteParentEntry(tid, dirtypages, leftPage, parent, parentEntry);
// 6. 刷新脏页缓存
dirtypages.put(leftPage.getId(), leftPage);
dirtypages.put(parent.getId(), parent);
}
test
BTreeFileDeleteTest.java