AeroSpike 4.x basic principles and processes to read and write

AeroSpike 4.x basic principles and processes to read and write

AeroSpike is based on the Hash NoSQL, is essentially a storage system KV

The simplest is to be understood that the memory stored in a red-black trees, red-black tree which is stored the index, which is stored after the position and the data stored hash values

classDiagram class Index{ ...... // offset: 4 cf_digest keyd;//20 bytes ...... // offset: 47 uint64_t rblock_id: 37; // can address 2^37 * 16b = 2Tb drive uint64_t n_rblocks: 19; // is enough for 8Mb/16b = 512K rblocks uint64_t file_id: 7; // can spec 2^7 = 128 drives ...... }

Key value is not stored in the index, but storing a hash value (calculated by the key + set cf_digest_compute2), in addition to storing the index number of the device (the actual file) and the specific value and rblock_id length (n_rblocks)

Probably the inquiry process

st=>start: Start
find_index=>operation: 查内存索引
hit_index=>condition: 索引命中
find_swb=>operation: 查写入缓冲区(SWB)
find_ssd=>operation: 查磁盘
hit=>condition: 命中
e=>end

st->find_index->hit_index
hit_index(yes)->find_swb->hit
hit_index(no)->e
hit(yes)->e
hit(no)->find_ssd
find_ssd->e

Understanding rblock, swb

Equipment (actually a file descriptor)

What is rblock

We put a file area divided blocks, each block size 16byte

| rblock0 | rblock1 | rblock2 | ... | rblockn |

Therefore, it is easy to determine where the value is stored by rblock_id

nblock this record is the number of occupied block

What is swb

swb is a write buffer

Because this requires a long time to write the disk, in order to reduce the number of system calls can be introduced into the buffer, each time to write this buffer filled after the painted plate

AeroSpike all operations are based writing, only the operation of the current buffer before written records do not change, such as deleting a record is written to a certain key is deleted

The buffer size is 8M

Invalid key processing

Background thread scan files, organize block