The first version of the RoseDB V2 refactoring is released!
RoseDB is a lightweight, fast and reliable KV storage engine based on the Bitcask storage model. The design of the Bitcask storage model is mainly inspired by log-structured file systems and log file merging.
If you are interested, please refer to the Bitcask paper: riak.com/assets/bitc…
The files in which RoseDB stores data have been redesigned using Write Ahead Logs, which are append-only files with a block cache.
I split the original Redis data structure and protocol in rosedb, and will form a separate project later to facilitate access to different storage engines, such as rosedb, badger, pebble, levledb, etc.
Therefore, rosedb only focuses on the functions of the stand-alone storage engine, and is currently under active maintenance. Everyone is welcome to issue or contribute, click start ⭐️.
Project address: github.com/rosedblabs/…
Here are some brief introductions and usage examples of RoseDB:
main feature
Advantage
- Read and write low latency
This is due to the additional write feature of the Bitcask storage model file, making full use of the advantages of sequential IO. High throughput, even when data is completely out of order
Data written to RoseDB does not need to be sorted on the disk, and Bitcask's log-structured file design reduces the movement of the disk head during the writing process.
- Capable of handling datasets larger than memory, with stable performance
Data access in RoseDB involves direct lookups to in-memory indexed data structures, which makes looking up data very efficient even for very large datasets.
- A disk IO can get any key-value pair
RoseDB's memory index data structure directly points to the disk location where the data is located. It does not require multiple disk seeks to read a value, and sometimes even does not need to seek, thanks to the operating system's file system cache and WAL's block cache.
- Fast and stable performance
The RoseDB write operation needs to address the end of the currently open file at most once, and then perform additional writing, and the memory will be updated after writing. This process will not be affected by the size of the database data, so the performance is stable.
- Crash recovery is fast
Crash recovery with RoseDB is easy and fast because RoseDB files are append-only written once. Recovery operations require checking records and verifying CRC data to ensure data consistency.
- easy backup
Backups can be very complex on most systems. RoseDB simplifies this process with its append-once on-disk format. Any tool that archives or copies files in disk block order will properly back up or copy RoseDB databases.
- Batch operations can guarantee atomicity, consistency, and durability
RoseDB supports batch operations that are atomic, consistent, and durable. New write operations in a batch are buffered in memory until committed. If the batch is successfully committed, all write operations in the batch are persisted to disk. If the batch fails, all write operations in the batch will be discarded. That is, all write operations in a batch operation either all succeed or all fail.
shortcoming
All keys must be maintained in memory
RoseDB keeps all keys in memory at all times, which means your system must have enough memory to hold all keys.
Get started quickly
basic operation
package main
import "github.com/rosedblabs/rosedb/v2"
func main() {
// 指定选项
options := rosedb.DefaultOptions
options.DirPath = "/tmp/rosedb_basic"
// 打开数据库
db, err := rosedb.Open(options)
if err != nil {
panic(err)
}
defer func() {
_ = db.Close()
}()
// 设置键值对
err = db.Put([]byte("name"), []byte("rosedb"))
if err != nil {
panic(err)
}
// 获取键值对
val, err := db.Get([]byte("name"))
if err != nil {
panic(err)
}
println(string(val))
// 删除键值对
err = db.Delete([]byte("name"))
if err != nil {
panic(err)
}
}
`
batch operation
// 创建批处理
batch := db.NewBatch(rosedb.DefaultBatchOptions)
// 设置键值对
_ = batch.Put([]byte("name"), []byte("rosedb"))
// 获取键值对
val, _ := batch.Get([]byte("name"))
println(string(val))
// 删除键值对
_ = batch.Delete([]byte("name"))
// 提交批处理
_ = batch.Commit()
`
The complete code can be viewed in the examples sample code.