Abstract: GaussDB (for Redis) (hereinafter referred to as the Gaussian Redis ) Huawei cloud database team independently developed compatible Redis protocol cloud native database, which is calculated using stored split architecture, breakthrough memory limit of open source Redis can be easily extended to PB level storage.
This article will introduce the storage architecture, four major features, competitiveness, and application scenarios.
Storage architecture
Gauss Redis is based on a separate computing and storage architecture. The computing layer implements hot data caching, and the storage layer implements full data storage. The RDMA high-speed network interconnection is used in the middle. The algorithm predicts the user's access rules and realizes the automatic cold and hot exchange of data. Extreme performance improvement.
Four characteristics
This architecture is based on Huawei’s powerful and widely used self-developed distributed storage system DFV, and implements a cloud-native architecture of Share Everything. It takes full advantage of cloud-native elastic scaling and resource sharing, making Gauss Redis possess strong consistency, The four major characteristics of second expansion, low cost, and ultra-availability perfectly avoid the problems of open source Redis's master-slave accumulation, master-slave inconsistency, fork jitter, memory utilization only 50%, big key blocking, gossip cluster management and other issues.
- Strong agreement
Data replication is a matter of storage, so professional matters are left to a professional team. Through distributed storage DFV, Gauss Redis easily achieves 3 copies of strong consistency, and can easily support 6 copies, which is the industry's first.
Under the strong consistency architecture, users no longer have to worry about the master-slave accumulation of open source Redis, which brings extreme problems such as data loss, inconsistency, and OOM, let alone business errors, such as counters, current limiters, access statistics, hash fields, etc. Inconsistent.
- Scaling in seconds
After the data scale expands, expansion is a high-risk and difficult operation. Gauss Redis is based on a cloud-native architecture and divides the expansion into a computing layer and a storage layer. Expansion of the computing layer does not require any data relocation, just modify the routing map, and it can be completed in seconds. The storage layer is a co-built super data lake with huge capacity, and the expansion is cut into delicate 64MB data partitions, which is almost insensitive to the upper database business.
Therefore, Gauss Redis can easily support large-scale business expansion, and truly achieve on-demand expansion and purchase of computing/storage tiers.
- low cost
Compared with open source Redis, Gauss Redis uses disk instead of memory on storage media. On the one hand, due to the storage-computing separation architecture, computing resources are reduced by half, that is, there is no slave node; on the other hand, storage resources are purchased on demand without any waste, and logical/physical compression is adopted. In the end, the overall cost per GB is less than one-tenth of open source Redis.
- Super usable
Regardless of whether it is a single shard or a cluster, open source Redis or friend Redis uses a master-slave architecture for data replication, resulting in a cluster of N nodes. If a pair of master-slave (ie, 2 nodes) is suspended at the same time, the entire cluster is unavailable. After Gauss Redis adopts the separation of storage and computing, each computing node can see and share all data, so N nodes can tolerate up to N-1 nodes, and it can truly achieve higher availability than high availability.
Competitive Analysis
Scene recommendation
Gauss Redis not only approaches cache performance, but its storage capabilities (scalability, high performance, ease of use) surpass databases. Therefore, in addition to Gaussian Redis for caching scenarios, Gaussian Redis can be selected for large-scale data storage up to the PB level. The scene reference is as follows:
Selection suggestion
Command compatibility
Compatible with 5.0 protocol, including string/hash/list/zset/set/stream/geo/ HyperLogLog/bitmap/pubsub, etc., but for performance and security considerations, individual dangerous commands are disabled. For details, refer to: https://support. huaweicloud.com/usermanual-nosql/nosql_09_0076.html
Capacity reference
In the past, when users choose open source Redis, they need to buy memory. For example, user data is 100G. Because the memory usage of open source Redis is halved, you need to buy 200G of memory, and for high availability, you have to buy 200G slave nodes, so the total is 400G Memory. However , when purchasing Gauss Redis , users only need to purchase storage space on demand according to the size of the actual data stored. The memory is only for performance acceleration (the larger the memory/disk ratio, the better the performance).
Attachment: Information reference
1. " Comparison of costs between Huawei Cloud GaussDB (for Redis) and self-built open source Redis "
2. " A timeout caused by fork, let us re-discuss the jitter problem of Redis" https://bbs.huaweicloud.com/blogs/227525
3. " When Redis meets the separation of computing and storage "
https://developer.huaweicloud.com/hero/forum/thread-83188-1-1.html
4. " Performance comparison between GaussDB (for Redis) and native Redis "
https://bbs.huaweicloud.com/blogs/236949
Click to follow and learn about Huawei Cloud's fresh technology for the first time~