Inspur Yunhai officially released a new generation of InCloud Rail G7 series of hyper-converged all-in-one machines in May 2023. Its built-in InCloud dSAN hyper-converged storage component is based on a new generation of hardware platform design and supports the full-stack RDMA protocol. It also has EC erasure function. It also brings a new experience and provides richer product functions for users in the new era.
This article will explain in detail how InCloud dSAN designs the EC erasure function on the new generation of InCloud Rail G7 series hyper-converged all-in-one machines.
What is EC erasure coding technology?
Erasure coding technology is a data protection method that mainly uses erasure coding algorithms to divide the original data into multiple segments, then performs expansion and encoding verification, and stores the segmented data and verification together on disks, storage nodes, or Stored elsewhere for fault tolerance purposes. Basic principle: total data block = original data block + check block, commonly expressed as, n = k + m. The basic idea is to calculate the original data elements of k blocks through certain encoding calculations to obtain m block check elements. For these k+m block elements, when any of the m block elements fails (including data and verification errors), the original k block data can be restored through the corresponding reconstruction algorithm.
Challenges of EC erasure coding in block storage design
Figure: Schematic diagram of EC erasure code data segmentation and calculation
The above scenario takes K = 4 and M = 2 as an example. First, the application data is split into 4 shards, then two parity strip shards are calculated, and then the 6 shards are downloaded according to the data distribution algorithm. Sent to 6 nodes for storage. Since the multi-copy strategy is a commonly used data redundancy strategy for customers, we have done relevant analysis on the read and write processes of EC erasure coding and three copies and the processes in fault scenarios: From the following table, it can be concluded that in the normal write process In this case, 7 IOs are needed to complete the normal writing process. At the same time, 7 IOs are required for reading and writing in a fault scenario, which results in a larger reading and writing penalty. In addition, in the small block IO read and write process, if the data block is smaller than the EC slice data size, there will be a waste of space.
| Data redundancy strategy |
Reading and writing scenarios |
Consumption of read and write IO times |
| EC（4+2） |
Normal reading |
1 _ |
| Write normally |
4 read IO, 3 write IO |
|
| Reads falling on the failed disk |
5 read IOs |
|
| Writes falling on the failed disk |
5 read IO + 2 write IO |
|
| three copies |
Normal reading |
1 _ |
| Write normally |
3 _ |
|
| Reads falling on the failed disk |
2 _ |
|
| Writes falling on the failed disk |
1 _ |
In hyper-converged application scenarios, users often use block storage to undertake application data, which is suitable for scenarios including virtualization, cloud platforms, databases, etc. This scenario will generate a lot of small-block random read and write loads. At the same time, users have high requirements for latency, which are required to be controlled within 5ms, and have little impact on the business in failure scenarios. These requirements are the shortcomings of EC erasure coding.
EC design of Inspur Yunhai hyper-fusion on the new generation G7 platform
Due to the obvious advantages and disadvantages of EC, how to use EC functions on hyper-converged block storage is an urgent need nowadays. Inspur Yunhai Hyper-Convergence has made a unique solution design on the new generation G7 platform:
Figure: EC erasure code design scheme
Highlights of the program include:
- Data hierarchical management: The data storage is divided into a cache layer and a data layer. The cache layer uses high-performance NVMe and Sata SSD storage. At the same time, a multi-copy design is used in the cache layer to provide high-performance data storage capabilities. The data layer provides EC storage, mainly stored by HDD hard disks, providing high-capacity storage space.
- Intelligent cache management: Design hot and cold tiered intelligent cache management to efficiently identify hot data and cold data. At the same time, we design global cache management functions to ensure that customers’ hot data is stored in the cache space.
- Efficient storage of data space: Due to copy storage in the cache layer, small block IO of the upper layer application is aggregated. When the data becomes cold data and is flushed to the EC storage layer, the flushing uses data alignment and is aggregated into large sequential strips. With reading and writing, efficient data storage can be performed in the EC data layer.
- EC accelerated computing: The design supports multiple EC acceleration engines and supports EC accelerated computing on different platforms. Among them, the ISA-L EC acceleration library is used on the Intel platform to give full play to the EC computing capabilities of Intel CPUs.
In order to verify the effect of the solution: compare the performance test of ISA-L accelerated computing and using Jerasure pure software without CPU accelerated computing. The test data shown in the figure below adopts database applications commonly used in industries such as finance and medical care. Using 1G test data, in the trigger data flash scenario, the EC algorithm is used to select the Reed-Solomon and Cauchy algorithms to test the EC erasure scenario. Found from experimental data:
Figure: Comparison of EC computing bandwidth of different K/M models ISA-L and Jerasure
- Under different K/M models of ISA-L , the EC calculation bandwidth performance is relatively stable, while software calculations show a linear downward trend as the amount of calculation increases;
- The throughput rate of ISA-L on the new generation G7 platform is 9.8 GB/s, which is 5.7 times higher than software computing.
Summary: Compared with the multi-copy strategy, ECerasure coding can improve storage utilization. For example, the utilization rate in the 4 + 2 scenario is 66%, 3 copies is only 33%, and 8 + 2 can achieve 80%. However, compared with multiple copies, there is still more room for optimization when it comes to EC erasure calculation, data verification, read and write penalties and other issues. InCloud dSAN distributed storage software is based on the new generation G7 platform, relying on new hardware (Intel new generation Xeon CPU) and ISA-L software acceleration library to design new EC functions and adopt unique data layering, intelligent cache management, and software Solutions such as hard collaboration solve the current widespread problems of data redundancy and storage space utilization.