Recently, CNCF (Cloud Native Computing Foundation) [1] released the latest version of the cloud native panorama [2]. HwameiStor, an independent and open-source cloud-native local storage system of "DaoCloud" , is included in the Cloud Native Storage (cloud-native storage) quadrant of the RunTime (runtime) layer in the CNCF cloud-native panorama, and has become the cloud-native local storage recommended by CNCF project.
The picture is excerpted from Cloud Native Panorama: https://landscape.cncf.io
HwameiStor
cloud-native local storage system
With the advent of the cloud-native era , more and more business systems adopt cloud-native architecture. As a core component for the stable operation of bearing services, the storage system is facing new challenges . First of all , in terms of ease of use , traditional storage service deployment, operation and maintenance are complicated, and the degree of cloud-native is low, so it cannot be integrated with mainstream cloud-native orchestration platforms. The second is performance . In cloud-native scenarios, traditional storage cannot meet the needs of a large number of application accesses, causing storage performance to become the bottleneck of application operating efficiency. The third is high availability . The cloud-native storage system that has been applied in the production environment needs to ensure high reliability/high availability and no single point of failure. The fourth is agility . Containers can be destroyed and generated at any time, so the persistence of container data is essential. Persistent Volume PV (Persistent Volume) can be quickly created, destroyed, smoothly expanded/shrunk, and quickly migrated with Pod migration, etc. The ability needs to be improved urgently.
HwameiStor is a Kubernetes-native local storage system with the advantages of production operation and maintenance, rapid deployment, high performance, high availability, and low cost. HwameiStor forms a local storage resource pool of HDD, SSD and NVMe disks for unified management, provides distributed local data volume services through standard CSI interfaces, and provides data persistence capabilities for stateful cloud-native applications or components, which can replace expensive Traditional storage area network Storage Area Network (SAN).
HwameiStor provides four types of local data volumes: 1) LVM-based single-copy local data volumes , which can be adapted to Kafka, ElasticSearch, Redis and other high-availability architecture middleware to meet high requirements for data access; 2) LVM-based Highly available data volumes with double copies can meet the requirements of OLTP databases such as MySQL for high availability of data storage and high performance of data access; 3) Variable local data volumes based on LVM can convert single-copy data volumes into dual The high-availability data volume of the copy meets the reliability requirements of the data volume as the security level of the application data increases; 4) The data volume based on the raw disk satisfies the direct use of the local disk by the application, such as distributed systems such as MinIO and Ceph . Applications can automatically use HwameiStor's local volumes through the CSI driver according to their own needs, quickly respond to the deployment, expansion, migration and other requirements of the business system, and realize automatic operation and maintenance based on K8s.
The core of HwameiStor lies in the definition and implementation of custom resource CRD. Based on the existing PersistentVolume (PV) and PersistentVolumeClaim (PVC) object classes in K8s, Hwameistor defines more abundant object classes to associate PV/PVC with local data disks , to help users obtain a consistent experience , and also facilitate users to view the system resource status of HwameiStor.
Recently, HwameiStor released several major updates. Added the important function of data volume migration and expulsion to ensure the continuous normal operation of HwameiStor in the production environment. When a Kubernetes system node needs to be maintained or goes offline due to various reasons, the resources on the node need to be released and evicted first. HwameiStor will automatically respond to this operation and maintenance operation in a timely manner, and actively migrate the copy of the data volume on the node to other On the node, on the one hand, the application data will not be lost or damaged. On the other hand, the evicted application Pod can quickly recover and access existing data on other nodes, ensuring business continuity. In addition, operation and maintenance personnel can also actively migrate data volumes to balance system resources and ensure smooth system operation. At the same time, in terms of high availability, HwameiStor has added more types of operating systems, including CentOS, Ubuntu, Kirin, ARM and other operating systems.
Next, HwameiStor will mainly focus on the following four aspects. 1. Observability , allowing operation and maintenance personnel to better understand what happened to the system in the past, present and future, making operation and maintenance work more convenient and easy. 2. The snapshot of the data volume , which copies the data at any point in time, provides online data backup and recovery, further improves the security and reliability of the data, and improves the disaster recovery capability of the data. 3. Disk life cycle management , including health management. Through the current health data of the disk, disk abnormalities are detected in time and alarms are issued; in addition, according to the historical health data of the disk, disk failures are predicted, and data reliability and reliability are ensured through early intervention. availability. 4. HwameiStor Operator , through the Operator, realizes the full life cycle management and operation and maintenance of HwameiStor, further improving the usability of the system.
At the same time, HwameiStor is an important storage component in "DaoCloud Daoke" a new generation of cloud-native operating system -- "cloud-native application cloud platform DaoCloud Enterprise 5.0 (DCE 5.0)". At present, there are also some customers (for example: customers in banks, automobiles, etc.) that have used HwameiStor to support middleware applications in their production environments. On GitHub, HwameiStor is well-received, has gained 346 stars, and has 13 permanent community contributors. Developers are welcome to download and use, participate in discussions and contribute to the community.
project address:
https://github.com/hwameistor/hwameistor