Native cloud storage and cloud storage What is the difference?

Author | Li Peng (Zhuanghuai) Ali cloud smart business group of senior technical experts

REVIEW : New issues of corporate load / intelligent workload of container, moved to cloud storage encountered performance, flexibility, high availability, encryption, isolation, observability and life cycle, not only need to improve levels of storage products , the need to improve the cloud native control / data plane, promote cloud storage and cloud storage native evolution. This article will explain the problem scenarios to explore possible solutions, and ultimately come to the native cloud storage and cloud storage is now what can be done and what needs to be done in the future.

introduction

Recently I had the opportunity to participate by Infra Meetup joint Kubernetes & Cloud Native Meetup co-organized cloud-oriented native Meetup persistent applications, open source storage thinking recent cloud storage, cloud native storage combination, what cloud native storage in the end that needs to be done What, native cloud storage What next challenge is to do more reflection and combing, one of the words, to share a few preliminary view.

As the cloud of native applications for mobility, scalability and dynamic characteristics may be, the corresponding, cloud native storage brings density requirements, speed, degree of mixing, so the cloud storage basic capability also proposed in efficiency, flexibility, autonomy, stability, application of low coupling, GuestOS optimization, security and other aspects of the appeal.

Health status of the original cloud

Container and native cloud computing is quickly accepted business

Forrester Forecast: 2022, the global organization / company in the build environment run container applications, increased from less than 30% today, will significantly raised to over 75%, container trend of enterprise application unstoppable.

On the other hand, according to IDC the growth trend of the future enterprise storage market forecast: cloud storage requirements compared to the 2015 to 2020 there will be more than three times the growth of the enterprise storage market, enterprise class data management core data consumption the proportion of occupied memory upgrade from 15% to 23%, structured data and DBMS data further strengthened in the enterprise storage market.

Cloud native, the core enterprise application / intelligence applications, the use of cloud storage to deploy native produce available stateful applications, accelerating the rise. Overseas storage giant EMC, NetApp embrace cloud native, active layout REX-Ray flexrex, Trident and other cloud storage native layout program.

Kubernetes cloud infrastructure has become the era of the native

Over the past year (2018-2019) in, Kubernetes cloud infrastructure is becoming a native of the times, more and more Internet, databases, message queues, which are state enterprise core applications, and gradually migrate to the cloud native platform Kubernetes, different performance on the cloud in the block storage throughput and delay, and the stability is different requirements, such as:

1. Millisecond delay NvME SSD stable level, to meet the demand for high performance database and KVstore;
2. With the deployment of stand-alone applications to enhance the density of the challenges of single block storage density;
3. This shared memory block, and isolation of the resilient block has a higher memory requirements.

In the cloud native environment, how to declaratively to meet different business scenarios, became a challenge to achieve the control plane and the data plane of native cloud storage.

In the intelligent application of AI scenario, high performance computing, flow calculations attempt to deploy by Kubernetes cloud native platform, using cloud storage to complete work training, calculations, reasoning, select Kubernetes environment, this cloud is stored in and the use of a challenge. For example, there is evidence that ecological Spark is gradually migrating to Kubernetes eg Gang Scheuler native scheduler and dispatcher extended from Hadoop YARN.

In a cloud computing environment: Since the model costs and storage computing isolated, HDFS will still exist in the manner of storage protocols, but storage will gradually migrate to the object store (OSS, S3) from 3 copies of HDFS; GPU multi-machine card MPI computing, Kubernetes of Flink flow has gradually become the mainstream of computing, storage access method is also more object-storage presented.

But in the process of using object storage, computational efficiency of large data / AI applications are still facing serious challenges:

1. Reduce network IO on the same node repeatedly pull generated by the same Block;
2. Shuffle reduce the data generated by the write IO;
3. Realization of data-aware computing, computing migration to nearby calculated data.

The current Kubernetes scheduler, and cloud storage features did not give a good solution, so it also gives native cloud storage in accelerating big data computing, provides a stage play to make up for deficiencies in IO throughput.

Big data offline computing such as gene calculate, has come a massive run computing tasks by Kubernetes cloud native platform: file storage peak throughput 10GBps - 30GBps peak rigidity payment, it requires a separate high throughput file storage form and mode of delivery in the cloud native evolution and change in the environment.

Container service to become a native cloud era infrastructure

With the cloud on enterprise applications are increasingly choosing to use a container of the way, has been a significant container services in different cloud vendors in business growth, container services has gradually become a new era of native cloud infrastructure and best use cloud entrance resources. Native cloud storage for cloud computing / cloud storage, it also has a new connotation, there is a need to rethink the nature of the differences and connections cloud storage and cloud storage native.

Thinking native cloud storage and cloud storage

Cloud Native Storage vs Cloud Storage：

• Or unified opposition?
• Link between the two?
• Differences and focus?

= 1. Cloud storage cloud storage native UI, declarative memory for the application layer of the application efficiency of combination of capabilities +

Six elements of cloud storage native declared:

1. Capacity Size;
2. IOPS ,, throughput performance, delay;
3. Accessibility, shared / exclusive;
4. IO observability;
5. QoS；
6. Multi-tenant isolation.

2. tiered storage, reuse infrastructure dividends, do not reinvent the wheel, shift load for a new type of storage forms part

3. In the control plane to achieve efficiency, the ability of autonomy, maximize storage stability and security

Native cloud storage market

To better understand how to build native cloud storage in the cloud environment, look at the cloud storage native Kubernetes several deployment in mainstream enterprise environments, as well as comparison of cloud storage form:

1. Ceph on Kubernetes with Rook
2. Portworx
3. OpenEBS

Ceph on Kubernetes with Rook

Ceph is Sage Weil developed by the University of California, Santa Cruz in 2003, is part of his PhD project. Ceph LTS mature and stable, high availability, ecological powerful, tightly integrated native in the cloud era and Kubernets. Based on Ceph RADOS (Reliable Autonomic Distributed Object Store) high availability storage, since before the release of native cloud era in 2003, has been widely deployed in the production of high-availability storage, supports the widest range of block storage RBD, file POSIX Cephfs, access and object store protocol.

RedHat / SUSE Ceph is currently the main commercial supporter, landing platforms in multiple containers case, RBD, CephFS have been adopted as the primary storage platform for the implementation of container, to compensate for the lack of foundation for cloud storage.

Rook is currently deploying and operation and maintenance scheduling tool Kubernetes Ceph available at the product level.

Ceph basic architecture of a data plane OSDs (RADOS) and control plane MON / RBD / RADOSGW / CEPHFS composition in CRUSH Algorithm core algorithm processing data redundancy and high availability, the upper application storage completion data directly through the data plane with OSDs librados read and write, snapshot support, backup, monitoring capability observability the like, can be output through the Kubernetes direct Rook, RedHat / SUSE also provides the ability to install separate clusters.

Some of the basic architecture of Ceph features and capabilities:

• Control plane: MON / RBD / RADOSGW / CEPHFS;
• Data plane: OSDs (RADOS);
• Snapshots, backup, support IO monitoring storage performance monitoring, support server RBD QoS rate limiting capability.

Portworx

Portworx container services deployed in a manner, each node is called PX, various public cloud butt block storage server bare metal or down, or up to provide file service block.

Form and does not bind the hardware vendors can access any public cloud or a self-built server cluster (only required to support iSCSI or FC protocol), currently Portworx flagship cloud disaster recovery capabilities DR, copy cloudy, with a complete snapshot (ROW), Cloudy management, synchronous replication (the RTO, seconds) asynchronous replication (RPO <= 15min), may be stated by way Kubernetes CRD, durable and elegant cloud data with the application automatically migrated cloud ability. PX may be independently deployed, the container is not strong dependence of Kubernetes network.

Portworx some basic functions / performance characteristics:

• Elastic expansion, ability to identify automatically PX server node, dynamically scheduling IO

• Control plane
  • Mainstream support vessel scheduling tool: Kubernetes, Mesos, Swarm, etc.
  • IO support level performance monitoring
• IO face
  • Scattered data blocks and metadata to a different node
  • RPC uses caching and high performance
  • QOS isolation: not supported
  • The characteristics of the underlying storage IOPS (4k) 768 - 65024
  • Delay (4k): 0.58ms - 23ms
• Value-added features
  • Encryption (tripartite keys hosting, transmission encryption, disk encryption off), and integrated support for cloud vendors KMS Vault
  • Snapshot (the ROW), management cloudy, synchronous replication (the RTO, second stage), asynchronous replication (RPO <= 15min)
  • Scalability> 1000 nodes,> 10,000 Volume
  • Support topology-aware computing

OpenEBS

OpenEBS Kubernetes build open source version of EBS, based on software defined PV: various media, including local disks, and other cloud storage pooling and unified management. Use iSCSI as the storage protocol. Not bound one vendor's storage, a variety of reasons flexible access storage. In a sense also more flexible, lightweight. However, the strong dependence vessel network abstraction layer increases OpenEBS layer, the writing operation through abstraction layer, and each has its own volume PV Controller, adds extra overhead, can be done while a more flexible, but compared to Portworx , Ceph, its relatively large disadvantage in performance.

OpenEBS some basic functions / performance characteristics:

• Control surfaces: the expansion vessel scheduling system, support ultra integration. Compared blocks, the number of volumes and the plurality of volumes arranged in any size, more flexible;
• High availability: Each volume can have multiple copies of real-time data synchronization, data synchronization is the synchronization between different storage pools;
• Snapshots, backup, storage performance monitoring function;
• Cloud-Native Tools and good integration: You can use the cloud native tools (such as Prometheus, Grafana, Fluentd, Weavescope, Jaeger, etc.) to configure, monitor and manage storage resources.

Understanding of cloud storage

Pangu vs RADOS

Compare more than three open source / storage company, in order to more easily understand cloud storage architecture, we stratified Pangu's layered architecture and Ceph storage to make a comparison.

Can put CS (Chunk Server) analogy Ceph OSDs service process, the Pangu Master process analogous to the Ceph MDSs process.

The cloud product analogous to the block is stored Ceph RBD, files are stored in the category CephFS, analogous to objects stored RADOSGW, the present storage block / high-performance file temporarily stored CPFS no corresponding product.

With the evolution of the architecture of Pangu, Pangu 2.0 and comprehensively promote, promote user mode TCP network protocol stack, comprehensive RDMA storage network, fully optimize the performance of the RPC, the upper product storage also enjoy a huge bonus underlying storage change, enter the level sub-millisecond latency, and one million IOPS of age, native cloud storage is also bound to be on top of the product storage hierarchy, you can inherit these capabilities.

Cloud storage native difference cloud and the public cloud dedicated (private)

By analyzing the native cloud storage market, we can find these store has common features is support for the statement of the API, you can achieve the metrics and statements of performance, capacity, and other aspects of function, more or less on the quality / stable / security has a different support.

Further, the cloud native load directly via the data plane without the loss of use of the product stored in the capacity, performance, capacity of accessibility, a multi-tenant control plane continue to improve user-oriented applications IO observability, application level of the QoS, the isolation, the storage interface through the control plane interface to CSI / Flexvolume like can be declared, and provides partial Operator storage life cycle, the container arrangement and the service application storing the actual load becomes the adhesive statements, it may be more correct to use the cloud storage position.

Since the complete infrastructure of public cloud storage product, you can use more lightweight data plane (virtio, nfs-utils, cpfs-sdk, oss-sdk) to access the stored product.

Proprietary cloud environment quite different, or no portion virtualization virtual environment, SAN, and the naked disc is the main storage, by the need to build ceph RADOS achieve similar or pangu SDS, and then via the data plane (librados / px / pv-controller) to achieve access to the stored.

For private cloud vSphere, OpenStack, flying constructed, there is provided closer to the public cloud storage mode, but because of differences in the deployment of the module, there are differences in the control / data plane supporting ability.

It is simply:

• Public cloud Cloud Native Storage = Declarative API + Cloud Storage
• Proprietary cloud Cloud Native Storage = Declarative API + Native Storage

Public cloud cloud storage native

1. Storage tiering, reuse infrastructure dividends, do not reinvent the wheel.

1. Native cloud storage

• Improve data consistency plane (kernel / OS / net / client / sdk version control and optimization parameters);
• Build a unified control plane CSI / Flexvolume / Operator, provide customer-oriented statement API;
• Topology-aware scheduling to achieve the level of choreography, achieved node awareness zone awareness cloud disk, local disk.

Block Storage

In the process of achieving the control plane level buffer IO control by using the Kernel Cgroup blkio binding Aliyun Linux 2 OS, to enhance the particle size in the application layer QoS local disk, the disk control cloud. LVM by the local disks cut points may be implemented to enhance the density of the cloud single disc. Through the mount point / IO index measuring apparatus to gather capability IO observability.

Native cloud storage - Main characteristic index storage block:

• Capacity: single disc 32TB
• Delay: 0.2ms - 10ms
• IOPS: 5K – 1M
• Throughput: 300Mbps - 4Gbps (local NvME ESSD: 2GBps)
• Accessibility: single availability zone exclusive
• QoS: single disc isolation, process isolation
• Multi-tenancy: single disc isolated

For details, see: cloud disk performance

File Storage

In the control plane can be controlled by Pod Security Policy and SecuritContext achieve force application UID / GID control, application control file system ACL. The control plane to achieve control of the file system life cycle, by measuring mount point indicator IO gather capability observability of IO.

Native cloud storage - the main features of index files are stored:

• Capacity: 10PB single file system
• Delay: 100 subtle - 10ms
• IOPS: 15K – 50K
• Throughput: 150Mbps - 20GBps
• Accessibility: multi-cluster shared much free zone
• QoS: IO competition
• Multi-tenant: PSP ACL (namespace)

CPFS Parallel File System

Provide life-cycle of configurability, the file system control plane implement the file system ACL control of QoS provides clients the speed limit declarative management Operator, and further, to achieve declarative CPFS file system deployed in the cloud-native environment .

Native cloud storage - the main features of high-performance file storage index:

• Capacity: Single File System 100PB
• Delay: 0.5ms - 10ms
• IOPS: 50K – 1M
• Throughput: 10Gbps - 1000GBps
• Accessibility: multi-cluster shared much free zone
• QoS: Support clients speed limit
• Multi-tenant: PSP ACL (namespace)

Summary: Cloud native storage v1 - Functional

Today's cloud storage native has been achieved in the control plane / control plane interface to support Ali cloud the whole category of products stored in the data plane also completed most of the system-level optimization and client layer. But with the migration of a large number of container persistence enterprise applications and intelligent applications, we are still faced with more problems and challenges.

---

In the development process of the entire cloud storage native v1, the team thanks to Ali cloud storage in file storage, block storage and object store co-operation and assistance, together to create native storage cloud era.

As the cloud of native applications for mobility, and dynamic characteristics may be extended, the cloud storage native brings corresponding density requirements, speed, degree of mixing, so the top of the cloud storage capacity has proposed a substantially efficiency , flexibility, autonomy, stability, application demands and low coupling, GuestOS optimization, security and other aspects. The new enterprise workloads / intelligent workload of container, moved to cloud storage encountered performance, flexibility, high availability, encryption, isolation, can be observational issues, life cycle and so on, not only the need to improve levels of storage products, more needs to improve the control of cloud native / data plane, and promote the evolution of cloud native storage and cloud storage, which is a cloud native storage v2 outlook and planning, we will further reveal these new scene in a future article, needs, programs and the direction of development.

"Alibaba Cloud native micro-channel public number (ID: Alicloudnative) focus on micro service, Serverless, container, Service Mesh and other technical fields, focusing popular technology trends in cloud native, cloud native large-scale landing practice, do most understand cloud native developers technology public number. "