With the rapid improvement of hardware performance and the continuous growth of information services, enterprise users have also put forward higher requirements for the flexibility and manageability of IT infrastructure deployment. The traditional architecture based on hardware server hosts can no longer meet the new requirements. In order to meet the demand, more and more companies choose to apply shared storage virtualization technology to integrate storage resources and conduct centralized management. In this issue, Zhihui Huayun, in particular, brings "Analysis of ArcherOS Stack Shared Storage Virtualization Technology" to accelerate the promotion of government and enterprise users' business and applications to the cloud.
1. Introduction to ArcherOS Stack
ArcherOS Stack, as a cloud operating system layer between hardware and business load, adopts virtualization technology supported by X86 and ARM dual stack to realize the abstraction of general server physical resources, and integrates server physical resources such as CPU, memory, I/O, etc. Transform into a set of logical resources that can be uniformly managed, scheduled, and allocated, and based on these logical resources, build multiple simultaneous and isolated virtual machine execution environments on a single physical server to achieve higher resource utilization while satisfying applications More flexible dynamic resource allocation requirements, such as providing high-availability features such as hot migration and hot upgrade, to achieve lower operating costs, higher flexibility and faster business response speed.
ArcherOS Stack focuses on the virtualization requirements of key businesses, and meets the business requirements of typical key business applications such as ERP, CRM, core database, Web, e-commerce, and multi-application integration for performance, high reliability, high security, and automatic operation and maintenance. Provide corresponding virtualization technology features to accelerate the promotion of cloudification of services and applications.
2.ArcherOS Stack system architecture
ArcherOS Stack is a horizontally extended software integrated infrastructure cloud operating system, including cloud management platform (ArManager), software-defined computing (ArCom), software-defined networking (ArNet), shared storage virtualization (ArSSV), software-defined security, etc. Features. It is the smallest set of software-defined data center (SDDC) architecture.
ArManager: ArcherOS Stack management platform, providing the ability to create, manage and monitor resources.
Deployment Tool (ArLCM): Wizard-style graphical deployment tool for life cycle management of ArcherOS clusters.
ArCom: Virtual machine management program, used to create and manage virtual machines.
ArNet: virtual switch management; security groups can be used to set security policies for virtual machines, rules support TCP/UDP/ICMP/EGP and other protocols; support IPV4/IPV6 dual stack; traffic isolation.
ArSSV: Shared storage virtualization component. Through the software layer, the SAN device is divided and mapped to the LUN of the physical server for unified pool management, and small logical volumes are divided from them. The virtual machine recognizes these logical volumes as block devices.
3. Introduction to ArSSV
A major challenge brought by cloud computing is to solve the storage problems faced by large-scale virtual machine deployment. The first is the storage I/O performance bottleneck problem, because the growth rate of storage performance is slower than that of computing power. In terms of virtualization, I/O bottlenecks and slow storage performance have become the main bottlenecks. ArcherOS Stack provides different I/O performance optimization methods through shared virtualization components (ArSSV) to effectively alleviate storage bottlenecks. The second is the problem of low storage utilization. ArSSV provides storage thin provisioning technology to improve storage utilization and reduce storage hardware procurement costs. The third is the efficiency of large-scale deployment of virtual machines. ArSSV provides virtual machine cloning technology, which can shorten the time for large-scale deployment and distribution of virtual machines.
In addition, despite the rapid growth of the hyper-converged and software-defined storage market in recent years, traditional enterprise-level customer IT architectures often still have some SAN equipment. These SAN equipment are often costly, so customers do not want to waste their existing investments. According to the IDC market report, as of 2020 Q3, traditional enterprise storage still accounts for 59.2% of the domestic storage market, but HCI and SDS are gradually eroding the traditional storage market and will also grow steadily in the next few years. Due to factors such as old age, there is a transition period from traditional SAN equipment to software-defined storage (SDS) in the cloud transformation of IT architecture.
ArcherOS Stack supports the docking of SAN equipment to protect customers' existing investments. Users do not need to separately manage and set storage concept objects such as LUNs and RAID, but only need to set virtual machines/virtual disks to use various storage functions, which reduces the user's threshold for use. ArcherOS Stack achieves the following features through shared storage virtualization components:
It has nothing to do with hardware, as long as it is SAN storage supported by Linux, it can be adapted
Fully integrated with self-developed computing virtualization component (ArCOM)
Unify all devices into the same software stack
Supports thin provisioning, and allocates actual space according to the written effective data capacity
3.1 ArSSV overall architecture
实现思路是：在FC-SAN设备上划分配置好LUN，通过多路径软件映射给物理服务器，在服务器端将映射过来的LUN经由软件层虚拟化出一个大的存储资源池，资源池内分出一个个小的逻辑卷（LV），虚拟机可以识别这些逻辑卷成一个个块设备（虚拟磁盘）。
3.2 ArSSV关键技术解析
ArSSV对接各类存储后端
ArSSV最大的优势在于：在对接不同厂家的SAN存储设备时，不需要花费大量的时间开发对应的存储驱动程序。而且由于各家存储系统有不同的功能集，针对A、B不同存储系统开发的功能集也不一致，从而导致平台层面功能的差异性。针对存储系统开发对应的驱动仅适用于定制化项目中，而且每次开发、测试周期都很长。ArSSV另辟蹊径：屏蔽后端存储系统细节，将存储池的部分存储空间，通过LUN直接作为主机节点的存储空间，主机节点再对其进行细粒度的划分后，分配给虚拟机使用。
ArSSV实现精简置备
由于LVM自身的精简置备是通过thin-provision target实现的，这种实现方式下在LV创建快照之后性能会严重下降，所以ArSSV采用的是通过qemu将qcow2文件的数据分布直接写入LV的方式来实现精简置备。
qcow2在创建时实际占用空间很小（目前为1G，此数值可以调整），只有在有真正数据写入后开始增长，增长速度会根据IO速度变化。虽然qcow2可以自动增长，但LV的大小是固定的，所以需要在合适的时间动态扩展LV。LV的动态扩展会通过以下两种方式触发：
其一：ArSSV主动检查：ArSSV主动监测LV的空间增长情况，当实际写入空间占到所创建LV的50%时，LV会主动增加1GB大小的存储空间，以此类推，直至实际数据与LV大小一致。
其二：通过事件达到对LV真实空间的扩容，即：当虚拟机内部读写速度大于磁盘的自动扩容速度，虚拟机会出现磁盘读写错误，系统服务会发送一个读写错误的事件，当ArSSV监听到此类事件后会立即发起一次LV扩容操作，扩容成功后虚拟机读写即可自动恢复正常。
ArSSV元数据管理
ArSSV realizes the unification of metadata through distributed lock services. When it comes to metadata modification operations, distributed locks ensure that only one node is operated at the same time. Each node in the cluster runs a distributed lock service, and the state of the lock is written to the shared storage. The Disk Paxos algorithm is used to read and write the shared storage to achieve the acquisition, release, and timeout of distributed locks. Since all data of the distributed lock service is stored on the SAN storage, reliability will not be affected even if the cluster host process crashes.
ArSSV snapshots and clones
ArSSV uses tags to uniquely identify the LV of the snapshot, and online snapshots are implemented by calling libvirt. ArSSV implements the complete cloning of the virtual machine through the cinder driver. After the cloning, the new virtual machine has no data reference relationship with the source virtual machine. Part of the data of the source virtual machine will not affect the data of the new virtual machine. The data remains independent and the reliability is relatively high. It is higher for linked clones.
3.3 ArSSV summary
ArSSV helps users manage and use FC-SAN storage of different brands and models with extremely high versatility and compatibility, eliminating the complicated process of docking with different FC-SAN storage manufacturers’ device drivers, and avoiding the fact that FC-SAN manufacturers do not provide hardware. In the case of driver, the dilemma of self-developed driver is required to unify the use experience of X86 and ARM on FC-SAN equipment from different manufacturers, and even by providing simplified provisioning, snapshot, and cloning functions, users can directly use high-end SAN storage. Function, so that users do not need to care about the docking of hardware drivers, nor do they need to have professional SAN storage management knowledge, and focus more on the construction of business on the virtualization platform. It is very friendly to small and medium-sized manufacturers with weak IT construction capabilities and insufficient IT talent echelon.