With the rapid development of distributed computing, containers, and multi-form intelligent hardware, edge devices have increased dramatically, and gradually become the main force of data production. As the bottom support of edge data, edge storage provides high-speed and low-latency data access for edge computing to fully release the value of edge data and enhance the competitiveness of enterprises. Recently, Sandstone USP V3.3 version of Shanyan unified storage platform was officially released. The new version has been supplemented and enhanced in terms of access scenarios, network transmission, container support, site management, deployment specifications, etc. to meet the needs of edge scenarios, to meet the needs of storage and management of edge data in edge scenarios, and help customers solve planning This article explains the related features in terms of deployment and management of edge storage.
1. In the era of Internet of Things, edge storage has become a must-have option
The amount of data generated by edge devices is large, and the underlying storage is required to support massive scale.
From the perspective of total data, the global total data will reach 180 ZB by 2025, of which 75% of the data comes from the edge and terminals, and 50% of the data needs to be in Edge processing. From the perspective of data traffic, by 2021, global IP data traffic will reach 3.3 ZB, of which 63% of the data generated by edge devices. The huge amount of edge data is inseparable from the strong support of the underlying storage.
Edge devices have high real-time requirements and need to store and process edge data nearby. With the application of deep learning and augmented reality technology, edge data processing needs to have real-time performance. The traditional cloud computing model uses centralized management. Edge data is processed after remote transmission. There are transmission delays and network fluctuations, making it difficult to meet the real-time requirements of edge applications. This requires dedicated storage for edge computing to enable data storage and processing nearby.
Under this background, the integration of edge storage and artificial intelligence technology will promote new business forms such as smart transportation, unattended power grid inspection, and unattended financial outlets.
2. What exactly is edge storage?
Edge storage is an edge computing service and provides real-time and reliable data access. Edge storage is an extended concept of edge computing. It is a new type of distributed storage architecture for edge big data storage. It mainly provides real-time and reliable data storage and access for edge computing. Unlike centralized cloud storage services, edge storage migrates data storage from a long-distance cloud server to an edge storage device or edge data center closer to the data, which greatly reduces the time between data generation, computing, and storage. Physical distance, lower network communication overhead, interactive delay and bandwidth cost, and higher scalability.
Hierarchical architecture of edge storage: edge device-> edge data center-> distributed data center From the perspective of the hierarchical architecture of edge storage, edge storage is composed of edge devices, edge data centers, and distributed data centers, as shown in Figure 1. Distributed data centers are deployed in areas far from the cloud platform, provide EB-level storage services, and cooperate with cloud data centers; edge data centers are deployed in areas where edge devices are concentrated, providing TB-level real-time storage services, multiple small physical data The center is combined into a logical data center; the bottom layer is composed of a large number of edge devices (such as various sensor terminals), and the devices are connected to each other to form an edge storage network.
Figure 1. Hierarchical architecture of edge storage
The value of edge storage: high efficiency and low latency, safe and reliable, and the value of promoting cloud-edge collaborative edge storage can be summarized as the following three aspects:
o Provide data prefetching and caching services to overcome the problems of high latency and network dependence caused by long-distance data transmission from cloud storage.
o Provide distributed data storage services adjacent to edge terminals, take advantage of rich storage features, ease storage and bandwidth pressures in cloud data centers, and reduce data risks.
o Cooperate with cloud storage to provide storage services, flexibly deploy multiple application loads, achieve collaborative processing of computing tasks at the edge and the cloud, and improve the real-time and reliability of data processing.
3. The rapid development of the Internet of Things has boosted the implementation of edge storage scenarios
As the Internet of Things technology matures, edge application scenarios have gradually reached many industries. In the power industry, in the past, power grid inspections required manual inspections, and each inspection of a substation took more than 1.5 hours, which was time-consuming and labor-intensive. Now using sensor equipment, the video, water, gas and wind data around the substation are aggregated to the edge data center for analysis and processing according to the Internet of Things transmission protocol. Edge storage is responsible for the interconnection, storage and other functions between the sensing device and the IoT platform, and uploads the data processed on the edge side to the upper-level data center to realize intelligent monitoring of the operation status of the substation.
In the transportation industry, a virtual station (gate system) is set up between each interchange and entrance / exit of the expressway to realize ETC (electronic non-stop toll collection system) vehicles and MTC (manual semi-automatic toll collection system) vehicles. The vehicle generates transaction flow (or pass certificate), ETC pass record and captured image information (including license plate number, license plate color, etc.) through the virtual site, and uploads it to the road section center and network center for reception, verification, detection, settlement, inspection, etc. Operation, to achieve the effect of unmanned expressway and intelligent toll collection. The edge storage can be deployed in the center of the road segment or the network center, responsible for storing the recorded information and massive images generated by the virtual site, and coordinated with the upper-level data center or cloud for scheduling.
In the construction industry, smart construction sites are also a good practice for edge scenes. Use the Internet of Things technology to collect data generated by various types of terminal equipment such as construction machinery, safety helmets, smart gates, environmental detectors, and surveillance cameras. By deploying edge storage on the construction site, a unified management platform can be built, and the data can be analyzed on the edge side Processing, no need to return to the data center for further processing, and efficiently achieve all-round, visual, and intelligent management of mechanical equipment scheduling, personnel safety monitoring, construction material inventory, and engineering environment monitoring, helping to solve difficult construction site management, frequent safety accidents, Problems such as inadequate environmental protection systems ensure the safety and efficiency of project construction.
In large enterprises, edge storage helps branch offices make better use of data, and enables data aggregation and distribution between headquarters and branches. In this scenario, branches in various locations are equivalent to edge sites. They have both data to be processed by themselves and data to be synchronized with the headquarters. By deploying edge storage, the data is stored locally for a long time after the branch office is processed, avoiding the consumption of link resources with the headquarters; the data that needs to be synchronized can be aggregated to the headquarters in a period of time. This not only avoids frequent data interaction between the headquarters and branches, but also enables branches to use data resources more efficiently and release value.
4. Shanyan unified storage and matching edge scenes to improve edge computing capabilities
In the SandStone USP V3.3 version, Shanyan has complemented and enhanced some of the features in conjunction with the requirements of edge scenarios to provide stronger support in terms of access interfaces, network transmission, intelligent management, and deployment specifications.
Integrate blocks, files, and objects, rely on IPv6 to support the Internet of Things to efficiently transmit new versions and provide access interfaces such as blocks, files, and objects, support multiple application loads, meet storage access requirements for multiple edge data types, and avoid storage selection challenges ; Distributed, decentralized architecture, there is no performance bottleneck, and can flexibly replace applications deployed on the edge side, not affected by hardware, more flexible; newly added support for IPv6 protocol, so that the data link can carry a wealth of IoT protocols , To meet the needs of customers in different industries for transmission standards, improve the efficiency of network communications between data centers.
Rich APIs for intelligent management, good docking of containers, and new versions of cloud-side collaboration support rich APIs to implement automated storage management strategies, help users conveniently manage the underlying storage resources, and provide enterprise-level storage such as lifecycle automation management and remote disaster recovery Features, realize intelligent operation and maintenance, reduce the pressure of operation and maintenance personnel; support container stateful storage, provide persistent storage capabilities, and promote cloud and edge resources, applications, and data in the context of flexible deployment of application loads Collaboration, to avoid problems such as resource shortages and insufficient computing power when newly deployed edge devices, and improve the real-time, reliability and security of data processing.
Unified view management of branch sites, miniaturized deployment reduces construction costs. The new version supports centralized management and control of branch sites, and provides a unified view management interface. The status of the site is clear at a glance, avoiding frequent screening of the operating status of edge terminals; supports multi-site standardization and small-scale deployment , Simple and easy to use, business online quickly, while ensuring low cost while ensuring continuity.
With the widespread use of edge scenarios, the value demands of enterprises for edge storage are becoming more specific and clear. Shanyan unified storage platform realizes a set of data storage supporting multiple edge terminals. The data storage nearby reduces the risk of data leakage. The unified deployment of resources helps customers reduce hardware consumption, while improving overall costs while increasing revenue for customers.
Edge computing has become an important technical support for Internet of Things applications, and edge scenarios will also blossom in the drive of industry demand. The new version of Sandstone unified storage platform SandStone USP is aimed at edge scenario requirements, helping customers solve the problems of edge storage planning, deployment and management, supporting faster edge computing with more powerful edge storage, and providing massive intelligent storage support for edge data.