Head picture.png

Author | Shuai Chang Alibaba Cloud Native Serverless Team Senior Technical Expert
Source | Alibaba Cloud Native Official Account

Experience document: Mirror pull acceleration document

FaaS and containers

Because of its disruptive innovation, container images have become the de facto standard for application deployment formats in the cloud-native era. Leading cloud vendors FaaS (Function-as-a-Service) services such as Alibaba Cloud Function Computing and AWS Lambda will also support the use of container image deployment functions in 2020, fully embracing the container ecosystem. Since its release, developers have successively used mirroring to quickly serverless multiple scenarios such as machine learning, audio and video processing, event-driven offline data processing, and front-end automation to improve efficiency and reduce costs. However, cold start has always been a problem that Serverless cannot circumvent. Container mirroring needs to download and decompress data remotely through the network. For GB-level mirrors, the pull time may be as high as minutes, which objectively magnifies the cold start side effect and hinders the serverless evolution of real-time applications.

Function calculation mirror acceleration function

Traditional image pull acceleration emphasizes "developer responsibility", such as streamlining the image, rationally assigning the image layer, multi-stage construction, using tools (such as docker-slim) to remove unnecessary data, and following the best practices of construction. These tasks not only increase the burden on users, but also have limited acceleration effects, and run-time stability risks. Alibaba Group's ultra-large-scale container environment with highly complex scenarios has a deep accumulation of mirror storage and acceleration technologies, and has outstandingly undertaken the rigorous challenges of three-year double eleven, double twelve, and Spring Festival promotion scenarios. Alibaba Cloud Serverless cooperates in-depth with services such as container mirroring and storage, and outputs internal innovations in function calculations: Hangzhou, Beijing, Shanghai, East America, and West America have officially released mirroring acceleration functions. This function transfers the burden of mirroring optimization that originally belonged to developers to function computing, and further helps developers improve production efficiency and focus on business innovation.

1. Acceleration effect

We selected the internal production environment and the workload of the open source community, covering 7 different combinations of image sizes, IO access modes, and startup commands, including machine learning, artificial intelligence, front-end automation, and web applications, as the benchmark, and deployed them in the FC Beijing area. As shown in the figure below, the acceleration of function computing after the mirror acceleration function is enabled generally exceeds 50%. For common bloated mirrors in machine learning scenarios (such as multiple teams sharing basic mirrors, ml-small-import, ml-large-import, ai- cat-or-dog) acceleration effect is more obvious (about 70%-86%), the larger the mirror image, the higher the optimization space.

2. How to use

Image acceleration can be enabled through the console, CLI tool or FC SDK. For detailed steps, please refer to the image pull acceleration document .

Method 1: Select "Enable Mirror Acceleration" under the function configuration of the function computing console .

Method 2: Use Funcraft tool to deploy .

Add AccelerationType: Default under the existing CustomContainerConfig configuration, configure AccelerationType if you want to close it:

        CustomContainerConfig:
          Image: registry-vpc.cn-beijing.aliyuncs.com/fc-demo/python-flask:v0.1
          AccelerationType: Default

3. Features

FC mirror acceleration has the following characteristics:

Easy to use: Just turn on the image acceleration on the function, and the function calculation will automatically create an accelerated image and cache. After the conversion is completed (within 5 minutes), the function automatically uses the accelerated image cache.
Focus on business innovation: Developers do not need to spend time deliberately simplifying and optimizing the image size or strictly distinguishing the construction methods of Serverless and Serverfull application images. FC is responsible for pulling and decompressing data according to the actual use of the application.
The acceleration is free, and the threshold for use is low: there is no additional cost incurred when the image acceleration is turned on, and there is no need for developers to purchase or upgrade any other services. In fact, as the image pull time becomes shorter, the corresponding request fee is also reduced.
Extremely fast elasticity, scaling to 0, event triggering: FaaS combined with container mirroring has greatly simplified the application migration to Serverless, and the acceleration function has further unlocked real-time and quasi-real-time workloads. Containers that once required minutes to start can now be started quickly within a few seconds. Start, and really realize the reduction to 0.

Why is mirror pull slow?

An OCI V1 container image contains multiple layers, and each layer is a compressed file system (folder), usually stored in a remote service (such as object and file storage) in tar.gz format. The steps to pull the image are as follows:

Download the tar.gz file corresponding to each layer to the local.
Decompress each layer sequentially.
Combine each layer (such as Overlay) as a rootfs startup container.

Although the above steps are simple, they are the main reason for the slow image pull:

File format defects, coarse-grained data stratification, and sequential decompression: The gzip layer prevents fine-grained random reading of the data actually needed by the application, and requires all layers to be decompressed sequentially in a single thread. Actual observation found that the image layer can increase the speed through concurrent download, but the decompression link is difficult to optimize in the gzip format.
Inefficient compression/decompression algorithm: gzip used in the image layer, benchmark gzip decompression speed is nearly 9 times slower than lz4 on average.
Full data download: Also due to the coarse-grained layering and gzip format (seek is not supported), the mirrored data must be downloaded to the local regardless of whether it is actually useful or not.

In summary, the image pull time is directly proportional to the size of the image, while running apt/yum install during the container image building process, useless tests, data files, and executing commands such as chmod/chown during the building process causes multiple copies of the same data. It is easy to introduce data that is not needed by a large number of applications.

Acceleration principle

Function computing applies Alibaba Group's mature image acceleration technology to public cloud services. The acceleration technology revolves around two core ideas:

Load on demand: Only read the data that the application actually needs, greatly reducing the amount of data transmission.
More efficient storage and algorithms: data of the same size can be decompressed faster.

1. Load on demand

The data load rate of the mirror included in Benchmark is between 12%-84%. Except for web applications with smaller mirrors, the data utilization rate of most scenes is less than 50%. The original image with layer as the data distribution unit is converted into a data format that supports fine-grained on-demand reading, and is stored in a storage with better latency and throughput.

2. Efficient decompression

In addition to the delay savings in the download step brought by on-demand loading, the image acceleration technology has also been extensively optimized in the data decompression step. The figure below shows that even when more than 70% of the full data is loaded, the optimization effect still exceeds 60%.

future plan

Function Computing has officially released container image acceleration, which accelerates 50%-80% in different scenarios through on-demand reading and more efficient decompression technology. Even GB-level images can be started end-to-end in a few seconds. The acceleration function, combined with the extreme flexibility of function calculation and the characteristics of event triggering, unlocks more workloads with high real-time requirements. Container applications can more easily enjoy the Serverless feature, truly scale down to zero and quickly expand on a large scale. In the future, FC will continue to optimize all aspects of the cold start to provide extreme flexibility, assume more user responsibilities, and enable developers to focus on business innovation.

Appendix: Experimental scene data

reference

About the Author

Shuai Chang, senior technical expert of the Alibaba Cloud native serverless team, led the ecological integration of functional computing and container technology and the native observability of FaaS cloud.

Start-up delay is reduced by 50%-80%, and function computing releases image acceleration function