This article is shared from the Huawei Cloud Community " Be the first to support Kuasar!" iSulad Sandbox API simplifies the call chain and doubles reliability ", author: Cloud Container Future.
Sandbox isolation technology is a technology that effectively isolates processes to run in an independent environment. With the rise of container technology, sandbox isolation technology has also been widely used in the cloud native field. iSulad is the first to support Kuasar through the Sandbox API, providing efficient and stable sandbox management capabilities.
However, due to the history of container technology, the concept of sandboxing does not receive sufficient support in container engines and container runtimes. The OCI standard [1] does not define sandbox management, so the container engine and container runtime can only use container management to manage the sandbox, causing performance and stability problems. For details, see the Kuasar series of articles [2].
In fact, the container field has been conducting in-depth research and exploration on the introduction of sandbox management interfaces. For example, the Containerd community integrated Sandbox API related functions into the mainline in April 2022 [3]. This move optimized Containerd's internal sandbox management logic. Unfortunately, however, it still uses the OCI standard interface to call the container runtime to manage the sandbox.
On April 21, 2023, Huawei released the multi-sandbox runtime Kuasar[4] at the Kubecon+CloudNativeCon Europe 2023 Cloud Native Summit, introducing sandbox management logic into the container runtime. So far, Kuasar has become the first container runtime to support Sandbox API. It becomes possible for container engines to directly manage sandboxes using the Sandbox API.
iSulad[5] is also the first to support Kuasar through the Sandbox API, providing efficient and stable sandbox management capabilities. openEuler 23.09 has completed the integration of iSulad+Kuasar+StratoVirt, providing users with an extremely fast and lightweight secure container solution. For details, please refer to the second Kuasar series article [6].
Introduction to the Sandbox API
service Controller {
rpc Create(ControllerCreateRequest) returns (ControllerCreateResponse);
rpc Start(ControllerStartRequest) returns (ControllerStartResponse);
rpc Platform(ControllerPlatformRequest) returns (ControllerPlatformResponse);
rpc Prepare(PrepareRequest) returns (PrepareResponse);
rpc Purge(PurgeRequest) returns (PurgeResponse);
rpc UpdateResources(UpdateResourcesRequest) returns (UpdateResourcesResponse);
rpc Stop(ControllerStopRequest) returns (ControllerStopResponse);
rpc Wait(ControllerWaitRequest) returns (ControllerWaitResponse);
rpc Status(ControllerStatusRequest) returns (ControllerStatusResponse);
rpc Shutdown(ControllerShutdownRequest) returns (ControllerShutdownResponse);
}
The introduction of the Sandbox API solves the long-standing pain point between container engines and container runtimes[2]:
- Introducing Sandbox semantics to enhance the coherence of cloud native architecture
- Reduce the redundancy of the shim process, reduce resource overhead, and speed up startup
- Shorten the call chain and double the reliability
- Eliminate Pause container redundancy
- Unified sandbox interface enables container runtime to support multiple sandboxes
life cycle and management
The Sandbox API[7] defines how the container engine interacts with the container runtime. The Controller Service defines the life cycle management interface of the sandbox, including Create, Start, Stop, and Wait to exit. , status query (Status), destruction (Shutdown), platform information query (Platform), etc.
Through the Sandbox API, the container engine can directly manage the sandbox without indirectly managing the sandbox through the OCI standard interface, which improves the performance and stability of the container engine.
Resource management
The Sandbox API also defines the resource management interface of the sandbox, including resource preparation (Prepare), resource cleanup (Purge), resource update (UpdateResources), etc. The container engine can manage container resources through these interfaces, such as preparing resources before creating a container, updating resources during operation, and cleaning up resources after the container exits.
iSulad new architecture
Figure 1. iSulad architecture comparison diagram
After the release of Kuasar, iSulad immediately adopted a new architecture to support the Sandbox API, allowing it to directly manage the sandbox through Kuasar.
In order to maintain the compatibility and stability of existing versions, iSulad has only reconstructed and upgraded the CRI V1 version, allowing users to use the Sandbox API to manage the sandbox. The CRI V1alpha version continues to use the OCI standard to handle sandbox management requests.
Decoupling sandboxes and containers
In the new architecture, iSulad introduces the semantics of Sandbox and adds a new core module Sandbox, making it a first-class citizen of the container engine and realizing the decoupling of container management and sandbox management. From the perspective of the overall cloud native architecture, sandbox management between container orchestration components, container engines and container runtimes has become more smooth and efficient, forming a complete sandbox management link.
Take the iSulad+Kuasar+StratoVirt extremely fast and lightweight secure container solution as an example. iSulad receives CRI requests from Kubernetes in the northbound direction, creates Sandbox objects to handle PodSandbox related calls, and uses the Executor module to handle CRI Container requests. In the southbound direction, the Controller module is used to call Kuasar's Sandboxer process through the Sandbox API to manage the sandbox, and the Shim V2 module in the Runtime is used to call Kuasar's Task process to manage the containers in the StratoVirt virtual machine.
sandbox controller
Figure 2. Sandbox controller class diagram
The implementation of the Sandbox API enables iSulad to manage the sandbox directly through the Controller. Therefore, there is no need to create a Pause container when the Kuasar container is running to be compatible with the OCI standard, avoiding the redundancy of Pause containers.
In the new architecture, the design of the Controller module takes full consideration of the compatibility with the original sandbox management function, that is, it supports users to create ordinary containers (Pause containers) as sandboxes through the Sandbox and Controller modules.
As shown in the figure above, the Controller module provides a unified Controller interface for Sandbox, as well as two different implementations: Sandboxer Controller and Shim Controller.
- The Sandboxer Controller is an encapsulation of the Sandbox API and forwards the user's sandbox management request to Kuasar's Sandboxer process through the gRPC interface, allowing the Sandboxer to execute the underlying sandbox management logic.
- Shim Controller is compatible with the original container-based management interface and forwards the Sandbox management request to the Executor module in order to create and manage the Pause container-based sandbox. The implementation of Shim Controller enables users to continue to use the OCI standard interface to manage the sandbox under the new architecture, so as to be compatible with the original deployed business and ensure the continuity of functions.
The detailed design of Sandbox and Controller can be found in the design document of iSulad community [8].
Simplify container call chain
Figure 3. Container startup flow chart
After supporting the Sandbox API, iSulad's container management process has also undergone some changes. The figure above takes the iSulad+Kuasar+StratoVirt solution as an example, showing iSulad's simplified process from starting the sandbox to starting the container.
In the figure, Kuasar Task acts as the init process in the virtual machine and is also the management process of the container in the virtual machine sandbox. It provides the container management interface Task API to iSulad. The implementation of the Task API interface in the current solution is similar to but not identical to Shim V2. According to the Shim V2 specification, the container engine will call a Shim V2 binary, create a Shim process and return the Shim address for managing sandboxes, containers and resources. However, by calling the Sandbox API, iSulad no longer needs to manage the sandbox through the Shim process. On the contrary, the Start interface of the Sandbox API will return a Task address after starting the sandbox, allowing iSulad to communicate directly with the Kuasar Task process in the virtual machine to manage the life cycle of the container. This design eliminates the Shim process to reduce the memory overhead of the management plane and shorten the call chain, thereby improving the performance and stability of the entire solution.
Summarize
The Sandbox API is the core link of the extremely fast, lightweight and secure container solution composed of the three components iSulad, Kuasar and StratoVirt. Through the Sandbox API, the container engine can directly manage the sandbox without indirectly managing the sandbox through the OCI standard interface, thus significantly improving the performance and stability of the container engine. The introduction of the Sandbox API also provides a standardized interface for sandbox management between container engines and container runtimes, providing new possibilities for the development of the container field. The current implementation of the Sandbox API has been integrated into the main line of the iSulad community. Users can experience this full-stack self-developed ultra-fast, lightweight and secure container solution through openEuler 23.09. For details, please refer to the Kuasar series of articles [6].
The openEuler community has always adhered to the concepts of openness, collaboration, and win-win, and welcomes more developers to participate in the construction of iSulad, Kuasar, and StratoVirt to jointly promote the prosperity and development of the container field.
reference
[1] OCI Runtime Spec: https://github.com/opencontainers/runtime-spec/tree/main
[2] iSulad+Kuasar: A new generation of unified container runtime solution that reduces management plane resource consumption by 99%: https://mp.weixin.qq.com/s/uxpsV2yLVobiuYeaAhA_Lg
[3] Sandbox API : https://github.com/containerd/containerd/issues/4131
[4] Kuasar technology for multi-sandbox container runtime unveiled! 100% startup speed improvement, 99% memory overhead optimization: https://mp.weixin.qq.com/s/4L7f8pAlt9KCoyDhkW0mUQ
[5] iSulad: https://gitee.com/openeuler/iSulad
[6] openEuler 23.09 deploys extremely fast and lightweight secure containers based on Kuasar with one click: https://mp.weixin.qq.com/s/oaK_uF_UGsucH7rczISg-g
[7] sandbox.proto: https://github.com/kuasar-io/rust-extensions/blob/main/crates/sandbox/src/protos/sandbox.proto
[8] iSulad Sandbox design documentation: https://gitee.com/openeuler/iSulad/tree/master/docs/design/detailed/Sandbox
Click to follow and learn about Huawei Cloud’s new technologies as soon as possible~
Alibaba Cloud suffered a serious failure and all products were affected (restored). Tumblr cooled down the Russian operating system Aurora OS 5.0. New UI unveiled Delphi 12 & C++ Builder 12, RAD Studio 12. Many Internet companies urgently recruit Hongmeng programmers. UNIX time is about to enter the 1.7 billion era (already entered). Meituan recruits troops and plans to develop the Hongmeng system App. Amazon develops a Linux-based operating system to get rid of Android's dependence on .NET 8 on Linux. The independent size is reduced by 50%. FFmpeg 6.1 "Heaviside" is released