Alibaba officially open sourced its self-developed container technology Pouch

Cloud Effect Platform 2017-11-27 13:44:21 Views 315 Comments 0 Posted in: Alibaba Cloud Effect Platform
Linux Security Architecture Docker Open Source Image E-commerce Container Data Center Cloud Effect Dragonfly Pouch

Abstract: After restarting and maintaining Dubbo, Alibaba technology has There are continuous developments in open source. At the China Open Source Annual Conference, Alibaba officially open sourced its self-developed container technology Pouch.

At the China Open Source Annual Conference, Alibaba officially open sourced Pouch, a container technology based on the Apache 2.0 protocol. Pouch is a lightweight container technology with features such as fast efficiency, high portability, and low resource consumption. It mainly helps Ali to deliver internal services faster, and at the same time improves the utilization of physical resources in ultra-large data centers. Rate. After open source, Pouch has become an inclusive technology, and everyone can get it on GitHub. The GitHub project address:

https://github.com/alibaba/pouch

0acdac1b1944a2a86825e571875f63e2be795b82

The open source of Pouch is a signal that Alibaba is optimistic about container technology. Today, it has become a consensus that container technology has landed in most enterprises worldwide. How to do a good job in the technical selection of containers and how to make the container technology controllable are believed to be issues that every enterprise must consider. Pouch undoubtedly adds another weapon to the container ecology, and has won a position for Chinese technology in the container open source ecology monopolized by global giants.

Status of Pouch Technology With

the open source Pouch, I believe that many experts in the industry will be interested in Alibaba's current container technology. In the end, is Alibaba Play Container a hero or a rising star? Looking at the future from the past, this is especially true in the field of technology. The precipitation and accumulation of technology can roughly see the technical strength of a company.

Pouch Evolution

Tracing the history of Pouch, we will find that Pouch originated in 2011. At that time, technologies such as namespaces and cgroups on the Linux kernel began to mature, and tools such as LXC were also born at the same time. As a technology company, Alibaba developed container technology t4 based on LXC, and provided services to the group in the form of products at that time. This move is regarded as Alibaba's first exploration of container technology, and it also accumulates initial experience for Alibaba's container technology. With the passage of time, two years later, Docker was born, and its mirroring technology has greatly solved the "software packaging" problem that has plagued the industry for many years. After the mirror technology became popular, there was no reason why Alibaba could not integrate this technology that brought great value to the industry. Therefore, in 2015, based on its own container technology, t4 gradually absorbed the Docker image technology in the community, and gradually evolved and polished it into Pouch.

Container technology with mirroring innovation is like a hurricane. Wherever it goes, it is applauded at home and abroad, and Alibaba is no exception. Since the end of 2015, Alibaba Group has also undergone changes at the infrastructure level. There are many reasons, one of which is the simplest, and I believe it is not difficult for everyone to understand that an Internet company with the size of Alibaba must be supported by a huge data center. The explosive growth of business will inevitably lead to an increase in infrastructure demand, which means resulting in a substantial increase in infrastructure costs. The lightweight and low resource consumption of containers, coupled with the rapid distribution of images, quickly made Alibaba determined to increase investment in the field of container technology to help comprehensively upgrade data centers.

Alibaba Container Scale

After more than two years of investment, Alibaba's container technology, Pouch, has played an extremely important role in the Group's basic technology. In 2017 Double 11, behind the huge transaction of 168.2 billion, Pouch did it in the "super project":

100% of online business Pouched
Container scale reached one million level
Back inside the Alibaba Group, Pouch's daily services have covered most of the business units, covering business scenarios including: e-commerce, advertising, search, etc.; covering technology stacks including: e-commerce applications, databases, big data, stream computing, etc. ; Covers programming languages: Java, C++, NodeJS, etc.

Pouch technical advantages

Alibaba 's container technology has such a wide range of applications, which is a great blessing to the industry, because Alibaba has demonstrated with facts that container technology has been proven in large-scale production environments. However, since Pouch originated from Alibaba, not the community, there are differences between the two systems in terms of container effects and technical implementation. In other words, Pouch has many unique technical advantages.

Strong

isolation Isolation is an unavoidable technical problem that enterprises cannot avoid in the process of cloudification. Strong isolation means that the technology has the preliminary conditions for commercial use; otherwise, it is almost impossible to roll out the business line. Even a technology company like Alibaba, at the beginning of practicing container technology, was not immune to security problems. As we all know, most of the container solutions in the industry are based on the cgroup and namespace provided by the Linux kernel to achieve isolation, and such lightweight solutions have drawbacks:

the same kernel is shared between containers, between containers and the host;
the isolation resources implemented by the kernel, dimensions insufficient.

Faced with the current situation of the kernel, Alibaba has adopted three aspects of work to solve the security problem of containers:

user mode enhances the isolation dimension of containers, such as network bandwidth, disk usage, etc.;
submit patches to the kernel to repair container resources Visibility issues, bugs in cgroups;
implement hypervisor-based containers, container isolation by creating new kernels.

Research on container security will continue for quite a long time in the industry. In the open source Pouch, Alibaba will continue to integrate features such as lxcfs and share it with the community on the basis of the original security. At the same time, Alibaba is also planning to open source the "Ali Kernel", which will give back to the industry the enhancement of Alibaba's Linux kernel over the years.

P2P image distribution

With the explosive growth of Alibaba's business and the rapid popularization of container technology after 2015, the distribution of Alibaba's container images has also become an urgent problem to be solved. Although container images have helped enterprises to do a lot of optimization in application file reuse and other aspects compared to traditional methods, the distribution efficiency is still maddening under the scale of tens of thousands of clusters. Take a simple example: if there are 10,000 physical nodes in the data center, and each node initiates an image download to the mirror warehouse at the same time, the network pressure and CPU pressure of the machine where the mirror warehouse is located can be imagined.

Based on the above scenarios, Alibaba's image distribution tool "Dragonfly" came into being. Dragonfly is a general file distribution system based on intelligent P2P technology. It solves the problems of time-consuming distribution, low success rate, and wasted bandwidth in large-scale file distribution scenarios. Significantly improve business capabilities such as release deployment, data preheating, and large-scale container image distribution. At present, "Dragonfly" and Pouch are open source at the same time, and the project address is:
https://github.com/alibaba/dragonfly

The usage architecture diagram of Pouch and Dragonfly is as follows:
34fcaae2985add9be896fbbca823d948ea16aa41

Rich container technology

Alibaba Group includes a variety of business scenarios, and almost every scenario has its own requirements for Pouch. If the external "single container, single process" solution is used, there is incredible resistance to containerization in the business sector. Inside Alibaba, basic technology plays a huge supporting role, and it needs to better support the operation of business at all times. When the business is running, it is almost difficult for technology to make the business change and adapt itself in turn. Therefore, a container technology that is not intrusive to application development and application operation and maintenance can be rapidly rolled out on a large scale. Otherwise, during the containerization process, on the one hand, the support of the business side will not be available, and on the other hand, a lot of manpower will be required to help the business side, and the non-standardized business operation and maintenance will be realized.

Alibaba is well aware of this, and the internal Pouch technology can be said to be non-invasive to the business, and it is precisely because of this that 100% containerization is achieved within the group. Such container technology is called "rich container" by countless Alibaba people.

The implementation of the "rich container" technology is mainly to create a container on the Linux kernel that is completely consistent with the virtual machine experience. In this way, it is more powerful than the general container, with a complete init process inside and any services required by business applications. Of course, this also proves why Pouch can be non-intrusive to applications. During the implementation of the technology, Pouch needs to define the execution entry of the container as systemd, and in the kernel mode, Pouch introduces the latest kernel patch of cgroup namespace, which satisfies the isolation of systemd in the rich container mode. From the perspective of enterprise operation and maintenance processes, rich containers also have obvious advantages. It can do some things before the application's Entrypoint starts, such as unified to do some security-related things, and the operation and maintenance-related agents are pulled up. These things that need to be done in a unified way, if they are put into the user's startup script or image, are intrusive to the user's application, and the rich container can handle these things transparently.

Kernel Compatibility The explosive development of

container technology has enabled many companies at the forefront of technology to enjoy technological dividends. Then, the "long tail effect" is also doomed to have a long cycle of technological evolution. The development of Pouch also encountered the same problem in the process of scaling.

As long as the scale reaches a certain amount, "Moore's Law" determines that there will be legacy resources in the data center, and how to utilize and process these physical resources is a big problem. The same is true within the Alibaba Group. Whether it is a different model of machine or a Linux kernel ranging from 2.6.32 to 3.10+, heterogeneity still exists. If all applications are to run in Pouch, Pouch must support all kernel versions, and the Linux kernels supported by existing container technologies are all above 3.10. Fortunately, technically, for old kernel versions such as 2.6.32, namespace support only lacks user namespace; other namespaces and common cgroup subsystems exist; but /proc/self/ns are used to record namespaces. Auxiliary files did not exist at the time, and system calls such as setns also needed to be supported in a later version of the kernel. Ali's technical strategy is to bypass some system calls and implement container support for older versions of the kernel through some other methods.

Of course, from another point of view, rich container technology has also been adapted to other operation and maintenance systems, monitoring systems, and user habits on the old version of the kernel to a large extent, ensuring that Pouch has a high degree of kernel compatibility. availability.

Therefore, in general, based on the technical advantages of Pouch, it is not difficult to find the application scenarios suitable for Pouch: the rapid containerization of traditional IT architecture, the deployment of large-scale business of enterprises, and the financial scenarios with high security isolation requirements and high stability requirements. Wait.

Pouch Architecture

With differentiated technical advantages, Pouch has been well proven in Alibaba's large-scale application scenarios. However, it has to be said that there are still some differences between Alibaba's internal Pouch and the current open source version.

Although the advantages are obvious, if the internal Pouch is directly open sourced, it is almost impossible. After years of development, the internal Pouch has been coupled with Alibaba's internal infrastructure and business scenarios while serving the business. The content of coupling is not very versatile for the industry, and it also involves some other issues. Therefore, in the open source process of Pouch, the first priority is to decouple internal dependencies and open source the core parts that are also of great value to the community. At the same time, Ali hopes to stand with the community at the very beginning of open source, and build Pouch's open source community together. Subsequently, the Pouch within the Alibaba Group was gradually replaced with the open source version of the Pouch, and finally the goal of consensus inside and outside the Pouch was achieved. Of course, in this process, the decoupling of the internal Pouch and the evolution of plug-in are equally important. In Pouch's open source plan, the end of March next year will be an important time point, when the 1.0 version of Pouch will be released.

From the first moment of planning to open source, the architecture diagram of Pouch in the ecosystem is designed as follows:

405bb7122a318e7f2f1398ab025a9fa6c5ab6c5c

The ecological architecture of Pouch can be viewed from two aspects: first, how to connect the container orchestration system; second, how to strengthen the container runtime .

The support of the container orchestration system is an important part of the Pouch open source plan. Therefore, at the beginning of the design, Pouch hopes that it can natively support orchestration systems such as Kubernetes. To achieve this, Pouch is the first in the industry to support container 1.0.0. At present, the container solution containerd in the industry is mainly in version 0.2.3, and the security and other functions of the new version are not yet available, and Pouch is already the first to eat crabs. At present, Docker is still the most popular container engine solution in the Kubernetes system, and Kubernetes' strategic plan at the runtime level is to use cri-containerd to reduce the coupling between itself and commercial products, and to take the road of being compatible with community solutions, such as cri-containerd and containerd Community Edition. In addition, it should be mentioned that the internal Pouch is an important part of Alibaba's dispatching system Sigma, and supports the realization of the "mixed department" project. In the Pouch open source route, it will also aim to support "mixed divisions". In the future, Sigma's scheduling and co-location capabilities are expected to serve the industry.

In terms of ecology, Pouch is based on openness; in terms of container runtime, Pouch advocates "richness" and "security". The support of runC can be said to go with the flow. The support of runV shows a difference from the ecology. Although docker supports runV by default, it is not compatible with "container" and "virtual machine" in docker's API, so Docker is not a unified management entry. As far as we know, there are still many scenarios of virtual machines in existing enterprises. Therefore, in the era of containers, how to manage containers and virtual machines through a unified operation and maintenance portal is bound to be a "virtual machine moving forward". In the transition period of "container", one of the solutions that enterprises are most concerned about. The open source form of Pouch covers this scenario well. runlxc is Alibaba's self-developed lxc container runtime. Pouch's support for it also means that runlxc will be open sourced in the near future, covering scenarios where enterprises have a large number of low-version Linux kernels.

The architecture of the Pouch docking ecosystem is as follows, and the internal architecture of Pouch can refer to the following figure:

484a4626f29deaaa119e6b3a428e937e5c91febe

is similar to the traditional container engine solution, and Pouch also presents a C/S software architecture. At the command line CLI level, both Pouch CLI and Docker CLI can be supported. Connected to the container runtime, Pouch internally calls containerd through gRPC through the container client. The interior of Pouch Daemon adopts the design concept of componentization, and extracts the corresponding System Manager, Container Manager, Image Manager, Network Manager, and Volume Manager to provide a unified object management solution.

Written at the end

Now that Pouch is open source, it means that the container technology accumulated by Alibaba will go out of Alibaba and face the industry. The technical advantages of Pouch determine that it will provide a differentiated container solution for users to choose from. When enterprises are taking the road of cloudification and embracing cloud native (Cloud Native), Pouch is committed to becoming a powerful software to help enterprises achieve the most stable support for their digital transformation.
Pouch is currently open source on GitHub, and any form of open source participation is welcome. The GitHub address is:
https://github.com/alibaba/pouch


Author introduction

Sun Hongliang, Alibaba technical expert, graduated from Zhejiang University, is currently responsible for the open source construction of the container project Pouch in Alibaba. He has been engaged in the field of cloud computing for several years. He is one of the first engineers in China to research and practice container technology, and plays an extremely important role in container technology preaching in China. With the book "Docker Source Code Analysis", he personally advocates the spirit of open source, and is the global maintainer of the Docker Swarm project.





This article is the original content of Yunqi Community, and cannot be reproduced without permission. If you need to reprint, please send an email to [email protected]; if you find any content suspected of plagiarism in this community, please send an email to: yqgroup@ service.aliyun.com reports and provides relevant evidence. Once verified, the community will immediately delete the allegedly infringing content.