[Interpretation of the latest technologies at DockerCon2017] Using Moby tools and Linuxkit to customize container systems

Abstract: In Yunqi TechDay 34: The latest technical interpretation of DockerCon2017, Alibaba technical expert Wang Bingshen explained the newly released Moby project and Linuxkit project in DockerCon2017, and introduced the difference between Moby and Docker in detail, as well as Linuxkit The advantages of , and introduce how to use Moby tools and Linuxkit to customize the container system with examples.

Abstract: In Yunqi TechDay 34: The latest technical interpretation of DockerCon2017, Alibaba technical expert Wang Bingshen explained the Moby project and Linuxkit project newly released in DockerCon2017, and introduced the difference between Moby and Docker in detail, as well as the Linuxkit project. The advantages of , and introduce how to use Moby tools and Linuxkit to customize the container system with examples.

The following content is organized based on the live video and shorthand of the speakers.

Speaker introduction:
Wang Bingshen (flower name: Xiheng), has rich experience in Linux development, has in-depth research on Docker technology, and submitted Docker Patch many times. Previously engaged in the design and implementation of Alibaba Cloud container service network solutions, focusing on the basic environment research of container technology.

This sharing mainly focuses on the following three aspects:
Why the Moby project
Linuxkit project is created
Using Moby tools and Linuxkit to customize the container system
515fb84e14253395b7d1a0d50d640f07a79723ab
This sharing will do some analysis on Moby and Linuxkit, two new projects released by Docker in DockerCon with more controversy. Everyone knows that at DockerCon, the Docker company renamed the Docker project to Moby and released the Linuxkit project. So in this sharing, I will first introduce the Moby project, then the Linuxkit project, and finally combine these two projects to build my own container system.

1. Why the Moby project
f0bd8e73b28aef0881e5893e1467c63b1b979678
At DockerCon, after Solomon, the former CEO of Docker, changed the PR, you can see a lot of "black question marks" expressions on the official PR. Everyone is very dissatisfied with this behavior of Docker, thinking that Docker is the The name was changed to Moby, whether Docker Run will be run through Moby in the future, and whether Docker will become a commercial product in the future, everyone is not very familiar with these issues. In many QQ exchange groups, everyone didn't understand this, so they thought that Docker had changed its name to Moby. In fact, in the introduction to Docker later, it is explained that in order to adapt to more scenarios, the scenario-based things in Docker are abstracted into the Moby project to build a more complete ecosystem.

Why did the Moby project come about? It can be seen that in recent years, after the Docker 1.17 version after 2015, many new features have been introduced, such as Network, runC, etc. Docker has more and more components, and the supporting scenarios are becoming more and more complex, such as Microservices, machine learning, the Internet of Things, etc., so the downloads of Docker images have also increased exponentially. In the end, the environment in which Docker runs is also becoming more and more complex, such as Linux, Windows, and embedded systems. on the device.
1c76dea63151c46958f822b68ad92eab199796a2
Therefore, the original release version of Docker is no longer suitable for these increasingly complex scenarios. For example, in IoT, the performance of Raspberry Pi is relatively good, but if it runs Docker, Docker will occupy A large part of the entire performance, so that it can no longer run some other own applications, how can users solve these things, should they write a Docker container engine by themselves? This cost is very high, and you need to build the wheels yourself. Therefore, in order to provide a more open ecosystem, Docker company abstracts some of the current components in the Docker project into a Moby project, so that system builders can assemble the existing components through the Moby project, and then assemble them into a container they need engine.
2d464737098ee55ba0bf5587c16f652514d0f261 The
Moby project now has over 80 components that allow users to avoid repetitive wheel building. Users can assemble components according to their own needs and make their own container system. These components have a standardized calling method, and they communicate with each other through gRPC. Its language can also be customized, and it will not have to be written in Go language as before. In a standardized way, these components can be combined through the Moby project to become a container system that they depend on.
The entire structure design of the a5b57df99ffe6be61da50474c705794db4868534
Moby project is shown in the figure below. Moby will provide an interface of a component library, and these components will implement the interface of this component library. These components are replaceable and can have different implementations on different platforms. Use Moby's tools to assemble the interfaces of these component libraries. For example, Docker now relies on the two components of network and Runtime, and then assembles these two components together, and then builds a software called Docker. It may be possible to make a container system that you need without relying on many Docker components.
135805cdf464eaa09ae3ffdb111246f18c432a76
Take the history of automobile development as an example. At the beginning, it was like a scooter. The whole system was completed by one person, and each component had to be closely coupled. This was the case before Docker 1.7, and there was only a single project. After 1.7, it is like an old-fashioned car. Many things will be divided into different factories for production, and eventually a product will be assembled. Of course, this method only means that these products are produced separately and will not affect each other, but there is no It is very general, and there is no way to adapt it in more scenarios. Therefore, it will eventually develop into the current car or a wider application scenario, that is, the parts factory first produces its own parts, and then the intermediate assembly factory purchases different parts to assemble the products they need. This is quite Moby project in Docker.
591c020884770c80949a58d44c30badc68fd4627
What are the benefits of Moby? First of all, it will provide a more open ecosystem. Previously, the components of Docker were distributed in different projects, so that the components can be donated to some foundations separately. For example, containerd has been donated to CNCF. After the Moby ecosystem, the follow-up will also be More components will be donated to the foundation, which will provide developers with a more open ecosystem. Secondly, components can be built in different languages, because the components communicate through standardized gRPC, and the components also allow customers to customize different container systems according to their own needs. For example, in the IoT environment, users can only Assemble the runtime and network into a container system of its own, without being limited by Docker.
4ee76fed579266c7308be60b50f9a611f9d38a32
What everyone is most concerned about is what impact this will have on Docker, whether Docker has become a closed-source product, and whether you will have to buy it if you want to use Docker in the future. In fact, Docker officially also gave the following picture, which means that the Docker project is now renamed the Moby project, but Docker will gradually be abstracted and separated from the Moby project, as a way of assembling Moby, such as Docker relies on this Some of the libraries in it are specialized into an assembly, which is assembled into its own version of Docker CE, which is the community version of Docker. The Docker Community Edition will continue to be open source, so users and developers do not need to worry about the charges for using Docker in the future. For Docker users, he does not need to perceive changes in the user interface. The commands used are Docker instead of Moby. For more operation and maintenance support, you can choose the Docker EE version, and let Docker's engineers do the operation and maintenance and more complex online support for you. If you develop it yourself, you can continue to choose the Docker community version. For architects, it is now possible not to rely heavily on the Docker project, but to assemble a container engine through these components to meet their own needs.
42189d2be93fdb4b2e51e5937006e3b94a58567e
Second, the Linuxkit project
Let's take a look at the latest released component - Linuxkit. Linuxkit is a secure, streamlined and portable operating system in the container era jointly released by many companies, so what are its features? First of all, Linuxkit is an operating system based on Moby. It can contain Moby's component ecology, and its system services are run in containers. Compared with the previous traditional container system architecture, it is possible to run the Docker engine on CentOS or Ubuntu, and run its own services on the Docker engine. In addition to these container engines, there are actually many system services. are not used, and they also bring a lot of risk and system load. Therefore, for this operating system in the current container era, it can be abstracted into a more streamlined system, that is, only the LinuxKit kernel and container engine, and system services can also be provided to the entire system by running on the container engine like Docker applications. , and all system services are also plugged, upgraded or replaced in the form of containers.
ec3bcc3c03d5d6b20e32e09905f9393010b54cee
security What are the benefits of
this approach? The first is to improve the security of the operating system. For many enterprises, they will buy the RedHat system, because the RedHat system has official support and many security features. In fact, its features are fundamentally solved problems for Linuxkit. For example, because services are all containers, you can only assemble some services you need in your own system, and these services are also in the sandbox of the container. To run, for example, there is a DHCP service. When it runs, it can specify which permissions it has. Even if there are some loopholes, it may not have permission to do those operations on the system at all, so it fundamentally solves the problem of security. question. Secondly, if this service is really vulnerable, and it can mobilize the network to affect the entire operating system, it is also very convenient to upgrade it, because it is a container, and it can also be replaced. Finally, its security is also reflected in the fact that it is jointly built by the community, all Linuxkit components are open source, and any problems will be solved in time.
540e407a98ea28606e72bd2f2383aebb7a3bc9e6
Streamlined system Another benefit
it brings is that it is more streamlined. You can look at the normal system size. Ubuntu is 1.4G, CentOS is 680M, and the size of the Linuxkit bare system is only 34M. Because it only has Kernel and a container engine, the system is very small. And the advantage of less services is that the startup is very fast, basically it can be started in 2 to 3 seconds, so that it can be used on IoT devices, and it does not depend on many system services, so it can be started quickly, and Will consume less resources. Linuxkit can also be used in cloud computing scenarios. This is mainly because of its dynamic scalability. For example, when a day is about to grab a red envelope, a lot of traffic suddenly comes. Using the Linuxkit system, you can expand an instance in 2 to 3 seconds. , you can better deal with it.
48be55d1c6afd82c46173b5f46d574703e3d9936
Portable
Linuxkit system may run in many environments, such as desktops, servers and mainframes, etc. The architecture can have Intel or ARM, and it may run on bare metal or virtual machines on cloud platforms. Because Linuxkit can use the same configuration and use Moby to build different parameters to print different images, for example, print ISO images to adapt to physical machines, and VHD to adapt to virtual machines.
8aeb9bb62fc1f97f517980c62c6ea5ca3fbe8354
Third, use Moby tools and Linuxkit to customize the container system
Then look at how the actual description file is written. It contains several components, which image is the kernel in its operating system, what is the container engine under it, and then the following describes the service containers that the system depends on. For example, DHCP is added in the example to allow It can dynamically obtain network configuration, and then use Moby's tools to build it with one click. It can also build different formats, such as ISO, RAW and VHD on cloud platforms.
bb1d654e8046afba412c22769db7093fa4fa7826
Below is an example of customizing a container OS with Moby tools and Linuxkit. On the left is an operating system description, using a Redis database service, exporting an ISO format, and building an image directly through the Moby command. The size of this image is only 41M, which is not an order of magnitude with other operating systems.
feb1dc3d71a187fc5be6d0a48bc6f2fd6ecfb7a3
and then configure its startup directly, you can see that it can be started within three seconds.

b4023c35d8f8a77540c155adad697c09210628cc

This article is the original content of Yunqi Community, and may not be reproduced without permission. If you want to reprint, please send an email to [email protected]