The development of Docker and container technology is changing with each passing day. This article attempts to sort out the current Docker ecosystem from a global perspective. Since it is an overview, it will not involve specific technical details.
Docker has changed a lot since its release, and in some ways it has changed a lot. For tech lovers, we love cool new features and improvements to old ones. But for users in production environments, they don't really like such frequent changes! In any case, it is necessary for us to sort out the many concepts in the docker ecosystem and the relationship between them, as well as the milestone events of docker since its birth (2018).
Container technology in full bloom
Although docker pushed container technology to its peak, container technology was not born from docker. In fact, container technology is not even a new technology, because its birth and use are indeed a few years old. You may not have heard of the following names, but they are all applications of container technology:
- Chroot Jail
- FreeBSD Jails
- Linux VServer
- Solaris Containers
- OpenVZ
- Process Containers
- LXC
- Warden
- LMCTFY
- Docker
- RKT
Chroot Jail
is our common chroot command usage. It appeared in 1979 and is considered one of the earliest containerization technologies. It can isolate the file system of a process.
The FreeBSD Jail
Freebsd Jail implements OS-level virtualization and is one of the pioneers of OS-level virtualization technology.
A Linux VServer
is a dedicated virtual server implemented using system-level virtualization capabilities added to the Linux kernel.
Solaris Containers
is also an OS-level virtualization technology designed for X86 and SPARC systems. Solaris containers are a combination of system resource control and boundary isolation provided through "zones".
OpenVZ
OpenVZ is an OS-level virtualization technology in Linux. It allows the creation of multiple securely isolated Linux containers known as VPS.
Process Containers
Process containers are developed by engineers at Google and are commonly referred to as cgroups.
LXC
LXC, also known as Linux containers, is also an operating system-level virtualization technology that allows multiple independent systems to run on the host using a single Linux kernel.
Warden
In its initial stages, Warden used LXC as the container runtime. It has now been replaced by CloudFoundy.
LMCTFY
LMCTY is the acronym for Let me contain that for you. It is an open source version of Google's container stack.
Google's engineers have been working with docker's libertainer team to abstract and port libertainer's core concepts into this project. The progress of the project is unknown and is expected to be replaced by libcontainer.
Docker
Docker is a tool for packaging applications and their dependencies into containers that can run on almost any server.
RKT
RKT is short for Rocket, an application container engine focused on security and open standards.
As we've seen, docker wasn't the first containerization technology, but it's certainly one of the most well-known. Docker was born in 2013 and has developed rapidly. The following figure shows the components in the current docker platform (this figure is from the Internet):
Docker sits on top of the system infrastructure and powers the application. It consists of an industry-standard container runtime component called containerd, a local orchestration tool called docker swarm, and an open source docker community version and docker enterprise version that provides commercial management services.
Important concepts related to docker
Docker & LXC
Docker's first execution environment was LXC, but since version 0.9 LXC was replaced by libcontainer.
Docker & libcontainer
Libcontainer encapsulates the basic functions provided by Linux for docker, such as cgroups, namespaces, netlink and netfilter, etc., as shown in the following figure (this figure is from the Internet):
2015 - Docker &
runC In 2015, docker released runC, a lightweight cross-platform container runtime. This is basically a command line gadget that can run containers directly with libcontainer without going through the docker engine. The goal of runC is to make standard containers available everywhere.
Docker & The Open Containers Initiative (OCI)
OCI is a lightweight open management architecture established in 2015 by docker, CoreOS and other leaders in the container industry. It maintains some projects, like runC, as well as container runtime specs and image specs. The purpose of OCI is to set standards around the container industry, such that containers created with docker can run on any other container engine.
2016 - Docker & containerd
In 2016, Docker spun off containerd and donated it to the community. Breaking this component out into a separate project allows docker to move the container management functionality out of docker's core engine and into a separate daemon process (ie, containerd).
After Docker Components
splits containerd, the relationship between Docker components is shown in the following figure (this figure comes from the Internet):
So far, docker has evolved from a single piece of software to a set of independent components and projects.
How does Docker run a container?
- Docker engine creates container images
- Pass the container image to containerd
- containerd 调用 containerd-shim
- containerd-shim uses runC to run containers
- containerd-shim allows the runtime (runC in this case) to exit after starting the container
The biggest benefit of this model is that the docker engine will not be interrupted when the docker engine is upgraded.
2017 - Containers Go Mainstream
2017 was the year that containers went mainstream, which is why docker supports so many platforms outside of Linux (Docker for Mac, Docker for Windows, Docker for AWS, GCP, etc.).
When container technology became popular, the Docker company realized the need for a new production model, which is why it started the Moby project.
Moby Project
The Moby Project opens a new chapter in enabling collaboration and production. It is an open source project that aims to advance the containerization of software. Project Moby provides dozens of Lego-like components and a framework for assembling them into custom container-based systems.
The Docker production model starts out like any other common single open source project (this diagram is from the internet):
And then split a single project into different open components (this picture is from the Internet):
Then evolve to a model that can share these components and the assembly of components (this diagram is from the Internet):
Eventually a model that provides more collaboration on components and a collection of generic components (this diagram is from the internet):
Now let's introduce some of the components in the Moby project.
Containerd
Containerd is the core container runtime created by docker based on industry standards. It can be used as a daemon for Linux and Windows and manages the entire container lifecycle.
Linuxkit
Linuxkit, another component in the Moby project, is a tool for building secure, cross-platform, lean systems for containers. Currently supported native hypervisors are hyper-v and vmware. The supported cloud platforms are AWS, Azure, etc.
Infrakit
Infrakit is also part of the Moby project. It is a toolkit for creating and managing declarative, immutable and self-healing infrastructure.
Infrakit is designed to automate the setup and management of infrastructure to support distributed systems and higher-level container orchestration systems. Infrakit is useful for use cases like orchestration tools like Docker Swarm and Kubernetes or for creating auto-scaling clusters across public clouds like AWS.
Libnetwork
Libnetwork is a container network management project implemented in Go language. Its goal is to define a Container Networking Model (CNM)
and provide a consistent programming interface and network abstraction for applications. This satisfies the "composable" requirement of container networking.
Docker & Docker Swarm
Docker Swarm is an orchestration tool built in the docker engine. It is natively included in docker engine as a standalone tool since docker 1.12. We can use docker cli to create clusters, deploy and manage applications and services through docker swarm. The following figure describes the role of docker swarm in the docker system (this figure comes from the Internet):
Docker & Kubernetes
In the competition between docker swarm and kubernetes, it is clear that kubernetes has the advantage. So docker made an emergency U-turn and started native support for integration with kubernetes. This is the big news in the container world in 2017! At this point, docker users and developers can freely choose to use kubernetes or swarm to perform container orchestration. We can think that docker and kubernetes are married (this picture is from the Internet):
The new docker version that supports kubernetes integration will allow users to deploy their docker compose applications as kubernetes local pods and services. Kubernetes is a very powerful and gradually recognized local orchestration tool (this picture comes from the Internet):
Hope you are not confused by the many names and concepts in this article, let's end this article with the following graph, which shows the trend of the number of image pulls from docker hub from 2013 to 2017:
参考：
An Overall View On Docker Ecosystem — Containers, Moby, Swarm, Linuxkit, containerd, Kubernetes