PS: Welcome everyone to pay attention to my public account: aCloudDeveloper, focus on technology sharing, and strive to build a dry goods sharing platform. The QR code can be scanned at the end of the article. Thank you.
I recommend everyone to read on the official account, where the reading experience is better, and a lot of dry goods have been accumulated.
In the previous two articles, we summarized the resource isolation technology Linux namespace used behind Docker.
The Linux namespace of Docker's basic technology explains in detail
the source code analysis of the Linux namespace of Docker's basic technology
This article will discuss another technique - resource quotas, which are implemented by Linux cgroups.
Cgroups is a mechanism provided by the Linux kernel. This mechanism can integrate (or separate) a series of tasks and subtasks into different groups based on resources according to requirements, thereby providing a unified framework for system resource management. (from "Docker Containers and Container Cloud")
In layman's terms, cgroups can limit and record the physical resources (including CPU, memory, IO, etc.) used by task groups (process groups or thread groups).
In order to facilitate the operation of users (programmers), cgroups are implemented in the form of a pseudo file system and provide external APIs. User operations on the file system are operations on cgroups.
In terms of implementation, cgroups actually hooks each execution task. When the allocation and use of resources are involved in the task execution process, the function on the hook will be triggered to detect the corresponding resources, so that the resources can be detected. Limits and priority assignments.
The role of cgroups
To sum up, cgroups provide the following four functions:
Resource limit : cgroups can limit the total amount of resources used by tasks, such as setting the upper limit of the memory used when the application is running, and once the quota is exceeded, an OOM (Out of Memory) prompt will be issued.
Priority assignment : By assigning the number of CPU time slices and the size of disk IO bandwidth, it is actually equivalent to controlling the priority of task running.
Resource statistics : cgroups can count the resource usage of the system, such as CPU usage time, memory usage, etc. This function is very suitable for billing.
Task control : cgroups can perform operations such as suspending and resuming tasks.
Subsystems for cgroups
Cgroups are divided into different subsystems according to different resource categories during design. A subsystem is essentially a resource controller. For example, CPU resources correspond to the CPU subsystem, which is responsible for controlling the allocation of CPU time slices, and memory corresponds to the memory subsystem, responsible for Limit memory usage. Further, a subsystem or multiple subsystems can form a cgroup, and resource control in cgroups is implemented in units of cgroups. A task (or process or thread) can join a cgroup, or move from one cgroup to another A cgroup, but there are some limitations here, so I won't go into details here. Check the relevant information for details.
For us, the most important thing is to know how to use it. Let's see how Docker is used for CPU, memory and IO resources.
For CPU, Docker uses the parameter -c or --cpu-shares to set the CPU weight used by a container. The size of the weight also affects the priority of CPU usage.
As follows, start two containers and assign different CPU weights to the final CPU usage:
docker run --name "container_A" -c 1024 ubuntu
docker run --name "container_B" -c 512 ubuntu
When there is only one container, even if less CPU weight is specified, it will occupy the entire CPU, indicating that this weight is only a relative weight. Stop the above "container_A" as follows, and "container_B" will be allocated to all available CPUs.
For memory, Docker uses -m (set memory limit) and --memory-swap (set memory and swap limit) to control the amount of memory used by the container, as follows, limit 200M of memory and 100M of swap to the container, and then give A worker thread in the container allocates 280M of memory, because 280M is within the allowable 300M range, there is no problem. The memory allocation process is continuously allocated and released, as follows:
If the worker thread uses more than 300M of memory, a memory overrun error occurs and the container exits, as follows:
For IO resources, it is used in the same way as the CPU. Use --blkio-weight to set its usage weight. The two indicators measured by IO are bps (byte per second, the amount of data read and written per second) and iops (io per second). , the number of IO per second), the actual use, generally use these two indicators to measure the bandwidth of IO read and write, several parameters are as follows:
--device-read-bps, limit reading bps of a device.
--device-write-bps, limit the bps of writing to a device.
--device-read-iops, restrict reading iops of a device.
--device-write-iops, limit write iops to a device.
If the bps write rate of the container to its file system /dev/sda is limited to 30MB/s, then use dd to test the disk write rate in the container as follows, it can be seen that it is less than 30MB/s.
Under normal circumstances, my machine can reach 56.7MB/s, which is generally over 1G.
The above examples of resource usage restrictions are essentially implemented by calling the cgroups mechanism of the Linux kernel. After each container is created, Linux will create a cgroup directory for each container, named after the container ID, and the directory is in / In sys/fs/cgroup/, for the above example of CPU resource limitation, we can see relevant information in /sys/fs/cgroup/cpu/docker, as follows:
Among them, the value saved in cpu.shares is the limit value, and there are many other items. If you are interested, you can try it out.
Summarize
The role of cgroups, the implementation of cgroups, the subsystem mechanism of cgroups, the usage of CPU, memory and IO, and the corresponding Linux cgroups file directory.
PS: Friends who are interested in cloud computing can follow my WeChat public account: aCloudDeveloper, focus on the field of cloud computing, and insist on sharing dry goods.
