Docker Cgroups resource control management

Cgroups is a mechanism provided by the Linux kernel that can limit the resources used by a single process or multiple processes. It can achieve fine-grained control over resources such as cpu and memory. Currently, the increasingly popular lightweight container Docker uses cgroups The resource limitation capability is provided to complete the resource control of cpu, memory and other parts.

In addition, developers can also use the fine-grained control capabilities provided by cgroups to limit the resource usage of a certain process or a certain group of processes. For example, on an eight-core server that deploys both front-end web services and back-end computing modules, you can use cgroups to restrict the web server to only use six of the cores, leaving the remaining two cores for back-end computing module.

The four major functions of cgroups:

Resource limit: You can limit the total amount of resources used by the task.
Priority allocation: through the number of allocated cpu time slices and the size of disk IO bandwidth, it is actually equivalent to controlling the task running priority.
Resource statistics: You can count the resource usage of the system, such as cpu duration, memory usage, etc.
Task control: cgroup can perform operations such as suspending and resuming tasks.

2. The concept of cpu time slice

The time slice is the time allocated by the CPU to each program. Each thread is allocated a time period, called its time slice, which is the time the process is allowed to run, so that the various programs appear to be running at the same time. If the process is still running when the time slice ends, the CPU will be taken away and assigned to another process. If the process blocks or ends before the time slice ends, the CPU switches immediately. Without causing a waste of CPU resources.

Macroscopically: We can open multiple applications at the same time, and each program runs in parallel and runs at the same time. But at the micro level: Since there is only one CPU, it can only process a part of the program's requirements at a time. How to deal with fairness, one way is to introduce time slices, and each program will execute in turn.

3. Restrictions on CPU usage

1. Set the upper limit of CPU usage

Linux uses CFS (Completely Fair Scheduler, Completely Fair Scheduler) to schedule the use of the CPU by each process. The default scheduling period of CFS is 100ms.

We can set the scheduling cycle of each container process and how much CPU time each container can use at most during this cycle.

Note:

Use --cpu-period to set the scheduling period, and use --cpu-quota to set the CPU time that the container can use in each period. The two can be used together.

The effective range of the CFS period is 1ms ~ 1s, and the corresponding value range of --cpu-period is 1000 ~ 1000000 (in microseconds).

The CPU quota of the container must not be less than 1ms, that is, the value of --cpu-quota must be >= 1000.

View the default CPU usage limit for containers

Do a stress test

Set CPU usage time limit when creating container

CPU limit for existing containers

Just modify the /sys/fs/cgroup/cpu/docker/container id/cpu.cfs_quota_us file directly

2. Set the CPU resource usage ratio

Create two containers and set the CPU resource usage ratio

Note: Docker specifies the CPU share through --cpu-shares, the default value is 1024, and the value is a multiple of 1024

Enter two containers separately for pressure test

View the running status of the container and observe the CPU usage ratio

4. Restrictions on memory usage

1. Limit the maximum memory that the container can use

docker run -itd --name tan3 -m 512m centos:7 /bin/bash
#-m(--memory=)选项用于限制容器可以使用的最大内存

2. Limit the swap size available to the container

docker run -itd --name tan3 -m 512m --memory-swap=1g centos:7 bash
#限制可用的swap 大小，--memory-swap

注：--memory-swap是必须要与 --memory（或-m）一起使用的

正常情况下， --memory-swap 的值包含容器可用内存和可用swap
所以 -m 512m --memory-swap=1g 的含义为：容器可以使用512M 的物理内存，并且可以使用512M (1G - 512M)的swap

设置为0或者不设置，则容器可以使用的 swap 大小为 -m 值的两倍。
如果 --memory-swap 的值和 -m 值相同，则容器不能使用swap。
如果 --memory-swap 值为 -1，它表示容器程序使用的内存受限，而可以使用的swap空间使用不受限制（宿主机有多少swap 容器就可以使用多少）

5. Restrictions on disk IO configuration control (blkio)

--device-read-bps: Limit the read speed bps (data volume) on a device, the unit can be kb, mb (M) or gb.

--device-write-bps : Limit the write speed bps (data volume) on a certain device, the unit can be kb, mb (M) or gb.

--device-read-iops : Limit the iops (number of times) to read a device

--device-write-iops : limit the iops (number of times) written to a certain device

1. Create a container without limiting the writing speed

docker run -it --name tt1 centos:7 /bin/bash
#创建容器tt1，不限制写入速度

dd if=/dev/zero of=/opt/test.out bs=10M count=5 oflag=direct
#通过dd来验证写速度，拷贝50M的数据

2. Create a container and limit the write speed

docker run -it --name tt2 --device-write-bps /dev/sda:1mb centos:7 bash
#创建容器，并限制写入速度为1MB/s

6. Clear the disk space occupied by docker

docker system prune -a
#用于清理磁盘，删除关闭的容器、无用的数据卷和网络

7. Summary

1. Limiting parameters for cpu

docker run -cpu-period     #设置调度周期时间1000~1000000
           -cpu-quota      #设置容器进程的CPU占用时间，要与调度周期时间成比例
           --cpu-shares    #设置多个容器之间的CPU资源占用比
           --cpuset-cpus   #绑核（第一个CPU编号从0开始）

2. Limitations on memory

-m 物理内存 [--memory-swap=总值]

3. Limitations on disk IO

--device-read-bps 设备文件:1mb/1M     #限制读速度
--device-write-bps 设备文件:1mb/1M    #限制写速度
--device-read-iops                   #限制读次数
--device-write-iops                  #限制写次数
 
docker system prune -a    #清理磁盘，删除关闭的容器、无用的数据卷和网络。