Hello Docker (6)-Docker resource quota
1. Docker resource quota
1. Introduction to Docker resource quota
Docker does not limit the hardware resources of the container by default. When the container load is too high, it will occupy the host resources as much as possible. Based on the cgroups function provided by the Linux kernel, Docker can limit the resources used by the container at runtime, such as memory, CPU, IO, and network.
On the Linux system, if the Linux kernel detects that the current host has no available memory, it will throw an OOME (Out Of Memory Exception), and it will turn on killing to kill some processes.
Once OOME occurs, any process may be killed, including the Docker daemon. Therefore, Docker specially adjusted the OOM_Odj priority of the Docker daemon to avoid being killed, but the priority of the container has not been adjusted.
After complex calculations within the Linux system, each system process will have an OOM_Score score. The higher the OOM_Odj, the higher the score, and the process with the highest score will be killed first.
Docker can adjust OOM_Odj when starting a container, or you can specify that important containers are prohibited from being killed by OMM, and use -oom-kill-disable=true to specify when starting the container.
2. Introduction to cgroup
Control Groups is a mechanism provided by the Linux kernel that can limit, record, and isolate physical resources (such as CPU, Memory, disk IO, etc.) used by process groups. It is used by projects such as LXC and Docker to control process resources. A cgroup is a basic structure that provides functions and interfaces for grouping processes for management, and is used to implement specific resource management functions such as IO or memory allocation control. The resource management function implemented by cgroup is called cgroup subsystem, as follows:
(1) blkio, set to limit the input and output control of each block device, such as disks, CDs, USB, etc.
(2) cpu, use the scheduler to provide CPU access for cgroup tasks.
(3) cpuacct, which generates CPU resource reports for cgroup tasks.
(4) cpuset, for multi-core CPUs, separate CPU and memory will be allocated for cgroup tasks.
(5) devices, allow or deny access to devices by cgroup tasks.
(6) Freezer, pause and resume cgroup tasks.
(7) memory, set the memory limit of each cgroup and generate memory resource reports.
(8) net_cls, mark each network packet for convenient use by cgroup.
(9) ns, the namespace subsystem.
(10) perf_event, increase the ability to monitor and track each group, that is, it can monitor all threads belonging to a specific group and threads running on a specific CPU.
Two, Docker memory quota
1. Docker memory limit
The memory limit functions provided by Docker are as follows:
(1) The memory and swap partition size that the container can use.
(2) The core memory size of the container.
(3) Exchange behavior of container virtual memory.
(4) The soft limit of container memory.
(5) Whether to kill containers that take up too much memory.
(6) The priority of the container being killed.
2. Docker memory limit parameters
(1) memory
-m, --memory, memory limit, format is number plus unit, unit can be b, k, m, g, minimum is 4M.
--memory-swap, the total limit of memory + swap partition size, must be larger than the memory setting.
If you do not set memory and memory-swap, the Docker container can use up all the memory and swap partitions of the host by default. After a Docker container has used up all the memory and swap partition of the host for a period of time, it will be killed by the host system.
If memory is set to a value n not less than 4M, memory-swap is not set or memory-swap is set to 0, the memory size that can be used by the Docker container is n, and the size of the swap partition that can be used is also n. Docker default container swap The size of the partition is the same as the memory; if you run a program that keeps applying for memory in a Docker container, the final memory size that the Docker container can occupy is 2n; if memory is set to a and memory-swap is set to b, the Docker container can The memory size used is a, and the swap partition size that the Docker container can use is ba; if memory is set to a and memory-swap is set to -1, the memory size that the Docker container can use is a, and there is no limit to the swap partition size used by the Docker container. .
--memory-reservation, the soft limit of memory.
Memory reservation is a soft mechanism that does not guarantee that the memory used by the container will not exceed the memory-reservation limit at any time, but only ensures that the container will not occupy the memory size exceeding the memory-reservation limit for a long time. Although the Docker container can use up to the memory size of the memory, when the host's memory resources are tight, the system will reclaim part of the container's memory pages during the next memory recovery of the system, forcing the container's memory usage to return to the memory-reservation size .
By default, the memory-reservation value is not set, and its value is the same as the memory limit value. If memory-reservation is set to 0, it is equivalent to not setting.
(2) OOM killer
--oom-kill-disable, whether to prevent OOM killer from killing the container, it is not set by default.
By default, when an out-of-memory (OOM) error occurs, the OOM killer process will kill the process in the Docker container to release memory. By setting the --oom-kill-disable option to prohibit OOM killer from killing processes in the container. Make sure to use the oom-kill-disable option to disable OOM killer only when using the memory option. If the memory option is not set, but OOM-killer is disabled, it may cause more memory by killing the host process when an OOM error occurs.
--oom-score-adj, the priority of the container being killed by OOM killer, the range is [-1000, 1000], the default is 0, the larger the value, the easier it is to be killed.
--memory-swappiness, used to set the virtual memory control behavior of the container. The value is an integer between 0 and 100.
(3) Kernel memory
--kernel-memory, the core memory limit, the minimum is 4M.
The difference between core memory and user memory is that core memory cannot be swapped out, so Docker containers can block some system services by consuming too much memory. Core memory includes: stack pages, slab pages, socket memory pressure, tcp memory pressure.
The core memory can be constrained by setting the core memory limit. For example, each process consumes some stack pages. By limiting the core memory, new processes can be prevented from being created when the core memory is used too much.
Core memory and user memory are not independent, and must be limited in the context of user memory limitation.
Assume that the limit value of user memory is U and the limit value of core memory is K. There are three possible ways to limit the core memory:
U != 0, does not limit the core memory, is the default standard setting
K <U, the core memory is a subset of the user memory, the total memory of each cgroup is overused, K can be set, and the total number of groups will not exceed the total memory.
K> U, changes in core memory will also cause changes in user counters, and both container core memory and user memory will trigger recycling.
docker run -it -m 500M --kernel-memory 50M ubuntu:16.04 /bin/bash
Start the container, the process of the Docker container can use up to 500M memory and 50M core memory.
docker run -it --kernel-memory 50M ubuntu:16.04 /bin/bash It is
useless to set the user memory limit. The process in the Docker container can use as much memory as possible, but can use up to 50M core memory.
(4) Swappiness
By default, the kernel of the Docker container can swap out a certain percentage of anonymous pages, which can be set using the memory-swappiness parameter. --memory-swappiness can be set from 0 to 100. 0 means to turn off anonymous page swap. 100 means that all anonymous pages can be exchanged. By default, if you set --memory-swappiness, it will be inherited from the parent process.
docker run -it --memory-swappiness=0 ubuntu:16.04 /bin/bash
memory-swappiness is set to 0 to maintain the working set of the container and avoid the performance loss of the exchange agent.
Three, Docker CPU quota
1. Docker CPU limit
By default, all Docker containers can use host CPU resources equally and there is no limit.
The CPU resource limit option provided by Docker can limit which CPUs the container can use on a multi-core system. There are two ways to limit the maximum CPU time that Docker containers can use: one is to set the relative proportion of the CPU time that each container can use when there are multiple CPU-intensive containers competing for the CPU; the other is to set the container in an absolute way in each scheduling The maximum CPU time that can be used in a cycle.
Docker can use -c or --cpu-shares to set the weight of the container's CPU usage. If not specified, the default value is 1024.
2. CPU limit parameters
The options related to docker run and CPU limit are as follows:
--cpuset-cpus="" Allowed CPU set, the value can be 0-3,0,1
-c,--cpu-shares=0 CPU weight--cpu
-period =0 Limit CPU CFS cycle, range from 100ms~1s, ie [1000, 1000000]
--cpu-quota=0 Limit CPU CFS quota, which must be no less than 1ms, ie greater than or equal to 1000
--cpuset-mems="" Allowed in The memory nodes (MEMs) executed on the above are only valid for NUMA systems
--cpus=value specifies the amount of CPU resources that the container can use. If the host has two CPUs and set --cpus="1.5", the container can use up to 1.5 CPUs,
--cap-add=sys_nice enables the CAP_SYS_NICE function of the Docker container, allows the container to trigger a good process value, and set real-time scheduling policies. Set CPU affinity and other operations.
--cpu-rt-runtime=value The maximum number of microseconds that the container can run with real-time priority during the real-time scheduler cycle of the Docker daemon. The CAP_SYS_NICE function needs to be enabled.
--ulimit rtprio=value The maximum real-time priority allowed by the container. The CAP_SYS_NICE function needs to be enabled. docker run --it --cpu-rt-runtime=950000 --ulimit rtprio=99 --cap-add=sys_nice ubuntu
To run the container, the real-time scheduler needs to run the docker daemon and set --cpu-rt-runtime to the maximum number of microseconds reserved for real-time tasks in each run time period. The default period is 1000000 microseconds (1 second), setting --cpu-rt-runtime=950000 can ensure that the container using the real-time scheduler can run 950,000 microseconds every 1000000 microseconds, and reserve at least 50,000 microseconds for non-real-time tasks .
Among them --cpuset-cpus is used to set the CPU core that the container can use.
-c,--cpu-shares is used to set the CPU weight that each container can allocate when multiple containers compete for CPU.
--cpu-period and --cpu-quata are used to absolutely set the CPU time that the container can use. docker run -it --cpuset-cpus="1,3" ubuntu:14.04 /bin/bash
The processes in the container can be executed on cpu 1 and cpu 3. docker run -it --cpuset-cpus="0-2" ubuntu:14.04 /bin/bash
Processes in the container can be executed on cpu 0, cpu 1, and cpu 2. docker run -it --cpuset-mems="1,3" ubuntu:14.04 /bin/bash
The process in the container can only use the memory on memory nodes 1 and 3 on a
multi-core system, and the CPU time weight is calculated on all CPU cores. Even if the CPU time limit of a certain container is less than 100%, 100% of the time of each CPU core can be used.
Four, Docker disk IO quota
Docker's control of disk IO needs to rely on the host device. The main parameters are as follows:
--device-read-bps limits the read speed on the device (bytes per second), the unit can be kb, mb or gb.
--device-read-iops Limit the read speed of the specified device by reading IO times per second.
--device-write-bps limit the write speed of the device (bytes per second), the unit can be kb, mb or gb.
--device-write-iops Limit the write speed of the specified device by writing IO times per second.
--blkio-weight The default disk IO weight value of the container, and the valid value range is 10-100.
--blkio-weight-device IO weight control for specific devices, the format is DEVICE_NAME:WEIGHTdocker run -tid –name disk1 –device-write-bps /dev/sda:1mb ubuntu:stress
Five, Docker container space quota
When docker uses devicemapper as a storage driver, the default maximum size of each container and image is 10G. You can use dm.basesize to specify when the docker daemon starts, but it will cause all local images and containers on the Docker Host to be cleaned up. There are no restrictions when using other storage drivers such as aufs or overlay.