Original Address: https://segmentfault.com/a/1190000008323952
In cgroup inside, with CPU-related subsystems cpusets, cpuacct and cpu.
Cpuset affinity which is mainly used to set the CPU, you can limit the process cgroup in the only run on a specific CPU, or can not run on the specified CPU, while cpuset can set memory affinity. Set affinity generally only need it only in special circumstances, it will not be discussed.
cpuacct includes a CPU cgroup statistics used by the current, less information, are interested can go and see its documentation, will not be discussed.
Benpian only describes cpu subsystems, including how cgroup limits CPU usage limit and relative value of other cgroup.
本篇所有例子都在ubuntu-server-x86_64 16.04下执行通过
Create a sub-cgroup
in ubuntu, systemd has helped us to mount a good cpu subsystem, we just need to create a subdirectory under the corresponding directory on it
#从这里的输出可以看到，cpuset被挂载在了/sys/fs/cgroup/cpuset，
#而cpu和cpuacct一起挂载到了/sys/fs/cgroup/cpu,cpuacct下面
dev@ubuntu:~$ mount|grep cpu
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
#进入/sys/fs/cgroup/cpu,cpuacct并创建子cgroup
dev@ubuntu:~$ cd /sys/fs/cgroup/cpu,cpuacct
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct$ sudo mkdir test
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct$ cd test
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ ls
cgroup.clone_children cpuacct.stat cpuacct.usage_percpu cpu.cfs_quota_us cpu.stat tasks
cgroup.procs cpuacct.usage cpu.cfs_period_us cpu.shares notify_on_release
In addition cgroup inside common cgroup.clone_children, tasks, cgroup.procs, notify_on_release these documents to cpuacct. Cpuacct files beginning with the sub-system, we only need to focus on cpu. The beginning of the file.
& cpu.cfs_quota_us cpu.cfs_period_us
cfs_period_us to set the time period length, cfs_quota_us cgroup to configure the number of CPU time in the cycle length can be used in the current set, together with the two documents set using the upper limit of the CPU. Unit two files are microseconds (us), cfs_period_us the range of 1 millisecond (ms) to 1 second (s), cfs_quota_us to values greater than 1ms, if cfs_quota_us is -1 (the default ), represents unlimited cpu time. Here are a few examples:
1.限制只能使用1个CPU（每250ms能使用250ms的CPU时间）
# echo 250000 > cpu.cfs_quota_us /* quota = 250ms */
# echo 250000 > cpu.cfs_period_us /* period = 250ms */
2.限制使用2个CPU（内核）（每500ms能使用1000ms的CPU时间，即使用两个内核）
# echo 1000000 > cpu.cfs_quota_us /* quota = 1000ms */
# echo 500000 > cpu.cfs_period_us /* period = 500ms */
3.限制使用1个CPU的20%（每50ms能使用10ms的CPU时间，即使用一个CPU核心的20%）
# echo 10000 > cpu.cfs_quota_us /* quota = 10ms */
# echo 50000 > cpu.cfs_period_us /* period = 50ms */
cpu.shares
shares used to set the relative value of the CPU, and is for all a CPU (core), the default value is 1024, if there are two systems cgroup, are A and B, shares value is 1024 A, B of value of shares is 512, then the obtained a 1024 / (1204 + 512) = 66% of CPU resources, and B will receive 33% of the CPU resources. shares has two characteristics:
如果A不忙，没有使用到66%的CPU时间，那么剩余的CPU时间将会被系统分配给B，即B的CPU使用率可以超过33%
如果添加了一个新的cgroup C，且它的shares值是1024，那么A的限额变成了1024/(1204+512+1024)=40%，B的变成了20%
It can be seen from the above two characteristics:
在闲的时候，shares基本上不起作用，只有在CPU忙的时候起作用，这是一个优点。
由于shares是一个绝对值，需要和其它cgroup的值进行比较才能得到自己的相对限额，而在一个部署很多容器的机器上，
cgroup的数量是变化的，所以这个限额也是变化的，自己设置了一个高的值，但别人可能设置了一个更高的值，
所以这个功能没法精确的控制CPU使用率。
cpu.stat
contains three statistics below
nr_periods: represents the number of elapsed time period cpu.cfs_period_us inside configuration
nr_throttled: In these periods the above, how many times is limited (ie cgroup in the process used up its quota in the specified time period)
throttled_time: cgroup in the process is limited to the use of CPU it lasted how long (ns)
Examples
here to cfs_period_us & cfs_quota_us, for example, show you how to control CPU usage.
#继续使用上面创建的子cgroup： test
#设置只能使用1个cpu的20%的时间
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ sudo sh -c "echo 50000 > cpu.cfs_period_us"
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ sudo sh -c "echo 10000 > cpu.cfs_quota_us"
#将当前bash加入到该cgroup
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ echo $$
5456
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ sudo sh -c "echo 5456 > cgroup.procs"
#在bash中启动一个死循环来消耗cpu，正常情况下应该使用100%的cpu（即消耗一个内核）
dev@ubuntu:/sys/fs/cgroup/cpu,cpuacct/test$ while :; do echo test > /dev/null; done
#--------------------------重新打开一个shell窗口----------------------
#通过top命令可以看到5456的CPU使用率为20%左右，说明被限制住了
#不过这时系统的%us+%sy在10%左右，那是因为我测试的机器上cpu是双核的，
#所以系统整体的cpu使用率为10%左右
dev@ubuntu:~$ top
Tasks: 139 total, 2 running, 137 sleeping, 0 stopped, 0 zombie
%Cpu(s): 5.6 us, 6.2 sy, 0.0 ni, 88.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 499984 total, 15472 free, 81488 used, 403024 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 383332 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5456 dev 20 0 22640 5472 3524 R 20.3 1.1 0:04.62 bash
#这时可以看到被限制的统计结果
dev@ubuntu:~$ cat /sys/fs/cgroup/cpu,cpuacct/test/cpu.stat
nr_periods 1436
nr_throttled 1304
throttled_time 51542291833
Conclusion
use cgroup limit CPU usage more tangled, with cfs_period_us & cfs_quota_us it, limiting the dead, and can not make full use of idle CPU, with shares it, they can not configure a percentage, is extremely difficult to control. In short, the use of cpu cgroup subsystems need to be cautious.
