Set up control groups
Background Information
The core of openGauss resource load management is the resource pool. To configure the resource pool, you must first implement the settings of the control group Cgroups in the environment. For more introduction to the principles of Cgroups, please check the product manual of the relevant operating system.
The class control group is the top-level control group where the database business runs. The default sub-class control group "DefaultClass" is automatically generated during cluster deployment. The Medium control group of DefaultClass will contain system-triggered jobs running. This control group does not allow resource modification, and the jobs running on this control group are not controlled by resource management, so it is recommended to create a new subclass and its Workload control. group to set the resource ratio.
Preconditions
Familiar with the use of the "Server Tools > gs_cgroup" and "Server Tools > gs_ssh" chapters in the Tool Reference.
Steps
illustrate:
- In openGauss, it is necessary to perform the creation, update, and deletion of the control group on each cluster node in order to control the entire cluster resources. Therefore, in the following steps, use "Server Tools > gs_ssh" in "Tool Reference" command execution.
- The naming requirements for control groups are as follows:
- Neither the subclass control group nor the Workload control group are allowed to contain the character ":" in the name.
- Control groups with the same name cannot be created.
Create sub-Class control group and Workload control group
1. Log in to the openGauss master node as the operating system user omm.
2. Create sub-Class control groups named "class_a" and "class_b", and set the CPU resource quota to 40% and 20% of Class respectively.
gs_ssh -c "gs_cgroup -c -S class_a -s 40"
gs_ssh -c "gs_cgroup -c -S class_b -s 20"
3. Create Workload control groups named "workload_a1" and "workload_a2" under the sub-class control group "class_a". The CPU resource quotas are 20% and 60% of the "class_a" control group, respectively.
gs_ssh -c "gs_cgroup -c -S class_a -G workload_a1 -g 20 "
gs_ssh -c "gs_cgroup -c -S class_a -G workload_a2 -g 60 "
4. Create workload control groups named "workload_b1" and "workload_b2" under the sub-class control group "class_b". The CPU resource quotas are 50% and 40% of the "class_b" control group, respectively.
gs_ssh -c "gs_cgroup -c -S class_b -G workload_b1 -g 50 "
gs_ssh -c "gs_cgroup -c -S class_b -G workload_b2 -g 40 "
Updating resource quotas for control groups
1. Update the CPU resource quota of the "class_a" control group to 30%.
gs_ssh -c "gs_cgroup -u -S class_a -s 30"
2. Update the CPU resource quota of "workload_a1" under "class_a" to 30% of "class_a".
gs_ssh -c "gs_cgroup -u -S class_a -G workload_a1 -g 30"
Notice: The CPU resource occupied by the adjusted Workload control group "workload_a1" should not be larger than the corresponding sub-Class control group "class_a". Also, this name cannot be the default name of the Timeshare Cgroup such as "Low", "Medium", "High" or "Rush".
delete control group
gs_ssh -c "gs_cgroup -d -S class_a"
The above operation can delete the control group "class_a".
Note: The root user or a user with root access specifies "-d" and "-U username" to delete the default Cgroups accessible by the normal user "username". Ordinary users can delete existing Class Cgroups by specifying "-d" and "-S classname".
View control group information
1. View the control group information in the configuration file.
gs_cgroup -p
Control group configuration information
gs_cgroup -p
Top Group information is listed:
GID: 0 Type: Top Percent(%): 1000( 50) Name: Root Cores: 0-47
GID: 1 Type: Top Percent(%): 833( 83) Name: Gaussdb:omm Cores: 0-20
GID: 2 Type: Top Percent(%): 333( 40) Name: Backend Cores: 0-20
GID: 3 Type: Top Percent(%): 499( 60) Name: Class Cores: 0-20
Backend Group information is listed:
GID: 4 Type: BAKWD Name: DefaultBackend TopGID: 2 Percent(%): 266(80) Cores: 0-20
GID: 5 Type: BAKWD Name: Vacuum TopGID: 2 Percent(%): 66(20) Cores: 0-20
Class Group information is listed:
GID: 20 Type: CLASS Name: DefaultClass TopGID: 3 Percent(%): 166(20) MaxLevel: 1 RemPCT: 100 Cores: 0-20
GID: 21 Type: CLASS Name: class1 TopGID: 3 Percent(%): 332(40) MaxLevel: 2 RemPCT: 70 Cores: 0-20
Workload Group information is listed:
GID: 86 Type: DEFWD Name: grp1:2 ClsGID: 21 Percent(%): 99(30) WDLevel: 2 Quota(%): 30 Cores: 0-5
Timeshare Group information is listed:
GID: 724 Type: TSWD Name: Low Rate: 1
GID: 725 Type: TSWD Name: Medium Rate: 2
GID: 726 Type: TSWD Name: High Rate: 4
GID: 727 Type: TSWD Name: Rush Rate: 8
Group Exception information is listed:
GID: 20 Type: EXCEPTION Class: DefaultClass
PENALTY: QualificationTime=1800 CPUSkewPercent=30
GID: 21 Type: EXCEPTION Class: class1
PENALTY: AllCpuTime=100 QualificationTime=2400 CPUSkewPercent=90
GID: 86 Type: EXCEPTION Group: class1:grp1:2
ABORT: BlockTime=1200 ElapsedTime=2400
The configuration information of the control group viewed in the preceding example is shown in Table 1.
Table 1 Control group configuration information
2. View the control group information in the tree structure in the operating system.
Run the following command to query the tree structure information of the control group.
gs_cgroup -P
The returned information is as follows, where shares represents the value of the dynamic resource quota "cpu.shares" of the CPU resources in the operating system, and cpus represents the value of the dynamic resource quota "cpuset.cpus" of the CPUSET resources in the operating system, which means that the control group can The number of cores used.
Mount Information:
cpu:/dev/cgroup/cpu
blkio:/dev/cgroup/blkio
cpuset:/dev/cgroup/cpuset
cpuacct:/dev/cgroup/cpuacct
Group Tree Information:
- Gaussdb:wangrui (shares: 5120, cpus: 0-20, weight: 1000)
- Backend (shares: 4096, cpus: 0-20, weight: 400)
- Vacuum (shares: 2048, cpus: 0-20, weight: 200)
- DefaultBackend (shares: 8192, cpus: 0-20, weight: 800)
- Class (shares: 6144, cpus: 0-20, weight: 600)
- class1 (shares: 4096, cpus: 0-20, weight: 400)
- RemainWD:1 (shares: 1000, cpus: 0-20, weight: 100)
- RemainWD:2 (shares: 7000, cpus: 0-20, weight: 700)
- Timeshare (shares: 1024, cpus: 0-20, weight: 500)
- Rush (shares: 8192, cpus: 0-20, weight: 800)
- High (shares: 4096, cpus: 0-20, weight: 400)
- Medium (shares: 2048, cpus: 0-20, weight: 200)
- Low (shares: 1024, cpus: 0-20, weight: 100)
- grp1:2 (shares: 3000, cpus: 0-5, weight: 300)
- TopWD:1 (shares: 9000, cpus: 0-20, weight: 900)
- DefaultClass (shares: 2048, cpus: 0-20, weight: 200)
- RemainWD:1 (shares: 1000, cpus: 0-20, weight: 100)
- Timeshare (shares: 1024, cpus: 0-20, weight: 500)
- Rush (shares: 8192, cpus: 0-20, weight: 800)
- High (shares: 4096, cpus: 0-20, weight: 400)
- Medium (shares: 2048, cpus: 0-20, weight: 200)
- Low (shares: 1024, cpus: 0-20, weight: 100)
- TopWD:1 (shares: 9000, cpus: 0-20, weight: 900)
3. Obtain the configuration information of the control group through the system view.
a. Use gsql to connect to the database.
b. Obtain the configuration information of all control groups in the system.
openGauss=# SELECT * FROM gs_all_control_group_info;
Create a resource pool
Background Information
openGauss supports the division of host resources by creating resource pools. After resource load management is enabled, only using the default resource pool cannot meet the demands of the business for resource load management. New resource pools must be created as needed to reallocate system resources to meet the needs of the actual business for fine management of system resources. The characteristics of common resource pools are shown in Table 1.
Table 1 Features of common resource pools
| Resource pool classification | Features |
|---|---|
| Common resource pool (common scenario) | 1. Associate the Workload control group. And there must be no group resource pool associated with the subclass corresponding to the control group. That is, if a business resource pool association control group 'class1:wd' is created, there must be no group resource pool association 'class1'. 2. The default value of mem_percent is 0%, and there is no limit that the sum of mem_percent is less than 100%. |
After the resource load management function is enabled, the system will automatically create a default_pool. When a session or user does not specify an associated resource pool, it will be associated with the default_pool by default. default_pool is bound to the DefaultClass:Medium control group by default, and does not limit the number of concurrent services associated with it. The detailed properties of default_pool are shown in Table 2.
Table 2 default_pool properties
| Attributes | attribute value | illustrate |
|---|---|---|
| respool_name | default_pool | Resource pool name. |
| mem_percent | 100 | Maximum occupied memory percentage. |
| cpu_affinity | -1 | CPU affinity, reserved parameters. |
| control_group | DefaultClass:Medium | The control group associated with the resource pool. |
| active_statements | -1 | The maximum number of concurrency allowed by the resource pool. -1 means unlimited concurrency. |
| max_dop | 1 | After SMP is enabled, the concurrency of operator execution is reserved. |
| memory_limit | 8GB | The upper limit of memory usage, reserved parameters. |
| parents | 0 | Parent resource pool OID. |
| io_limits | 0 | The upper limit of the number of times to trigger IO per second. The row storage unit is 10,000 times/s, and the column storage unit is times/s. 0 means no control. |
| io_priority | None | When the IO utilization rate is as high as 90%, the priority level associated with heavy consumption of IO jobs for IO resource management and control. None means no control. |
| nodegroup | installation | The name of the logical cluster where the resource pool is located. |
| is_foreign | f | Resource pools are not used for users outside the logical cluster. |
Notice: openGauss does not allow modification of the default_pool parameter.
Preconditions
You are familiar with the syntax of CREATE RESOURCE POOL, ALTER RESOURCE POOL, and DROP RESOURCE POOL.
Operation process
Create a resource pool
1. Use gsql to connect to the database.
2. Create a group resource pool associated with the specified sub-Class control group. For example the following: The group resource pool named "resource_pool_a" is associated with the "class_a" control group.
openGauss=# CREATE RESOURCE POOL resource_pool_a WITH (control_group='class_a');
openGauss=# CREATE RESOURCE POOL resource_pool_b WITH (control_group='class_b');
CREATE RESOURCE POOL
3. Create a business resource pool and associate it with the specified Workload control group. For example, the following: the service resource pool named "resource_pool_a1" is associated with the "workload_a1" control group.
openGauss=# CREATE RESOURCE POOL resource_pool_a1 WITH (control_group='class_a:workload_a1');
openGauss=# CREATE RESOURCE POOL resource_pool_a2 WITH (control_group='class_a:workload_a2');
openGauss=# CREATE RESOURCE POOL resource_pool_b1 WITH (control_group='class_b:workload_b1');
openGauss=# CREATE RESOURCE POOL resource_pool_b2 WITH (control_group='class_b:workload_b2');
CREATE RESOURCE POOL
illustrate:
- If the associated control group is not specified when creating a resource pool, the resource pool will be associated with the default control group ("Medium" Timeshare control group under the DefaultClass control group).
- The value of control_group is case-sensitive and should be specified with single quotation marks.
- If the database user specifies the string represented by the Timeshare control group, that is, one of "Rush", "High", "Medium" or "Low", such as the string of control_group is "High", it means that the resource pool is assigned to the DefaultClass control group under the "High" Timeshare control group.
- control_group can specify a user to create a Workload control group, namely 'class1:wd', or it can have a control group level, for example: 'class1:wd:2', this level must be in the range of 1-10, but this Levels will not make any distinction. In older versions, it was allowed to create Workload Control Groups with the same name and differentiate them by level. However, after the new version is upgraded, it is not allowed to create a control group with the same name. If a user has created a Workload control group with the same name in the old version, the level will not be distinguished during use. This may cause the problem that the control group is not clearly used. Users can delete the old control group with the same name by themselves to clear the use of the control group.
Manage resource pools
Modify the properties of the resource pool. For example, the following: Modify the control group associated with the resource pool "resource_pool_a2" to "class_a:workload_a1" (assuming that class_a:workload_a1 is not associated with other resource pools).
openGauss=# ALTER RESOURCE POOL resource_pool_a2 WITH (control_group="class_a:workload_a1");
ALTER RESOURCE POOL
delete resource pool
Delete the resource pool. For example, delete the resource pool "resource_pool_a2" below.
openGauss=# DROP RESOURCE POOL resource_pool_a2;
DROP RESOURCE POOL
illustrate:
- If a role is already associated with the resource pool, it cannot be deleted.
- In a multi-tenant scenario, if you delete a group resource pool, all its business resource pools will be deleted. A resource pool can be deleted only when no user is associated.
View information about resource pools
Notice:
1. It is not allowed to use INSERT, UPDATE, DELETE, TRUNCATE to operate the system table pg_resource_pool of resource load management.
2. It is not allowed to modify the memory_limit and cpu_affinity attributes of the resource pool.
- View information about all resource pools in the current cluster.
openGauss=# SELECT * FROM PG_RESOURCE_POOL;
respool_name | mem_percent | cpu_affinity | control_group | active_statements | max_dop | memory_limit | parentid | io_limits | io_priority | nodegroup | is_foreign | max_worker
------------------+-------------+--------------+---------------------+-------------------+---------+--------------+----------+-----------+--------------+--------------+------------
default_pool | 100 | -1 | DefaultClass:Medium | -1 | 1 | 8GB | 0 | 0 | None | installation | f |
resource_pool_a | 20 | -1 | class_a | 10 | 1 | 8GB | 0 | 0 | None | installation | f |
resource_pool_b | 20 | -1 | class_b | 10 | 1 | 8GB | 0 | 0 | None | installation | f |
resource_pool_a1 | 20 | -1 | class_a:workload_a1 | 10 | 1 | 8GB | 16970 | 0 | None | installation | f |
resource_pool_a2 | 20 | -1 | class_a:workload_a2 | 10 | 1 | 8GB | 16970 | 0 | None | installation | f |
resource_pool_b1 | 20 | -1 | class_b:workload_b1 | 10 | 1 | 8GB | 16971 | 0 | None | installation | f |
resource_pool_b2 | 20 | -1 | class_b:workload_b2 | 10 | 1 | 8GB | 16971 | 0 | None | installation | f |
(7 rows)
- View the control group information associated with a resource pool. For details, refer to the gs_control_group_info(pool text) function in the Statistics Functions chapter.
In the following command, "resource_pool_a1" is the name of the resource pool.
openGauss=# SELECT * FROM gs_control_group_info('resource_pool_a1');
name | class | workload | type | gid | shares | limits | rate | cpucores
---------------------+---------+-------------+-------+-----+--------+--------+------+----------
class_a:workload_a1 | class_a | workload_a1 | DEFWD | 87 | 30 | 0 | 0 | 0-3
(1 row)
Table 3 gs_control_group_info attributes
| Attributes | attribute value | illustrate |
|---|---|---|
| name | class_a:workload_a1 | class and workload name |
| class | class_a | Class control group name |
| workload | workload_a1 | Workload Control Group Name |
| type | DEFWD | Control group type (Top, CLASS, BAKWD, DEFWD, TSWD) |
| gid | 87 | control group id |
| shares | 30 | The percentage of CPU resources of the parent node |
| limits | 0 | Percentage of the parent node's CPU cores |
| rate | 0 | Distribution ratio in Timeshare |
| cpucores | 0-3 | Number of CPU cores |