background

For instance for a database, CPU and memory resources are limited, and those resources can greatly affect the database query performance. Therefore, when the database load is high to a certain extent, may compete for CPU and memory resources among each individual queries, resulting in lower overall query performance. This is for those delay-sensitive business, the situation is intolerable. In AnalyticDB for PostgreSQL (referred to as ADB PG), it provides a system resource load management tools --resource queue. Users can according to its own business, the number of concurrent queries the specified database can run, you can use the memory size of each query, and CPU resources that can be used. This ensures that queries have expected system resources, resulting query performance in line with expectations.

Use resource queue management and load isolation

This section describes how to create and use the resource queue in ADB PG.

Create a resource queue

Syntax ADB for PG in the resource queue created as follows

CREATE RESOURCE QUEUE name WITH (queue_attribute=value [, ... ])

    ACTIVE_STATEMENTS=integer
        [ MAX_COST=float [COST_OVERCOMMIT={TRUE|FALSE}] ]
        [ MIN_COST=float ]
        [ PRIORITY={MIN|LOW|MEDIUM|HIGH|MAX} ]
        [ MEMORY_LIMIT='memory_units' ]

 | MAX_COST=float [ COST_OVERCOMMIT={TRUE|FALSE} ]
        [ ACTIVE_STATEMENTS=integer ]
        [ MIN_COST=float ]
        [ PRIORITY={MIN|LOW|MEDIUM|HIGH|MAX} ]
        [ MEMORY_LIMIT='memory_units' ]

Which, ACTIVE_STATEMENTS specified resource queue at some point in time most active queries (queries being performed) number. MEMORY_LIMIT set resource queue size of all the memory that can be used up to a query on a single computing node (segment), the unit may be KB, MB or GB, default value is -1, no limit. MAX_COST set maximum resource queue query cost, where the cost of the query optimizer refers to the ADB for PG estimated cost of a query, the default value is -1, no limit. COST_OVERCOMMIT parameters associated with MAX_COST parameters, if the MAX_COST parameter is set, then when COST_OVERCOMMIT true, indicates that the query query cost greater than MAX_COST can be executed in the system is idle, and if it is false, it means that query cost greater than MAX_COST of those queries It will be rejected. MIN_COST set a minimum query cost resource queue, when the query cost less than MIN_COST query, but will not wait in line to be executed immediately. PRIORITY Set priority, high priority resource queue inquiry will be allocating more CPU resources to perform, if you do not specify a priority, the default is MEDIUM.

When create resource queue, ACTIVE_STATEMENTS MAX_COST and a parameter which must be specified.

You can specify the number of queries the resource queue where users can execute concurrently when creating a resource queue
```
CREATE RESOURCE QUEUE adhoc WITH (ACTIVE_STATEMENTS=3);
```
Here created a resource queue called adhoc, and this resource queue where a user in a given point in time, at most, only three query execution. If the user has launched a new query, the query will be in a waiting state until the previous query is finished.
You can specify the memory limit for this resource queue in the query to use when creating resource queue
```
CREATE RESOURCE QUEUE myqueue WITH (ACTIVE_STATEMENTS=20, MEMORY_LIMIT='2000MB');
```
Here we created a resource queue named myqueue all in this query can be used in the resource queue up to 2000MB of memory size. Each of those queries in the course of implementation, will take MEMORY_LIMIT / ACTIVE_STATEMENTS memory in compute node (segment), for myqueue this resource queue for each query will take up 2000MB / 20 = 100MB of memory. If for some queries queries require a separate memory size available, can be used statement_mem to set up, but the need to ensure that no more than resource queue of MEMORY_LIMIT and max_statement_mem . Examples are as follows:
```
SET statement_mem='1GB';
SELECT * FROM test_adbpg WHERE col='adb4pg' ORDER BY id;
RESET statement_mem;
```
Set the priority queue
to a different resource queue priority can be set to control resource queue queries using CPU resources. For example, when large-scale system of concurrent queries, high priority resource queue queries will use more CPU resources than low-priority resource queue queries, thus ensuring high priority queries more fully CPU resources to execute. resource queue priority queue settings or you can use alter after created when you create ... with the statement to be modified. Examples are as follows:
```
CREATE RESOURCE QUEUE executive WITH (ACTIVE_STATEMENTS=3, PRIORITY=MAX);

ALTER RESOURCE QUEUE adhoc WITH (PRIORITY=LOW);
ALTER RESOURCE QUEUE reporting WITH (PRIORITY=HIGH);
```

Different mechanisms and priorities and ACTIVE_STATEMENTS MEMORY_LIMIT are: ACTIVE_STATEMENTS and MEMORY_LIMIT will determine whether it is allowed to be executed before query execution. The priority mechanism after a query is started, the operating state of the system to dynamically query the corresponding CPU resources available to it in accordance with the priority queue assignment. For example, at some point ADB for PG There are some low priority query execution, then a high priority inquiry into the ADB for PG and ready to perform. So ADB for PG queries to allocate more resources for the CPU and reduce CPU resources low priority query. CPU resource allocation rules are as follows: Consistent same priority queries can be assigned to CPU resources. When the system is also available in high, medium and low priority query when the high-priority query distribution system to get 90 percent of the CPU resources, and the remaining 10% will be reserved for medium and low priority queries to distribution, in which 10% of the CPU resources of the system, the priority queries will be 90% of the CPU resources, the low priority query the remaining 10% is obtained.

After the resource queue is created, the system may be used gp_toolkit.gp_resqueue_status Limit settings and view the current state of resource queues, as follows:

postgres=> SELECT * from gp_toolkit.gp_resqueue_status WHERE
postgres->   rsqname='adhoc';
 queueid | rsqname | rsqcountlimit | rsqcountvalue | rsqcostlimit | rsqcostvalue | rsqmemorylimit | rsqmemoryvalue | rsqwaiters | rsqholders
---------+---------+---------------+---------------+--------------+--------------+----------------+----------------+------------+------------
   19283 | adhoc   |             3 |             0 |           -1 |            0 |             -1 |              0 |          0 |          0
(1 行记录)

In addition, resource queue can not be created within a transaction block:

postgres=> begin;
BEGIN
postgres=> CREATE RESOURCE QUEUE test_q WITH (ACTIVE_STATEMENTS=3, PRIORITY=MAX);
ERROR:  CREATE RESOURCE QUEUE cannot run inside a transaction block

Not all sql will be limited resource queue when resource_select_only is set to on, when only SELECT, SELECT INTO, CREATE TABLE AS SELECT, and DECLARE CURSOR will be constrained. The resource_select_only is off when, INSERT, UPDATE, and DELETE will be a queue to manage resources. In the ADB for PG in, resource_select_only default settings become off.

The role assigned to the resource queue

After creating the resource queue, we have to assign one or more users to the resource queue, so this resource queue resource management will be where the user query. If a user does not appear to specify which allocate a resource queue, then the ADB for PG will default to assign him to pg_default resource queue. pg_default can run with 500 active query, no limit cost limit, the priority of medium:

postgres=> SELECT * from gp_toolkit.gp_resqueue_status WHERE rsqname='pg_default';
 queueid |  rsqname   | rsqcountlimit | rsqcountvalue | rsqcostlimit | rsqcostvalue | rsqmemorylimit | rsqmemoryvalue | rsqwaiters | rsqholders
---------+------------+---------------+---------------+--------------+--------------+----------------+----------------+------------+------------
    6055 | pg_default |           500 |             1 |           -1 |          126 |             -1 |   2.096128e+09 |          0 |          1
(1 行记录)

A role assigned to a resource queue syntax is as follows:

ALTER ROLE name RESOURCE QUEUE queue_name;
CREATE ROLE name WITH LOGIN RESOURCE QUEUE queue_name;

Users can modify the resource queue to which they belong after the role is created, you can assign resource queue when creating the role. It should be noted that at any time a role can only be attributed to a resource queue.

To delete a resource queue ownership role

A role must belong to a resouce queue, when we need to remove the role of a designated resouce queue, and then put back pg_default this resouce queue, you can use

ALTER ROLE role_name RESOURCE QUEUE none;

Modify resource queue configuration

resource queue Once created, you can use the resource queue resource allocation resource queue to be modified, as follows:

修改活跃的查询数：
ALTER RESOURCE QUEUE adhoc WITH (ACTIVE_STATEMENTS=5);
修改resource queue优先级：
ALTER RESOURCE QUEUE exec WITH (PRIORITY=MAX);
修改resource queue内存和计划代价约束：
ALTER RESOURCE QUEUE adhoc WITH (MAX_COST=-1.0, MEMORY_LIMIT='2GB');

Delete resource queue

Before deleting a resouce queue, we need to ensure that role this resource queue is not assigned, and no inquiry is in waiting state. Otherwise, delete does not succeed.

DROP RESOURCE QUEUE name;

Reference material

https://gp-docs-cn.github.io/docs/ref_guide/sql_commands/CREATE_RESOURCE_QUEUE.html
http://gpdb.docs.pivotal.io/43330/admin_guide/workload_mgmt.html#topic9

AnalyticDB for PostgreSQL resource use queue management and load isolation