Detailed explanation of Apache Doris resource isolation

1. Brief introduction

Mainly for BE nodes

The main purpose of Doris's multi-tenancy and resource isolation scheme is to reduce mutual interference when multiple users perform data operations in the same Doris cluster, and to allocate cluster resources to each user more reasonably.

2. Test experience

2.1 BE node setting label

Note: One BE only supports setting one Tag.

BE nodes in a Doris cluster can be tagged (Tag), and BE nodes with the same tag form a resource group (Resource Group); resource group can be regarded as a management unit for data storage and computing.

2.1.1 Before grouping

2.1.2 Start grouping

-- 3个节点划分成2个资源组
alter system modify backend "be01:9050" set ("tag.location" = "group_a");
alter system modify backend "be02:9050" set ("tag.location" = "group_b");
alter system modify backend "be03:9050" set ("tag.location" = "group_c");

2.1.3 After grouping

2.2 Distributing data according to resource groups

After resource groups are divided, different copies of user data can be distributed in different resource groups.

2.2.1 Create a test table

-- 以SQL99其中一张维表为例，每个资源组放一个副本
create table tpcds.catalog_returns_duplicate
(
    cr_item_sk                integer               not null,
    cr_order_number           integer               not null,
    cr_returned_date_sk       integer                       ,
    cr_returned_time_sk       integer                       ,
    cr_ship_date_sk           integer                       ,
    cr_refunded_customer_sk   integer                       ,
    cr_refunded_cdemo_sk      integer                       ,
    cr_refunded_hdemo_sk      integer                       ,
    cr_refunded_addr_sk       integer                       ,
    cr_returning_customer_sk  integer                       ,
    cr_returning_cdemo_sk     integer                       ,
    cr_returning_hdemo_sk     integer                       ,
    cr_returning_addr_sk      integer                       ,
    cr_call_center_sk         integer                       ,
    cr_catalog_page_sk        integer                       ,
    cr_ship_mode_sk           integer                       ,
    cr_warehouse_sk           integer                       ,--    cr_reason_sk              integer                       ,
    cr_return_quantity        integer                       ,
    cr_return_amount          decimal(7,2)                  ,
    cr_return_tax             decimal(7,2)                  ,
    cr_return_amt_inc_tax     decimal(7,2)                  ,
    cr_fee                    decimal(7,2)                  ,
    cr_return_ship_cost       decimal(7,2)                  ,
    cr_refunded_cash          decimal(7,2)                  ,
    cr_reversed_charge        decimal(7,2)                  ,
    cr_store_credit           decimal(7,2)                  ,
    cr_net_loss               decimal(7,2)                 
)ENGINE=olap 
DUPLICATE KEY(`cr_item_sk`,`cr_order_number`) 
DISTRIBUTED BY HASH(`cr_item_sk`,`cr_order_number`) BUCKETS 10 
PROPERTIES("replication_allocation"
    = "tag.location.group_a:1, tag.location.group_b:1, tag.location.group_c:1");

2.2.2 Import test data

-- 随机测试写入几条数据验证
insert into tpcds.catalog_returns_duplicate values
(.....)

2.3 User Resource Usage Authority Control

By setting the resource usage permission of the user, the query of a certain user can only be executed by using the nodes in the specified resource group.

2.3.1 Create a test user

-- 用户名@用户端连接所在的主机地址（测试不设置密码）
-- 默认为 '%'，即表示该用户可以从任意host连接到 DorisDB
CREATE USER 't_rg_user01'@'%';
CREATE USER 't_rg_user02'@'%';
CREATE USER 't_rg_user03'@'%'; 

2.3.2 Authorized users

-- GRANT授权（授予所有库和表的权限给用户）
GRANT SELECT_PRIV ON *.* TO 't_rg_user01'@'%';
GRANT SELECT_PRIV ON *.* TO 't_rg_user02'@'%';
GRANT SELECT_PRIV ON *.* TO 't_rg_user03'@'%';

2.3.3 Setting User Resource Usage Permissions

set property for 't_rg_user01' 'resource_tags.location' = 'group_a';
set property for 't_rg_user02' 'resource_tags.location' = 'group_b';
set property for 't_rg_user03' 'resource_tags.location' = 'group_a, group_b, group_c';

2.3.4 Authentication permissions

1. Prepare test query SQL

select * from (
select * from catalog_returns_duplicate crd0402
) t1
JOIN
(
select * from catalog_returns_duplicate crd0402
) t2 on t1.cr_item_sk = t2.cr_item_sk
JOIN
(
select * from catalog_returns_duplicate crd0402
) t3 on t2.cr_item_sk = t3.cr_item_sk
JOIN
(
select distinct cr_item_sk from catalog_returns_duplicate crd0402
) t4 on t3.cr_item_sk = t4.cr_item_sk

2. start query

From the [be scan rows] icon in grafana, it can be clearly seen that the resources used by each user have corresponding memory isolation based on the settings.

2.4 Read and write permission verification

Read only without write or write only without read permission verification

-- 用户创建
CREATE USER 't_only_read_user'@'%';
CREATE USER 't_only_write_user'@'%';

-- 对指定的库或表的读取权限
GRANT SELECT_PRIV ON *.* TO 't_only_read_user'@'%';

-- 对指定的库或表的导入权限
GRANT LOAD_PRIV ON *.* TO 't_only_write_user'@'%';

3. Optimize configuration

The resource group approach is to isolate and limit resources at the node level.
In the resource group, resource preemption may still occur; the resource limit function can be used for single query

3.1 Memory Limits

-- 设置会话变量 exec_mem_limit。则之后该会话内（连接内）的所有查询都使用这个内存限制。
set exec_mem_limit=1G;

-- 设置全局变量 exec_mem_limit。则之后所有新会话（新连接）的所有查询都使用这个内存限制。
set global exec_mem_limit=1G;

-- 在 SQL 中设置变量 exec_mem_limit。则该变量仅影响这个 SQL。
select /*+ SET_VAR(exec_mem_limit=1G) */ id, name from tbl where xxx;

3.2 CPU limit

-- 设置会话变量 cpu_resource_limit。则之后该会话内（连接内）的所有查询都使用这个CPU限制。
set cpu_resource_limit = 2

-- 设置用户的属性 cpu_resource_limit，则所有该用户的查询情况都使用这个CPU限制。该属性的优先级高于会话变量 cpu_resource_limit
set property for 'user1' 'cpu_resource_limit' = '3';

Four. Summary

1. Physical isolation of query resources can be achieved for users , and resource isolation is not required for writing
2. You can control the authority of database and table granularity for users, refer to user account management