Introduction
Test the effect of partitioning and bucketing.
Basic operation of partition
add partition
ALTER TABLE v2x_olap_database.government_car
ADD PARTITION p20221203 VALUES LESS THAN ("2022-12-04");
A dynamic partition table cannot add partitions and needs to be converted to a manual partition table.
view partition
show partitions from <表名>
delete partition
alter table <表名>
drop partition <分区名>
Switch between manual partition table and dynamic partition table
Manual to automatic
alter table <表名> set ("dynamic_partition.enable" = "true")NOTE: If dynamic_partition.start is set, historical partitions whose partition range is before the offset will be deleted
Automatic to manual
alter table <表名> set ("dynamic_partition.enable" = "false")Partitions and Buckets
Doris supports two layers of data division. The first layer is Partition, which supports the division of Range and List. The second layer is Bucket (Tablet), which only supports Hash division.
It is also possible to use only one level of partitioning. When using one-level partitioning, only bucket partitioning is supported. Let's introduce the partition and bucket respectively:
-
Partition
- The Partition column can specify one or more columns, and the partition column must be a KEY column. The usage of multi-column partitioning will be introduced later in the summary of multi-column partitioning .
- Regardless of the type of the partition column, double quotes are required when writing the partition value.
- There is theoretically no upper limit to the number of partitions.
- When you do not use Partition to create a table, the system will automatically generate a Partition with the same name as the table name and a full range of values. The Partition is invisible to users and cannot be deleted.
- Partitions with overlapping ranges cannot be added when creating partitions .
Range partition
-
The partition column is usually a time column to facilitate the management of old and new data.
-
Partition supports
VALUES LESS THAN (...)only specifying the upper bound, and the system will use the upper bound of the previous partition as the lower bound of the partition to generate a left-closed and right-open interval. It also supportsVALUES [...)generating a left-closed and right-open interval by specifying upper and lower bounds.
Partition
Range Partition
Static Range Partition
create table
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)
(
PARTITION `p201701` VALUES LESS THAN ("2017-02-01"),
PARTITION `p201702` VALUES LESS THAN ("2017-03-01"),
PARTITION `p201703` VALUES LESS THAN ("2017-04-01")
)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 5
PROPERTIES
(
"replication_num" = "1"
);easy to understand. Here is an example to illustrate the changes in the range of partitions when the `VALUES LESS THAN (...)` statement is used to add or delete partitions:
- As in the `example_range_tbl` example above, when the table is created, the following three partitions will be automatically generated:
```text
p201701: [MIN_VALUE, 2017-02-01)
p201702: [2017-02-01, 2017-03-01)
p201703: [2017-03-01, 2017-04-01)
```
Test inserting data (insert the corresponding partitions below)
insert into example_range_tbl values(10001,'2017-01-05','北京',30,1,'2017-10-01 17:05:45',2,22,22);
insert into example_range_tbl values(10000,'2017-02-01','北京',20,0,'2017-10-01 06:00:00',20,10,10);
insert into example_range_tbl values(10000,'2017-03-03','北京',20,0,'2017-10-01 07:00:00',15,2,2);Query created partitions
SHOW PARTITIONS FROM example_range_tbl;Dynamic Range Partition
create table
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)()
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "10"
);dynamic_partition.enable: Whether to enable the dynamic partition feature, which can be specified as true or false. The default is true.
dynamic_partition.time_unit: The unit of dynamic partition scheduling, which can be specified as DAY WEEK MONTH. When DAY is specified, the suffix format of the dynamically created partition name is yyyyMMdd, for example - - 20200325. When WEEK is specified, the suffix format of the dynamically created partition name is yyyy_ww, that is, the week of the current date in this year. When MONTH is specified, the dynamically created partition name suffix format is yyyyMM, for example, 202003.
dynamic_partition.start: The start time of the dynamic partition, based on the current day, partitions beyond this time range will be deleted. If not filled, it defaults to Integer.MIN_VALUE which is -2147483648.
dynamic_partition.end: The end time of the dynamic partition, based on the current day, a partition range of N units will be created in advance.
dynamic_partition.prefix: Dynamically created partition name prefix.
dynamic_partition.buckets: The number of buckets corresponding to dynamically created partitions.
view partition
show partitions from example_range_tbl;+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 12985 | p20230302 | 1 | 2023-03-02 16:47:18 | NORMAL | date | [types: [DATE]; keys: [2023-03-02]; ..types: [DATE]; keys: [2023-03-03]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13006 | p20230303 | 1 | 2023-03-02 16:47:18 | NORMAL | date | [types: [DATE]; keys: [2023-03-03]; ..types: [DATE]; keys: [2023-03-04]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13027 | p20230304 | 1 | 2023-03-02 16:47:18 | NORMAL | date | [types: [DATE]; keys: [2023-03-04]; ..types: [DATE]; keys: [2023-03-05]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13048 | p20230305 | 1 | 2023-03-02 16:47:19 | NORMAL | date | [types: [DATE]; keys: [2023-03-05]; ..types: [DATE]; keys: [2023-03-06]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
4 rows in set (0.00 sec)You can see that the above created 3 partitions starting from today.
Insert historical data to see
insert into example_range_tbl values(10001,'2017-01-05','beijing',30,1,'2017-10-01 17:05:45',2,22,22);The above operation will report an error in the future (indicating that if there is no corresponding historical partition, the default is that the insertion is unsuccessful)
[root@doris1 ~]# cat _load_error_log\?file\=__shard_3%2Ferror_log_insert_stmt_75a4a5394a0a4c4a-a813211947164ecc_75a4a5394a0a4c4a_a813211947164ecc
Reason: no partition for this tuple. tuple=+---------------+---------------+------------------+-----------------+----------------+--------------------+-----------------+-----------------+-----------------+
|(Int128) |(Date) |(Nullable(String))|(Nullable(Int16))|(Nullable(Int8))|(Nullable(DateTime))|(Nullable(Int64))|(Nullable(Int32))|(Nullable(Int32))|
+---------------+---------------+------------------+-----------------+----------------+--------------------+-----------------+-----------------+-----------------+
| 10001| 2017-01-05| beijing| 30| 1| 2017-10-01 17:05:45| 2| 22| 22|
+---------------+---------------+------------------+-----------------+----------------+--------------------+-----------------+-----------------+-----------------+
. src line []; Enable history partition
create_history_partition = true
1) dynamic_partition.history_partition_num is not set, ie -1; expect_create_partition_num = end - start;
2) dynamic_partition.history_partition_num is set expect_create_partition_num = end - max(start, -histoty_partition_num);
create_numate_partition = false will not create a partition = end - 0;
When expect_create_partition_num is greater than max_dynamic_partition_num (500 by default), it is forbidden to create too many partitions.
Note: If dynamic_partition.start and `expect_create_partition_num` are not set, historical partitions cannot be created
Create table statement
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)()
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.end" = "5",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "10",
"dynamic_partition.create_history_partition" = "true",
"dynamic_partition.start" = "-10"
);view partition
show partitions from example_range_tbl;+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 13071 | p20230220 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-20]; ..types: [DATE]; keys: [2023-02-21]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13092 | p20230221 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-21]; ..types: [DATE]; keys: [2023-02-22]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13113 | p20230222 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-22]; ..types: [DATE]; keys: [2023-02-23]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13134 | p20230223 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-23]; ..types: [DATE]; keys: [2023-02-24]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13155 | p20230224 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-24]; ..types: [DATE]; keys: [2023-02-25]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13176 | p20230225 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-25]; ..types: [DATE]; keys: [2023-02-26]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13197 | p20230226 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-26]; ..types: [DATE]; keys: [2023-02-27]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13218 | p20230227 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-27]; ..types: [DATE]; keys: [2023-02-28]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13239 | p20230228 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-02-28]; ..types: [DATE]; keys: [2023-03-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13260 | p20230301 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-01]; ..types: [DATE]; keys: [2023-03-02]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13281 | p20230302 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-02]; ..types: [DATE]; keys: [2023-03-03]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13302 | p20230303 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-03]; ..types: [DATE]; keys: [2023-03-04]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13323 | p20230304 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-04]; ..types: [DATE]; keys: [2023-03-05]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13344 | p20230305 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-05]; ..types: [DATE]; keys: [2023-03-06]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13365 | p20230306 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-06]; ..types: [DATE]; keys: [2023-03-07]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13386 | p20230307 | 1 | 2023-03-02 17:05:30 | NORMAL | date | [types: [DATE]; keys: [2023-03-07]; ..types: [DATE]; keys: [2023-03-08]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+(Note that today's time is March 2, and the partitions for the next 3 days and the partitions for the past 10 days are created above)
Insert data test (if there is no partition or you can’t create it yourself, the history partition is also the function of automatically creating the previous partition)
insert into example_range_tbl values(10001,'2017-01-05','beijing',30,1,'2017-10-01 17:05:45',2,22,22); The following data can be created successfully
insert into example_range_tbl values(10001,'2023-02-27','beijing',30,1,'2017-10-01 17:05:45',2,22,22); Create partitions in batches
The function of creating partitions in batches has fully investigated the needs of users in the early stage. Based on the design goals of simplicity, power and ease of use, the core of the design is locked in several elements:
-
Time interval range (opening and closing issues will be considered)
-
Timespan (i.e. the size of the time dimension of each partition)
-
Time unit (year, month, day, hour, week, etc.)
Create table statement
CREATE TABLE IF NOT EXISTS example_range_tbl1
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)
(
FROM ("2000-01-01") TO ("2021-01-01") INTERVAL 1 YEAR,
FROM ("2021-01-01") TO ("2022-01-01") INTERVAL 1 MONTH,
FROM ("2022-01-01") TO ("2023-01-01") INTERVAL 1 WEEK,
FROM ("2023-01-01") TO ("2023-02-01") INTERVAL 1 DAY
)
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);View partition data
mysql> show partitions from example_range_tbl1;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 13408 | p_2000 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2000-01-01]; ..types: [DATE]; keys: [2001-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13409 | p_2001 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2001-01-01]; ..types: [DATE]; keys: [2002-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13410 | p_2002 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2002-01-01]; ..types: [DATE]; keys: [2003-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13411 | p_2003 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2003-01-01]; ..types: [DATE]; keys: [2004-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13412 | p_2004 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2004-01-01]; ..types: [DATE]; keys: [2005-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13413 | p_2005 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2005-01-01]; ..types: [DATE]; keys: [2006-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13414 | p_2006 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2006-01-01]; ..types: [DATE]; keys: [2007-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13415 | p_2007 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2007-01-01]; ..types: [DATE]; keys: [2008-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13416 | p_2008 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2008-01-01]; ..types: [DATE]; keys: [2009-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13417 | p_2009 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2009-01-01]; ..types: [DATE]; keys: [2010-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13418 | p_2010 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2010-01-01]; ..types: [DATE]; keys: [2011-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13419 | p_2011 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2011-01-01]; ..types: [DATE]; keys: [2012-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13420 | p_2012 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2012-01-01]; ..types: [DATE]; keys: [2013-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13421 | p_2013 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2013-01-01]; ..types: [DATE]; keys: [2014-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13422 | p_2014 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2014-01-01]; ..types: [DATE]; keys: [2015-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13423 | p_2015 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2015-01-01]; ..types: [DATE]; keys: [2016-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13424 | p_2016 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2016-01-01]; ..types: [DATE]; keys: [2017-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13425 | p_2017 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2017-01-01]; ..types: [DATE]; keys: [2018-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13426 | p_2018 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2018-01-01]; ..types: [DATE]; keys: [2019-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13427 | p_2019 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2019-01-01]; ..types: [DATE]; keys: [2020-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13428 | p_2020 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2020-01-01]; ..types: [DATE]; keys: [2021-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13429 | p_202101 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-01-01]; ..types: [DATE]; keys: [2021-02-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13430 | p_202102 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-02-01]; ..types: [DATE]; keys: [2021-03-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13431 | p_202103 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-03-01]; ..types: [DATE]; keys: [2021-04-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13432 | p_202104 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-04-01]; ..types: [DATE]; keys: [2021-05-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13433 | p_202105 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-05-01]; ..types: [DATE]; keys: [2021-06-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13434 | p_202106 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-06-01]; ..types: [DATE]; keys: [2021-07-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13435 | p_202107 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-07-01]; ..types: [DATE]; keys: [2021-08-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13436 | p_202108 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-08-01]; ..types: [DATE]; keys: [2021-09-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13437 | p_202109 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-09-01]; ..types: [DATE]; keys: [2021-10-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13438 | p_202110 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-10-01]; ..types: [DATE]; keys: [2021-11-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13439 | p_202111 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-11-01]; ..types: [DATE]; keys: [2021-12-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13440 | p_202112 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2021-12-01]; ..types: [DATE]; keys: [2022-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13441 | p_2022_01 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-01]; ..types: [DATE]; keys: [2022-01-03]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13442 | p_2022_02 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-03]; ..types: [DATE]; keys: [2022-01-10]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13443 | p_2022_03 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-10]; ..types: [DATE]; keys: [2022-01-17]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13444 | p_2022_04 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-17]; ..types: [DATE]; keys: [2022-01-24]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13445 | p_2022_05 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-24]; ..types: [DATE]; keys: [2022-01-31]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13446 | p_2022_06 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-01-31]; ..types: [DATE]; keys: [2022-02-07]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13447 | p_2022_07 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-02-07]; ..types: [DATE]; keys: [2022-02-14]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13448 | p_2022_08 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-02-14]; ..types: [DATE]; keys: [2022-02-21]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13449 | p_2022_09 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-02-21]; ..types: [DATE]; keys: [2022-02-28]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13450 | p_2022_10 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-02-28]; ..types: [DATE]; keys: [2022-03-07]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13451 | p_2022_11 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-03-07]; ..types: [DATE]; keys: [2022-03-14]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13452 | p_2022_12 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-03-14]; ..types: [DATE]; keys: [2022-03-21]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13453 | p_2022_13 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-03-21]; ..types: [DATE]; keys: [2022-03-28]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13454 | p_2022_14 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-03-28]; ..types: [DATE]; keys: [2022-04-04]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13455 | p_2022_15 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-04-04]; ..types: [DATE]; keys: [2022-04-11]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13456 | p_2022_16 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-04-11]; ..types: [DATE]; keys: [2022-04-18]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13457 | p_2022_17 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-04-18]; ..types: [DATE]; keys: [2022-04-25]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13458 | p_2022_18 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-04-25]; ..types: [DATE]; keys: [2022-05-02]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13459 | p_2022_19 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-05-02]; ..types: [DATE]; keys: [2022-05-09]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13460 | p_2022_20 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-05-09]; ..types: [DATE]; keys: [2022-05-16]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13461 | p_2022_21 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-05-16]; ..types: [DATE]; keys: [2022-05-23]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13462 | p_2022_22 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-05-23]; ..types: [DATE]; keys: [2022-05-30]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13463 | p_2022_23 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-05-30]; ..types: [DATE]; keys: [2022-06-06]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13464 | p_2022_24 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-06-06]; ..types: [DATE]; keys: [2022-06-13]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13465 | p_2022_25 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-06-13]; ..types: [DATE]; keys: [2022-06-20]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13466 | p_2022_26 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-06-20]; ..types: [DATE]; keys: [2022-06-27]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13467 | p_2022_27 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-06-27]; ..types: [DATE]; keys: [2022-07-04]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13468 | p_2022_28 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-07-04]; ..types: [DATE]; keys: [2022-07-11]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13469 | p_2022_29 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-07-11]; ..types: [DATE]; keys: [2022-07-18]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13470 | p_2022_30 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-07-18]; ..types: [DATE]; keys: [2022-07-25]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13471 | p_2022_31 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-07-25]; ..types: [DATE]; keys: [2022-08-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13472 | p_2022_32 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-08-01]; ..types: [DATE]; keys: [2022-08-08]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13473 | p_2022_33 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-08-08]; ..types: [DATE]; keys: [2022-08-15]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13474 | p_2022_34 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-08-15]; ..types: [DATE]; keys: [2022-08-22]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13475 | p_2022_35 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-08-22]; ..types: [DATE]; keys: [2022-08-29]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13476 | p_2022_36 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-08-29]; ..types: [DATE]; keys: [2022-09-05]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13477 | p_2022_37 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-09-05]; ..types: [DATE]; keys: [2022-09-12]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13478 | p_2022_38 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-09-12]; ..types: [DATE]; keys: [2022-09-19]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13479 | p_2022_39 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-09-19]; ..types: [DATE]; keys: [2022-09-26]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13480 | p_2022_40 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-09-26]; ..types: [DATE]; keys: [2022-10-03]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13481 | p_2022_41 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-10-03]; ..types: [DATE]; keys: [2022-10-10]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13482 | p_2022_42 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-10-10]; ..types: [DATE]; keys: [2022-10-17]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13483 | p_2022_43 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-10-17]; ..types: [DATE]; keys: [2022-10-24]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13484 | p_2022_44 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-10-24]; ..types: [DATE]; keys: [2022-10-31]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13485 | p_2022_45 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-10-31]; ..types: [DATE]; keys: [2022-11-07]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13486 | p_2022_46 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-11-07]; ..types: [DATE]; keys: [2022-11-14]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13487 | p_2022_47 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-11-14]; ..types: [DATE]; keys: [2022-11-21]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13488 | p_2022_48 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-11-21]; ..types: [DATE]; keys: [2022-11-28]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13489 | p_2022_49 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-11-28]; ..types: [DATE]; keys: [2022-12-05]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13490 | p_2022_50 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-12-05]; ..types: [DATE]; keys: [2022-12-12]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13491 | p_2022_51 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-12-12]; ..types: [DATE]; keys: [2022-12-19]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13492 | p_2022_52 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-12-19]; ..types: [DATE]; keys: [2022-12-26]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13493 | p_2022_53 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2022-12-26]; ..types: [DATE]; keys: [2023-01-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13494 | p_20230101 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-01]; ..types: [DATE]; keys: [2023-01-02]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13495 | p_20230102 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-02]; ..types: [DATE]; keys: [2023-01-03]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13496 | p_20230103 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-03]; ..types: [DATE]; keys: [2023-01-04]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13497 | p_20230104 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-04]; ..types: [DATE]; keys: [2023-01-05]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13498 | p_20230105 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-05]; ..types: [DATE]; keys: [2023-01-06]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13499 | p_20230106 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-06]; ..types: [DATE]; keys: [2023-01-07]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13500 | p_20230107 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-07]; ..types: [DATE]; keys: [2023-01-08]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13501 | p_20230108 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-08]; ..types: [DATE]; keys: [2023-01-09]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13502 | p_20230109 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-09]; ..types: [DATE]; keys: [2023-01-10]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13503 | p_20230110 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-10]; ..types: [DATE]; keys: [2023-01-11]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13504 | p_20230111 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-11]; ..types: [DATE]; keys: [2023-01-12]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13505 | p_20230112 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-12]; ..types: [DATE]; keys: [2023-01-13]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13506 | p_20230113 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-13]; ..types: [DATE]; keys: [2023-01-14]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13507 | p_20230114 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-14]; ..types: [DATE]; keys: [2023-01-15]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13508 | p_20230115 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-15]; ..types: [DATE]; keys: [2023-01-16]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13509 | p_20230116 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-16]; ..types: [DATE]; keys: [2023-01-17]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13510 | p_20230117 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-17]; ..types: [DATE]; keys: [2023-01-18]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13511 | p_20230118 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-18]; ..types: [DATE]; keys: [2023-01-19]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13512 | p_20230119 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-19]; ..types: [DATE]; keys: [2023-01-20]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13513 | p_20230120 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-20]; ..types: [DATE]; keys: [2023-01-21]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13514 | p_20230121 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-21]; ..types: [DATE]; keys: [2023-01-22]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13515 | p_20230122 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-22]; ..types: [DATE]; keys: [2023-01-23]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13516 | p_20230123 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-23]; ..types: [DATE]; keys: [2023-01-24]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13517 | p_20230124 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-24]; ..types: [DATE]; keys: [2023-01-25]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13518 | p_20230125 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-25]; ..types: [DATE]; keys: [2023-01-26]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13519 | p_20230126 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-26]; ..types: [DATE]; keys: [2023-01-27]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13520 | p_20230127 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-27]; ..types: [DATE]; keys: [2023-01-28]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13521 | p_20230128 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-28]; ..types: [DATE]; keys: [2023-01-29]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13522 | p_20230129 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-29]; ..types: [DATE]; keys: [2023-01-30]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13523 | p_20230130 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-30]; ..types: [DATE]; keys: [2023-01-31]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 13524 | p_20230131 | 1 | 2023-03-02 17:22:31 | NORMAL | date | [types: [DATE]; keys: [2023-01-31]; ..types: [DATE]; keys: [2023-02-01]; ) | user_id | 10 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
117 rows in set (0.02 sec)List partition
-
The partition column supports
BOOLEAN, TINYINT, SMALLINT, INT, BIGINT, LARGEINT, DATE, DATETIME, CHAR, VARCHARdata types, and the partition value is an enumeration value. Only when the data is one of the enumerated values of the target partition, the partition can be hit. -
Partition supports passing
VALUES IN (...)to specify the enumeration values contained in each partition. -
The following uses an example to illustrate the changes of partitions when adding and deleting partitions.
Test build table
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) not NULL COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY LIST(city)
(
PARTITION `p_huabei` VALUES IN ("beijing", "tianjin", "shijiazhuang"),
PARTITION `p_dongbei` VALUES IN ("shenyang", "dalian"),
PARTITION `p_huazhong` VALUES IN ("wuhan", "changsha"),
PARTITION `p_xinan` VALUES IN ("chengdu", "chongqing")
)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 5
PROPERTIES
(
"replication_num" = "1"
);The city field in LIST(city) cannot be empty
Test Data
insert into example_range_tbl values(10001,'2017-01-05','beijing',30,1,'2017-10-01 17:05:45',2,22,22);
insert into example_range_tbl values(10000,'2017-02-01','tianjin',20,0,'2017-10-01 06:00:00',20,10,10);
insert into example_range_tbl values(10000,'2017-03-03','shenyang',20,0,'2017-10-01 07:00:00',15,2,2);Effect
When we have a suitable partition and bucket, after the imported data is exported to Doris, the data will be stored according to the partition and bucket column in the table creation statement. The storage example of the above website site data is as shown in the figure:
Figure 2: Data storage after Doris partitioning and bucketing
At this point, if you execute the SQL query:
select * from test_tbl where date = "2020-03-23" and site = 1According to the predicate date = "2020-03-23", the partition p20200323 can be located, and the predicate site = 1 can be located to bucket_1 under this partition. Assuming that there are 30 days of data, the number of buckets calculated by automatic bucketing is 20. Then, after the definite predicate of partition and bucket is pushed down, the scan amount of the full table of data can be reduced to 1/600 of the original (30 days * 20 buckets = 600), which greatly reduces the scope of data scan and improves the efficiency of query .
experiment
dynamic partition
Create a dynamic partition (now March 2)
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)()
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "3",
"dynamic_partition.create_history_partition" = "true",
"dynamic_partition.end" = "2",
"dynamic_partition.start" = "-3"
);View the current partition (you can see the data created for the past 3 days and the next 2 days)
mysql> show partitions from example_range_tbl;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 15869 | p20230227 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-02-27]; ..types: [DATE]; keys: [2023-02-28]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15876 | p20230228 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-02-28]; ..types: [DATE]; keys: [2023-03-01]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15883 | p20230301 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-01]; ..types: [DATE]; keys: [2023-03-02]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15890 | p20230302 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-02]; ..types: [DATE]; keys: [2023-03-03]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15897 | p20230303 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-03]; ..types: [DATE]; keys: [2023-03-04]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15904 | p20230304 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-04]; ..types: [DATE]; keys: [2023-03-05]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
6 rows in set (0.00 sec)
Looking at the situation on the second day (now March 3), you can see that automatic partitioning automatically creates future partitions and deletes historical partitions.
mysql> show partitions from example_range_tbl;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 15876 | p20230228 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-02-28]; ..types: [DATE]; keys: [2023-03-01]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15883 | p20230301 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-01]; ..types: [DATE]; keys: [2023-03-02]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15890 | p20230302 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-02]; ..types: [DATE]; keys: [2023-03-03]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15897 | p20230303 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-03]; ..types: [DATE]; keys: [2023-03-04]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15904 | p20230304 | 1 | 2023-03-02 17:44:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-04]; ..types: [DATE]; keys: [2023-03-05]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15958 | p20230305 | 1 | 2023-03-03 00:09:48 | NORMAL | date | [types: [DATE]; keys: [2023-03-05]; ..types: [DATE]; keys: [2023-03-06]; ) | user_id | 3 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
6 rows in set (0.00 sec)Manual Partition Experiment
Create table statement
CREATE TABLE IF NOT EXISTS example_range_tbltest
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)
(
PARTITION `p201701` VALUES LESS THAN ("2017-02-01"),
PARTITION `p201702` VALUES LESS THAN ("2017-03-01"),
PARTITION `p201703` VALUES LESS THAN ("2017-04-01")
)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 5
PROPERTIES
(
"replication_num" = "1"
);view partition
mysql> show partitions from example_range_tbltest;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 15911 | p201701 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [0000-01-01]; ..types: [DATE]; keys: [2017-02-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15912 | p201702 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-02-01]; ..types: [DATE]; keys: [2017-03-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15913 | p201703 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-03-01]; ..types: [DATE]; keys: [2017-04-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
3 rows in set (0.01 sec)insert data
insert into example_range_tbltest values(10000,'2017-04-02','shenyang',20,0,'2017-10-01 07:00:00',15,2,2);The following error is reported because there is no corresponding partition
mysql> insert into example_range_tbltest values(10000,'2017-04-02','shenyang',20,0,'2017-10-01 07:00:00',15,2,2);
ERROR 5025 (HY000): Insert has filtered data in strict mode, tracking_url=http://10.240.0.51:8040/api/_load_error_log?file=__shard_235/error_log_insert_stmt_749a91311755470e-8b2b8b059bce5df2_749a91311755470e_8b2b8b059bce5df2Add a partition manually
ALTER TABLE example_range_tbltest ADD PARTITION p201704 VALUES LESS THAN ("2017-05-01");mysql> show partitions from example_range_tbltest;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 15911 | p201701 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [0000-01-01]; ..types: [DATE]; keys: [2017-02-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15912 | p201702 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-02-01]; ..types: [DATE]; keys: [2017-03-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15913 | p201703 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-03-01]; ..types: [DATE]; keys: [2017-04-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15946 | p201704 | 1 | 2023-03-02 17:57:34 | NORMAL | date | [types: [DATE]; keys: [2017-04-01]; ..types: [DATE]; keys: [2017-05-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
4 rows in set (0.00 sec)Then insert data (you can see that the new partition inserts data successfully)
mysql> insert into example_range_tbltest values(10000,'2017-04-02','shenyang',20,0,'2017-10-01 07:00:00',15,2,2);
Query OK, 1 row affected (0.03 sec)
{'label':'insert_9ef4039c342c494c_aa49e0c045fbcf55', 'status':'VISIBLE', 'txnId':'1030'}Looking at the situation on the second day (now March 3), you can see that the manually created partition has increased the visibleVersion by 1, and the others have not changed. The added 1 is the impact on the table when inserting data.
mysql> show partitions from example_range_tbltest;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 15911 | p201701 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [0000-01-01]; ..types: [DATE]; keys: [2017-02-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15912 | p201702 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-02-01]; ..types: [DATE]; keys: [2017-03-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15913 | p201703 | 1 | 2023-03-02 17:52:52 | NORMAL | date | [types: [DATE]; keys: [2017-03-01]; ..types: [DATE]; keys: [2017-04-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 1 |
| 15946 | p201704 | 2 | 2023-03-02 17:58:36 | NORMAL | date | [types: [DATE]; keys: [2017-04-01]; ..types: [DATE]; keys: [2017-05-01]; ) | user_id | 5 | 1 | HDD | 9999-12-31 23:59:59 | | NULL | 1.363 KB | false | tag.location.default: 1 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
4 rows in set (0.00 sec)Test start without dynamic partition
-- 1.创建表
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)()
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 2",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "5",
"dynamic_partition.end" = "2"
);
-- 2.下面的start和创建历史分区要打开才行
-- "dynamic_partition.start" = "-3"
-- "dynamic_partition.create_history_partition" = "true"
-- 3.先创建一个动态分区不设置start,看明天是否会删除今天的分区
CREATE TABLE IF NOT EXISTS example_range_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
ENGINE=OLAP
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
PARTITION BY RANGE(`date`)()
DISTRIBUTED BY HASH(`user_id`)
PROPERTIES (
"replication_allocation" = "tag.location.default: 2",
"dynamic_partition.enable" = "true",
"dynamic_partition.time_unit" = "DAY",
"dynamic_partition.prefix" = "p",
"dynamic_partition.buckets" = "5",
"dynamic_partition.end" = "2"
);
-- 4.查看表的分区情况，也就是现在是4-06号，看4-07号的时候是否会创建4-08号，并且删除4-06号的数据（由于上面没有设置dynamic_partition.start的参数，默认保留）
show partitions from example_range_tbl;
mysql> show partitions from example_range_tbl;
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| PartitionId | PartitionName | VisibleVersion | VisibleVersionTime | State | PartitionKey | Range | DistributionKey | Buckets | ReplicationNum | StorageMedium | CooldownTime | RemoteStoragePolicy | LastConsistencyCheckTime | DataSize | IsInMemory | ReplicaAllocation |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
| 341668 | p20230406 | 1 | 2023-04-06 13:27:17 | NORMAL | date | [types: [DATE]; keys: [2023-04-06]; ..types: [DATE]; keys: [2023-04-07]; ) | user_id | 5 | 2 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 2 |
| 341684 | p20230407 | 1 | 2023-04-06 13:27:17 | NORMAL | date | [types: [DATE]; keys: [2023-04-07]; ..types: [DATE]; keys: [2023-04-08]; ) | user_id | 5 | 2 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 2 |
| 341700 | p20230408 | 1 | 2023-04-06 13:27:17 | NORMAL | date | [types: [DATE]; keys: [2023-04-08]; ..types: [DATE]; keys: [2023-04-09]; ) | user_id | 5 | 2 | HDD | 9999-12-31 23:59:59 | | NULL | 0.000 | false | tag.location.default: 2 |
+-------------+---------------+----------------+---------------------+--------+--------------+----------------------------------------------------------------------------+-----------------+---------+----------------+---------------+---------------------+---------------------+--------------------------+----------+------------+-------------------------+
3 rows in set (0.00 sec)
5.查看第二天是否有新的分区产生，和不删除久的分区。
References
One article teaches you how to play with the new function of Apache Doris partition and bucket
https://www.cnblogs.com/tgzhu/p/14751013.html
Data Center Articles