表分区简介2

文档用途

了解Postgresql表分区

详细信息

Postgreslq内核中支持表分区（Table partitioning）包括：范围（range）、列表（list）。而对于其他分区类型及复杂分区操作。PG如何支持呢？扩展插件pg_pathman可用于支持复杂发杂分区操作。pg_pathman支持范围（range）、哈希（hash）及分区的分裂、合并等复杂操作

1、安装配置

$ git clone https://github.com/postgrespro/pg_pathman
$ export PATH=/home/digoal/pgsql9.6:$PATH

$ cd pg_pathman
$ make USE_PGXS=1
$ make USE_PGXS=1 install

$ cd $PGDATA
$ vi postgresql.conf
shared_preload_libraries = 'pg_pathman,pg_stat_statements'

$ pg_ctl restart -m fast

$ psql
postgres=# create extension pg_pathman;
CREATE EXTENSION

注：

Hgdb561自带pg_pathman

直接配置使用

# alter system set shared_preload_libraries = 'pg_pathman';

$ pg_ctl restart -m fast

$ psql
postgres=# create extension pg_pathman;
CREATE EXTENSION

2、hash分区

hash分区

建立测试表

CREATE TABLE items (

id SERIAL PRIMARY KEY,

name TEXT,

code BIGINT);

插入测试数据

INSERT INTO items (id, name, code)

SELECT g, md5(g::text), random() * 100000

FROM generate_series(1, 10000) as g;

分区并迁移

SELECT create_hash_partitions('items', 'id', 10);

查询

SELECT * FROM items WHERE id = 1234;

id | name | code

------+----------------------------------+-------

1234 | 81dc9bdb52d04dc20036dbd8313ed055 | 87938

(1 row)

EXPLAIN SELECT * FROM items WHERE id = 1234;

QUERY PLAN

-------------------------------------------------------------------------------------

Append (cost=0.28..2.50 rows=1 width=44)

-> Index Scan using items_11_pkey on items_11 (cost=0.28..2.50 rows=1 width=44)

Index Cond: (id = 1234)

(3 rows)

3、范围分区

建立测试表：

CREATE TABLE test_range_pathman (

id SERIAL,

dt TIMESTAMP NOT NULL,

level INTEGER,

msg TEXT);

CREATE INDEX ON test_range_pathman(dt);

插入测试数据：

INSERT INTO test_range_pathman (dt, level, msg) SELECT g, random() * 6,

md5(g::text) FROM generate_series('2019-01-01'::date, '2019-12-31'::date, '1 minute') as g;

INSERT INTO test_range_pathman (dt, level, msg) SELECT g, random() * 6,

md5(g::text) FROM generate_series('2019-01-01'::date, '2019-12-31'::date, '60 minute') as g;

创建分区表：

SELECT create_range_partitions(

'test_range_pathman',--主表名

'dt', --分区字段

'2019-01-01'::date, --分区起始日期

'1 month'::interval, --分区间隔

null, --不指定分区数量，根据时间与间隔会自动计算出数量

false --默认tue立即迁移数据，false是不迁移数据

);

查看数据：

只统计主表数据量（分区，但数据未迁移）

select count(*) from only test_range_pathman;

count

--------

524161

(1 row)

非堵塞式数据迁移，并查看数据：

使用非堵塞式的迁移接口
partition_table_concurrently(relation   REGCLASS,              -- 主表OID
                             batch_size INTEGER DEFAULT 1000, -- 一个事务批量迁移多少记录
                             sleep_time FLOAT8 DEFAULT 1.0)    -- 获得行锁失败时，休眠多久再次获取，重试60次退出任务。

select partition_table_concurrently('test_range_pathman',10000,1.0);

select count(*) from only test_range_pathman;

count

-------

(1 row)

#父表中数据已经为0，迁移全部完毕

数据迁移完成后，建议禁用主表，这样执行计划就不会出现主表了
postgres=# select set_enable_parent('part_test'::regclass, false);

#查看子表数据

postgres=> select * from test_range_pathman_10 limit 10;

id | dt | level | msg

------+---------------------+-------+----------------------------------

6553 | 2019-10-01 00:00:00 | 5 | eee5c09b0e683d62ce1fbfb591963341

6554 | 2019-10-01 01:00:00 | 5 | d65d91ca0d52f1b8b79a7621498030cf

6555 | 2019-10-01 02:00:00 | 4 | 3f5c4e0bb66847e219f5943b7fdd8a02

6556 | 2019-10-01 03:00:00 | 0 | 389176c45d5e9bcdd37db5ec441d9542

6557 | 2019-10-01 04:00:00 | 3 | 174240f01991d66ccf38a2322d0e01e2

6558 | 2019-10-01 05:00:00 | 3 | d0b1194e3ed234759d83913809a2bb3c

6559 | 2019-10-01 06:00:00 | 3 | 64e716bd201ff17e2f9edb597f2bb005

6560 | 2019-10-01 07:00:00 | 5 | d588ec8ca1f9c894cffd2beea0de9734

6561 | 2019-10-01 08:00:00 | 5 | d8f5b8b84bba3b05466ffb6526d3378a

6562 | 2019-10-01 09:00:00 | 1 | 64f81817c1b7d620a98c3ca95d90700f

(10 rows)

查看分区表执行计划：

explain select * from journal where dt between '2019-03-29 06:00:00' and '2019-03-29 10:00:00' ;

QUERY PLAN

-------------------------------------------------------------------------------------------------------------------------------------------------

Append (cost=0.00..11.61 rows=242 width=49)

-> Seq Scan on journal (cost=0.00..0.00 rows=1 width=49)

Filter: ((dt >= '2019-03-29 06:00:00'::timestamp without time zone) AND (dt <= '2019-03-29 10:00:00'::timestamp without time zone))

-> Index Scan using journal_88_dt_idx on journal_88 (cost=0.28..10.40 rows=241 width=49)

Index Cond: ((dt >= '2019-03-29 06:00:00'::timestamp without time zone) AND (dt <= '2019-03-29 10:00:00'::timestamp without time zone))

(5 rows)

注意:

分区列必须有not null约束
分区个数必须能覆盖已有的所有记录

4、分区表高级管理

自动扩展分区

范围分区表，允许自动扩展分区。

如果新插入的数据不在已有的分区范围内，会自动创建分区。

set_auto(relation REGCLASS, value BOOLEAN)

Enable/disable auto partition propagation (only for RANGE partitioning).

It is enabled by default.

例子

postgres=> \d+ test_range_pathman;

Table "test.test_range_pathman"

Description

--------+-----------------------------+-----------+----------+------------------------------------------------+----------+--------------+-

------------

Indexes:

"test_range_pathman_dt_idx" btree (dt)

Child tables: test_range_pathman_1,

test_range_pathman_10,

test_range_pathman_11,

test_range_pathman_12,

test_range_pathman_13,

test_range_pathman_2,

test_range_pathman_3,

test_range_pathman_4,

test_range_pathman_5,

test_range_pathman_6,

test_range_pathman_7,

test_range_pathman_8,

test_range_pathman_9

postgres=> \d+ test_range_pathman_13;

Table "test.test_range_pathman_13"

Description

--------+-----------------------------+-----------+----------+------------------------------------------------+----------+--------------+-

------------

Indexes:

"test_range_pathman_13_dt_idx" btree (dt)

Check constraints:

"pathman_test_range_pathman_13_check" CHECK (dt >= '2020-01-01 00:00:00'::timestamp without time zone AND dt < '2020-02-01 00:00:00'::

timestamp without time zone)

Inherits: test_range_pathman

postgres=>

postgres=> select * from test_range_pathman limit 10;

id | dt | level | msg

----+---------------------+-------+----------------------------------

1 | 2019-01-01 00:00:00 | 2 | e79ef3f3199f43435b8b002edf493c06

2 | 2019-01-01 01:00:00 | 3 | 6fe4e5108fae8be678353f4eb91926f3

3 | 2019-01-01 02:00:00 | 4 | 5df5f73b9ec0e9639a4fabf1162ca25f

4 | 2019-01-01 03:00:00 | 2 | ec19a5e22418e26492adaf1e52abb213

5 | 2019-01-01 04:00:00 | 4 | daef3e76471068995b49ef559a5363ad

6 | 2019-01-01 05:00:00 | 2 | cb2188eb040b27b0e9318c138d26f882

7 | 2019-01-01 06:00:00 | 3 | 6ef7f60f6104d17488ccdd72d3f77061

8 | 2019-01-01 07:00:00 | 0 | b89bddb8988fc0ff36800677852fdc61

9 | 2019-01-01 08:00:00 | 5 | 2e6c897cd130377fffc24c1989803894

10 | 2019-01-01 09:00:00 | 4 | 26a82826be29d79c4bc1bf9892524798

(10 rows)

更多详细信息请登录【瀚高技术支持平台】查看

PostgreSQL_HighGoDB

发布了399 篇原创文章 · 获赞 108 · 访问量 65万+

他的留言板关注

猜你喜欢