pg_receivewal实践

测试从pg_receivewal的日志中恢复从库为主库:

主从配置async模式,配置pg_receivewal接收日志pg_receivewal -D /dbaas/pg/data/pg_receivewal_data -v -h 10.9.10.202

主插入1000万数据,当插入一半时,停止从库

主库插完数据,停止

将pg_receivewal的日志拷贝到从库/dbaas/pg/data/pg_receivewal_data下

修改从库recovery.conf文件,添加内容:

restore_command = 'cp /dbaas/pg/data/pg_receivewal_data/%f "%p"'

启动数据库,数据库成功恢复到10000000行。

另外:

restore_command = 'cp /dbaas/pg/data/pg_receivewal_data/%f "%p"'

standby_mode = on

primary_conninfo = 'host=10.9.10.203 port=5432 user=postgres connect_timeout=60'

recovery_target_timeline = 'latest'

trigger_file = '/dbaas/pg/data/.tfile'

这种把恢复和备库设置放一起,是不会先去从归档恢复,会直接当备库启动。

测试pg_receivewal的同步机制:

-S slotname

--slot=slotname

Require pg_receivewal to use an existing replication slot (see Section 26.2.6). When this option is used, pg_receivewal will report a flush position to the server, indicating when each segment has been synchronized to disk so that the server can remove that segment if it is not otherwise needed.

When the replication client of pg_receivewal is configured on the server as a synchronous standby, then using a replication slot will report the flush position to the server, but only when a WAL file is closed. Therefore, that configuration will cause transactions on the primary to wait for a long time and effectively not work satisfactorily. The option --synchronous (see below) must be specified in addition to make this work correctly.

--synchronous

Flush the WAL data to disk immediately after it has been received. Also send a status packet back to the server immediately after flushing, regardless of --status-interval.

This option should be specified if the replication client of pg_receivewal is configured on the server as a synchronous standby, to ensure that timely feedback is sent to the server.

在这个说明中,使用了--slot=for_pgreceivewal --synchronous后,理论上是有点像sync同步一样,数据库会等待接收端进行回放,回放结束,再进行commit。

1)使slotsynchronous参数:

数据库端:

修改postgresql.conf参数synchronous_standby_name=‘pg_receivewal’

重启数据库

postgres=# select pg_create_physical_replication_slot('for_pgreceivewal');

 pg_create_physical_replication_slot

-------------------------------------

 (for_pgreceivewal,)

(1 row)

postgres=# select pg_get_replication_slots();

        pg_get_replication_slots

----------------------------------------

 (for_pgreceivewal,,physical,,f,f,,,,,)

(1 row)

postgres=# select pg_get_replication_slots();

                pg_get_replication_slots

--------------------------------------------------------

 (for_pgreceivewal,,physical,,f,t,629488,,,0/9DD77B00,)

(1 row)

接收端:

pg_receivewal -D /dbaas/pg/data/pg_receivewal_data --slot=for_pgreceivewal --synchronous -v -h 10.9.10.202

数据库端:

postgres=# select * from pg_stat_replication ;

pid   | usesysid | usename  | application_name | client_addr | client_hostname | client_port |         backend_start         | backend_xmin |   state   |  sent_lsn  | write_lsn  | flush_lsn  | replay_lsn |    write_lag    |    flush_lag    |   replay_lag    | sync_priority | sync_state

--------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+--------------+-----------+------------+------------+------------+------------+-----------------+-----------------+-----------------+---------------+------------

 912703 |       10 | postgres | pg_receivewal    | 10.9.10.203 |                 |       63103 | 2019-05-15 02:52:30.029814+00 |              | streaming | 0/D9E44528 | 0/D9E44528 | 0/D9E44528 |            | 00:00:00.010972 | 00:00:00.010972 | 00:06:49.964729 |             1 | sync

(1 row)

在数据库端,使用pgbench测试:

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf /dbaas/pg/data]$ pgbench -c 10 -j 5 -r -T 100 pgbench

starting vacuum...end.

transaction type: <builtin: TPC-B (sort of)>

scaling factor: 100

query mode: simple

number of clients: 10

number of threads: 5

duration: 100 s

number of transactions actually processed: 15290

latency average = 65.426 ms

tps = 152.845128 (including connections establishing)

tps = 152.855112 (excluding connections establishing)

statement latencies in milliseconds:

         0.002  \set aid random(1, 100000 * :scale)

         0.001  \set bid random(1, 1 * :scale)

         0.001  \set tid random(1, 10 * :scale)

         0.000  \set delta random(-5000, 5000)

         0.046  BEGIN;

         0.300  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;

         0.111  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;

         0.417  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;

         2.878  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;

         0.092  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);

        61.539  END;

2)不使用slotsynchronous参数

数据库端,删除slot

postgres=# select pg_drop_replication_slot('for_pgreceivewal');

 pg_drop_replication_slot

--------------------------

(1 row)

postgres=# select pg_get_replication_slots();

 pg_get_replication_slots

--------------------------

(0 rows)

接收端:

pg_receivewal -D /dbaas/pg/data/pg_receivewal_data  -v -h 10.9.10.202

压测:

[postgres@90027282-dd74-4642-b5dd-359b6cc67aaf ~]$ pgbench -c 10 -j 5 -r -T 100 pgbench

starting vacuum...end.

transaction type: <builtin: TPC-B (sort of)>

scaling factor: 100

query mode: simple

number of clients: 10

number of threads: 5

duration: 100 s

number of transactions actually processed: 27015

latency average = 37.021 ms

tps = 270.120226 (including connections establishing)

tps = 270.138206 (excluding connections establishing)

statement latencies in milliseconds:

         0.002  \set aid random(1, 100000 * :scale)

         0.001  \set bid random(1, 1 * :scale)

         0.001  \set tid random(1, 10 * :scale)

         0.000  \set delta random(-5000, 5000)

         0.042  BEGIN;

         0.242  UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid;

         0.110  SELECT abalance FROM pgbench_accounts WHERE aid = :aid;

         0.291  UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid;

         1.778  UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid;

         0.091  INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP);

        34.396  END;

同步模式下,停止pg_receivewal,会导致主hang住:

postgres=# create table test(id int, info varchar(8), crt_time timestamp);

^CCancel request sent

WARNING:  canceling wait for synchronous replication due to user request

DETAIL:  The transaction has already committed locally, but might not have been replicated to the standby.

CREATE TABLE

pg_receivewal的使用场景思考:

1)当配置高可用集群时,一个备库很危险,容易出现单点故障,当同步节点死掉后,主节点会hang住(这是没有专业高可用工具的情况下,ecox这种高可用工具,会自动处理为单节点可写)。而部署两个同步节点又会浪费资源(不做读写分离的情况下),这个时候,可以创建一个pg_receivewal节点,使用同步模式,当一个同步节点停止后,它可以充当sync节点,给主节点报告LSN。

为了减少网络开销,还可以将pg_receivewal节点设置为主库本地,即起到了归档作用,又起到了同步节点的功能。唯一注意的是,要定期清理日志空间。

2)第二种情况,数据库集群不使用sync模式,想使用更加高性能的async模式。这时,使用pg_receivewal来做日志归档,至少比aync节点同步的日志要多。这样基于他的恢复,也将丢更少的数据。当数据库更新插入极端的大时,pg_receivewal归档的日志,可能还没有归档命令归档的多。归档是到本地,如果服务器挂了,那么归档也没有了,可以使用pgbackrest或者rync工具将归档同步到其他服务器。

接下来要思考的场景:

那如果遇到一个落后很久的备库,想不重建的情况下恢复。有归档日志,我们应该怎么做?

1)先从归档恢复,再建立同步?不行,恢复后,日志线就+1了,不能再和主建立同步了。

2)将归档日志,从库缺少的地方开始,拷贝到主库pg_wal下,再进行启动从库。待验证。

猜你喜欢

转载自www.cnblogs.com/kuang17/p/10868556.html