postgres流复制类似于oracle的goldengate，在PostgreSQL在9.0之后引入了主备流复制机制，通过流复制，备库不断的从主库同步相应的数据，并在备库apply每个WAL record，这里的流复制每次传输单位是WAL日志的record。而PostgreSQL9.0之前提供的方法是主库写完一个WAL日志文件后，才把WAL日志文件传送到备库，这样的方式导致主备延迟特别大。
PG主备流复制的核心部分由walsender，walreceiver和startup三个进程组成。
walsender进程是用来发送WAL日志记录的
walreceiver进程是用来接收WAL日志记录的
startup进程是用来apply日志的
PG主备流复制的核心部分由walsender，walreceiver和startup三个进程组成。
walsender进程是用来发送WAL日志记录的
walreceiver进程是用来接收WAL日志记录的
startup进程是用来apply日志的
现在通过参考网上资料，把流复制的搭建过程重新整理如下：
配置环境
主机名 IP地址 角色
qxy 192.168.40.170 primary
qxy1 192.168.40.171 standby
主数据库配置
1、首先配置replica用户进行主从同步,并且具有登录和备份权限。
[postgres@qxy pgsql]$ psql psql.bin (9.6.4) Type "help" for help. postgres=# CREATE ROLE replica login replication encrypted password 'replica'; CREATE ROLE postgres=#2、修改pg_hba.conf文件添加新增replica用户用于同步
# TYPE DATABASE USER ADDRESS METHOD # "local" is for Unix domain socket connections only local all all trust # IPv4 local connections: host all all 127.0.0.1/32 trust #host all all 192.168.40.1/24 md5 host replication replica 192.168.40.1/24 md5 <====指定192.168.40网段的replica用户都可以连接
3、修改postgresql.conf 文件
wal_level = hot_standby # 备用服务器上增加了运行只读查询所需的信息 max_wal_senders = 5 # 这个设置了可以最多有几个流复制连接 wal_keep_segments = 128 # 设置流复制保留的最多的xlog数目 wal_sender_timeout = 60s # 设置流复制主机发送数据的超时时间
4、启动postgres数据库
[postgres@qxy data]$ pg_ctl stop -D $PGDATA waiting for server to shut down....LOG: received fast shutdown request LOG: aborting any active transactions LOG: autovacuum launcher shutting down LOG: shutting down LOG: database system is shut down done server stopped [postgres@qxy data]$ pg_ctl start -D $PGDATA server starting [postgres@qxy data]$ LOG: database system was shut down at 2018-06-07 12:08:50 GMT LOG: MultiXact member wraparound protections are now enabled LOG: database system is ready to accept connections LOG: autovacuum launcher started
5、备数据库配置
备库执行
使用pg_basebackup建备库
[postgres@QXY1 data]$ pg_basebackup -F p --progress -D /spark/pgsql/data -h 192.168.40.170 -p 5432 -U replica --password Password: 21856/21856 kB (100%), 1/1 tablespace [postgres@QXY1 data]$ [postgres@QXY1 data]$ ls -ltr total 116 drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_stat_tmp drwx------. 4 postgres dba 4096 Jun 7 20:23 pg_logical -rw-------. 1 postgres dba 1636 Jun 7 20:23 pg_ident.conf drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_dynshmem -rw-------. 1 postgres dba 206 Jun 7 20:23 backup_label drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_clog drwx------. 3 postgres dba 4096 Jun 7 20:23 pg_xlog drwx------. 4 postgres dba 4096 Jun 7 20:23 pg_multixact -rw-------. 1 postgres dba 22592 Jun 7 20:23 postgresql.conf -rw-------. 1 postgres dba 4 Jun 7 20:23 PG_VERSION drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_twophase drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_subtrans drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_snapshots drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_serial drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_notify -rw-------. 1 postgres dba 4606 Jun 7 20:23 pg_hba.conf drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_commit_ts drwx------. 5 postgres dba 4096 Jun 7 20:23 base -rw-------. 1 postgres dba 88 Jun 7 20:23 postgresql.auto.conf drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_tblspc drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_stat drwx------. 2 postgres dba 4096 Jun 7 20:23 pg_replslot drwx------. 2 postgres dba 4096 Jun 7 20:23 global [postgres@QXY1 data]$6、修改备库recovery.conf，该文件在$PGHOME/share/postgres,把该文件copy到/spark/pgsql/data下面
[postgres@QXY1 postgresql]$ pwd /spark/pgsql/share/postgresql [postgres@QXY1 postgresql]$ ls -ltr recovery.conf.sample -rw-r--r--. 1 postgres postgres 5587 Aug 9 2016 recovery.conf.sample [postgres@QXY1 postgresql]$ 修改如下内容： standby_mode = on # 指定为从库 primary_conninfo = 'host=192.168.40.170 port=5432 user=replica password=replica' # 对应的主库信息 recovery_target_timeline = 'latest' # 这个说明这个流复制同步到最新的数据
7、修改备库的postgresql.conf中几个参数
max_connections = 200 hot_standby = on max_standby_streaming_delay = 30s wal_receiver_status_interval = 10s hot_standby_feedback = on
8、启动备库
[postgres@QXY1 pgsql]$ FATAL: database files are incompatible with server DETAIL: The data directory was initialized by PostgreSQL version 9.6, which is not compatible with this version 9.4.9. [postgres@QXY1 data]$ /spark/pgsql/pgsql/bin/pg_ctl start -D /spark/pgsql/data/ server starting [postgres@QXY1 data]$ LOG: database system was shut down in recovery at 2018-06-07 14:29:47 GMT LOG: entering standby mode LOG: redo starts at 0/2000028 LOG: consistent recovery state reached at 0/3000000 LOG: database system is ready to accept read only connections LOG: started streaming WAL from primary at 0/3000000 on timeline 1
9、主库查询replicate状态
[postgres@qxy spark]$ psql psql.bin (9.6.4) Type "help" for help. postgres=# \x Expanded display is on. postgres=# select * from pg_stat_replication ; -[ RECORD 1 ]----+------------------------------ pid | 5382 usesysid | 16385 usename | replica application_name | walreceiver client_addr | 192.168.40.171 client_hostname | client_port | 58399 backend_start | 2018-06-07 14:33:04.238161+00 backend_xmin | 548 state | streaming sent_location | 0/3001720 write_location | 0/3001720 flush_location | 0/3001720 replay_location | 0/3001720 sync_priority | 0 sync_state | async <======异步方式 postgres=#10、修改主库postgresql.conf配置文件
synchronous_standby_names = 'standby' # standby servers that provide sync rep11、修改备库recovery.conf配置文件
primary_conninfo = 'application_name=standby host=192.168.40.170 port=5432 user=replica password=replica' # 对应的主库信息12、重启主库和备库
主库查看replicate状态
postgres=# select * from pg_stat_replication ; -[ RECORD 1 ]----+------------------------------ pid | 5482 usesysid | 16385 usename | replica application_name | standby client_addr | 192.168.40.171 client_hostname | client_port | 58404 backend_start | 2018-06-07 14:45:09.909992+00 backend_xmin | 548 state | streaming sent_location | 0/3001988 write_location | 0/3001988 flush_location | 0/3001988 replay_location | 0/3001988 sync_priority | 1 sync_state | sync <=====更新为同步 postgres=#13、测试同步情况
主库创建一个测试表T postgres=# postgres=# create table t(id int); CREATE TABLE postgres=# insert into t values (10); INSERT 0 1 postgres=# 备库查询 [postgres@QXY1 ~]$ psql psql.bin (9.6.4) Type "help" for help. postgres=# postgres=# postgres=# select * from t; id ---- 10 (1 row) postgres=#
14、对应后台进程
主库 [postgres@qxy data]$ ps -ef |grep postgres root 3820 2904 0 11:14 pts/0 00:00:00 su - postgres postgres 3821 3820 0 11:14 pts/0 00:00:00 -bash postgres 5473 1 0 14:45 pts/0 00:00:00 /spark/pgsql/pgsql/bin/postgres -D /spark/pgsql/data postgres 5476 5473 0 14:45 ? 00:00:00 postgres: checkpointer process postgres 5477 5473 0 14:45 ? 00:00:00 postgres: writer process postgres 5478 5473 0 14:45 ? 00:00:00 postgres: wal writer process postgres 5479 5473 0 14:45 ? 00:00:00 postgres: autovacuum launcher process postgres 5480 5473 0 14:45 ? 00:00:00 postgres: stats collector process postgres 5482 5473 0 14:45 ? 00:00:00 postgres: wal sender process replica 192.168.40.171(58404) streaming 0/3017528 postgres 5516 3821 17 14:48 pts/0 00:00:00 ps -ef postgres 5517 3821 0 14:48 pts/0 00:00:00 grep postgres [postgres@qxy data]$ 备库 [postgres@QXY1 ~]$ ps -ef |grep postgres root 2564 2019 0 20:49 pts/0 00:00:00 su - postgres postgres 2565 2564 0 20:49 pts/0 00:00:00 -bash root 2922 2901 0 22:29 pts/1 00:00:00 su - postgres postgres 2923 2922 0 22:29 pts/1 00:00:00 -bash postgres 3033 1 0 22:43 pts/0 00:00:00 /spark/pgsql/pgsql/bin/postgres -D /spark/pgsql/data postgres 3034 3033 0 22:43 ? 00:00:00 postgres: startup process recovering 000000010000000000000003 postgres 3035 3033 0 22:43 ? 00:00:00 postgres: checkpointer process postgres 3036 3033 0 22:43 ? 00:00:00 postgres: writer process postgres 3037 3033 0 22:43 ? 00:00:00 postgres: stats collector process postgres 3049 3033 0 22:44 ? 00:00:00 postgres: wal receiver process streaming 0/3017528 postgres 3061 2565 0 22:49 pts/0 00:00:00 ps -ef postgres 3062 2565 0 22:49 pts/0 00:00:00 grep postgres [postgres@QXY1 ~]$