Barman installation and backup PostgreSQL

barman Features

Zero data loss backup. To ensure that users achieve zero data loss in the case of only one backup server.
Cooperation with the backup server. Allowing the backup server is unavailable during replication of the streaming master server, acquires the file from the wal barman.
Reliable monitoring integration. To monitor and report on the creation Barman configuration state.
Parallel backup and recovery. Copy faster execution speed up the entire backup and recovery process.
Backup directory. It allows the user to list on the same screen, hold, delete, archive and restore multiple full backups.
Support for multiple servers. It allows you to manage backup and recovery of multiple servers from the same location.
To store backup data. PostgreSQL to store backup data from multiple instances, support different versions.
Full hot backup. Perform a full hot backup PostgreSQL server from other servers, designed for backup and disaster recovery design.
Incremental Backup and Restore. Modified for the new and different generation of backup data, thus speeding up the processing speed.
Retention policy. Configure a custom backup retention policy to meet the needs of your organization.
Remote recovery. Remote execution PostgreSQL server backup recovery.
Wal file compression. Custom WAL file archiving and compression and backup.
Backup information. Generate regular backups of all disks and use information.
Geographic redundancy. DBA and system administrators to add another layer of protection for disaster recovery solutions PostgreSQL database.

barman installation

192.168.2.2 pg (postgresql server server)

192.168.2.3 barman (barman backup server)

postgresql version 11.2

barman version 2.9.1

yum install postgresql

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/pgdg-centos11-11-2.noarch.rpm
yum install -y postgresql11-server-11.2 postgresql11-contrib-11.2

/usr/pgsql-11/bin/postgresql-11-setup initdb  #执行数据库初始化

Modify the configuration postgres.conf

listen_addresses = '*'
port = 5432
wal_level = replica  #11版本后改为replica archive_mode = on archive_command = 'cp %p /var/lib/pgsql/11/data/pg_archive/%f' max_wal_senders = 10 #建议大于等于3 wal_keep_segments = 64 synchronous_standby_names = '*' hot_standby = on log_filename = 'postgresql-%Y%m%d_%H%M.log'

Modify pg_hba.conf configuration

local   all             all                                       trust
host    all             all               0.0.0.0/0               md5
host    all             all               ::1/128                 trust
host    replication     barman            0.0.0.0/0               md5
host    replication     streaming_barman  0.0.0.0/0               md5

Creating pg_archive, and set up user groups

mkdir /var/lib/pgsql/11/data/pg_archive
chown postgres:postgres /var/lib/pgsql/11/data/pg_archive -R

Start postgresql

systemctl enable postgresql-11
systemctl start postgresql-11

Create a barman and barman_streaming user

su postgres
createuser -s -P barman
#设置密码123456
createuser -P --replication streaming_barman
#设置密码123456

#创建.pgpass
touch ~/.pgpass #添加 pg:5432:*:streaming_barman:123456 pg:5432:*:barman:123456 #更改权限 chmod 600 ~/.pgpass #验证 psql -c 'SELECT version()' -U barman -h pg postgres

yum install barman

As the barman is written in python and postgresql Some features require the use of tools, some will depend on the installation

yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/postgresql11-libs-11.2-1PGDG.rhel7.x86_64.rpm
yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/python2-psycopg2-2.8.2-1.rhel7.x86_64.rpm
yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-2.9-1.rhel7.noarch.rpm
yum install -y https://download.postgresql.org/pub/repos/yum/11/redhat/rhel-7.6-x86_64/barman-cli-2.9-1.rhel7.noarch.rpm

Modify barman common configuration

vi /etc/barman.conf
#修改以下配置
parallel_jobs = 2   #并发数
immediate_checkpoint = true #在执行back时是否会自动执行checkpoint network_compression = true #网络传输压缩

Modify barman private configure postgresql server's

cd /etc/barman.d/
cp streaming-server.conf-template pg.conf
#将pg.conf中的[streaming]修改为[pg]

Adding to 192.168.2.2 pg / etc / hosts of

barman use

1. Create a copy groove

barman receive-wal --create-slot pg

2.barman check

[root@25ff7df1b128 pg]# barman check pg
Server pg:
	PostgreSQL: OK
	is_superuser: OK
	PostgreSQL streaming: OK
	wal_level: OK
	replication slot: OK
	directories: OK
	retention policy settings: OK
	backup maximum age: OK (no last_backup_maximum_age provided) compression settings: OK failed backups: OK (there are 0 failed backups) minimum redundancy requirements: FAILED (have 0 backups, expected at least 1) pg_basebackup: OK pg_basebackup compatible: OK pg_basebackup supports tablespaces mapping: OK pg_receivexlog: OK pg_receivexlog compatible: OK receive-wal running: OK archiver errors: OK

Where the minimum redundancy requirements: FAILED negligible, not all equipment after initial configuration, perform a full backup becomes OK

[root@25ff7df1b128 pg]# barman backup pg
Starting backup using postgres method for server pg in /var/lib/barman/pg/base/20190821T171724 Backup start at LSN: 0/90000D0 (000000010000000000000009, 000000D0) Starting backup copy via pg_basebackup for 20190821T171724 Copy done (time: 2 seconds) Finalising the backup. This is the first backup for server pg WAL segments preceding the current backup have been found: 000000010000000000000008 from server pg has been removed Backup size: 166.6 MiB Backup end at LSN: 0/B000060 (00000001000000000000000B, 00000060) Backup completed (start time: 2019-08-21 17:17:24.956763, elapsed time: 2 seconds) Processing xlog segments from streaming for pg 000000010000000000000009 00000001000000000000000A

First you have configured will have minimum redundancy requirements: FAILED, receive-wal running: FAILED, execution barman cron can become OK, barman performs the following process

[root@25ff7df1b128 pg]# barman cron
Starting WAL archiving for server pg
Starting streaming archiver for server pg Starting check-backup for backup 20190821T172217 of server pg #正常流复制时的进程 [root@25ff7df1b128 pg]# ps -ef|grep barman root 1278 1 0 17:30 ? 00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg root 1281 1278 0 17:30 ? 00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous #执行全备时的进程 [root@25ff7df1b128 ~]# ps -ef|grep barman root 1325 1 1 17:33 ? 00:00:00 /usr/bin/python2 /usr/bin/barman -c /etc/barman.conf -q receive-wal pg root 1327 1325 0 17:33 ? 00:00:00 /usr/pgsql-11/bin/pg_receivewal --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_receive_wal --verbose --no-loop --no-password --directory=/var/lib/barman/pg/streaming --slot=barman --synchronous root 1331 1035 12 17:33 pts/0 00:00:00 /usr/bin/python2 /usr/bin/barman backup pg root 1334 1331 0 17:33 pts/0 00:00:00 /usr/pgsql-11/bin/pg_basebackup --dbname=dbname=replication host=pg options=-cdatestyle=iso replication=true user=streaming_barman application_name=barman_streaming_backup -v --no-password --pgdata=/var/lib/barman/pg/base/20190821T173331/data --no-slot --wal-method=none --checkpoint=fast

Therefore barman is perfect pg_basebackup implemented using increments using pg_receivewal achieved.

barman advantage

1. may be separate from the master and slaver postgresql deployment, but require some tools barman postgresql server installation, such as pg_basebackup,pg_receivewal

2. wal can flow through the synchronization pull copying, if the current wal postgresql not close, barman .partial generates a real-time synchronization file, the file needs to be removed when using the suffix extended recovery

3.barman one server can manage multiple postgresql server, providing local and remote recovery (recovery need to configure the remote ssh-free dense)

4. Provide detailed inquiry detection and backup-related files

Some features barman

1. View the current replication status

[root@25ff7df1b128 ~]# barman replication-status pg
Status of streaming clients for server 'pg': Current LSN on master: 0/130001A8 Number of streaming clients: 1 1. #1 Sync WAL streamer Application name: barman_receive_wal Sync stage : 3/3 Remote write Communication : TCP/IP IP Address : 192.168.2.3 / Port: 33256 / Host: - User name : streaming_barman Current state : streaming (sync) Replication slot: barman WAL sender PID : 1731 Started at : 2019-08-21 17:33:16.495522+08:00 Sent LSN : 0/130001A8 (diff: 0 B) Write LSN : 0/130001A8 (diff: 0 B) Flush LSN : 0/130001A8 (diff: 0 B)

2. A list of backups

[root@25ff7df1b128 ~]# barman list-backup pg
pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 294.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS
pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 64.0 MiB
pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB

3. Review postgresql server

[root@25ff7df1b128 ~]# barman show-server pg
Server pg:
	active: True
	archive_timeout: 3600
	archiver: False
	archiver_batch_size: 0
	backup_directory: /var/lib/barman/pg
	backup_method: postgres
	backup_options: BackupOptions(['concurrent_backup']) bandwidth_limit: None barman_home: /var/lib/barman barman_lock_directory: /var/lib/barman basebackup_retry_sleep: 30 basebackup_retry_times: 3 basebackups_directory: /var/lib/barman/pg/base check_timeout: 30 checkpoint_timeout: 300 compression: None config_file: /var/lib/pgsql/11/data/postgresql.conf connection_error: None conninfo: host=pg user=barman dbname=postgres current_size: 23808077 current_xlog: 000000010000000000000015 custom_compression_filter: None custom_decompression_filter: None data_checksums: off data_directory: /var/lib/pgsql/11/data description: Example of PostgreSQL Database (Streaming-Only) disabled: False errors_directory: /var/lib/barman/pg/errors hba_file: /var/lib/pgsql/11/data/pg_hba.conf hot_standby: on ident_file: /var/lib/pgsql/11/data/pg_ident.conf immediate_checkpoint: True incoming_wals_directory: /var/lib/barman/pg/incoming is_in_recovery: False is_superuser: True last_backup_maximum_age: None max_incoming_wals_queue: None max_replication_slots: 10 max_wal_senders: 10 minimum_redundancy: 1 msg_list: [] name: pg network_compression: False parallel_jobs: 1 passive_node: False path_prefix: /usr/pgsql-11/bin pg_basebackup_bwlimit: True pg_basebackup_compatible: True pg_basebackup_installed: True pg_basebackup_path: /usr/pgsql-11/bin/pg_basebackup pg_basebackup_tbls_mapping: True pg_basebackup_version: 11.2 pg_receivexlog_compatible: True pg_receivexlog_installed: True pg_receivexlog_path: /usr/pgsql-11/bin/pg_receivewal pg_receivexlog_supports_slots: True pg_receivexlog_synchronous: True pg_receivexlog_version: 11.2 pgespresso_installed: False post_archive_retry_script: None post_archive_script: None post_backup_retry_script: None post_backup_script: None post_delete_retry_script: None post_delete_script: None post_recovery_retry_script: None post_recovery_script: None post_wal_delete_retry_script: None post_wal_delete_script: None pre_archive_retry_script: None pre_archive_script: None pre_backup_retry_script: None pre_backup_script: None pre_delete_retry_script: None pre_delete_script: None pre_recovery_retry_script: None pre_recovery_script: None pre_wal_delete_retry_script: None pre_wal_delete_script: None primary_ssh_command: None recovery_options: RecoveryOptions([]) replication_slot: Record(slot_name='barman', active=True, restart_lsn='0/150000C8') replication_slot_support: True retention_policy: None retention_policy_mode: auto reuse_backup: None server_txt_version: 11.2 slot_name: barman ssh_command: None streaming: True streaming_archiver: True streaming_archiver_batch_size: 0 streaming_archiver_name: barman_receive_wal streaming_backup_name: barman_streaming_backup streaming_conninfo: host=pg user=streaming_barman streaming_supported: True streaming_wals_directory: /var/lib/barman/pg/streaming synchronous_standby_names: ['*'] systemid: 6727448561719488729 tablespace_bandwidth_limit: None timeline: 1 wal_compression: off wal_level: replica wal_retention_policy: main wals_directory: /var/lib/barman/pg/wals xlog_segment_size: 16777216 xlogpos: 0/150000C8

4. Remote Execution switch-wal

[root@25ff7df1b128 ~]# barman switch-wal pg
The WAL file 000000010000000000000015 has been closed on server 'pg'

5. Check backupid

[root@25ff7df1b128 ~]# barman list-backup pg
pg 20190821T174304 - Wed Aug 21 17:43:08 2019 - Size: 326.6 MiB - WAL Size: 0 B - WAITING_FOR_WALS
pg 20190821T173331 - Wed Aug 21 17:33:35 2019 - Size: 310.6 MiB - WAL Size: 16.0 MiB - WAITING_FOR_WALS
pg 20190821T172217 - Wed Aug 21 17:22:20 2019 - Size: 230.6 MiB - WAL Size: 80.0 MiB
pg 20190821T171724 - Wed Aug 21 17:17:27 2019 - Size: 198.6 MiB - WAL Size: 32.0 MiB
[root@25ff7df1b128 ~]# barman check-backup pg 20190821T171724 [root@25ff7df1b128 ~]# barman check-backup pg 20190821T171724s ERROR: Unknown backup '20190821T171724s' for server 'pg'

6. View Files

-target parameters include:

data: All data files listed in the directory

standalone: Lists all equipment and documents required wal

wal: List all equipment start to the current time wal file

full: Equal to data + wal

[root@25ff7df1b128 ~]# barman list-files --target wal pg 20190821T173331
/var/lib/barman/pg/wals/0000000100000000/000000010000000000000012
/var/lib/barman/pg/wals/0000000100000000/000000010000000000000013

7. Backup Recovery

#备份恢复
barman recover <server_name> <backup_id> <postgres server data path> #时间点恢复 barman recover <server_name> <backup_id> <postgres server data path> --target-time TARGET_TIME

Also --target-xid TARGET_XIDavailable: --target-lsn TARGET_LSN, --target-name TARGET_NAME, ,--target-immediate

8. Review the backup details

[root@25ff7df1b128 ~]# barman show-backup pg 20190821T173331
Backup 20190821T173331:
  Server Name            : pg
  Status                 : WAITING_FOR_WALS PostgreSQL Version : 110002 PGDATA directory : /var/lib/pgsql/11/data Base backup information: Disk usage : 294.6 MiB (310.6 MiB with WALs) Incremental size : 294.6 MiB (-0.00%) Timeline : 1 Begin WAL : 000000010000000000000012 End WAL : 000000010000000000000012 WAL number : 1 Begin time : 2019-08-21 17:33:31+08:00 End time : 2019-08-21 17:33:35.321271+08:00 Copy time : 3 seconds Estimated throughput : 75.6 MiB/s Begin Offset : 96 End Offset : 0 Begin LSN : 0/12000060 End LSN : 0/13000000 WAL information: No of files : 1 Disk usage : 16.0 MiB WAL rate : 12.58/hour Last available : 000000010000000000000013 Catalog information: Retention Policy : not enforced Previous Backup : 20190821T172217 Next Backup : 20190821T174304

reference: