PG backup and recovery (1)

1. Logical backup

1.1 Logical backup tool

PG provides pg_dump and pg_dumpall for logical backup. The difference is that pg_dumpall can only dump all data sets of the database into a script file, while pg_dump can choose to specify the database for logical backup. Both can ensure the consistency of the dump data and do not block the normal read and write operations of other sessions.

1. The basic syntax of pg_dump

1) Basic grammar

pg_dump [connection_option] [option] [dbname]

connection_option

parameter	meaning
-h/–host	Host
-p/–port	Port
-U	Username username
-w	Never prompt for a password. If the server request requires password authentication and the password cannot be obtained by other means, the command connection will fail.
-W	Force pg_dump to prompt for a password before connecting to a database, generally you don’t need to pay attention to this parameter
–role	If the authentication user authority is not enough, this command can guarantee to switch to the role of the corresponding authority
dbname	Specify logical backup database

option

parameter	meaning
-a/–data-only	Only output data, do not output table definition SQL statements, only meaningful for plain text format
-b/–blob	Whether dump contains large objects, this parameter is turned on by default
-c/–clean	Whether to include the cleanup object sentence, it is only meaningful for the plain text format
-C/–create	Whether to output the create database statement, only meaningful for the plain text format
-E/–encoding	Specify the character set encoding format
-f/–file	Output to specified file
-F/–format	p, c, d, t four formats, the details are as follows
-n/–schema	Only dump objects that meet the matching schema, multiple patterns can be specified, and wildcard matching can also be used
-N/–exclude-schema	Do not dump objects in the specified schema that meet the matching conditions
-O/–no-owner	Indicates that the object ownership is not set to the owner in the source database
-s/–schema-only	Only export the object definition, not the data
-t/–table	Only dump the tables, views, and sequences that meet the match
-T/–exclude-table	Do not dump tables that meet the matching conditions
-Z/–compress=0…9	Compression level, 0 means no compression. No compression by default
–insert	Dump the data into SQL statement format, the recovery speed under this parameter will be very slow
–column-inserts	Dump into the format of Insert into table(col,...) values, avoiding the inconsistency of the source and target table column order under -insert, resulting in data recovery errors
–lock-wait-timeout	Specify the timeout period for waiting to acquire the shared table lock during dump
–no-tablespaces	Dump data does not output the specified table space, which means that the restored data uses the default table space

pg_dump supports 4 formats

• p, plain：

The default format, the backup output is readable text text. When restoring, you can directly execute the SQL of the backup text in the database.

• c, custom

Customizable archive format, and data compression is turned on by default. When restoring, you can adjust the restore order of the backup objects and support the selection of objects to be restored.

The backup is written to a file. You need to pay attention to the size of a single file supported by the file system.

This format must be restored using the pg_restore command.

• d, directory

The catalog archive format, similar to the custom format, needs to be restored with pg_restore. However, a catalog will be created in the catalog archive format, and then each table or each large object corresponds to a backup output file.

With the TOC file name describing the details of the backup, this format supports compression by default and also supports parallel export.

• t, tar

The tar archive format does not support compression. At the same time, the maximum limit of each table cannot exceed 8GB.It also needs to use pg_restore to restore.

2. Examples of backup commands

• Back up all objects in the specified database

# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1 -f db1.sql

• Back up the specified table under the specified database

# 备份public下的t1、t2表
# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1  -n public -t t1 -t t2

# 备份public下的满足t1*的所有表
# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1  -n public -t 't1*'

• Dump logical backups into custom-format archive files

# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1  -Fc -f db1.dump

• Archive logical backups through directory-format, and start certain concurrent threads

# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1  -Fd -j 5 -f ./aa

# cd aa/
# ll
total 16
-rw-r--r-- 1 root root   35 Sep  3 22:44 3107.dat.gz
-rw-r--r-- 1 root root   35 Sep  3 22:44 3108.dat.gz
-rw-r--r-- 1 root root   25 Sep  3 22:44 3109.dat.gz
-rw-r--r-- 1 root root 3887 Sep  3 22:44 toc.dat

1.2 Logical backup and recovery

PG provides the pg_restore command to logically restore the data dumped by pg_dump. For SQL scripts can be restored through psql

1. The basic syntax of pg_dump

pg_restore [connection_option] [option] [filename]

connection_option is similar to pg_dump, except that pg_restore needs to use -d dbname to specify when restoring a specific database.

options

parameter	meaning
filename	Backup files that need to be restored
-a/–no-data	Only restore data, not data definition
-c/–clean	Clear the corresponding data before restoring the data
-C/–create	Create before restoring data
-d/–dbname	Specify to restore to a specific database
-e/–exit-on-error	Indicates that the recovery database will exit when it encounters an error. The default error will continue and an error count will be displayed eventually
-F/–format	Specify the format of the restored backup file. Generally speaking, pg_restore will judge by itself. If you need to specify it, you can specify t, d, c
-I/–index	Restore only named indexes
-j	Open multiple concurrency for recovery
-n namespaces or --schema	Only restore the schema or table data with the specified name, can be used with -t
-O/–no-owner	The default recovery object is not to specify the owner
–no-tablespaces	Restored data are restored to the default tablespace
-P	Restore only specified functions
-s/–schema-only	Restore only the specified table structure
-t/–table	Restore only the specified table
-T/–trigger	Restore only specified triggers
-f	Specify recovery file

2. psql restores the specified SQL script

# psql -U postgres -p 5433 -h 127.0.0.1 -d db1 -f /root/db1.sql


postgres=# \c db1
psql (12.4, server 12.2)
You are now connected to database "db1" as user "postgres".
db1=# \d
            List of relations
 Schema |   Name    |   Type   |  Owner
--------+-----------+----------+----------
 public | t1        | table    | postgres
 public | t1_id_seq | sequence | postgres
 public | t2        | table    | postgres
 public | v_t1      | view     | postgres
(4 rows)

3. pg_restore usage example

• Restore the data under format=c

# pg_dump -U postgres -p 5432 -h 127.0.0.1 db1  -Fc -f db1.dump
# pg_restore -U postgres -p 5433 -h 127.0.0.1 -d postgres -C  /root/db1.dump        //-C指定创建对应的dbname，若不使用-C则会恢复带-d指定数据库下

#  psql -U postgres -p 5433 -h127.0.0.1 postgres
psql (12.4, server 12.2)
Type "help" for help.

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges
-----------+----------+----------+-------------+-------------+-----------------------
 db1       | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(4 rows)

postgres=# \c db1
psql (12.4, server 12.2)
You are now connected to database "db1" as user "postgres".
db1=# \d
            List of relations
 Schema |   Name    |   Type   |  Owner
--------+-----------+----------+----------
 public | t1        | table    | postgres
 public | t1_id_seq | sequence | postgres
 public | t2        | table    | postgres
 public | v_t1      | view     | postgres
(4 rows)

• Restore the dump file to new_dbname

# pg_restore -U postgres -p 5433 -h 127.0.0.1 -d db2  /root/db1.dump        //需要提前创建好new_dbname


db1=# \c db2
psql (12.4, server 12.2)
You are now connected to database "db2" as user "postgres".
db2=# \d
            List of relations
 Schema |   Name    |   Type   |  Owner
--------+-----------+----------+----------
 public | t1        | table    | postgres
 public | t1_id_seq | sequence | postgres
 public | t2        | table    | postgres
 public | v_t1      | view     | postgres
(4 rows)

• Restore the specified table

# pg_restore -U postgres -p 5433 -h 127.0.0.1 -d postgres -n public -t t1  -C /root/db1.dump

• Restore the backup file in the -d format to the database of some objects that already exist

# pg_restore -U postgres -p 5433 -h 127.0.0.1 -d db1  -c  /root/aa/     //-c表示先做清理后恢复