逻辑备份主要是使用pg_dump、pg_dumpall命令实现。其中pg_dumpall可以备份全局元数据，比如：用户密码、表空间等。pg_dump备份格式可以为二进制、文本等，并且在备份的过程中可以压缩。copy命令可以把表的数据备份出来，在对某个表快速备份的时候，可以使用。
1、copy命令使用
postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+---------+-------+-----------------------
postgres | postgres | UTF8 | C | C |
qxy | postgres | UTF8 | C | C |
template0 | postgres | UTF8 | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | C | C | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+---------+-------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | C | C | | 7071 kB | pg_default | default administrative connection database
qxy | postgres | UTF8 | C | C | | 7079 kB | tblsp1 |
template0 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
postgres=# \c qxy scott
You are now connected to database "qxy" as user "scott".
qxy=# select * from t;
id | name
----+----------------------------------
1 | 900150983cd24fb0d6963f7d28e17f72
2 | 900150983cd24fb0d6963f7d28e17f72
3 | 900150983cd24fb0d6963f7d28e17f72
4 | 900150983cd24fb0d6963f7d28e17f72
5 | 900150983cd24fb0d6963f7d28e17f72
6 | 900150983cd24fb0d6963f7d28e17f72
7 | 900150983cd24fb0d6963f7d28e17f72
8 | 900150983cd24fb0d6963f7d28e17f72
9 | 900150983cd24fb0d6963f7d28e17f72
10 | 900150983cd24fb0d6963f7d28e17f72
(10 rows)
qxy=# copy (select * from t) to stdout; <=====copy到标准输出
1 900150983cd24fb0d6963f7d28e17f72
2 900150983cd24fb0d6963f7d28e17f72
3 900150983cd24fb0d6963f7d28e17f72
4 900150983cd24fb0d6963f7d28e17f72
5 900150983cd24fb0d6963f7d28e17f72
6 900150983cd24fb0d6963f7d28e17f72
7 900150983cd24fb0d6963f7d28e17f72
8 900150983cd24fb0d6963f7d28e17f72
9 900150983cd24fb0d6963f7d28e17f72
10 900150983cd24fb0d6963f7d28e17f72
qxy=# copy (select * from t) to '/spark/pgsql/data_test/t.txt'; <=====输出到具体文件
COPY 10
qxy=#
qxy=# \q
[postgres@qxy pg_clog]$ cat /spark/pgsql/data_test/t.txt
1 900150983cd24fb0d6963f7d28e17f72
2 900150983cd24fb0d6963f7d28e17f72
3 900150983cd24fb0d6963f7d28e17f72
4 900150983cd24fb0d6963f7d28e17f72
5 900150983cd24fb0d6963f7d28e17f72
6 900150983cd24fb0d6963f7d28e17f72
7 900150983cd24fb0d6963f7d28e17f72
8 900150983cd24fb0d6963f7d28e17f72
9 900150983cd24fb0d6963f7d28e17f72
10 900150983cd24fb0d6963f7d28e17f72
[postgres@qxy pg_clog]$
2、pg_dump命令的使用
[postgres@qxy pg_clog]$ pg_dump --help
pg_dump dumps a database as a text file or to other formats.
Usage:
pg_dump [OPTION]... [DBNAME]
General options:
-f, --file=FILENAME output file or directory name
-F, --format=c|d|t|p output file format (custom, directory, tar,
plain text (default))
-j, --jobs=NUM use this many parallel jobs to dump
-v, --verbose verbose mode
-V, --version output version information, then exit
-Z, --compress=0-9 compression level for compressed formats
--lock-wait-timeout=TIMEOUT fail after waiting TIMEOUT for a table lock
-?, --help show this help, then exit
Options controlling the output content:
-a, --data-only dump only the data, not the schema
-b, --blobs include large objects in dump
-c, --clean clean (drop) database objects before recreating
-C, --create include commands to create database in dump
-E, --encoding=ENCODING dump the data in encoding ENCODING
-n, --schema=SCHEMA dump the named schema(s) only
-N, --exclude-schema=SCHEMA do NOT dump the named schema(s)
-o, --oids include OIDs in dump
-O, --no-owner skip restoration of object ownership in
plain-text format
-s, --schema-only dump only the schema, no data
-S, --superuser=NAME superuser user name to use in plain-text format
-t, --table=TABLE dump the named table(s) only
-T, --exclude-table=TABLE do NOT dump the named table(s)
-x, --no-privileges do not dump privileges (grant/revoke)
--binary-upgrade for use by upgrade utilities only
--column-inserts dump data as INSERT commands with column names
--disable-dollar-quoting disable dollar quoting, use SQL standard quoting
--disable-triggers disable triggers during data-only restore
--enable-row-security enable row security (dump only content user has
access to)
--exclude-table-data=TABLE do NOT dump data for the named table(s)
--if-exists use IF EXISTS when dropping objects
--inserts dump data as INSERT commands, rather than COPY
--no-security-labels do not dump security label assignments
--no-synchronized-snapshots do not use synchronized snapshots in parallel jobs
--no-tablespaces do not dump tablespace assignments
--no-unlogged-table-data do not dump unlogged table data
--quote-all-identifiers quote all identifiers, even if not key words
--section=SECTION dump named section (pre-data, data, or post-data)
--serializable-deferrable wait until the dump can run without anomalies
--snapshot=SNAPSHOT use given snapshot for the dump
--strict-names require table and/or schema include patterns to
match at least one entity each
--use-set-session-authorization
use SET SESSION AUTHORIZATION commands instead of
ALTER OWNER commands to set ownership
Connection options:
-d, --dbname=DBNAME database to dump
-h, --host=HOSTNAME database server host or socket directory
-p, --port=PORT database server port number
-U, --username=NAME connect as specified database user
-w, --no-password never prompt for password
-W, --password force password prompt (should happen automatically)
--role=ROLENAME do SET ROLE before dump
If no database name is supplied, then the PGDATABASE environment
variable value is used.
Report bugs to <[email protected]>.
[postgres@qxy pg_clog]$
备份qxy数据库
[postgres@qxy data_test]$
[postgres@qxy data_test]$ mkdir -p /spark/pgsql/data_test/backup <====创建备份目录
[postgres@qxy data_test]$ cd
[postgres@qxy ~]$ pg_dump -f /spark/pgsql/data_test/backup/qxy.dmp -F c -C -h 192.168.40.170 -U scott qxy
pg_dump: [archiver (db)] connection to database "qxy" failed: could not connect to server: Connection refused
Is the server running on host "192.168.40.170" and accepting
TCP/IP connections on port 5432?
[postgres@qxy ~]$ pg_dump -f /spark/pgsql/data_test/backup/qxy.dmp -F c -C -h 192.168.40.170 -U scott qxy -p 5433
Password:
[postgres@qxy ~]$
[postgres@qxy backup]$ ls -ltr
total 4
-rw-rw-r--. 1 postgres postgres 2153 Jun 10 04:10 qxy.dmp
[postgres@qxy backup]$
3、pg_restore命令的使用
[postgres@qxy ~]$ pg_restore --help
pg_restore restores a PostgreSQL database from an archive created by pg_dump.
Usage:
pg_restore [OPTION]... [FILE]
General options:
-d, --dbname=NAME connect to database name
-f, --file=FILENAME output file name
-F, --format=c|d|t backup file format (should be automatic)
-l, --list print summarized TOC of the archive
-v, --verbose verbose mode
-V, --version output version information, then exit
-?, --help show this help, then exit
Options controlling the restore:
-a, --data-only restore only the data, no schema
-c, --clean clean (drop) database objects before recreating
-C, --create create the target database
-e, --exit-on-error exit on error, default is to continue
-I, --index=NAME restore named index
-j, --jobs=NUM use this many parallel jobs to restore
-L, --use-list=FILENAME use table of contents from this file for
selecting/ordering output
-n, --schema=NAME restore only objects in this schema
-O, --no-owner skip restoration of object ownership
-P, --function=NAME(args) restore named function
-s, --schema-only restore only the schema, no data
-S, --superuser=NAME superuser user name to use for disabling triggers
-t, --table=NAME restore named relation (table, view, etc.)
-T, --trigger=NAME restore named trigger
-x, --no-privileges skip restoration of access privileges (grant/revoke)
-1, --single-transaction restore as a single transaction
--disable-triggers disable triggers during data-only restore
--enable-row-security enable row security
--if-exists use IF EXISTS when dropping objects
--no-data-for-failed-tables do not restore data of tables that could not be
created
--no-security-labels do not restore security labels
--no-tablespaces do not restore tablespace assignments
--section=SECTION restore named section (pre-data, data, or post-data)
--strict-names require table and/or schema include patterns to
match at least one entity each
--use-set-session-authorization
use SET SESSION AUTHORIZATION commands instead of
ALTER OWNER commands to set ownership
Connection options:
-h, --host=HOSTNAME database server host or socket directory
-p, --port=PORT database server port number
-U, --username=NAME connect as specified database user
-w, --no-password never prompt for password
-W, --password force password prompt (should happen automatically)
--role=ROLENAME do SET ROLE before restore
The options -I, -n, -P, -t, -T, and --section can be combined and specified
multiple times to select multiple objects.
If no input file name is supplied, then standard input is used.
Report bugs to <[email protected]>.
[postgres@qxy ~]$
4、使用pg_restore命令查看备份内容
[postgres@qxy backup]$ pg_restore qxy.dmp | less
--
-- PostgreSQL database dump
--
-- Dumped from database version 9.6.4
-- Dumped by pg_dump version 9.6.4
SET statement_timeout = 0;
SET lock_timeout = 0;
SET idle_in_transaction_session_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SET check_function_bodies = false;
SET client_min_messages = warning;
SET row_security = off;
--
-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner:
--
CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;
--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner:
--
COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';
SET search_path = public, pg_catalog;
SET default_tablespace = '';
SET default_with_oids = false;
--
-- Name: t; Type: TABLE; Schema: public; Owner: scott
--
CREATE TABLE t (
id integer,
name text
);
ALTER TABLE t OWNER TO scott;
--
-- Name: test; Type: TABLE; Schema: public; Owner: scott
--
CREATE TABLE test (
id integer
);
ALTER TABLE test OWNER TO scott;
--
-- Data for Name: t; Type: TABLE DATA; Schema: public; Owner: scott
--
COPY t (id, name) FROM stdin;
1 900150983cd24fb0d6963f7d28e17f72
2 900150983cd24fb0d6963f7d28e17f72
3 900150983cd24fb0d6963f7d28e17f72
4 900150983cd24fb0d6963f7d28e17f72
5 900150983cd24fb0d6963f7d28e17f72
6 900150983cd24fb0d6963f7d28e17f72
7 900150983cd24fb0d6963f7d28e17f72
8 900150983cd24fb0d6963f7d28e17f72
9 900150983cd24fb0d6963f7d28e17f72
10 900150983cd24fb0d6963f7d28e17f72
\.
--
-- Data for Name: test; Type: TABLE DATA; Schema: public; Owner: scott
--
COPY test (id) FROM stdin;
\.
--
-- PostgreSQL database dump complete
--
(END)
5、删除qxy数据库
[postgres@qxy backup]$ psql -p 5433
psql.bin (9.6.4)
Type "help" for help.
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+---------+-------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | C | C | | 7071 kB | pg_default | default administrative connection database
qxy | postgres | UTF8 | C | C | | 7079 kB | tblsp1 |
template0 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
postgres=# drop database qxy;
DROP DATABASE
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+---------+-------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | C | C | | 7071 kB | pg_default | default administrative connection database
template0 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(3 rows)
postgres=#
6、还原qxy数据库
[postgres@qxy backup]$ pg_restore -d qxy /spark/pgsql/data_test/backup/qxy.dmp -p 5433
pg_restore: [archiver (db)] connection to database "qxy" failed: FATAL: database "qxy" does not exist <===通过第四步查看的备份内容可以看到，脚本里面没有执行create database 操作，所以需要自己手动创建数据库
[postgres@qxy backup]$ psql -p 5433
psql.bin (9.6.4)
Type "help" for help.
postgres=# create database qxy;
CREATE DATABASE
postgres=# \q
[postgres@qxy backup]$ pg_restore -d qxy /spark/pgsql/data_test/backup/qxy.dmp -p 5433
[postgres@qxy backup]$ psql -p 5433
psql.bin (9.6.4)
Type "help" for help.
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+---------+-------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | C | C | | 7071 kB | pg_default | default administrative connection database
qxy | postgres | UTF8 | C | C | | 7079 kB | pg_default |
template0 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
postgres=# \c qxy scott
You are now connected to database "qxy" as user "scott".
qxy=# select * from t; <======数据还原回来
id | name
----+----------------------------------
1 | 900150983cd24fb0d6963f7d28e17f72
2 | 900150983cd24fb0d6963f7d28e17f72
3 | 900150983cd24fb0d6963f7d28e17f72
4 | 900150983cd24fb0d6963f7d28e17f72
5 | 900150983cd24fb0d6963f7d28e17f72
6 | 900150983cd24fb0d6963f7d28e17f72
7 | 900150983cd24fb0d6963f7d28e17f72
8 | 900150983cd24fb0d6963f7d28e17f72
9 | 900150983cd24fb0d6963f7d28e17f72
10 | 900150983cd24fb0d6963f7d28e17f72
(10 rows)
qxy=#
注：数据库还原之前，可以把备份集导出成doc文本模式，然后可以直接修改该文本，把不需要的表或者其他对象过滤掉。
如:
1、把备份集导出一份doc文本
[postgres@qxy ~]$ pg_restore -l -f /spark/pgsql/data_test/backup/qxy.toc /spark/pgsql/data_test/backup/qxy.dmp [postgres@qxy ~]$ cat /spark/pgsql/data_test/backup/qxy.toc ; ; Archive created at [unknown] ; dbname: qxy ; TOC Entries: 11 ; Compression: -1 ; Dump Version: 1.12-0 ; Format: CUSTOM ; Integer: 4 bytes ; Offset: 8 bytes ; Dumped from database version: 9.6.4 ; Dumped by pg_dump version: 9.6.4 ; ; ; Selected TOC Entries: ; 2128; 1262 24576 DATABASE - qxy postgres 3; 2615 2200 SCHEMA - public postgres 2129; 0 0 COMMENT - SCHEMA public postgres 1; 3079 12390 EXTENSION - plpgsql 2130; 0 0 COMMENT - EXTENSION plpgsql 186; 1259 32782 TABLE public t scott 185; 1259 32778 TABLE public test scott 2123; 0 32782 TABLE DATA public t scott 2122; 0 32778 TABLE DATA public test scott
过滤掉表T，过滤的方法是在最前面加上';'
把其中关于表t的行注释掉，注释之后结果如下 2128; 1262 24576 DATABASE - qxy postgres 3; 2615 2200 SCHEMA - public postgres 2129; 0 0 COMMENT - SCHEMA public postgres 1; 3079 12390 EXTENSION - plpgsql 2130; 0 0 COMMENT - EXTENSION plpgsql ;186; 1259 32782 TABLE public t scott <=====注释掉，前面加';' 185; 1259 32778 TABLE public test scott ;2123; 0 32782 TABLE DATA public t scott <====注释掉， 前面加';' 2122; 0 32778 TABLE DATA public test scott
删除数据库重新还原
[postgres@qxy ~]$ pg_restore -F c -L /spark/pgsql/data_test/backup/qxy.toc -d qxy /spark/pgsql/data_test/backup/qxy.dmp -p 5433
pg_restore: [archiver (db)] connection to database "qxy" failed: FATAL: database "qxy" does not exist
postgres=# create database qxy;
CREATE DATABASE
postgres=# \q
[postgres@qxy ~]$ pg_restore -F c -L /spark/pgsql/data_test/backup/qxy.toc -d qxy /spark/pgsql/data_test/backup/qxy.dmp -p 5433
[postgres@qxy ~]$
[postgres@qxy backup]$ psql -p 5433
psql.bin (9.6.4)
Type "help" for help.
postgres=# \l+
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace | Description
-----------+----------+----------+---------+-------+-----------------------+---------+------------+--------------------------------------------
postgres | postgres | UTF8 | C | C | | 7071 kB | pg_default | default administrative connection database
qxy | postgres | UTF8 | C | C | | 7063 kB | pg_default |
template0 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | C | C | =c/postgres +| 6953 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
postgres=# \c qxy scott
You are now connected to database "qxy" as user "scott".
qxy=# \dt
List of relations
Schema | Name | Type | Owner
--------+------+-------+-------
public | test | table | scott
(1 row)
qxy=# select * from t; <=====表t已经被过滤掉。
ERROR: relation "t" does not exist
LINE 1: select * from t;
^
qxy=# \d t
Did not find any relation named "t".
qxy=# \d test
Table "public.test"
Column | Type | Modifiers
--------+---------+-----------
id | integer |
qxy=#