Second, from the main building
2.1 Test target
Test structures from the main installation process postgresql
2.2 Environmental ready
Examples of level replication
Streaming Replication main library can read and write, but the library is not allowed to write only allow queries of people, and the logic of the read-write copy from the library
Streaming Replication experimental environment
| Host computer |
CPU name |
Ip Address |
operating system |
Postgresql version |
| The master node |
pgsql |
192.168.231.131 |
Redhat7.2 |
PostgreSQL10 |
| Standby node |
pgstandby |
192.168.231.132 |
Redhat7.2 |
PostgreSQL10 |
Creating user operating system and related directory on pgsql and pgstandby, as follows:
# groupadd pgsql
# useradd pgsql -g pgsql
# passwd pgsql
# mkdir -p /database/pg10/pg_root
# mkdir -p /database/pg10/pg_tbs
# chown -R pgsql:pgsql /database/pg10
/ Database / pg10 / pg_root directory to store the database file system data,
/ Database / pg10 / pg_tbs store user-defined table space file.
Postgres operating system user to set environment variables,
/home/postgres/.bash _profile file add the following:
export PGPORT = 5432
export PGUSER=pgsql
export PGDATA=/database/pg10/pg_root
export LANG = en_US.utf8
export PGHOME=/usr/pgsql-10
export LD_LIBRARY_PATH=$PGHOME/lib:/lib64:/usr/lib64:/usr/local/lib64:/lib:/usr/lib:/usr/local/lib
export PATH=$PGHOME/bin:$PATH:.
export MANPATH＝$PGHOME/share/man:$MANPATH
2.3 initialize the database
[Pgsql @ pgstandby ~] $ initdb -D / database / pg10 / pg_root -E UTF8 --locale = C -U pgsql -W (initialize create a database instance)
-A authmethod This option specifies the local user authentication method used in pg_hba.conf inside. Unless you believe that all local users, do not use the trust (the default).
-D directory This option specifies which database cluster should be stored in the directory. This is the only information required by initdb, but you can set the environment variable PGDATA to avoid typing, some of which can be convenient since the database server (postgres) can find the database directory by the same variable.
-E encoding encoding selected template database. This will be the default encoding of the database you create later, unless you override it to create the database. The default is obtained from the locale if no locale is SQL_ASCII.
--locale = locale sets the default for the database cluster area. If this option is not specified, the locale is inherited from the environment initdb running over.
-U username database super user choose the user name. The default is the effective user running initdb. Super user name is what is not important, but can choose to keep the customary name postgres, even if the operating system user's name is not the same it does not matter.
-W Makes initdb prompt for the database superuser password. If you do not plan to use password authentication, this stuff is not important. Otherwise you will not be able to use password authentication until you set up a password.
--pwfile = filename Makes initdb read the database superuser's password from a file. The first line of the file is treated as a password to use.
Other commonly used parameters are:
-d print debug output as well as some information on some other lesser interest from the bootstrap backend. Bootstrap backend is the program initdb used to create the system tables. This option generates a large number of very boring output.
-L directory Specifies where initdb should find the input files to initialize the database needed. Usually unnecessary. If you need a clear statement, the program will prompt you to enter.
When -n By default, when initdb determines that an error prevented it from completely creating the database cluster's work, it removes any files it may have created before the end of the work can not be detected. This option inhibits any tidying thus useful for debugging.
After you configure $ PGDATA / postgresql.conf set the following parameters:
listen_addresses = ‘*’
wal_level = replica # minimal, replica, or logical
archive_mode = on # enables archiving; off, on, or always
archive_command = ’/ bin/date’ # command to use to archive a logfile segment
max_wal_senders = 10 # max number of walsender processes
wal_keep_segments = 512 # in logfile segment s, 16MB each; 0 disables
hot_standby = on
listen_addresses parameter specifies which ip can access, '*' on behalf of all ip can access the database.
Wal_level control parameters WAL log information output level, there is minimal, replica, logical three modes, the log WAL least minimal information recorded, in addition to recording information database shuts WAL restored when needed, and other operational information is not recorded; recording Replica the WAL more information than minimal information, support will be recorded WAL archiving, replication and backup library read-only queries and other information to enable WAL operations required; up to WAL log logical record of information, including support for parsing logic (10 version of the new properties, the use of such logical copy mode, described later in this chapter) of the desired WAL; WAL replica mode information recorded in the recording contains minimal information, WAL logical pattern information includes information recorded in the recording replica, this parameter defaults replica, adjusting this parameter requires a reboot to take effect database, copy open stream need to set this parameter to at least level replica
Archive_mode parameter controls whether to enable archiving, off said they did not enable archiving, on representation to enable archiving and use archive_command parameter configuration commands to WAL log archiving to archive storage, you need to restart the database after the entry into force of this parameter is set, there is usually set to on.
archive_command WAL archive command parameters may be archived to a local directory WAL may be archived to other hosts on the remote, since the flow does not necessarily replication-dependent configuration archiving configuration command, the command will be temporarily set to pseudo archive archive command / bin / date, if the latter need to open the archive directly configure the archive command.
max_wal_senders parameter controls the maximum number of transmission processes WAL on the master library, the process will consume WAL through pg_base_backup commands for backup on the main reference library, this parameter is set not higher than max_connections parameter values, the default value is 10, a backup copy stream library usually only need to consume the main library to copy a stream WAL sending process.
wal_keep_segments parameter setting main directory database pg_wal WAL minimum reserved number of log files to prepare for the library may be behind the main library retained by the main library WAL recovery, this parameter is set to be larger, in theory, to recover the standby database when an abnormality OFF the greater the probability level of the main database, the archive storage if sufficient space is recommended to configure this parameter get bigger, since each WAL default file is 16MB (WAL file size can be set by parameters --with-walsegsize compile-time), Thus pg_wal space directory approximate value × 16MB of wal_keep_segments parameter, here a 512 × 16MB = 8GB, the value of the actual number of files in the WAL directory pg_wal this parameter is slightly larger than.
hot_standby database recovery process parameter control whether to enable a read operation, this parameter is usually used in stream copy standby database, copy the backup stream read-only SQL database support When you turn this parameter, but by the library does not support a write operation, also set this parameter on the main library It is on.
The above is the flow of the main process of copying the configuration postgresqI. Conf parameters, other parameters not listed, postgresqI main and backup database library. The conf identical configuration recommendations.
Configuring the main library pg_hba.conf file, add the following:
host replication repuser 192.168.231.131/24 md5
host replication repuser 192.168.231.132/24 md5
Means to allow a user to connect any repuser database password from the external address, the configuration information is a two convenient because after the switchover.
2.4 asynchronous replication flow
2.4.1 copy data files in the form of deployment stream replication (Scheme I)
# Chown -R pgsql: pgsql / var / run / postgresql / - If the open postgresql database fails, the next / postgresql / insufficient rights situation report / var / run, please empowerment
drwxr-xr-x. 2 pgsql pgsql 80 Aug 9 11:03 postgresql
$ pg ctl start
Log on to the database to create a stream copying user repuser, stream copy the user needs to have permission REPLICATION and LOGIN permission to use the superuser pgsql
[pgsql@pgsql ~]$ psql postgres
psql (10.9)
Type "help" for help.
postgres=# create user repuser replication login connection limit 5 encrypted password 'hufj123';
CREATE ROLE
postgres=# SELECT pg_start_backup('frans_bk1');
pg_start_backup
-----------------
0/2000028
(1 row)
pg_ start_ backup () function initiates an online backup on the main library
[Pgsql @ pgsql pg10] $ tar czvf pg_root.tar.gz pg_root --exclude = pg_root / pg_wal - the main thing in the library $ PGDATA backup path, the pg_wal excluded because these logs are synchronized by the library
[pgSQL @ pgSQL PG10] $ scp pg_root.tar.gz [email protected]: / Database / PG10 / - good backup tar files to the standby database up
Preparation of library operations:
[Pgsql @ pgstandby pg10] $ tar xvf pg_root.tar.gz - prepared by the library to decompress, at $ PGDATA directory
Main library operations:
[pgsql@pgsql pg10]$ psql postgres
postgres = # SELECT pg_stop_backup (); - the main library is closed back up, in fact, after a good backup tar file backups can also be turned off
NOTICE: pg_stop_backup complete, all required WAL segments have been archived
pg_stop_backup
----------------
0/2000130
(1 row)
[Pgsql @ pgstandby pg_root] $ cp /usr/pgsql-10/share/recovery.conf.sample $ PGDATA / recovery.conf - prepared by the library copy of the document recovery.conf
[pgsql@pgstandby pg_root]$ vi recovery.conf
recovery_target_timeline = 'latest'
standby_mode = on
primary_conninfo = 'host=192.168.231.131 port=5432 user=repuser'
[Pgsql @ pgstandby ~] $ touch .pgpass - .pgpass home directory create a file, the purpose is not required to enter a password when connecting the main library
[Pgsql @ pgstandby ~] $ chmod 0600 .pgpass - attention given permission 0600
[pgsql@pgstandby ~]$ cat .pgpass
192.168.231.132:5432:replication:repuser:hufj123
192.168.231.131:5432:replication:repuser:hufj123
.Pgpass configuration file is not a warning log shows the following information
2019-08-05 11:37:42.556 CST [76967] FATAL: could not connect to the primary ser
ver: fe_sendauth: no password supplied
$ PGDATA encountered an error log to view the log / log / path.
If this step is not given, and the main reservoir can view WAL transmission process, it can be seen simultaneously by the library described WAL receiving process stream configured successfully copied, on viewing the main library WAL transmission process, as follows:
[Pgsql @ pgsql ~] $ ps -ef | grep wal
pgsql 35001 34996 0 22:27 ? 00:00:00 postgres: wal writer process
pgsql 35006 34996 0 22:27 ? 00:00:00 postgres: wal sender process repuser 192.168.231.132(55152) streaming 0/7000098
pgsql 35008 34790 0 22:27 pts/1 00:00:00 grep --color=auto wal
See Preparation WAL receiving process on the library as follows:
[Pgsql @ pgstandby ~] $ ps -ef | grep wal
pgsql 59336 59310 1 22:27 ? 00:00:00 postgres: wal receiver process streaming 0/70000D0
pgsql 59347 58742 0 22:27 pts/3 00:00:00 grep --color=auto wal
Primary library is then created on a test table and insert the data as follows:
postgres=# create table test_sr(id int4);
CREATE TABLE
postgres=# insert into test_sr values(1);
INSERT 0 1
Authentication data is synchronized on the backup database:
postgres=# select * from test_sr;
id
----
1
(1 row)
2.4.2 pg_basebackup (Examples level backup) deployed stream replication (Solution 2)
Library equipment library will stop, delete the database for testing
[pgsql@pgstandby ~]$ pg_ctl stop
waiting for server to shut down.... done
server stopped
[pgsql@pgstandby ~]$ rm -rf /database/pg10/pg_root/*
[pgsql@pgstandby ~]$ rm -rf /database/pg10/pg_tbs/*
[pgsql@pgstandby ~]$ pg_basebackup -D /database/pg10/pg_root -Fp -Xs -v -P -h 192.168.231.131 -p 5432 -U repuser
-D Indicates that the specified target node backup path for receiving the primary data repository, and where consistent with the primary database is still / database / pg10 / pg_root directory.
-F parameter specifies the format of the backup data pg_basebackup generated commands, has two formats, p (plain) format and t (tar) format, p (plain) refers to the data file format of the generated backup data and layout as the primary library, That is similar to the operating system commands to the database $ PGDATA system data file, a complete copy of the tablespace files to the standby node; T (tar) format means to make a backup file tar package and stored in a designated directory, the system files are packaged into base.tar, other tablespace files are packaged into oid.tar, wherein the OID OID tablespace.
The -X-WAL log parameters generated during the backup process contained in the backup mode, there are two alternative ways, f (fetch) and s (stream), f (fetch) refers to the log base was completed WAL transferred to the standby node, wal_keep_segments parameters on the case main library needs to be large, the backup process in order to avoid WAL generated before the standby node has not sent to the master database is overwritten, if this occurs will create a baseline backup fails, the f (fetch) the master mode will launch a reference library backup WAL transmission process; s (stream) mode, in addition to a reference to start the backup-process WAL transmission will start a WAL additional transmission process for transmitting the primary library on master WAL incremental library generated log stream, this approach avoids the situation WAL process f (fetch) mode in the main library is overwritten, the production stream replication deployment recommend this approach, especially busier libraries or large library.
-v parameter indicates verbose mode is enabled, the process of executing the command to print out the various stages of logging, it is recommended to enable this parameter, the command execution to understand what stage.
-P parameter display data file, table space file transfer approximate percentage, due to the execution pg_basebackup command during the main library data file will change, so this is just an estimate; recommends that this option is enabled, about the progress of data replication.
[Pgsql @ pgstandby ~] $ cp /usr/pgsql-10/share/recovery.conf.sample $ PGDATA / recovery.conf - This file pghome / share / below
[Pgsql @ pgstandby ~] $ we /database/pg10/pg_root/recovery.conf
recovery_target_timeline = 'latest'
standby_mode = on
primary_conninfo = 'host=192.168.231.131 port=5432 user=repuser'
[pgsql@pgstandby ~]$ pg_ctl start
postgres = # select usename, application_name, client_addr, sync_state from pg_stat_replication; - the main database query View sync mode
usename | application_name | client_addr | sync_state
---------+------------------+-----------------+------------
repuser | walreceiver | 192.168.231.132 | async
2.5 synchronous replication flow
Asynchronous replication flow refers to succeed when they return without waiting for the standby database receiving well written WAL log when a transaction is committed on the primary database to see if the main library exceptions machine, the main library on the transaction has been submitted may not have had time to send to the standby database, you library equipment can cause loss of data, prepared by the library and the amount of data lost WAL replication latency, the greater WAL replication delay, the greater the amount of data lost on the standby database. Deferred copy to wait while the transaction is committed on the primary database library equipment to receive and WAL log, when the main library receives at least confirmation a backup library sent back will return to success, synchronization stream replication ensures that at least one standby database received a master library WAL log transmitted, on the one hand to protect the integrity of the data, while increasing the response time of the transaction, thus replicating the master database synchronization stream throughput asynchronous stream throughput lower than the primary database copy.
Configure the synchronization stream copy
Preparation of the library profile recovery.conf set the following parameters, as follows:
primary_conninfo=’host=192.168.231.131 port=5432 user=repuser application_name=node2’
primary_conninfo application_name option parameters are added, the alias option specified application_name standby node, synchronous_standby_names main library postgresql.conf parameter values may be referenced by the library application_name option is provided here as node2.
postgresql.conf profile on the master library set the following parameters, other parameters and configuration consistent asynchronous stream replication.
synchronous_commit = on or remote_apply - The recommended setting is on, remote_apply application is prepared by the library before returning Information
synchronous_standby_narnes = ’node2’
wal_level configuration is also consistent replication and asynchronous stream configuration, provided to replica or logical.
Restart the main library and library equipment as sync replication status query
[pgsql@pgsql pg_root]$ psql postgres
psql (10.9)
Type "help" for help.
postgres=# select usename,application_name ,client_addr ,sync_state from pg_stat_replication;
usename | application_name | client_addr | sync_state
---------+------------------+-----------------+------------
repuser | node2 | 192.168.231.132 | sync
(1 row)
postgres=#
Copy flow monitoring
pg_stat_replication view
application_name: alias WAL transmission connection application process, this parameter value is displayed by the library configuration file primary_conninfo recovery.conf application_name parameter options.
client_addr: WAL connected to the sending process of the client IP address, which is prepared by the library of IP.
backend_start: WAL sending start time for the process.
state: display state WAL transmission process, Startup represents WAL process during startup; Catchup represented by the library are catching the main library; Streaming represented by the library has caught up with the main gallery, and the primary library sends a WAL log stream to the standby database, the state is the normal state of the stream replication; backup means that the backup by pg_basebackup ongoing; stopping represents WAL sending process is shutting down.
sent_lsn: WAL WAL log location recently sent the sending process.
write_lsn: library equipment recently wrote human WAL log position where WAL log stream is still the operating system cache, not written into the standby database WAL log files.
flush_lsn: library equipment recently wrote human WAL log position where WAL log stream has been written by the library WAL log files.
replay_lsn: WAL log position by the library recently applied.
write_lag: the main library WAL log in after-hours drop by the library waiting to receive WAL log (WAL log stream at this time people did not write the WAL log files prepared by the library, is still the operating system cache) and returns an acknowledgment time information.
flush_lag: the main library WAL log in after-hours drop by the library waiting to receive WAL log (WAL log stream is then written into the standby database WAL log files, but has not been applied WAL log) and return confirmation time information.
replay_lag: the main library WAL log off-hours wait WAL log received by the library (WAL time log stream has been written by the library WAL log file and log hexyl application WAL) and return acknowledgment time information.
sync_priority: priority-based mode standby database is selected as the priority synchronization equipment repository, there was no impact on the electoral quorum this field.
sync_state: synchronization state, following the state value, the async expressed by the library asynchronous synchronous mode; potential represented by the library currently asynchronous mode, if the current synchronization standby database rock drill, asynchronous standby database can be upgraded to become synchronized standby database; sync represents Preparation of library current synchronous mode; Quorum expressed by the library of candidate quorum standbys.
pg_stat_wal_receiver displays detailed information wal sending process
The main field information
pid: process ID WAL receiving process.
status: WAL state of the receiving process.
Receive_start_lsn: The first position after use WAL WAL log receiving process started.
Received_lsn: Well recently received write position of the person WAL WAL log files.
last_msg_send_time: Preparation of library sending process after receiving the last message, send time confirmation message back to the master database.
last_msg_receipt_time: Preparation of library receives the transmission process the last received time of the message.
conninfo: WAL receiving process uses the connection string, the connection parameter information configured by primary_conninfo recovery.conf profiles of library $ PGDATA directory.
Third, the copy standby switching flow
3.1 determine the role of standby
mode a: ps -ef | grep wal wal View course descriptions, judge standby
Second way: the master database query SELECT * from pg_stat_replication, Kuchar prepared in this view is not recorded; standby database query select * from pg_stat_wal_receiver, this view with the main Kuchar no record.
Three ways: SELECT pg_is_in_recovery (); return t is illustrated as the standby database, the main database returns f.
Four ways: by pg_controldata view database control information; Database cluster state information is determined by; The results are shown in production-oriented database; The results are shown in archive recovery is by the library.
Way five: View by recover.conf profile.
Determining whether the normal standby 3.2
(1) view the process standby database
Main library sending process:
[Pgsql @ pgsql ~] $ ps -ef | grep wal
pgsql 35001 34996 0 22:27 ? 00:00:00 postgres: wal writer process
pgsql 35006 34996 0 22:27 ? 00:00:00 postgres: wal sender process repuser 192.168.231.132(55152) streaming 0/7000098
pgsql 35008 34790 0 22:27 pts/1 00:00:00 grep --color=auto wal
Library equipment for the receiving process:
[Pgsql @ pgstandby ~] $ ps -ef | grep wal
pgsql 59336 59310 1 22:27 ? 00:00:00 postgres: wal receiver process streaming 0/70000D0
pgsql 59347 58742 0 22:27 pts/3 00:00:00 grep --color=auto wal
(2) is inserted into the primary data repository, to observe whether data is received by the library.
(3) replication status main database query parameters are normal.
[pgsql@pgsql pg_root]$ psql postgres
psql (10.9)
Type "help" for help.
postgres=# select usename,application_name ,client_addr ,sync_state from pg_stat_replication;
usename | application_name | client_addr | sync_state
---------+------------------+-----------------+------------
repuser | node2 | 192.168.231.132 | sync
(1 row)
3.3 switching manner
Triggered by the trigger master papers and switchover is triggered by the switchover command pg_ctl promot
First, the trigger file
1) Preparation of configuration parameters tigger_file recovery.conf library file is provided to activate the standby database trigger file path and name. Add tigger_file parameters.
trigger_file = ‘/database/pg10/pg_root/.postgresql.trigger.5432’
2) Close the master library, using -m fast mode off. (First do a checkpoint when the database is shut down, wal process will be sent to the standby database as of the checkpoint wal log stream, the standby database applications wal.)
3) Create a trigger to activate the standby database files in the standby database, when recovery.conf become recovery.done expressed by the library has switched the main library.
[pgsql@pgstandby ~]$ cat /database/pg10/pg_root/recovery.conf |grep ^[^#]
recovery_target_timeline = 'latest'
standby_mode = on
primary_conninfo='host=192.168.231.131 port=5432 user=repuser application_name=node2'
trigger_file='/database/pg10/pg_root/.postgresql.trigger.5432'
[Pgsql @ pgstandby ~] $ touch /database/pg10/pg_root/.postgresql.trigger.5432 - create a trigger file, it will automatically switch to the master database can not be detected after the main library.
4) created under the old main library recovery.conf file, and configure the appropriate parameters, like parameter backup library. Under the root directory arranged .pgpass Free confidential documents.
[Pgsql @ pgstandby ~] $ vi /database/pg10/pg_root/recovery.conf - recovery.conf parameters based primarily on the original standby database to be set.
recovery_target_timeline = 'latest'
standby_mode = on
primary_conninfo = 'host=192.168.231.131 port=5432 user=repuser'
[pgsql@pgstandby ~]$ cat .pgpass
192.168.231.132:5432:replication:repuser:hufj123
192.168.231.131:5432:replication:repuser:hufj123
5) Start the old main library, observe the standby database is normal.
Two, pg_ctl promote the master mode switchover
pg_ctl promote [ D datadir]
-D refers to data directory, if you do not specify the use value $ PGDATA set environment variables. After promote command issued, prepared by the library will stop the recovery operation mode is switched to the write mode and the main library. pg_ctl promote standby switching step and substantially the same trigger file.
1) Turn off the main libraries, using -m fast mode
2) Preparation of libraries for performing pg_ctl promote activation standby command library
3) the original primary database to switch to the standby database, create recovery.conf files, configuration parameters.
4) restart the original primary database to view the primary and backup process is normal.
Fourth, the delay library equipment
Configuration parameters recovery_min_apply_delay (integer) units support days d, h, the minutes min, s s, milliseconds ms.
Delay operation is mainly to prevent misuse, and the data can be retrieved directly by the library. Recovery_min_apply_delay parameter set too large, will lead to wal logs take up more space, recovery_min_apply_delay parameter set too small, would not achieve the purpose of data recovery.
recovery_min_apply_delay attention and parameters synchronous_commit reuse.
If synchronous_commit = on; No effect, as long as the standby database receives the log, without waiting for the master repository can affect delay parameters. If synchronous_commit = remote_apply, prepared by the library needs to receive the log and the log information wal-latency applications, the main library will receive a message on the appropriate library prepared only after a delay, in order to continue the next transaction.