GreenPlum数据库对于数据的冗余，也支持类似Oracle数据库物理DataGuard的数据冗余机制，Master的数据库镜像称为Standby，Segment节点的数据库镜像称之为Mirror，本文主要介绍如何为没有做Standby的Master节点添加Standby。
需要注意，在为Master节点搭建Standby的过程中，GreenPlum会自动关闭数据库，并以utility模式打开Master节点，然后修改gp_segment_configuration字典中增加Standby的信息，然后再关闭Master节点，将Master的数据拷贝到Standby节点，最后启动数据库，所以，在为Master节点添加Standby时，需要在空闲时段进行，否则会影响业务。
在搭建Standby的时候，Standby主机需要先安装GreenPlum数据库软件，这和正常安装GreenPlum数据库软件没有区别，从规范化考虑，尽量使用和Master主机同样的安装路径，下面简单结束安装过程。
1.创建gpadmin用户及安装目录
01 |
[root@mdw-std ~]# groupadd -g 520 gpadmin |
02 |
[root@mdw-std ~]# useradd -u 520 -g gpadmin gpadmin |
03 |
[root@mdw-std ~]# passwd gpadmin |
04 |
Changing password for user gpadmin. |
06 |
BAD PASSWORD: it is based on a dictionary word |
07 |
BAD PASSWORD: is too simple |
09 |
passwd: all authentication tokens updated successfully. |
10 |
[root@mdw-std ~]# mkdir -p /gpdb/app |
11 |
[root@mdw-std ~]# mkdir -p /gpdb/gpdata |
12 |
[root@mdw-std ~]# chown -R gpadmin:gpadmin /gpdb/ |
2.配置hosts文件，添加所有主机的信息
1 |
[gpadmin@mdw config]$ vi /etc/hosts |
除Standby主机外，其他所有主机(Master节点和所有Segment节点)的hosts文件均需要添加Standby主机的信息，也就是所有GreenPlum数据库包含的主机的hosts文件都需要包含以上hosts文件中的所有主机信息。
3.上传安装文件及解压
01 |
[gpadmin@mdw-std gpdb]$ scp 10.9.15.20:/gpdb/*.zip . |
02 |
The authenticity of host '10.9.15.20 (10.9.15.20)' can't be established. |
03 |
RSA key fingerprint is 61:72:68:57:16:28:40:d4:bc:9e:68:f0:bc:ac:65:e9. |
04 |
Are you sure you want to continue connecting (yes/no)? yes |
05 |
Warning: Permanently added '10.9.15.20' (RSA) to the list of known hosts. |
07 |
greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip 100% 134MB 133.7MB/s 00:01 |
08 |
[gpadmin@mdw-std gpdb]$ unzip greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip |
09 |
Archive: greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip |
10 |
inflating: README_INSTALL |
11 |
inflating: greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.bin |
4.配置内核参数，添加以下内容
01 |
[root@mdw-std ~]# vi /etc/sysctl.conf |
03 |
kernel.shmmax = 500000000 |
04 |
kernel.shmall = 4000000000 |
06 |
kernel.sem = 250 512000 100 2048 |
08 |
kernel.core_uses_pid = 1 |
12 |
net.ipv4.tcp_syncookies = 1 |
13 |
net.ipv4.ip_forward = 0 |
14 |
net.ipv4.conf.default.accept_source_route = 0 |
15 |
net.ipv4.tcp_tw_recycle = 1 |
16 |
net.ipv4.tcp_max_syn_backlog = 4096 |
17 |
net.ipv4.conf.all.arp_filter = 1 |
18 |
net.ipv4.conf.default.arp_filter = 1 |
19 |
net.ipv4.ip_local_port_range = 1025 65535 |
20 |
net.core.netdev_max_backlog = 10000 |
21 |
vm.overcommit_memory = 2 |
通过sysctl命令使之生效。
1 |
[root@mdw-std ~]# sysctl -p |
5.配置资源限额(limit)
1 |
[root@mdw-std ~]# vi /etc/security/limits.conf |
2 |
gpadmin soft nofile 65536 |
3 |
gpadmin hard nofile 65536 |
4 |
gpadmin soft nproc 131072 |
5 |
gpadmin hard nproc 131072 |
6.修改磁盘模式，/boot/grub/menu.lst文件增加elevator=deadline。
01 |
[root@mdw-std ~]# echo deadline > /sys/block/sda/queue/scheduler |
02 |
[root@mdw-std ~]# cat /sys/block/sda/queue/scheduler |
05 |
[root@mdw-std ~]# vi /boot/grub/menu.lst |
06 |
# grub.conf generated by anaconda |
08 |
# Note that you do not have to rerun grub after making changes to this file |
09 |
# NOTICE: You have a /boot partition. This means that |
10 |
# all kernel and initrd paths are relative to /boot/, eg. |
12 |
# kernel /vmlinuz-version ro root=/dev/sda3 |
13 |
# initrd /initrd-[generic-]version.img |
17 |
splashimage=(hd0,0)/grub/splash.xpm.gz |
19 |
title Oracle Linux Server Unbreakable Enterprise Kernel (3.8.13-16.2.1.el6uek.x86_64) |
21 |
kernel /vmlinuz-3.8.13-16.2.1.el6uek.x86_64 ro root=UUID=1fb9c4cf-3cf4-47db-bf4a-0a87958d477d rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet |
22 |
initrd /initramfs-3.8.13-16.2.1.el6uek.x86_64.img |
23 |
title Oracle Linux Server Red Hat Compatible Kernel (2.6.32-431.el6.x86_64) |
25 |
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=1fb9c4cf-3cf4-47db-bf4a-0a87958d477d rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM elevator=deadline rhgb quiet |
26 |
initrd /initramfs-2.6.32-431.el6.x86_64.img |
7.配置磁盘预读块
1 |
[root@mdw-std ~]# /sbin/blockdev --setra 65535 /dev/sda3 |
8.安装GreenPlum数据库软件
1 |
[gpadmin@mdw-std gpdb]$ /bin/bash greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.bin |
跳过一堆的许可信息后，会提示license相关的信息。
1 |
******************************************************************************** |
2 |
Do you accept the Pivotal Database license agreement? [yes|no] |
3 |
******************************************************************************** |
输入yes确认后，会提示安装路径相关信息。
3 |
******************************************************************************** |
4 |
Provide the installation path for Greenplum Database or press ENTER to |
5 |
accept the default installation path: /usr/local/greenplum-db-4.3.6.2 |
6 |
******************************************************************************** |
这里使用和Master相同的安装路径。
3 |
******************************************************************************** |
4 |
Install Greenplum Database into </gpdb/app>? [yes|no] |
5 |
******************************************************************************** |
输入yes确认安装路径。
03 |
Extracting product to /gpdb/app |
05 |
******************************************************************************** |
06 |
Installation complete. |
07 |
Greenplum Database is installed in /gpdb/app |
09 |
Pivotal Greenplum documentation is available |
11 |
******************************************************************************** |
GreenPlum数据库软件安装完成，需要配置gpadmin用户的环境变量，增加以下内容。
1 |
[gpadmin@mdw-std app]$ vi /home/gpadmin/.bash_profile |
2 |
source /gpdb/app/greenplum_path.sh |
使之生效。
1 |
[gpadmin@mdw-std app]$ . /home/gpadmin/.bash_profile |
9.Standby主机创建数据库初始化数据存放目录及文件空间目录，需要和Master目录一样。
1 |
[gpadmin@mdw-std gpdata]$ mkdir /gpdb/gpdata/master |
2 |
[gpadmin@mdw-std gpdata]$ mkdir /gpdb/gpdata/fspc_master |
以上均是在Standby节点的主机上操作，接下来是在Master节点的主机上操作。
10.在Master节点配置所有节点于Standby节点的ssh互信。
01 |
[gpadmin@mdw config]$ vi hostlist |
09 |
[gpadmin@mdw config]$ gpssh-exkeys -f hostlist |
10 |
[STEP 1 of 5] create local ID and authorize on local host |
11 |
... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped |
13 |
[STEP 2 of 5] keyscan all hosts and update known_hosts file |
15 |
[STEP 3 of 5] authorize current user on remote hosts |
18 |
*** Enter password for mdw-std: |
23 |
[STEP 4 of 5] determine common authentication file content |
25 |
[STEP 5 of 5] copy authentication files to all remote hosts |
26 |
... finished key exchange with mdw-std |
27 |
... finished key exchange with sdw1 |
28 |
... finished key exchange with sdw2 |
29 |
... finished key exchange with sdw3 |
31 |
[INFO] completed successfully |
通过gpssh工具测试互信设置是否成功。
01 |
[gpadmin@mdw config]$ gpssh -f hostlist -e 'date' |
03 |
[mdw-std] Tue Feb 16 17:50:16 CST 2016 |
05 |
[ mdw] Tue Feb 16 17:50:16 CST 2016 |
07 |
[ sdw2] Tue Feb 16 17:50:16 CST 2016 |
09 |
[ sdw1] Tue Feb 16 17:50:16 CST 2016 |
11 |
[ sdw3] Tue Feb 16 17:50:16 CST 2016 |
11.在Master节点通过gpinitstandby命令添加Standby。
01 |
[gpadmin@mdw config]$ gpinitstandby -s mdw-std |
03 |
The filespace locations on the master must be mapped to |
04 |
locations on the standby. These locations must be empty on the |
05 |
standby master host. The default provided is the location of |
06 |
the filespace on the master (except if the master and the |
07 |
standby are hosted on the same node or host). In most cases the |
10 |
Enter standby filespace location for filespace fspc1 (default: /gpdb/gpdata/fspc_master/gpseg-1): |
这里会提示Master节点中记录的文件空间的目录，默认和Master相同，这里可以使用和Master节点不同的路径，建议使用一样的路径。需要注意，在这个步骤之前，一定要创建和Master一样的数据库初始化目录，这里是/gpdb/gpdata/master目录，否则会报这个路径找不到的错误。如果文件空间直接使用默认路径，直接回车就可以。
02 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Validating environment and parameters for standby initialization... |
03 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking for filespace directory /gpdb/gpdata/master/gpseg-1 on mdw-std |
04 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking for filespace directory /gpdb/gpdata/fspc_master/gpseg-1 on mdw-std |
05 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------ |
06 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master initialization parameters |
07 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------ |
08 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master hostname = mdw |
09 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master data directory = /gpdb/gpdata/master/gpseg-1 |
10 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master port = 5432 |
11 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master hostname = mdw-std |
12 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master port = 5432 |
13 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master data directory = /gpdb/gpdata/master/gpseg-1 |
14 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum update system catalog = On |
15 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------ |
16 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:- Filespace locations |
17 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------ |
18 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-pg_system -> /gpdb/gpdata/master/gpseg-1 |
19 |
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-fspc1 -> /gpdb/gpdata/fspc_master/gpseg-1 |
20 |
Do you want to continue with standby master initialization? Yy|Nn (default=N): |
输入Y确认搭建Standby。
02 |
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Syncing Greenplum Database extensions to standby |
03 |
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-The packages on mdw-std are consistent. |
04 |
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Adding standby master to catalog... |
05 |
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Database catalog updated successfully. |
06 |
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Updating pg_hba.conf file... |
07 |
20160216:17:55:02:011967 gpinitstandby:mdw:gpadmin-[INFO]:-pg_hba.conf files updated successfully. |
08 |
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Updating filespace flat files... |
09 |
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Filespace flat file updated successfully. |
10 |
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Starting standby master |
11 |
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking if standby master is running on host: mdw-std in directory: /gpdb/gpdata/master/gpseg-1 |
12 |
20160216:17:55:05:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Cleaning up pg_hba.conf backup files... |
13 |
20160216:17:55:11:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully. |
14 |
20160216:17:55:11:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Successfully created standby master on mdw-std |
搭建完成，可以使用gpstate命令查看Standby的状态。
01 |
[gpadmin@mdw config]$ gpstate -s |
02 |
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Starting gpstate with args: -s |
03 |
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1' |
04 |
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28' |
05 |
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Obtaining Segment details from master... |
06 |
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Gathering data from segments... |
08 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
09 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:--Master Configuration & Status |
10 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
11 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master host = mdw |
12 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master postgres process ID = 2474 |
13 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master data directory = /gpdb/gpdata/master/gpseg-1 |
14 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master port = 5432 |
15 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master current role = dispatch |
16 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Greenplum initsystem version = 4.3.6.2 build 1 |
17 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Greenplum current version = PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28 |
18 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Postgres version = 8.2.15 |
19 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master standby = mdw-std |
20 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Standby master state = Standby host passive |
21 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
22 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-Segment Instance Status Report |
23 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
24 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info |
25 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw1 |
26 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw1 |
27 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg0 |
28 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000 |
29 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status |
30 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2378 |
31 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up |
32 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up |
33 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
34 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info |
35 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw2 |
36 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw2 |
37 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg1 |
38 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000 |
39 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status |
40 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2362 |
41 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up |
42 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up |
43 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:----------------------------------------------------- |
44 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info |
45 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw3 |
46 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw3 |
47 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg2 |
48 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000 |
49 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status |
50 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2384 |
51 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up |
52 |
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up |
也可以在Standby节点的主机上查看启动的进程来查看Standby的状态。
1 |
[gpadmin@mdw-std gpdata]$ ps -ef | grep gpadmin |
2 |
gpadmin 4362 1 0 17:55 ? 00:00:00 /gpdb/app/bin/postgres -D /gpdb/gpdata/master/gpseg-1 -p 5432 -b 5 -z 3 --silent-mode=true -i -M master -C -1 -x 0 -y -E |
3 |
gpadmin 4378 4362 0 17:55 ? 00:00:00 postgres: port 5432, master logger process |
4 |
gpadmin 4379 4362 0 17:55 ? 00:00:00 postgres: port 5432, startup process recovering 000000010000000000000002 |
5 |
gpadmin 4390 4362 0 17:55 ? 00:00:00 postgres: port 5432, wal receiver process |
第一个进程是Standby的启动进程也就是守护进程，第二个进程是日志传输进程，第三个进程是数据同步进程也就是数据恢复进程，第四个进程是Standby与Master之间的通信检测进程，相当于心跳，以上进程的作用均是个人的猜测，有可能不准。
Standby搭建完成，即可在数据库中查询到Standby的信息。
01 |
[gpadmin@mdw config]$ psql -d dbdream |
05 |
dbdream=# select * from gp_segment_configuration; |
06 |
dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port | san_mounts |
07 |
------+---------+------+----------------+------+--------+-------+----------+---------+------------------+-------- |
08 |
1 | -1 | p | p | s | u | 5432 | mdw | mdw | | |
09 |
2 | 0 | p | p | s | u | 40000 | sdw1 | sdw1 | | |
10 |
3 | 1 | p | p | s | u | 40000 | sdw2 | sdw2 | | |
11 |
4 | 2 | p | p | s | u | 40000 | sdw3 | sdw3 | | |
12 |
5 | -1 | m | m | s | u | 5432 | mdw-std | mdw-std | | |
