HG_Repmgr支持Linux操作系统，本文演示过程使用的操作系统是RedHat7，推荐使用Redhat/CentOs 7.0及以上版本。
本文档中的示例环境为两节点。
一、 安装检查
要验证系统是否满足Highgo Database集群的最低要求，以root用户身份登录并运行以下命令。要查看可用 RAM 和交换空间大小，运行以下命令：
| [root@node1 ~]# df -h [root@node1 ~]# free -h [root@node1 ~]# grep MemTotal /proc/meminfo MemTotal:512236 kB [root@node1 ~]# grep SwapTotal /proc/meminfo SwapTotal:1574360 kB |
内存及对应swap建议值参照表：
| MemTotal |
SwapTotal |
| 8G |
2~4G |
| 8~16G |
4~8G |
| 16~64G |
8~32G |
| >=64G |
32G |
2、检查内核参数
| [root@node1 ~]# cat /proc/sys/kernel/shmmax [root@node1 ~]# cat /proc/sys/kernel/shmall [root@node1 ~]# cat /proc/sys/kernel/shmmni |
内存大小及对应内核参数建议值：
| MemTotal |
shmall |
shmmax |
||
| 8G |
7~8G |
4G |
||
| 8~64G |
M*50% |
M*90~95% |
||
| >=64G |
32G |
M*90%~95% |
||
二、 安装前准备
1) 关闭防火墙和NetworkManager
在NeoKylin 7.X（当前项目中的通用机大部分为7.x）执行如下命令
| [root@node1 ~]# systemctl stop firewalld.service [root@node1 ~]# systemctl disable firewalld.service [root@node1 ~]# systemctl status firewalld.service [root@node1 ~]# systemctl stop NetworkManager.service [root@node1 ~]# systemctl disable NetworkManager.service [root@node1 ~]# systemctl status NetworkManager.service |
2) 关闭SELINUX
| [root@node1 ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config [root@node1 ~]# setenforce 0 [root@node1 ~]# cat /etc/selinux/config | grep SELINUX=disabled [root@node1 ~]# getenforce |
3) 修改主机名
两节点分别修改主机名，节点一为node1，节点二为node2，主机名可根据实际情况修改。
| [root@node1 ~]# hostnamectl set-hostname node1 [root@node2 ~]# hostnamectl set-hostname node1 |
4) 检查时间和时区
集群所有节点时间要一致，建议配置时间同步服务器。
| [root@node1 ~]$ timedatectl Local time: Fri 2018-10-19 18:52:17 CST Universal time: Fri 2018-10-19 10:52:17 UTC RTC time: Fri 2018-10-19 10:52:18 Time zone: Asia/Shanghai (CST, +0800) NTP enabled: no NTP synchronized: no RTC in local TZ: no DST active: n/a |
NeoKylin 7.X下如何修改时区
| [root@node1 ~]# timedatectl list-timezones [root@node1 ~]# timedatectl set-timezone Asia/Shanghai 修改时间 [root@node1 ~]# date -s "20170622 10:26:00" |
5) 安装系统软件包
配置yum源，所有节点都要配置
| [root@node1 ~]# cat /etc/yum.repos.d/highgo.repo [rhel] name=rhel baseurl=file:///media/cdrom enabled=1 gpgcheck=0 [root@node1 ~]# mkdir /media/cdrom [root@node1 ~]# mount /dev/cdrom /media/cdrom mount: /dev/sr0 is write-protected, mounting read-only |
安装依赖包，所有节点都要安装。
| [root@node1 ~]# yum clean all [root@node1 ~]# yum list [root@node1 ~]# yum install vim wget readline readline-devel zlib zlib-devel openssl openssl-devel pam-devel libxml2-devel libxslt-devel python-devel tcl-devel gcc gcc-c++ rsync -y |
6) 配置hosts
所有节点都要添加。
| [root@node1 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.80.10 node1 192.168.80.11 node2 |
7) 所有节点创建highgo用户并修改密码
| [root@node1 ~]# groupadd -g 5866 highgo [root@node1 ~]# useradd -u 5866 -g highgo highgo [root@node1 ~]# passwd highgo |
8) 配置操作系统limits
| [root@node1 ~]# vi /etc/security/limits.conf #for highgo db 5.6.4 highgo soft core unlimited highgo hard nproc unlimited highgo soft nproc unlimited highgo hard memlock unlimited highgo hard nofile 1024000 highgo soft memlock unlimited highgo soft nofile 1024000 highgo hard stack 65536 highgo soft stack 65536 |
9) 所有节点设置highgo用户环境变量
| [root@node1 ~]# vi /home/highgo/.bash_profile export PGHOME=/opt/highgo/5.6.4 export PGUSER=highgo export PGPORT=5866 export PGDATA=$PGHOME/data export PATH=$PGHOME/bin:$PATH:$HOME/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PGHOME/lib [root@node1 ~]#source ~/.bash_profile |
10) 添加sudo权限（所有节点）
hg_repmgr的virtual IP功能是借助操作行功能实现，需要使用root权限，所以要给highgo户配置免密sudo权限。
| vi /etc/sudoers root ALL=(ALL) ALL highgo ALL=(ALL) NOPASSWD:ALL %highgo ALL=(ALL) NOPASSWD:ALL |
测试权限是否生效
| [highgo@node12 ~]$ sudo tail -n 50 /var/log/messages |
三、 安装
1) 所有节点上传并安装HGDB
| [root@node1 upload]# rpm -ivh hgdb5.6.4-sec-rhel7.x-x86-64-20190130_195124.tar.gz |
2) 删除备节点的data目录
| [highgo@node2 ~]# pg_ctl stop [highgo@node2 ~]# rm -r $PGDATA |
3) 主节点设置数据库参数
| cd $PGDATA vi postgresql.replication.conf listen_addresses = '*'; --修改最大连接数 max_connections = 2000; --修改shared_buffers，建议设置为物理内存的25%，最大不超过40%，通用机一般为32G，设置为10G shared_buffers = '10240MB'; checkpoing_timeout=30min; checkpoint_completion_target = 0.8; --设置hgdb生成的日志格式 log_destination = 'csvlog'; --开启日志 logging_collector = on; --修改日志存放路径 log_directory = 'hgdb_log'; --修改日志文件名称格式 log_filename = 'highgodb_%d.log'; --设置每天生成一个新的日志文件 log_rotation_age = '1d'; --不限制单个日志文件大小 log_rotation_size = 0; --覆盖同名文件 log_truncate_on_rotation = on; --设置记录ddl语句 log_statement = 'ddl'; --开启归档 wal_level = replica; archive_mode = on; --红色字体部分为归档存放路径 archive_command = 'test ! -f /hgdbbak/archive/%f && cp %p /hgdbbak/archive/%f' --设置用户连接与断开数据库的信息 log_connections=on; log_disconnections=on; maintenance_work_mem=’1GB’; shared_preload_libraries = ' pg_pathman,timescaledb,repmgr ' 创建对应的归档目录。 mkdir -p /hgdbbak/archive 将该文件包含到 postgresql.conf 中： vi postgresql.conf include 'postgresql.replication.conf' 重启数据库 pg_ctl restart |
4) 安装license
注意：
1、 激活license之前一定要确认合同中是测试license还是永久license
2、 使用license之前请确保是否与当前数据库版本匹配
未安装license，数据库启动时会显示剩余时间：
| [highgo@node1 ~]$ 日志: 00000: This is a trial edition, validate until 2018-11-21 21:58:57, database will not be able to start up after that time,please apply an official license by that time. |
上传license并复制到data目录修改名称为hgdb.lic
| [root@node1 uplaod]# cp hgdb_476_3m.lic /opt/highgo/5.6.4/data/hgdb.lic [root@node1 uplaod]# chown highgo:highgo -R /opt/highgo/5.6.4/data/hgdb.lic [root@node1 uplaod]# ls -l /opt/highgo/5.6.4/data/hgdb.lic -rw-r--r-- 1 highgo highgo 239 Oct 22 22:21 /opt/highgo/5.6.4/data/hgdb.lic |
激活license
| [highgo@node1 data]$ pg_ctl reload server signaled 日志: 00000: 接收到 SIGHUP, 重载配置文件 日志: 00000: 注意 :检测到授权文件已更改，该license为试用版授权，有效期截止到2019-01-20 21:58:57 |
检查license
| [highgo@node1 data]$ check_lic -D $PGDATA
############## HIGHGODB LICENSE ################### License status:Normal License validity:2019-1-21 Database Version:Highgo Database 5.6.4 Trail Edition ################################################### |
5) 为 HG_Repmgr 创建用户和数据库（仅主节点）
| createuser -P -s hgrepmgr -->将密码设置为 highgo createdb hgrepmgr -O hgrepmgr |
6) 配置 pg_hba.conf（仅主节点）
| 在pg_hba.conf 中追加以下内容 # cus host replication hgrepmgr 192.168.90.0/24 md5 host hgrepmgr hgrepmgr 192.168.90.0/24 md5 重新加载 pg_ctl reload |
7) 配置密码文件（所有节点）
在所有节点使用 highgo 用户创建密码文件：
| vi ~/.pgpass node1:5866:replication:hgrepmgr:highgo node1:5866:hgrepmgr:hgrepmgr:highgo node12:5866:replication:hgrepmgr:highgo node12:5866:hgrepmgr:hgrepmgr:highgo 修改文件权限 chmod 0600 .pgpass |
8) 测试主端数据库是否可达（所有备节点）
备节只需安装软件，调用psql可以连接主库。
| psql 'host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2'
[highgo@node12 ~]$ psql 'host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2' psql (5.6.4) Type "help" for help.
hgrepmgr=# |
9) 配置 HG_Repmgr
修改配置文件（仅主节点） 修改$HGDB_HOME/conf 中的 hg_repmgr.conf 文件，内容如下：
| node_id=1 node_name=node1 conninfo='host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2' data_directory='/opt/highgo/5.6.4/data' pg_bindir='/opt/highgo/5.6.4/bin' passfile='/home/highgo/.pgpass' failover=automatic promote_command='repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf standby promote' follow_command='repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf standby follow --upstream-node-id=%n' virtual_ip = '192.168.80.12' network_card = 'enp0s3' gateway = '192.168.6.254' |
10) 注册主节点（仅主节点）
| [highgo@node1 conf]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf primary register INFO: connecting to primary database... NOTICE: attempting to install extension "repmgr" NOTICE: "repmgr" extension successfully installed NOTICE: primary node record (id: 1) registered [highgo@node1 conf]$
[highgo@node1 conf]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf cluster show ID | Name | Role | Status | Upstream | Location | Connection string ----+------+---------+-----------+----------+----------+----------------------------------------------------------- 1 | node1 | primary | * running | | default | host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2 [highgo@node1 conf]$ |
11) 克隆并注册备节点（所有备节点）
1）修改配置文件 修改 node2 的$HGDB_HOME/conf 中 hg_repmgr.conf 文件，内容如下：
| node_id=2 node_name=node12 conninfo='host=node12 user=hgrepmgr dbname=hgrepmgr connect_timeout=2' data_directory='/opt/highgo/5.6.4/data' pg_bindir='/opt/highgo/5.6.4/bin' passfile='/home/highgo/.pgpass' |
2）检查备节点是否可被克隆
| repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -h node1 -U hgrepmgr -d hgrepmgr standby clone --dry-run
[highgo@node12 ~]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -h node1 -U hgrepmgr -d hgrepmgr standby clone --dry-run NOTICE: destination directory "/opt/highgo/5.6.4/data" provided INFO: connecting to source node DETAIL: connection string is: host=node1 user=hgrepmgr dbname=hgrepmgr DETAIL: current installation size is 1437 MB NOTICE: standby will attach to upstream node 1 HINT: consider using the -c/--fast-checkpoint option INFO: all prerequisites for "standby clone" are met [highgo@node12 ~]$ |
3）克隆备节点
| repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -h node1 -U hgrepmgr -d hgrepmgr standby clone
[highgo@node12 ~]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -h node1 -U hgrepmgr -d hgrepmgr standby clone NOTICE: destination directory "/opt/highgo/5.6.4/data" provided INFO: connecting to source node DETAIL: connection string is: host=node1 user=hgrepmgr dbname=hgrepmgr DETAIL: current installation size is 1437 MB INFO: checking and correcting permissions on existing directory "/opt/highgo/5.6.4/data" NOTICE: starting backup (using pg_basebackup)... HINT: this may take some time; consider using the -c/--fast-checkpoint option INFO: executing: /opt/highgo/5.6.4/bin/pg_basebackup -l "repmgr base backup" -D /opt/highgo/5.6.4/data -h node1 -p 5866 -U hgrepmgr -X stream
NOTICE: standby clone (using pg_basebackup) complete NOTICE: you can now start your PostgreSQL server HINT: for example: pg_ctl -D /opt/highgo/5.6.4/data start HINT: after starting the server, you need to register this standby with "repmgr standby register" [highgo@node12 ~]$ |
4）启动并注册备节点
| pg_ctl -D /opt/highgo/5.6.4/data start repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf standby register
[highgo@node12 ~]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf standby register INFO: connecting to local node "node12" (ID: 2) INFO: connecting to primary database WARNING: --upstream-node-id not supplied, assuming upstream node is primary (node ID 1) INFO: standby registration complete NOTICE: standby node "node12" (id: 2) successfully registered [highgo@node12 ~]$ |
5）查看集群状态
| repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf cluster show
[highgo@node1 ~]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf cluster show ID | Name | Role | Status | Upstream | Location | Connection string ----+-------+---------+-----------+----------+----------+------------------------------------------------------------ 1 | node1 | primary | * running | | default | host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2 2 | node12 | standby | running | node1 | default | host=node12 user=hgrepmgr dbname=hgrepmgr connect_timeout=2 [highgo@node1 ~]$ |
6）确认复制关系
在主节点查看复制关系：
| select * from pg_stat_replication;
highgo=# select * from pg_stat_replication; -[ RECORD 1 ]----+------------------------------ pid | 24650 usesysid | 17400 usename | hgrepmgr application_name | node12 client_addr | 192.168.6.17 client_hostname | client_port | 63418 backend_start | 2019-05-06 13:45:51.851471+08 backend_xmin | state | streaming sent_lsn | 1/AE0007B8 write_lsn | 1/AE0007B8 flush_lsn | 1/AE0007B8 replay_lsn | 1/AE0007B8 write_lag | flush_lag | replay_lag | sync_priority | 0 sync_state | async
highgo=# |
在备节点查看流复制状态
| select * from pg_stat_wal_receiver;
highgo=# select * from pg_stat_wal_receiver; -[ RECORD 1 ]---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- pid | 21859 status | streaming receive_start_lsn | 1/AE000000 receive_start_tli | 1 received_lsn | 1/AE0007B8 received_tli | 1 last_msg_send_time | 2019-05-06 13:48:29.919004+08 last_msg_receipt_time | 2019-05-06 13:48:29.91983+08 latest_end_lsn | 1/AE0007B8 latest_end_time | 2019-05-06 13:45:59.574451+08 slot_name | sender_host | node1 sender_port | 5866 conninfo | user=hgrepmgr passfile=/home/highgo/.pgpass connect_timeout=2 dbname=replication host=node1 port=5866 application_name=node12 fallback_application_name=walreceiver sslmode=prefer sslcompression=0 target_session_attrs=any
highgo=# |
12) 开启 repmgrd 进程（所有节点）
| [highgo@node1 conf]$ repmgrd -d -p /tmp/hg_repmgrd.pid [2019-05-06 14:02:42] [NOTICE] repmgrd (repmgrd 4.2) starting up [2019-05-06 14:02:42] [INFO] connecting to database "" [2019-05-06 14:02:43] [ERROR] repmgr extension not found on this node [2019-05-06 14:02:43] [DETAIL] repmgr extension is available but not installed in database "highgo" [2019-05-06 14:02:43] [HINT] check that this node is part of a repmgr cluster [highgo@node1 conf]$
highgo=# \c You are now connected to database "highgo" as user "highgo". create extension repmgr; [highgo@node1 conf]$ repmgrd -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -d -p /tmp/hg_repmgrd.pid [2019-05-06 14:21:21] [NOTICE] repmgrd (repmgrd 4.2) starting up [2019-05-06 14:21:21] [INFO] connecting to database "host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2" [highgo@node1 conf]$ хϢ: set_repmgrd_pid(): provided pidfile is /tmp/hg_repmgrd.pid [2019-05-06 14:21:21] [NOTICE] starting monitoring of node "node1" (ID: 1) [2019-05-06 14:21:21] [NOTICE] monitoring cluster primary "node1" (node ID: 1)
[highgo@node12 conf]$ repmgrd -f /opt/highgo/5.6.4/conf/hg_repmgr.conf -d -p /tmp/hg_repmgrd.pid [2019-05-06 14:21:50] [NOTICE] repmgrd (repmgrd 4.2) starting up [2019-05-06 14:21:50] [INFO] connecting to database "host=node12 user=hgrepmgr dbname=hgrepmgr connect_timeout=2" [highgo@node12 conf]$ хϢ: set_repmgrd_pid(): provided pidfile is /tmp/hg_repmgrd.pid [2019-05-06 14:21:50] [NOTICE] starting monitoring of node "node12" (ID: 2) [2019-05-06 14:21:50] [INFO] monitoring connection to upstream node "node1" (node ID: 1)
[highgo@node1 conf]$ ls -atl /tmp/hg_repmgrd.pid -rw-rw-r--. 1 highgo highgo 5 May 6 14:21 /tmp/hg_repmgrd.pid [highgo@node1 conf]$ [highgo@node12 conf]$ ls -atl /tmp/hg_repmgrd.pid -rw-rw-r--. 1 highgo highgo 5 May 6 14:21 /tmp/hg_repmgrd.pid [highgo@node12 conf]$ |
13) 查看集群状态
| [highgo@node1 conf]$ repmgr -f /opt/highgo/5.6.4/conf/hg_repmgr.conf cluster show ID | Name | Role | Status | Upstream | Location | Connection string ----+-------+---------+-----------+----------+----------+------------------------------------------------------------ 1 | node1 | primary | * running | | default | host=node1 user=hgrepmgr dbname=hgrepmgr connect_timeout=2 2 | node12 | standby | running | node1 | default | host=node12 user=hgrepmgr dbname=hgrepmgr connect_timeout=2 [highgo@node1 conf]$ |
更多详细信息请登录【瀚高技术支持平台】查看
