使用资源管理工具pcs配置Corosync+pacemaker
一、Corosync+pacemaker架构
1、高可用集群结构层：(从下往上)
1).Messaging Layer:消息层，传递心跳以及集群事务信息
corosync v1, v2(votequorum)
2).CRM:Cluster Resource Manager:集群资源管理器
作用：使本身不具备的高可用能力的服务也能够运行在高可用集群中，并以高可用模式提供服务
传递方式：CRM-->LRM:Local Resource Manager -->Pengine：Policy Engine(只运行在主节点上，决策)
CIB:Cluster Information Base:集群信息库
集群的全生命周期管理工具：
pcs: agent(pcsd)
crmsh: agentless (pssh)
独立服务：
pacemaker:CRM-->LRM-->Pengine-->CIB
重要的基本组件：cib/crmd/lrmd/pengine
配置接口：crmsh (agentless), pssh 并行ssh
pcs (agent、a/s), pcsd：由pcsd守护进程负责接收pcs发出的命令并执行
实现方式：group(资源组、资源类型), constraint(约束)
3).RA: Resource Agent:资源代理
作用：运行在各节点中的crm中的lrm负责管理本地资源，包括资源的启动、停止、重启、监控等（主要是一些控制脚本）
脚本实现方式：
LSB: /etc/rc.d/init.d/目录下的脚本； centos5/6/7
systemd：/usr/lib/systemd/system 真正读取：/etc/systemd/system/multi-user.wants 处于enable状态的服务；
OCF:Open Cluster Framework开放集群框架，公司或组织自行研发的脚本： [provider]
heartbeat、pacemaker、linbit
service：通用服务类型: /etc/ha.d/haresources.d/目录下的脚本；
stonith：STONITH：shoot the other node on the head 节点级别隔离：专用于stonith设备的类别
Fence: 资源级别的隔离
2、集群资源定义
1).资源类型：
primitive：主资源，原始资源；在集群中只能运行一个实例；
clone：克隆资源，在集群中可运行多个实例；
匿名克隆、全局惟一克隆、状态克隆(主动、被动)
multi-state(master/slave)：克隆资源的特殊实现；多状态资源；
group: 组资源；
启动或停止；资源监视；相关性：
2).资源属性：
priority: 优先级；
target-role：started, stopped, master;
is-managed: 是否允许集群管理此资源；
resource-stickiness: 资源粘性；
allow-migrate: 是否允许迁移；
3).约束：constraint
#“inf:”表示无穷大，以下配置表示mysqlip和(mynfs mysqld)间的黏性值无穷大，即三个资源始终在同一节点上运行。
#此外“-inf:”表示黏性值负无穷，定义-inf:的两个资源始终不在同一节点运行。
#mandatory默认值，表示强制约束，
location位置约束：资源对节点的倾向性,指定资源首选在哪些节点上运行
(-oo, +oo):
任何值+无穷大=无穷大
任何值+负无穷大=负无穷大
无穷大+负无穷大=负无穷大
colocation排列约束:资源彼此间是否能运行于同一节点的倾向性；指定哪些资源捆绑一起，在同一节点上运行
(-oo, +oo)
order顺序约束:多个资源启动顺序依赖关系;指定排列约束中的资源启动顺序，该顺序和colocation顺序相反
(-oo, +oo),Mandatory
3、配置属性
全局属性：property, stonith-enable等等；
高可用服务：资源，通过RA
webip, webstore, webserver
节点粘性：设置默认值m,共n个节点，则当前节点粘性为n*m
如：定义node1粘性100，其它为0，则各节点黏性值
node1: 100 + 0 + 0
node2: 0 + 0 + 0
node3: 0 + 0 + 0
4、pcs：集群的全生命周期管理工具：agent(pcsd)
属性定义参数：
cluster：集群
auth
setup
resource：资源
describe 描述
list 列出
create 创建
delete 删除
constraint：约束
colocation 倾向性
order 顺序
location 位置
property 全局属性
list 列出
set 设置
status：状态查看
config：配置
5、配置基础
CentOS 7： corosync v2 + pacemaker
corosync v2: vote system
pacemaker: 独立服务
配置集群的前提
(1).时间同步；
(2).基于当前正在使用的主机名互相访问；
(3).是否会用到仲裁设备；
二、配置pcs安装Corosync和pacemaker
1、配置集群的前提
1).设定各节点可以基于密钥进行ssh通信
[root@cen7 ~]# ssh-keygen -t rsa
[root@cen7 ~]# cd ~/.ssh
[root@cen7 .ssh]# ll
总用量 16
-rw-r--r--. 1 root root 401 6月 13 22:29 authorized_keys
-rw-------. 1 root root 1675 6月 13 22:28 id_rsa
-rw-r--r--. 1 root root 401 6月 13 22:28 id_rsa.pub
-rw-r--r--. 1 root root 1759 6月 13 23:08 known_hosts
[root@cen7 .ssh]# chmod 600 authorized_keys
[root@cen7 .ssh]# scp authorized_keys 192.168.88.133:/root/.ssh
2).确保时间点同步
[root@cen7 ~]# date;ssh node2 'date'
2018年 08月 02日 星期四 01:36:35 CST
2018年 08月 02日 星期四 01:36:36 CST
3).确保网络顺畅
[root@cen7 ~]# ping cen7.field.com
PING localhost (127.0.0.1) 56(84) bytes of data.
64 bytes from localhost (127.0.0.1): icmp_seq=1 ttl=64 time=0.127 ms
64 bytes from localhost (127.0.0.1): icmp_seq=2 ttl=64 time=0.069 ms
^C
--- localhost ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.069/0.098/0.127/0.029 ms
[root@cen7 ~]# ping node2.field.com
PING node2.field.com (192.168.88.133) 56(84) bytes of data.
64 bytes from node2.field.com (192.168.88.133): icmp_seq=1 ttl=64 time=1.46 ms
64 bytes from node2.field.com (192.168.88.133): icmp_seq=2 ttl=64 time=0.643 ms
^C
--- node2.field.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/m
4).所有节点的主机名称和对应的IP地址解析服务可以正常工作，且每个节点的主机名称需要跟"uname -n"命令的结果保持一致；
[root@cen7 ~]# uname -n ;ssh node2 'uname -n'
cen7.field.com
node2.field.com
因此，需要保证两个节点上的/etc/hosts文件均为下面的内容：
192.168.88.132 cen7.field.com cen7
192.168.88.133 node2.field.com node2
为了使得重新启动系统后仍能保持如上的主机名称，还分别需要在各节点执行类似如下的命令：
cen7:
[root@cen7 ~]# sed -i 's@\(HOSTNAME=\).*@\1cen7.field.com@g' /etc/sysconfig/network
[root@cen7 ~]# hostname cen7.field.com
Node2：
[root@node2 ~]# sed -i 's@\(HOSTNAME=\).*@\1node2.field.com@g' /etc/sysconfig/network
[root@node2 ~]# hostname node2.field.com
2、yum安装pcs
[root@cen7 ~]# cat /etc/resolv.conf
nameserver 8.8.8.8
nameserver 172.28.1.26
nameserver 172.28.1.28
[root@cen7 ~]# yum install pcs -y
已安装:
pcs.x86_64 0:0.9.162-5.el7.centos.1
作为依赖被安装:
avahi-libs.x86_64 0:0.6.31-19.el7 bc.x86_64 0:1.06.95-13.el7 cifs-utils.x86_64 0:6.2-10.el7 clufter-bin.x86_64 0:0.77.0-2.el7
clufter-common.noarch 0:0.77.0-2.el7 corosync.x86_64 0:2.4.3-2.el7_5.1 corosynclib.x86_64 0:2.4.3-2.el7_5.1 cups-libs.x86_64 1:1.6.3-35.el7
device-mapper-event.x86_64 7:1.02.146-4.el7 device-mapper-event-libs.x86_64 7:1.02.146-4.el7 device-mapper-persistent-data.x86_64 0:0.7.3-3.el7 gnutls.x86_64 0:3.3.26-9.el7
gssproxy.x86_64 0:0.7.0-17.el7 keyutils.x86_64 0:1.5.8-3.el7 libaio.x86_64 0:0.3.109-13.el7 libbasicobjects.x86_64 0:0.1.1-29.el7
libcgroup.x86_64 0:0.41-15.el7 libcollection.x86_64 0:0.7.0-29.el7 liberation-fonts-common.noarch 1:1.07.2-16.el7 liberation-sans-fonts.noarch 1:1.07.2-16.el7
libevent.x86_64 0:2.0.21-4.el7 libini_config.x86_64 0:1.3.1-29.el7 libldb.x86_64 0:1.2.2-1.el7 libnfsidmap.x86_64 0:0.25-19.el7
libpath_utils.x86_64 0:0.2.1-29.el7 libqb.x86_64 0:1.0.1-6.el7 libref_array.x86_64 0:0.1.5-29.el7 libtalloc.x86_64 0:2.1.10-1.el7
libtdb.x86_64 0:1.3.15-1.el7 libtevent.x86_64 0:0.9.33-2.el7 libtirpc.x86_64 0:0.2.4-0.10.el7 libverto-libevent.x86_64 0:0.2.5-4.el7
libwbclient.x86_64 0:4.7.1-6.el7 lvm2.x86_64 7:2.02.177-4.el7 lvm2-libs.x86_64 7:2.02.177-4.el7 net-snmp-libs.x86_64 1:5.7.2-33.el7_5.2
net-tools.x86_64 0:2.0-0.22.20131004git.el7 nettle.x86_64 0:2.7.1-8.el7 nfs-utils.x86_64 1:1.3.0-0.54.el7 overpass-fonts.noarch 0:2.1-1.el7
pacemaker.x86_64 0:1.1.18-11.el7_5.3 pacemaker-cli.x86_64 0:1.1.18-11.el7_5.3 pacemaker-cluster-libs.x86_64 0:1.1.18-11.el7_5.3 pacemaker-libs.x86_64 0:1.1.18-11.el7_5.3
perl-TimeDate.noarch 1:2.30-2.el7 psmisc.x86_64 0:22.20-15.el7 python-clufter.noarch 0:0.77.0-2.el7 python-lxml.x86_64 0:3.2.1-4.el7
quota.x86_64 1:4.01-17.el7 quota-nls.noarch 1:4.01-17.el7 resource-agents.x86_64 0:3.9.5-124.el7 rpcbind.x86_64 0:0.2.0-44.el7
ruby.x86_64 0:2.0.0.648-33.el7_4 ruby-irb.noarch 0:2.0.0.648-33.el7_4 ruby-libs.x86_64 0:2.0.0.648-33.el7_4 rubygem-bigdecimal.x86_64 0:1.2.0-33.el7_4
rubygem-io-console.x86_64 0:0.4.2-33.el7_4 rubygem-json.x86_64 0:1.7.7-33.el7_4 rubygem-psych.x86_64 0:2.0.0-33.el7_4 rubygem-rdoc.noarch 0:4.0.0-33.el7_4
rubygems.noarch 0:2.0.14.1-33.el7_4 samba-client-libs.x86_64 0:4.7.1-6.el7 samba-common.noarch 0:4.7.1-6.el7 samba-common-libs.x86_64 0:4.7.1-6.el7
tcp_wrappers.x86_64 0:7.6-77.el7 trousers.x86_64 0:0.3.14-2.el7
完毕！
可以发现，yum安装pcs后默认安装了众多依赖包，包括Corosync和pacemaker
3、使用pcs配置Corosync和pacemaker，配置集群
方便配置，在ansible中定义ha
[root@cen7 ~]# vim /etc/ansible/hosts
[ha]
192.168.88.132
192.168.88.133
1）.启动集群中的pscd服务
[root@cen7 ~]# ansible ha -m service -a 'name=pcsd state=started enabled=yes'
192.168.88.132 | SUCCESS => {
"changed": true,
"enabled": true,
"name": "pcsd",
"state": "started",
"status": {
.....
}
}
192.168.88.133 | SUCCESS => {
"changed": true,
"enabled": true,
"name": "pcsd",
"state": "started",
"status": {
......
}
}
2）.确认pcsd进程运行
[root@cen7 ~]# systemctl status pcsd
● pcsd.service - PCS GUI and remote configuration interface
Loaded: loaded (/usr/lib/systemd/system/pcsd.service; enabled; vendor preset: disabled)
Active: active (running) since 四 2018-08-02 01:53:52 CST; 49s ago
Docs: man:pcsd(8)
man:pcs(8)
Main PID: 2122 (pcsd)
CGroup: /system.slice/pcsd.service
└─2122 /usr/bin/ruby /usr/lib/pcsd/pcsd > /dev/null &
8月 02 01:53:49 cen7.field.com systemd[1]: Starting PCS GUI and remote configuration interface...
8月 02 01:53:52 cen7.field.com systemd[1]: Started PCS GUI and remote configuration interface...
3）.修改pcs安装时默认用户hacluster密码，两个节点密码设置一致。
[root@cen7 home]# ansible ha -m shell -a 'echo hacluster |passwd --stdin hacluster'
192.168.88.132 | SUCCESS | rc=0 >>
更改用户 hacluster 的密码 。
passwd：所有的身份验证令牌已经成功更新。
192.168.88.133 | SUCCESS | rc=0 >>
更改用户 hacluster 的密码 。
passwd：所有的身份验证令牌已经成功更新。
4）.集群节点间认证
注意认证时用户必须是pcs安装时的默认用户hacluster，否则无法认证通过
[root@cen7 home]# pcs cluster auth cen7.field.com node2.field.com -u hacluster
Password:
cen7.field.com: Authorized
Error: Unable to communicate with node2.field.com
此处认证不通过，逐一排查原因，可能是：iptables、setLinux、Firewall，centos7默认开启Firewall
[root@node2 home]# uname -n
node2.field.com
[root@node2 home]# service iptables stop
Redirecting to /bin/systemctl stop iptables.service
Failed to stop iptables.service: Unit iptables.service not loaded.
[root@node2 home]# getenforce
Disabled
[root@node2 home]# systemctl enable pcsd.service
[root@node2 home]# firewall-cmd --state
Running
#centos7默认开启Firewall
[root@node2 home]# systemctl stop firewalld
[root@node2 home]# firewall-cmd --state
not running
[root@node2 home]# chkconfig firewalld off
注意：正在将请求转发到“systemctl disable firewalld.service”。
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
关闭firewalld后认证通过
[root@cen7 home]# pcs cluster auth cen7.field.com node2.field.com -u hacluster
Password:
cen7.field.com: Authorized
node2.field.com: Authorized
5）.创建并启动集群
[root@cen7 home]# pcs cluster setup --name mycluster cen7.field.com node2.field.com
Destroying cluster on nodes: cen7.field.com, node2.field.com...
cen7.field.com: Stopping Cluster (pacemaker)...
node2.field.com: Stopping Cluster (pacemaker)...
node2.field.com: Successfully destroyed cluster
cen7.field.com: Successfully destroyed cluster
Sending 'pacemaker_remote authkey' to 'cen7.field.com', 'node2.field.com'
cen7.field.com: successful distribution of the file 'pacemaker_remote authkey'
node2.field.com: successful distribution of the file 'pacemaker_remote authkey'
Sending cluster config files to the nodes...
cen7.field.com: Succeeded
node2.field.com: Succeeded
Synchronizing pcsd certificates on nodes cen7.field.com, node2.field.com...
cen7.field.com: Success
node2.field.com: Success
Restarting pcsd on the nodes in order to reload the certificates...
cen7.field.com: Success
node2.field.com: Success
由以上信息，可以看到已经启动成功
[root@cen7 home]# cd /etc/corosync/
[root@cen7 corosync]# ls
corosync.conf corosync.conf.example corosync.conf.example.udpu corosync.xml.example uidgid.d
6）.编辑corosync.conf，配置如下：（单播配置）
说明：此处只是简单配置，使集群启动。
[root@cen7 corosync]# cat corosync.conf
totem {
version: 2
cluster_name: mycluster
secauth: off
transport: udpu
#传播协议：单播传输参数udpu
}
#在 nodelist 之下可以为某一节点设置只与该节点相关的信息，这些设置项只能包含在 node 之中，即只能对属于集群的节点服务器进行设置，
#而且只应包括那些与默认设置不同的参数。每台服务器都必须配置 ring0_addr。
nodelist {
node {
ring0_addr: cen7.field.com
nodeid: 1
}
node {
ring0_addr: node2.field.com
nodeid: 2
}
}
quorum {
provider: corosync_votequorum
two_node: 1
}
logging {
to_logfile: yes
logfile: /var/log/cluster/corosync.log
to_syslog: yes
}
7）.生成节点间通信时用到的认证密钥文件
[root@cen7 corosync]# corosync-keygen
[root@cen7 corosync]# scp corosync.conf authkey node2:/etc/corosync
8）.启动集群所有节点
[root@cen7 corosync]# pcs cluster start --all
cen7.field.com: Starting Cluster...
node2.field.com: Starting Cluster...
9）.确认pcsd进程运行
[root@cen7 corosync]# ps -aux |tail
root 5857 6.7 22.1 196292 95992 ? SLsl 03:34 0:06 corosync
root 5864 0.3 1.6 104252 7196 ? Ss 03:34 0:00 /usr/sbin/pacemakerd -f
haclust+ 5865 1.8 3.2 106016 13996 ? Ss 03:34 0:01 /usr/libexec/pacemaker/cib
root 5866 0.3 1.3 106764 6004 ? Ss 03:34 0:00 /usr/libexec/pacemaker/stonithd
root 5867 0.1 1.0 98776 4364 ? Ss 03:34 0:00 /usr/libexec/pacemaker/lrmd
haclust+ 5868 0.4 1.2 127960 5620 ? Ss 03:34 0:00 /usr/libexec/pacemaker/attrd
haclust+ 5869 1.9 4.5 116520 19824 ? Ss 03:34 0:01 /usr/libexec/pacemaker/pengine
haclust+ 5870 1.7 1.9 181324 8604 ? Ss 03:34 0:01 /usr/libexec/pacemaker/crmd
root 5892 1.0 0.4 155324 1880 pts/0 R+ 03:35 0:00 ps -aux
root 5893 0.0 0.1 108200 664 pts/0 S+ 03:35 0:00 tail
查看corosync-cfgtool工具使用帮助
[root@cen7 corosync]# corosync-cfgtool --help
corosync-cfgtool: invalid option -- '-'
corosync-cfgtool [-i <interface ip>] [-s] [-r] [-R] [-k nodeid] [-a nodeid] [-h] [-H]
A tool for displaying and configuring active parameters within corosync.
options:
-i Finds only information about the specified interface IP address.
-s Displays the status of the current rings on this node.
-r Reset redundant ring state cluster wide after a fault to
re-enable redundant ring operation.
-R Tell all instances of corosync in this cluster to reload corosync.conf.
-k Kill a node identified by node id.
-a Display the IP address(es) of a node
-h Print basic usage.
-H Shutdown corosync cleanly on this node.
corosync-cfgtool: invalid option -- 'e'
corosync-cfgtool: invalid option -- 'l'
corosync-cfgtool: invalid option -- 'p'
[root@cen7 corosync]# corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id = 127.0.0.1
status = ring 0 active with no faults
[root@cen7 corosync]#
注意：id = 127.0.0.1 表示集群失败，使用“pcs status”可以发现Node node2.field.com: UNCLEAN (offline)
原因是/etc/hosts中配置了127.0.0.1
编辑两台机的/etc/hosts,注销如下行并重启
[root@cen7 ~]# vim /etc/hosts
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 cen7.field.com
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.88.132 cen7.field.com cen7
192.168.88.133 node2.field.com node2
10）.查看集群节点的启动状态
[root@cen7 corosync]# pcs status
Cluster name: mycluster
WARNING: no stonith devices and stonith-enabled is not false
Stack: corosync
Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition WITHOUT quorum
Last updated: Thu Aug 2 03:47:09 2018
Last change: Thu Aug 2 03:34:33 2018 by hacluster via crmd on cen7.field.com
2 nodes configured
0 resources configured
Node node2.field.com: UNCLEAN (offline)
Online: [ cen7.field.com ]
No resources
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
从上面的信息可以看出只启动一个节点，处于不正常工作状态。
[root@cen7 corosync]# reboot
可以看到，重启已经正常启动节点
[root@cen7 ~]# pcs cluster start --all
cen7.field.com: Starting Cluster...
node2.field.com: Starting Cluster...
[root@cen7 ~]# corosync-cfgtool -s
Printing ring status.
Local node ID 1
RING ID 0
id = 192.168.88.132
status = ring 0 active with no faults
[root@cen7 ~]# corosync-cmapctl |grep members
runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.88.132)
runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.1.status (str) = joined
runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.88.133)
runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.2.status (str) = joined
[root@cen7 ~]# pcs status
Cluster name: mycluster
WARNING: no stonith devices and stonith-enabled is not false
Stack: corosync
Current DC: cen7.field.com (version 1.1.18-11.el7_5.3-2b07d5c5a9) - partition with quorum
Last updated: Thu Aug 2 03:59:40 2018
Last change: Thu Aug 2 03:34:33 2018 by hacluster via crmd on cen7.field.com
2 nodes configured
0 resources configured
Online: [ cen7.field.com node2.field.com ]
No resources
Daemon Status:
corosync: active/disabled
pacemaker: active/disabled
pcsd: active/enabled
从上面的信息可以看出两个节点都已经正常启动，并且集群已经处于正常工作状态。
[root@node2 ~]# corosync-cfgtool -s
Printing ring status.
Local node ID 2
RING ID 0
id = 192.168.88.133
status = ring 0 active with no faults
[root@node2 ~]# corosync-cmapctl |grep members
runtime.totem.pg.mrp.srp.members.1.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.88.132)
runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.1.status (str) = joined
runtime.totem.pg.mrp.srp.members.2.config_version (u64) = 0
runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.88.133)
runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1
runtime.totem.pg.mrp.srp.members.2.status (str) = joined
查看properties帮助
[root@cen7 ~]# pcs properties -h
Usage: pcs [-f file] [-h] [commands]...
Control and configure pacemaker and corosync.
Options:
-h, --help Display usage and exit.
-f file Perform actions on file instead of active CIB.
--debug Print all network traffic and external commands run.
--version Print pcs version information. List pcs capabilities if
--full is specified.
--request-timeout Timeout for each outgoing request to another node in
seconds. Default is 60s.
--force Override checks and errors, the exact behavior depends on
the command. WARNING: Using the --force option is
strongly discouraged unless you know what you are doing.
Commands:
cluster Configure cluster options and nodes.
resource Manage cluster resources.
stonith Manage fence devices.
constraint Manage resource constraints.
property Manage pacemaker properties.
acl Manage pacemaker access control lists.
qdevice Manage quorum device provider on the local host.
quorum Manage cluster quorum settings.
booth Manage booth (cluster ticket manager).
status View cluster status.
config View and manage cluster configuration.
pcsd Manage pcs daemon.
node Manage cluster nodes.
alert Manage pacemaker alerts.
查看所有全局设定
[root@cen7 ~]# pcs property list --all
Cluster Properties:
batch-limit: 0
cluster-delay: 60s
cluster-infrastructure: corosync
cluster-ipc-limit: 500
cluster-name: mycluster
cluster-recheck-interval: 15min
concurrent-fencing: false
crmd-finalization-timeout: 30min
crmd-integration-timeout: 3min
crmd-transition-delay: 0s
dc-deadtime: 20s
dc-version: 1.1.18-11.el7_5.3-2b07d5c5a9
default-action-timeout: (null)
default-resource-stickiness: (null)
election-timeout: 2min
enable-acl: false
enable-startup-probes: true
have-watchdog: false
is-managed-default: (null)
load-threshold: 80%
maintenance-mode: false
migration-limit: -1
no-quorum-policy: stop
node-action-limit: 0
node-health-base: 0
node-health-green: 0
node-health-red: -INFINITY
node-health-strategy: none
node-health-yellow: 0
notification-agent: /dev/null
notification-recipient:
pe-error-series-max: -1
pe-input-series-max: 4000
pe-warn-series-max: 5000
placement-strategy: default
remove-after-stop: false
shutdown-escalation: 20min
start-failure-is-fatal: true
startup-fencing: true
stonith-action: reboot
stonith-enabled: true
stonith-max-attempts: 10
stonith-timeout: 60s
stonith-watchdog-timeout: (null)
stop-all-resources: false
stop-orphan-actions: true
stop-orphan-resources: true
symmetric-cluster: true
11）.检查全局配置是否有问题
[root@cen7 ~]# crm_verify -L -V
error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
以上错误是因为默认开启stonish,配置集群的工作属性，禁用stonith，可消除该错误。
[root@cen7 ~]# pcs property set stonith-enabled=false
pcs property list ：只显示修改项
[root@cen7 ~]# pcs property list
Cluster Properties:
cluster-infrastructure: corosync
cluster-name: mycluster
dc-version: 1.1.18-11.el7_5.3-2b07d5c5a9
have-watchdog: false
stonith-enabled: false
[root@cen7 ~]# crm_verify -L -V
[root@cen7 ~]#