_MyCat high availability load balancing cluster implementation (HAProxy+Keepalived+MyCat)

Questions to be addressed in this course:

1. Software version

Operating System: CentOS-6.6-x86_64
JDK Version: jdk1.7.0_72
Keepalived Version: keepalived-1.2.18.tar.gz
HAProxy Version: haproxy-1.5.16.tar.gz
MyCat Version: Mycat-server-1.4-release- 20151019230038-linux.tar.gz
MySQL version: mysql-5.6.26.tar.gz

2. Deployment environment planning

name	IP	CPU name	configuration
HAProxy host 1	VIP:192.168.175.220\|192.168.175.202	mini2 (default Master)	-
HAProxy host 2	VIP:192.168.1.220\|192.168.175.203	mini3 (default Backup)	-
MyCat host 1	192.168.175.200	Huachao	-
MyCat host 2	192.168.175.201	mini1	-
MySQL master node	192.168.175.200	Huachao	-
MySQL slave node	192.168.175.201	mini1	-

3. MyCat high availability load balancing cluster deployment architecture diagram is as follows:

MySQL high availability cluster separation architecture

Graphical explanation:
(1) HAProxy realizes MyCat multi-node cluster high availability and load balancing, while HAProxy's own high availability can be realized through Keepalived. Therefore, HAProxy and Keepalived should be installed on the HAProxy host at the same time. Keepalived is responsible for preempting the vip (virtual ip) for the server. After the vip is preempted, the host can be accessed through the original ip or directly through the vip (virtual) .
(2) Keepalived preempts vip with priority, which is determined by the priority attribute in keepalived.conf configuration. But generally, the Keepalived service on whichever host starts first will grab the vip, even if it is a slave, as long as it starts first, it can grab it (pay attention to avoid Keepalived resource preemption).
(3) HAProxy is responsible for distributing the requests for VIPs to MyCat cluster nodes, which plays a role of load balancing. At the same time, HAProxy can also detect whether MyCat is alive, and HAProxy will only forward the request to the surviving MyCat.
(4) If one server in the Keepalived+HAProxy high-availability cluster goes down, Keepalived on another server in the cluster will immediately preempt the VIP and take over the service. At this time, the HAProxy node that preempted the VIP can continue to provide services.
(5) If a MyCat server is down, HAPorxy will not forward the request to the down MyCat, so MyCat is still available. To sum up: MyCat's high availability and load balancing are implemented by HAProxy, and HAProxy's high availability is implemented by Keepalived.

4. Deployment of HAProxy node 2

HAProxy host 2 (edu-haproxy-02, 192.168.175.201) Please refer to the previous course "High Availability Architecture - Section 16 -
MyCat Cluster Deployment (HAProxy+MyCat)" for peer-to-peer deployment and corresponding configuration .
Pay attention to the adjustment of the configuration file:
when deploying multiple nodes, the node and description configuration values in the haproxy.cfg configuration file should be adjusted accordingly.
HAProxy node 1 status information page: http://192.168.175.202:48800/admin-status

HAProxy node 2 status information page: http://192.168.175.203:48800/admin-status

5. Introduction to Keepalived (official website: http://www.haproxy.org/ )

Keepalived is a high-performance server high-availability or hot standby solution. Keepalived can be used to prevent server single-point failures. By cooperating with Haproxy, it can achieve high availability of web front-end services.
Keepalived is based on the VRRP protocol and uses the VRRP protocol to achieve high availability (HA). The VRRP (Virtual Router Redundancy Protocol) protocol is a protocol used to implement router redundancy. The VRRP protocol virtualizes two or more router devices into one device, and provides virtual router IP (one or more) externally. , if the router that actually owns this external IP works normally, it is the MASTER, or it is elected through an algorithm. MASTER implements various network functions for the virtual router IP, such as ARP request, ICMP, and data forwarding, etc.; other devices do not own the virtual IP, and the status is BACKUP. Except for receiving the VRRP status notification information from MASTER, no external communication is performed. Internet function. When the master fails, BACKUP will take over the network functions of the original MASTER.
The VRRP protocol uses multicast data to transmit VRRP data. VRRP data uses a special virtual source MAC address to send data instead of the MAC address of its own network card. When VRRP is running, only the MASTER router regularly sends VRRP notification information, indicating that the MASTER is working normally and the virtual router IP (group), BACKUP only receives VRRP data and does not send data. If no notification information from MASTER is received within a certain period of time, each BACKUP will declare itself as a MASTER, send notification information, and re-run the MASTER election state.

6. Installation of Keepalived (192.168.1.191, 192.168.1.192)

Keepalived download address

Upload or download keepalived (keepalived-1.2.18.tar.gz) to /usr/local/src directory
Unzip and
install Openssl is required to install keepalived
# yum install gcc gcc-c++ openssl openssl-devel
# cd /usr/local/src
# tar -zxvf keepalived-1.2.18.tar.gz
# cd keepalived-1.2.18
# ./configure --prefix=/usr/local/keepalived
# make && make install

Install keepalived as a Linux system service:
because the default path of keepalived is not installed (default is /usr/local), after the installation is complete, some work needs to be done. Copy the
default configuration file to the default path.
# mkdir /etc/keepalived
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
Copy the keepalived service script to the default address
# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/
# ln -s /usr/local/keepalived/sbin/keepalived /sbin/
to set keepalived service start
# chkconfig keepalived on
Modify the Keepalived configuration file
(1) MASTER node configuration file (192.168.175.202)

# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
## keepalived 自带的邮件提醒需要开启 sendmail 服务。建议用独立的监控或第三方 SMTP
router_id edu-haproxy-01 ## 标识本节点的字条串，通常为 hostname
}
## keepalived 会定时执行脚本并对脚本执行的结果进行分析，动态调整 vrrp_instance 的优先级。
## 如果脚本执行结果为 0，并且 weight 配置的值大于 0，则优先级相应的增加。
## 如果脚本执行结果非 0，并且 weight 配置的值小于 0，则优先级相应的减少。
## 其他情况，维持原本配置的优先级，即配置文件中 priority 对应的值。
vrrp_script chk_haproxy {
script "/etc/keepalived/haproxy_check.sh" ## 检测 haproxy 状态的脚本路径
interval 2 ## 检测时间间隔
weight 2 ## 如果条件成立，权重+2
}
## 定义虚拟路由， VI_1 为虚拟路由的标示符，自己定义名称
vrrp_instance VI_1 {
state BACKUP ## 默认主设备（ priority 值大的）和备用设备（ priority 值小的）都设置为 BACKUP，
## 由 priority 来控制同时启动情况下的默认主备，否则先启动的为主设备
interface eth1 ## 绑定虚拟 IP 的网络接口，与本机 IP 地址所在的网络接口相同，我的是 eth1
virtual_router_id 91 ## 虚拟路由的 ID 号，两个节点设置必须一样，可选 IP 最后一段使用,
## 相同的 VRID 为一个组，他将决定多播的 MAC 地址
priority 120 ## 节点优先级，值范围 0-254， MASTER 要比 BACKUP 高
nopreempt ## 主设备（ priority 值大的）配置一定要加上 nopreempt，否则非抢占也不起作用
advert_int 1 ## 组播信息发送间隔，两个节点设置必须一样，默认 1s
## 设置验证信息，两个节点必须一致
authentication {
auth_type PASS
auth_pass 1111 ## 真实生产，按需求对应该过来
}
## 将 track_script 块加入 instance 配置块
track_script {
chk_haproxy ## 检查 HAProxy 服务是否存活
}
## 虚拟 IP 池, 两个节点设置必须一样
virtual_ipaddress {
192.168.175.220 ## 虚拟 ip，可以定义多个，每行一个
}
}

(2) BACKUP node configuration file (192.168.175.203):
# vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
router_id edu-haproxy-02
}
vrrp_script chk_haproxy {
script "/etc/keepalived/haproxy_check.sh"
interval 2
weight 2
}
vrrp_instance VI_1 {
state BACKUP
interface eth1
virtual_router_id 91
priority 110
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_haproxy
}
virtual_ipaddress {
192.168.175.220
}
}

Special attention: If the non-preemptive mode does not take effect, after the faulty node of Keepalived recovers, it will lead to preemptive vip again, so there is a
risk of flash disconnection due to vip switching (video explanation). According to the above configuration, the Keepalived non-preemptive mode is configured. The configuration and attention points are as follows:
(1) The state of the master device and the slave device are both set to BACKUP
(2) Do not configure mcast_src_ip (the local IP address) on the master device and the slave device )
(3) The default master device (Keepalived node with a large priority value) must be configured with nopreempt, otherwise non-preemption will not
work. 224.0.0.18 is
the communication IP of the Master and Backup health checks)
# iptables -I INPUT -i eth0 -d 224.0.0.0/8 -p vrrp -j ACCEPT
# iptables -I OUTPUT -o eth0 -d 224.0.0.0/8 -p vrrp -j ACCEPT
(eth0 is the name of the network card device of the host, and the server in the production environment can use an independent network card to handle multicast and heartbeat detection, etc.)
# service iptables save
Restart the firewall:
# service iptables restart

Write the Haproxy status detection script /etc/keepalived/haproxy_check.sh (already configured in keepalived.conf)
The script requires: if haproxy stops running, try to start it, if it fails to start, kill the local keepalived process, keepalived
binds the virtual ip Set to the BACKUP machine. The content is as follows:
# mkdir -p /usr/local/keepalived/log
# vi /etc/keepalived/haproxy_check.sh

#!/bin/bash
START_HAPROXY="/etc/rc.d/init.d/haproxy start"
STOP_HAPROXY="/etc/rc.d/init.d/haproxy stop"
LOG_FILE="/usr/local/keepalived/log/haproxy-check.log"
HAPS=`ps -C haproxy --no-header |wc -l`
date "+%Y-%m-%d %H:%M:%S" >> $LOG_FILE
echo "check haproxy status" >> $LOG_FILE
if [ $HAPS -eq 0 ];then
echo $START_HAPROXY >> $LOG_FILE
$START_HAPROXY >> $LOG_FILE 2>&1
sleep 3
if [ `ps -C haproxy --no-header |wc -l` -eq 0 ];then
echo "start haproxy failed, killall keepalived" >> $LOG_FILE
killall keepalived
fi
fi

After saving, give the script execution permission:
# chmod +x /etc/keepalived/haproxy_check.sh

启动 Keepalived
# service keepalived start
Starting keepalived: [ OK ]

Keepalived service management command:
Stop: service keepalived stop
Start: service keepalived start
Restart: service keepalived restart
View status:service keepalived status

7. High availability test of Keepalived + Haproxy

Shut down Haproxy on 192.168.1.191, Keepalived will restart it
# service haproxy stop
Turn off Keepalived in 192.168.1.191, VIP (192.168.1.190) will be preempted by 192.168.1.192
# service keepalived stop

image.png

After Keepalived is stopped, the VIP (192.168.1.190) in the network interface of the 192.168.1.191 node will disappear

image.png

To view the MAC corresponding to the VIP at this time, use the CMD command to view it under Windows:

image.png

It means that the VIP has drifted to the physical host 192.168.1.192 at this time,
and then access the Haproxy cluster through the VIP, and the access is also 192.168.1.192
http://192.168.1.190:48800/admin-status