030：Cetus中间件和MHA读写分离

其他 2018-08-31 18:35:04 阅读次数: 0

030：Cetus中间件和MHA读写分离

一、主机环境

  • 虚拟机配置
CPU 内存 硬盘 OS版本 MySQL版本 MHA版本 Cetus版本
2-core 4G 500G CentOS 7.5.1804 5.7.18 0.57 v1.0.0-44
  • 主机信息
主机名 IP地址 Server_ID MHA Manager MHA Node Cetus 备注
node05 192.168.222.175 部署 部署 部署 监控/MySQL主库的故障转移
node02 192.168.222.172 172 - 部署 -
node03 192.168.222.173 173 - 部署 -
node04 192.168.222.174 174 - 部署 -

二、搭建主从

1、配置主从

  • 搭建基于GTID主从复制环境
  • 若开启主从延迟检测需创建库proxy_heart_beat和表tb_heartbeat
  • 创建用户和密码（默认用户对tb_heartbeat有读写权限）
  • 确认Cetus可以远程登录MySQL
0.主库创建同步帐号grant RELOAD,REPLICATION SLAVE, REPLICATION CLIENT on *.* to repl@'%' identified by password 'xxxxxx'; flush privileges;
1.mysqldump 出主库数据，增加 master-data=2；
2.导入从库；
3.change master to master_host="192.168.222.171", master_port=3306, master_user='repl',master_password='repl', master_auto_position=1;
4.start slave;
5.创建mysql cetus测试账号 grant all on *.* to gcdb@'%' identified by password 'xxxxxx'; flush privileges;
6.备库设置set global read_only=1;
7.创建延迟校库proxy_heart_beat和表tb_heartbeat
CREATE DATABASE proxy_heart_beat;
USE proxy_heart_beat;
CREATE TABLE tb_heartbeat ( p_id varchar(128) NOT NULL, p_ts timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), PRIMARY KEY (p_id) ) ENGINE = InnoDB DEFAULT CHARSET = utf8;
8.创建测试库ttt
create database if not exists ttt;
create table ttt.t1(id int(4)primary key not null auto_increment,nums int(20) not null);
insert into ttt.t1(nums) values(1),(2),(3),(4),(5);
update ttt.t1 set nums=100 where id =3;
delete from ttt.t1 where id =4;
select * from ttt.t1;
9.安装sendmail服务
yum install sendmail -y

2、 配置 hosts

  • 添加hosts
cat <<EOF >/etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.222.245 node01.mysql.com  node01
192.168.222.246 mycat01.mysql.com mycat01
192.168.222.247 node02.mysql.com  node02
192.168.222.248 node03.mysql.com  node03
192.168.222.249 node04.mysql.com  node04
192.168.222.59 node05.mysql.com  node05
192.168.222.251 redis01.mysql.com  redis01
192.168.222.252 redis02.mysql.com  redis02
EOF

3、配置免秘钥

  • mha管理节点：
ssh-keygen -t rsa

ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
  • mysql节点各自生成公私钥，并将公钥拷贝给其他mysql节点
# 192.168.222.172
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"

# 192.168.222.173
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"

# 192.168.222.174
ssh-keygen -t rsa
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
ssh-copy-id -i /root/.ssh/id_rsa.pub "[email protected]"
  • 测试ssh是否免密码登录

三、安装MHA和Cetus

1、下载包和安装依赖包

mha4mysql-manager-0.57.tar.gz和mha4mysql-node-0.57.tar.gz

下载地址

cetus源码下载
git clone https://github.com/Lede-Inc/cetus.git
# 安装Cetus依赖包
yum -y install cmake gcc glib2-devel flex libevent-devel mysql-devel gperftools-libs 
# 安装mha依赖包
yum -y install perl-DBD-MySQL perl-Config-Tiny perl-Log-Dispatch perl-Parallel-ForkManager perl-ExtUtils-CBuilder perl-ExtUtils-MakeMaker perl-CPAN perl-ExtUtils-Manifest

2、安装MHA Node

tar zxvf mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57
perl Makefile.PL
make && make install

-- Node安装完成后会在/usr/local/bin/下生成这些文件
#save_binary_logs : 保存和复制master的二进制日志。 
#apply_diff_relay_logs : 识别差异的中继日志事件并应用于其它slave。 
#filter_mysqlbinlog : 去除不必要的ROLLBACK事件(MHA已不再使用这个工具)。 
#purge_relay_logs : 清除中继日志(不会阻塞SQL线程)。

3、安装MHA manager

  • 在所有的 MySQL上安装 Node 节点；在 MHA 的管理节点安装manager 节点。
tar zvxf mha4mysql-manager-0.57.tar.gz 
cd mha4mysql-manager-0.57
perl Makefile.PL
make && make install

-- Manage安装完成后会在/usr/local/bin/下生成这些文件
#masterha_check_ssh : 检查MHA的SSH配置。 
#masterha_check_repl : 检查MySQL复制。 
#masterha_manager : 启动MHA。 
#masterha_check_status : 检测当前MHA运行状态。 
#masterha_master_monitor : 监测master是否宕机。 
#masterha_master_switch : 控制故障转移(自动或手动)。 
#masterha_conf_host : 添加或删除配置的server信息。

 cp -r /software/mha4mysql-manager-0.57/samples/scripts/* /usr/local/bin/
-- 拷贝/software/mha4mysql-manager-0.57/samples/scripts/下的脚本到/usr/local/bin/

#master_ip_failover ：自动切换时vip管理的脚本，不是必须，如果我们使用keepalived的，我们可以自己编写脚本完成对vip的管理，比如监控mysql，如果mysql异常，我们停止keepalived就行，这样vip就会自动漂移
#master_ip_online_change：在线切换时vip的管理，不是必须，同样可以可以自行编写简单的shell完成
#power_manager：故障发生后关闭主机的脚本，不是必须
#send_report：因故障切换后发送报警的脚本，不是必须，可自行编写简单的shell完成。

4、安装Cetus

1.安装说明

Cetus利用自动化建构系统CMake进行编译安装，其中描述构建过程的构建文件CMakeLists.txt已经在源码中的主目录和子目录中，下载源码并解压后具体安装步骤如下：

  • 创建编译目录：在源码主目录下创建独立的目录build，并转到该目录下
mkdir build/
cd build/
  • 编译：利用cmake进行编译，指令如下
读写分离版本：
cmake ../ -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/home/user/cetus_install -DSIMPLE_PARSER=ON

分库版本：
cmake ../ -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/home/user/cetus_install -DSIMPLE_PARSER=OFF

其中CMAKE_BUILD_TYPE变量可以选择生成 debug 版和或release 版的程序，CMAKE_INSTALL_PREFIX变量确定软件的实际安装目录的绝对路径，安装目录建议以/home/user/日期.编译版本.分支.commit_id的方式命名；SIMPLE_PARSER变量确定软件的编译版本，设置为ON则编译读写分离版本，否则编译分库版本。

该过程会检查您的系统是否缺少一些依赖库和依赖软件，可以根据错误代码安装相应依赖。

  • 安装：执行make install进行安装
make install
  • 配置：Cetus运行前还需要编辑配置文件
cd /home/user/cetus_install/conf/
cp XXX.json.example XXX.json
cp XXX.conf.example XXX.conf
vi XXX.json
vi XXX.conf

配置文件在make insatll后存在示例文件，以.example结尾，目录为/home/user/cetus_install/conf/，包括用户设置文件（users.json）、变量处理配置文件（variables.json）、分库版本的分片规则配置文件（sharding.json）、读写分离版本的启动配置文件（proxy.conf）和分库版本的启动配置文件（shard.conf）。

根据具体编译安装的版本编辑相关配置文件，若使用读写分离功能则需配置users.json和proxy.conf，若使用sharding功能则需配置users.json、sharding.json和shard.conf，其中两个版本的variables.json均可选配。

配置文件的具体说明见Cetus 读写分离版配置文件说明和Cetus 分库(sharding)版配置文件说明。

  • 启动：Cetus可以利用bin/cetus启动
读写分离版本：
bin/cetus --defaults-file=conf/proxy.conf [--conf-dir＝/home/user/cetus_install/conf/]

分库版本：
bin/cetus --defaults-file=conf/shard.conf [--conf-dir＝/home/user/cetus_install/conf/]

其中Cetus启动时可以添加命令行选项，--defaults-file选项用来加载启动配置文件（proxy.conf或者shard.conf），且在启动前保证启动配置文件的权限为660；--conf-dir是可选项，用来加载其他配置文件(.json文件)，默认为当前目录下conf文件夹。

Cetus可起动守护进程后台运行，也可在进程意外终止自动启动一个新进程，可通过启动配置选项进行设置。

2.安装实施

#进入源码目录
[root@node05 software]# git clone https://github.com/Lede-Inc/cetus.git
[root@node05 software]# cd cetus/
[root@node05 cetus]# mkdir build/ && cd build
[root@node05 build]# cmake ../ -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/usr/local/cetus -DSIMPLE_PARSER=ON
[root@node05 build]# make install
[root@node05 build]# ll /usr/local/cetus/
total 0
drwxr-xr-x 2 root root  19 Aug 31 09:29 bin
drwxr-xr-x 2 root root 143 Aug 31 09:29 conf
drwxr-xr-x 4 root root 210 Aug 31 09:29 lib
drwxr-xr-x 2 root root  19 Aug 31 09:29 libexec
drwxr-xr-x 2 root root  23 Aug 31 09:29 logs

[root@node05 build]# ll /usr/local/share/perl5/MHA/
total 388
-r--r--r-- 1 root root 12997 May 31  2015 BinlogHeaderParser.pm
-r--r--r-- 1 root root 18783 May 31  2015 BinlogManager.pm
-r--r--r-- 1 root root  2251 May 31  2015 BinlogPosFinderElp.pm
-r--r--r-- 1 root root  1648 May 31  2015 BinlogPosFinder.pm
-r--r--r-- 1 root root  3130 May 31  2015 BinlogPosFinderXid.pm
-r--r--r-- 1 root root  5684 May 31  2015 BinlogPosFindManager.pm
-r--r--r-- 1 root root 17480 May 31  2015 Config.pm
-r--r--r-- 1 root root 27019 May 31  2015 DBHelper.pm
-r--r--r-- 1 root root  3075 May 31  2015 FileStatus.pm
-r--r--r-- 1 root root 20370 May 31  2015 HealthCheck.pm
-r--r--r-- 1 root root 10560 May 31  2015 ManagerAdmin.pm
-r--r--r-- 1 root root  3679 May 31  2015 ManagerAdminWrapper.pm
-r--r--r-- 1 root root  3508 May 31  2015 ManagerConst.pm
-r--r--r-- 1 root root  4612 May 31  2015 ManagerUtil.pm
-r--r--r-- 1 root root 74140 May 31  2015 MasterFailover.pm
-r--r--r-- 1 root root 23654 May 31  2015 MasterMonitor.pm
-r--r--r-- 1 root root 23466 May 31  2015 MasterRotate.pm
-r--r--r-- 1 root root  1308 May 31  2015 NodeConst.pm
-r--r--r-- 1 root root  6689 May 31  2015 NodeUtil.pm
-r--r--r-- 1 root root 44325 May 31  2015 ServerManager.pm
-r--r--r-- 1 root root 33213 May 31  2015 Server.pm
-r--r--r-- 1 root root  6531 May 31  2015 SlaveUtil.pm
-r--r--r-- 1 root root  4874 May 31  2015 SSHCheck.pm
[root@node05 build]#

5、cetus替换mha部分文件

#使用 mha_ld/src 替换所有文件/usr/share/perl5/vendor_perl/MHA/目录的所有同名文件
[root@node05 build]# cd ../mha_ld/src/
[root@node05 src]# ls
BinlogHeaderParser.pm  BinlogPosFindManager.pm  HealthCheck.pm          MasterFailover.pm         NodeUtil.pm      ServerManager.pm
BinlogManager.pm       cetus.cnf                ManagerAdmin.pm         masterha_secondary_check  ProxyManager.pm  Server.pm
BinlogPosFinderElp.pm  Config.pm                ManagerAdminWrapper.pm  MasterMonitor.pm          sample.cnf       SlaveUtil.pm
BinlogPosFinder.pm     DBHelper.pm              ManagerConst.pm         MasterRotate.pm           sendMail.sh      SSHCheck.pm
BinlogPosFinderXid.pm  FileStatus.pm            ManagerUtil.pm          NodeConst.pm              sendmail.txt

[root@node05 src]# ls /usr/local/share/perl5/MHA/
BinlogHeaderParser.pm  BinlogPosFinderXid.pm    FileStatus.pm           ManagerConst.pm    MasterRotate.pm   Server.pm
BinlogManager.pm       BinlogPosFindManager.pm  HealthCheck.pm          ManagerUtil.pm     NodeConst.pm      SlaveUtil.pm
BinlogPosFinderElp.pm  Config.pm                ManagerAdmin.pm         MasterFailover.pm  NodeUtil.pm       SSHCheck.pm
BinlogPosFinder.pm     DBHelper.pm              ManagerAdminWrapper.pm  MasterMonitor.pm   ServerManager.pm

[root@node05 src]# rsync /software/cetus/mha_ld/src/* /usr/local/share/perl5/MHA/
[root@node05 src]#  ls /usr/local/share/perl5/MHA/
BinlogHeaderParser.pm  BinlogPosFindManager.pm  HealthCheck.pm          MasterFailover.pm         NodeUtil.pm      ServerManager.pm
BinlogManager.pm       cetus.cnf                ManagerAdmin.pm         masterha_secondary_check  ProxyManager.pm  Server.pm
BinlogPosFinderElp.pm  Config.pm                ManagerAdminWrapper.pm  MasterMonitor.pm          sample.cnf       SlaveUtil.pm
BinlogPosFinder.pm     DBHelper.pm              ManagerConst.pm         MasterRotate.pm           sendMail.sh      SSHCheck.pm
BinlogPosFinderXid.pm  FileStatus.pm            ManagerUtil.pm          NodeConst.pm              sendmail.txt

#使用 mha_ld/masterha_secondary_check替换masterha_secondary_check命令 which masterha_secondary_check
[root@node05 src]# which masterha_secondary_check
/usr/local/bin/masterha_secondary_check
[root@node05 src]# rsync /software/cetus/mha_ld/src/masterha_secondary_check /usr/local/bin/
[root@node05 src]# chmod +x /usr/local/bin/masterha_secondary_check
[root@node05 src]# ll /usr/local/bin/masterha_secondary_check
-r-xr-xr-x 1 root root 5186 Aug 31 11:48 /usr/local/bin/masterha_secondary_check

四、MHA和cetus配置

1、创建和修改配置文件

  • 在manager节点上创建cetus配置文件（cetus.cnf）
[root@node05 src]#  cp /software/cetus/mha_ld/cetus.cnf /etc/cetus.cnf
[root@node05 src]#  vim  /etc/cetus.cnf
[root@node05 src]#  cat /etc/cetus.cnf 
middle_ipport=192.168.222.175:23306
middle_user=admin
middle_pass=admin
  • 在manager节点上创建cetus的user和proxy.配置文件（users.json和proxy.conf）
[root@node05 src]# cp /usr/local/cetus/conf/proxy.conf.example /usr/local/cetus/conf/proxy.conf
[root@node05 src]# cp /usr/local/cetus/conf/users.json.example  /usr/local/cetus/conf/users.json
[root@node05 src]# vim /usr/local/cetus/conf/users.json
[root@node05 src]# cat /usr/local/cetus/conf/users.json
{
    "users":    [{
            "user": "gcdb",
            "client_pwd":   "iforgot",
            "server_pwd":   "iforgot"
        }, {
            "user": "cetus_app1",
            "client_pwd":   "cetus_app1",
            "server_pwd":   "cetus_app1"
        }]
}

[root@node05 src]# vim  /usr/local/cetus/conf/proxy.conf
[root@node05 src]# cat /usr/local/cetus/conf/proxy.conf
[cetus]
# For mode-switch
daemon = true

# Loaded Plugins
plugins=proxy,admin

# Proxy Configuration, For example: MySQL master and salve host ip are both 192.0.0.1
proxy-address=192.168.222.175:13306
proxy-backend-addresses=192.168.222.172:3306
proxy-read-only-backend-addresses=192.168.222.173:3306,192.168.222.174:3306


# Admin Configuration
admin-address=192.168.222.175:23306
admin-username=admin
admin-password=admin

# Backend Configuration, use test db and username created
default-db=ttt
default-username=gcdb
default-pool-size=100
max-resp-size=10485760
long-query-time=100

# File and Log Configuration, put log in /data and marked by proxy port, /data/cetus needs to be created manually and has rw authority for cetus os user
max-open-files = 65536
pid-file = cetus6001.pid
plugin-dir=lib/cetus/plugins
log-file=/var/log/cetus.log
log-level=debug

# Check salve delay
disable-threads=false
check-slave-delay=true
slave-delay-down=5
slave-delay-recover=1

# For trouble
keepalive=true
verbose-shutdown=true
log-backtrace-on-crash=true
  • 在manager节点创建全局配置文件(masterha_default.cnf)
[root@node05 software]# cat /etc/masterha_default.cnf
[server default]
#用于指定mha manager产生相关状态文件全路径
manager_workdir=/var/log/masterha

#指定mha manager的绝对路径的文件名日志文件
manager_log=/var/log/masterha/mha.log

# cetus porxy文件绝对路径
proxy_conf=/etc/cetus.cnf

# 登陆mysql数据库账户及密码
user=gcdb
password=iforgot

# ssh用户
ssh_user=root
ssh_port=22

# mysql数据库master节点binlog的位置，该参数用于当master节点死掉后通过ssh方式顺序读取binlog event，需要配置，因为master节点死掉后无法通过replication机制来自动获取binlog日志位置
master_binlog_dir=/r2/mysqldata
#master_binlog_dir= /r2/mysqldata

# 用于检测各节点间的连接性，此处详细可参考MHA parameters描述部分
secondary_check_script= masterha_secondary_check  -s 192.168.222.172 -s 192.168.222.173  -s 192.168.222.174
ping_interval=3
ping_type=select
remote_workdir=/tmp

#定义用于实现VIP漂移的脚本，后面的是shutdown以及report脚本
master_ip_failover_script=/usr/local/bin/master_ip_failover
#shutdown_script=/usr/local/bin/power_manager
shutdown_script=
report_script="/usr/local/share/perl5/MHA/sendMail.sh"


#MySQL用于复制的账号
repl_user=repl
repl_password=repl
  • 在manager节点上创建配置文件(mha01.cnf)
[root@node05 software]#  cat /etc/mha.cnf
[server default]
manager_log=/var/log/masterha/mha.log
manager_workdir=/var/log/masterha

[server1]
hostname=192.168.222.172
master_binlog_dir=/r2/mysqldata
port=3306

[server2]
hostname=192.168.222.173
master_binlog_dir=/r2/mysqldata
port=3306

[server3]
hostname=192.168.222.174
master_binlog_dir=/r2/mysqldata
port=3306
  • 在manager节点上修改故障转移脚本（master_ip_failover）
[root@node05 mha4mysql-manager-0.57]# vim  /usr/local/bin/master_ip_failover
#!/usr/bin/env perl

use strict;
use warnings FATAL => 'all';
use Getopt::Long;
use MHA::DBHelper;

my (
    $command,          $ssh_user,        $orig_master_host, $orig_master_ip,
    $orig_master_port, $new_master_host, $new_master_ip,    $new_master_port
);

#添加切换vip
my $vip = '192.168.222.99/24';
my $key = '1';
my $ssh_start_vip = "/sbin/ifconfig ens224:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig ens224:$key down";
$ssh_user = "root";


GetOptions(
    'command=s'          => \$command,
    'ssh_user=s'         => \$ssh_user,
    'orig_master_host=s' => \$orig_master_host,
    'orig_master_ip=s'   => \$orig_master_ip,
    'orig_master_port=i' => \$orig_master_port,
    'new_master_host=s'  => \$new_master_host,
    'new_master_ip=s'    => \$new_master_ip,
    'new_master_port=i'  => \$new_master_port,
);

exit &main();

sub main {

    print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

    if ( $command eq "stop" || $command eq "stopssh" ) {

        my $exit_code = 1;
        eval {
            print "Disabling the VIP on old master: $orig_master_host \n";
            &stop_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn "Got Error: $@\n";
            exit $exit_code;
        }
        exit $exit_code;
    }
    elsif ( $command eq "start" ) {

        my $exit_code = 10;
        eval {
            print "Enabling the VIP - $vip on the new master - $new_master_host \n";
            &start_vip();
            $exit_code = 0;
        };
        if ($@) {
            warn $@;
            exit $exit_code;
        }
        exit $exit_code;
    }
    elsif ( $command eq "status" ) {
        print "Checking the Status of the script.. OK \n";
        exit 0;
    }
    else {
        &usage();
        exit 1;
    }
}

sub start_vip() {
    `ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
sub stop_vip() {
     return 0  unless  ($ssh_user);
    `ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
    print
    "Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

2、检查 SSH/复制/MHA Manager 的配置

  • 检查SSH 情况：masterha_check_ssh --conf=/etc/mha.cnf
[root@node05 src]# masterha_check_ssh --conf=/etc/mha.cnf
Fri Aug 31 13:55:51 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Fri Aug 31 13:55:51 2018 - [info] Reading application default configuration from /etc/mha.cnf..
Fri Aug 31 13:55:51 2018 - [info] Reading server configuration from /etc/mha.cnf..
Fri Aug 31 13:55:51 2018 - [info] Starting SSH connection tests..
Fri Aug 31 13:55:52 2018 - [debug] 
Fri Aug 31 13:55:51 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.172:22) to [email protected](192.168.222.173:22)..
Fri Aug 31 13:55:51 2018 - [debug]   ok.
Fri Aug 31 13:55:51 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.172:22) to [email protected](192.168.222.174:22)..
Fri Aug 31 13:55:51 2018 - [debug]   ok.
Fri Aug 31 13:55:52 2018 - [debug] 
Fri Aug 31 13:55:51 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.173:22) to [email protected](192.168.222.172:22)..
Fri Aug 31 13:55:51 2018 - [debug]   ok.
Fri Aug 31 13:55:51 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.173:22) to [email protected](192.168.222.174:22)..
Fri Aug 31 13:55:52 2018 - [debug]   ok.
Fri Aug 31 13:55:53 2018 - [debug] 
Fri Aug 31 13:55:52 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.174:22) to [email protected](192.168.222.172:22)..
Fri Aug 31 13:55:52 2018 - [debug]   ok.
Fri Aug 31 13:55:52 2018 - [debug]  Connecting via SSH from [email protected](192.168.222.174:22) to [email protected](192.168.222.173:22)..
Fri Aug 31 13:55:52 2018 - [debug]   ok.
Fri Aug 31 13:55:53 2018 - [info] All SSH connection tests passed successfully.
[root@node05 src]#
  • 检查复制情况：masterha_check_repl --conf=/etc/mha.cnf
[root@node05 src]# masterha_check_repl --conf=/etc/mha.cnf
Fri Aug 31 13:59:17 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Fri Aug 31 13:59:17 2018 - [info] Reading application default configuration from /etc/mha.cnf..
Fri Aug 31 13:59:17 2018 - [info] Reading server configuration from /etc/mha.cnf..
Fri Aug 31 13:59:17 2018 - [info] MHA::MasterMonitor version 0.56.
Fri Aug 31 13:59:17 2018 - [info] g_workdir: /var/log/masterha
Fri Aug 31 13:59:17 2018 - [info] proxy_conf: /etc/cetus.cnf
Fri Aug 31 13:59:17 2018 - [info] -------------ManagerUtil::check_node_version-----------
Fri Aug 31 13:59:18 2018 - [info] GTID failover mode = 1
Fri Aug 31 13:59:18 2018 - [info] Dead Servers:
Fri Aug 31 13:59:18 2018 - [info] Alive Servers:
Fri Aug 31 13:59:18 2018 - [info]   192.168.222.172(192.168.222.172:3306)
Fri Aug 31 13:59:18 2018 - [info]   192.168.222.173(192.168.222.173:3306)
Fri Aug 31 13:59:18 2018 - [info]   192.168.222.174(192.168.222.174:3306)
Fri Aug 31 13:59:18 2018 - [info] Alive Slaves:
Fri Aug 31 13:59:18 2018 - [info]   192.168.222.173(192.168.222.173:3306)  Version=5.7.18-log (oldest major version between slaves) log-bin:enabled
Fri Aug 31 13:59:18 2018 - [info]     GTID ON
Fri Aug 31 13:59:18 2018 - [info]     Replicating from 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 13:59:18 2018 - [info]   192.168.222.174(192.168.222.174:3306)  Version=5.7.18-log (oldest major version between slaves) log-bin:enabled
Fri Aug 31 13:59:18 2018 - [info]     GTID ON
Fri Aug 31 13:59:18 2018 - [info]     Replicating from 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 13:59:18 2018 - [info] Current Alive Master: 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 13:59:18 2018 - [info] Checking slave configurations..
Fri Aug 31 13:59:18 2018 - [info] Checking replication filtering settings..
Fri Aug 31 13:59:18 2018 - [info]  binlog_do_db= , binlog_ignore_db= 
Fri Aug 31 13:59:18 2018 - [info]  Replication filtering check ok.
Fri Aug 31 13:59:18 2018 - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking.
Fri Aug 31 13:59:18 2018 - [info] Checking SSH publickey authentication settings on the current master..
Fri Aug 31 13:59:18 2018 - [info] HealthCheck: SSH to 192.168.222.172 is reachable.
Fri Aug 31 13:59:18 2018 - [info] 
192.168.222.172(192.168.222.172:3306) (current master)
 +--192.168.222.173(192.168.222.173:3306)
 +--192.168.222.174(192.168.222.174:3306)

Fri Aug 31 13:59:18 2018 - [info] Checking replication health on 192.168.222.173..
Fri Aug 31 13:59:18 2018 - [info]  ok.
Fri Aug 31 13:59:18 2018 - [info] Checking replication health on 192.168.222.174..
Fri Aug 31 13:59:18 2018 - [info]  ok.
Fri Aug 31 13:59:18 2018 - [info] Checking master_ip_failover_script status:
Fri Aug 31 13:59:18 2018 - [info]   /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.172 --orig_master_ip=192.168.222.172 --orig_master_port=3306 


IN SCRIPT TEST====/sbin/ifconfig ens224:1 down==/sbin/ifconfig ens224:1 192.168.222.99/24===

Checking the Status of the script.. OK 
Fri Aug 31 13:59:18 2018 - [info]  OK.
Fri Aug 31 13:59:18 2018 - [warning] shutdown_script is not defined.
Fri Aug 31 13:59:18 2018 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.
[root@node05 src]#
  • 检查MHA 状态：masterha_check_status --conf=/etc/mha.cnf
[root@node05 src]# masterha_check_status --conf=/etc/mha.cnf
mha is stopped(2:NOT_RUNNING).
  • 检查masterha_secondary_check脚本中mysql客户端安装位置
[root@node05 software]# which mysql
/usr/local/mysql/bin/mysql
[root@node05 software]# grep -w /usr/bin/mysql  /usr/local/bin/*
/usr/local/bin/masterha_secondary_check:        . "/usr/bin/mysql -u$master_user -p$master_password -h$master_host -P$master_port "
[root@node05 software]# ln -s /usr/local/mysql/bin/mysql /usr/bin/mysql
[root@node05 software]# which mysql
/usr/bin/mysql
[root@node05 software]#

3、启动MHA manager

  • session A
[root@node05 src]# mkdir -p /var/log/masterha/
  • session B
[root@node05 mha4mysql-manager-0.57]# nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1/manager.log 2>&1 &
[1] 19246
  • session A
[root@node05 ~]# tail -f /var/log/masterha/app1/manager.log

IN SCRIPT TEST====/sbin/ifconfig ens224:1 down==/sbin/ifconfig ens224:1 192.168.222.99/24===

Checking the Status of the script.. OK 
Fri Aug 31 14:06:45 2018 - [info]  OK.
Fri Aug 31 14:06:45 2018 - [warning] shutdown_script is not defined.
Fri Aug 31 14:06:45 2018 - [info] Set master ping interval 3 seconds.
Fri Aug 31 14:06:45 2018 - [info] Set secondary check script: masterha_secondary_check  -s 192.168.222.172 -s 192.168.222.173  -s 192.168.222.174
Fri Aug 31 14:06:45 2018 - [info] Starting ping health check on 192.168.222.172(192.168.222.172:3306)..
Fri Aug 31 14:06:45 2018 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..
[root@node05 src]# cat /var/log/masterha/mha.log
Fri Aug 31 14:06:44 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Fri Aug 31 14:06:44 2018 - [info] Reading application default configuration from /etc/mha.cnf..
Fri Aug 31 14:06:44 2018 - [info] Reading server configuration from /etc/mha.cnf..
Fri Aug 31 14:06:44 2018 - [info] MHA::MasterMonitor version 0.56.
Fri Aug 31 14:06:44 2018 - [info] g_workdir: /var/log/masterha
Fri Aug 31 14:06:44 2018 - [info] proxy_conf: /etc/cetus.cnf
Fri Aug 31 14:06:44 2018 - [info] -------------ManagerUtil::check_node_version-----------
Fri Aug 31 14:06:45 2018 - [info] GTID failover mode = 1
Fri Aug 31 14:06:45 2018 - [info] Dead Servers:
Fri Aug 31 14:06:45 2018 - [info] Alive Servers:
Fri Aug 31 14:06:45 2018 - [info]   192.168.222.172(192.168.222.172:3306)
Fri Aug 31 14:06:45 2018 - [info]   192.168.222.173(192.168.222.173:3306)
Fri Aug 31 14:06:45 2018 - [info]   192.168.222.174(192.168.222.174:3306)
Fri Aug 31 14:06:45 2018 - [info] Alive Slaves:
Fri Aug 31 14:06:45 2018 - [info]   192.168.222.173(192.168.222.173:3306)  Version=5.7.18-log (oldest major version between slaves) log-bin:enabled
Fri Aug 31 14:06:45 2018 - [info]     GTID ON
Fri Aug 31 14:06:45 2018 - [info]     Replicating from 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 14:06:45 2018 - [info]   192.168.222.174(192.168.222.174:3306)  Version=5.7.18-log (oldest major version between slaves) log-bin:enabled
Fri Aug 31 14:06:45 2018 - [info]     GTID ON
Fri Aug 31 14:06:45 2018 - [info]     Replicating from 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 14:06:45 2018 - [info] Current Alive Master: 192.168.222.172(192.168.222.172:3306)
Fri Aug 31 14:06:45 2018 - [info] Checking slave configurations..
Fri Aug 31 14:06:45 2018 - [info] Checking replication filtering settings..
Fri Aug 31 14:06:45 2018 - [info]  binlog_do_db= , binlog_ignore_db= 
Fri Aug 31 14:06:45 2018 - [info]  Replication filtering check ok.
Fri Aug 31 14:06:45 2018 - [info] GTID (with auto-pos) is supported. Skipping all SSH and Node package checking.
Fri Aug 31 14:06:45 2018 - [info] Checking SSH publickey authentication settings on the current master..
Fri Aug 31 14:06:45 2018 - [info] HealthCheck: SSH to 192.168.222.172 is reachable.
Fri Aug 31 14:06:45 2018 - [info] 
192.168.222.172(192.168.222.172:3306) (current master)
 +--192.168.222.173(192.168.222.173:3306)
 +--192.168.222.174(192.168.222.174:3306)

Fri Aug 31 14:06:45 2018 - [info] Checking master_ip_failover_script status:
Fri Aug 31 14:06:45 2018 - [info]   /usr/local/bin/master_ip_failover --command=status --ssh_user=root --orig_master_host=192.168.222.172 --orig_master_ip=192.168.222.172 --orig_master_port=3306 


IN SCRIPT TEST====/sbin/ifconfig ens224:1 down==/sbin/ifconfig ens224:1 192.168.222.99/24===

Checking the Status of the script.. OK 
Fri Aug 31 14:06:45 2018 - [info]  OK.
Fri Aug 31 14:06:45 2018 - [warning] shutdown_script is not defined.
Fri Aug 31 14:06:45 2018 - [info] Set master ping interval 3 seconds.
Fri Aug 31 14:06:45 2018 - [info] Set secondary check script: masterha_secondary_check  -s 192.168.222.172 -s 192.168.222.173  -s 192.168.222.174
Fri Aug 31 14:06:45 2018 - [info] Starting ping health check on 192.168.222.172(192.168.222.172:3306)..
Fri Aug 31 14:06:45 2018 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..  --表示已开启监听
  • session B
[root@node05 software]# masterha_check_status --conf=/etc/mha.cnf
mha (pid:25386) is running(0:PING_OK), master:192.168.222.172
[root@node05 software]#

4、启动cetus

  • 启动前保证启动配置文件的权限为660
[root@node05 src]# chmod 660 /usr/local/cetus/conf/*
[root@node05 src]# ll /usr/local/cetus/conf/
total 28
-rw-rw---- 1 root root 1053 Aug 31 12:12 proxy.conf
-rw-rw---- 1 root root 1011 Aug 31 09:21 proxy.conf.example
-rw-rw---- 1 root root 1199 Aug 31 09:21 shard.conf.example
-rw-rw---- 1 root root 1030 Aug 31 09:21 sharding.json.example
-rw-rw---- 1 root root  189 Aug 31 12:04 users.json
-rw-rw---- 1 root root  198 Aug 31 09:21 users.json.example
-rw-rw---- 1 root ro
  • 守护进程模式启动Cetus
  • session A
[root@node01 software]# ll mha*
-rw-r--r-- 1 root root 118521 Aug 28 15:41 mha4mysql-manager-0.57.tar.gz
-rw-r--r-- 1 root root  54484 Aug 28 15:41 mha4mysql-node-0.57.tar.gz
[root@node05 src]# /usr/local/cetus/bin/cetus --defaults-file=/usr/local/cetus/conf/proxy.conf
[root@node05 src]#  ps -ef |grep cetusb
root     26580     1  0 14:23 ?        00:00:00 /usr/local/cetus/libexec/cetus --defaults-file=/usr/local/cetus/conf/proxy.conf
root     26581 26580  0 14:23 ?        00:00:00 /usr/local/cetus/libexec/cetus --defaults-file=/usr/local/cetus/conf/proxy.conf
root     26624  2272  0 14:23 pts/0    00:00:00 grep --color=auto cetus
[root@node05 src]#
  • 查看Cetus状态
  • session B
[root@node05 src]# mysql -ugcdb  -piforgot -h192.168.222.175 -P13306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7.18-log (cetus) MySQL Community Server (GPL)

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

([email protected]) 14:26:55 [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| proxy_heart_beat   |
| sys                |
| ttt                |
+--------------------+
6 rows in set (0.00 sec)

([email protected]) 14:27:02 [(none)]> exit
Bye
[root@node05 src]# mysql -uadmin  -padmin -h192.168.222.175 -P23306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7 admin

Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

([email protected]) 14:27:41 [(none)]> select conn_details from backends;select * from backends;                                         
+-------------+----------+------------+------------+-------------+
| backend_ndx | username | idle_conns | used_conns | total_conns |
+-------------+----------+------------+------------+-------------+
| 0           | gcdb     | 100        | 0          | 100         |
| 1           | gcdb     | 100        | 0          | 100         |
| 2           | gcdb     | 100        | 0          | 100         |
+-------------+----------+------------+------------+-------------+
3 rows in set (0.00 sec)

+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| backend_ndx | address              | state | type | slave delay | uuid | idle_conns | used_conns | total_conns |
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| 1           | 192.168.222.172:3306 | up    | rw   | NULL        | NULL | 100        | 0          | 100         |
| 2           | 192.168.222.173:3306 | up    | ro   | 426         | NULL | 100        | 0          | 100         |
| 3           | 192.168.222.174:3306 | up    | ro   | 427         | NULL | 100        | 0          | 100         |
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
3 rows in set (0.00 sec)

([email protected]) 14:58:37 [(none)]> cetus;
+--------------------------+---------------------+
| Status                   | Value               |
+--------------------------+---------------------+
| Cetus version            | v1.0.0-44-g5b1bd43  |
| Startup time             | 2018-08-31 14:23:07 |
| Loaded modules           | proxy admin         |
| Idle backend connections | 300                 |
| Used backend connections | 0                   |
| Client connections       | 4                   |
| Query count              | 5                   |
| QPS (1min, 5min, 15min)  | 0.00, 0.01, 0.00    |
| TPS (1min, 5min, 15min)  | 0.00, 0.00, 0.00    |
+--------------------------+---------------------+
9 rows in set (0.00 sec)

([email protected]) 14:58:39 [(none)]>

5、 验证cetus读写分离功能

  • session A
[root@node05 software]# mysql -ugcdb  -piforgot -h192.168.222.175 -P13306
([email protected]) 15:18:37 [(none)]> select sleep(10) from ttt.t1;
  • session B
([email protected]) 15:18:44 [(none)]> select conn_details from backends;select * from backends;
+-------------+----------+------------+------------+-------------+
| backend_ndx | username | idle_conns | used_conns | total_conns |
+-------------+----------+------------+------------+-------------+
| 0           | gcdb     | 100        | 0          | 100         |
| 1           | gcdb     | 100        | 0          | 100         |
| 2           | gcdb     | 99         | 1          | 100         | --读负载到backend_ndx=3这台
+-------------+----------+------------+------------+-------------+
3 rows in set (0.00 sec)

+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| backend_ndx | address              | state | type | slave delay | uuid | idle_conns | used_conns | total_conns |
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| 1           | 192.168.222.172:3306 | up    | rw   | NULL        | NULL | 100        | 0          | 100         |
| 2           | 192.168.222.173:3306 | up    | ro   | 403         | NULL | 100        | 0          | 100         |
| 3           | 192.168.222.174:3306 | up    | ro   | 404         | NULL | 99         | 1          | 100         | --读负载到192.168.222.174
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
3 rows in set (0.00 sec)

([email protected]) 15:18:46 [(none)]>
  • session A
([email protected]) 15:18:37 [(none)]> select sleep(10) from ttt.t1;
+-----------+
| sleep(10) |
+-----------+
|         0 |
|         0 |
|         0 |
|         0 |
+-----------+
4 rows in set (40.00 sec)

([email protected]) 15:19:20 [(none)]> 

#再次执行查询看后端负载到那个服务器
([email protected]) 15:19:20 [(none)]> ([email protected]) 15:19:20 [(none)]> select sleep(10) from ttt.t1;
+-----------+
| sleep(10) |
+-----------+
|         0 |
|         0 |
|         0 |
|         0 |
+-----------+
4 rows in set (40.01 sec)
  • session B
([email protected]) 15:19:22 [(none)]> select conn_details from backends;select * from backends;
+-------------+----------+------------+------------+-------------+
| backend_ndx | username | idle_conns | used_conns | total_conns |
+-------------+----------+------------+------------+-------------+
| 0           | gcdb     | 100        | 0          | 100         |
| 1           | gcdb     | 99         | 1          | 100         |  --读负载到backend_ndx=2这台
| 2           | gcdb     | 100        | 0          | 100         |
+-------------+----------+------------+------------+-------------+
3 rows in set (0.00 sec)

+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| backend_ndx | address              | state | type | slave delay | uuid | idle_conns | used_conns | total_conns |
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
| 1           | 192.168.222.172:3306 | up    | rw   | NULL        | NULL | 100        | 0          | 100         |
| 2           | 192.168.222.173:3306 | up    | ro   | 783         | NULL | 99         | 1          | 100         |  --读负载到192.168.222.173
| 3           | 192.168.222.174:3306 | up    | ro   | 432         | NULL | 100        | 0          | 100         |
+-------------+----------------------+-------+------+-------------+------+------------+------------+-------------+
3 rows in set (0.00 sec)
  • 验证简单读写
([email protected]) 15:23:11 [(none)]> select sleep(5);insert into ttt.t1(nums) values(6),(7),(8),(9),(10);
+----------+
| sleep(5) |
+----------+
|        0 |
+----------+
1 row in set (5.00 sec)

Query OK, 5 rows affected (0.05 sec)
Records: 5  Duplicates: 0  Warnings: 0

([email protected]) 15:30:35 [(none)]> select * from ttt.t1;
+----+------+
| id | nums |
+----+------+
|  1 |    1 |
|  2 |    2 |
|  3 |  100 |
|  5 |    5 |
|  6 |    6 |
|  7 |    7 |
|  8 |    8 |
|  9 |    9 |
| 10 |   10 |
+----+------+
9 rows in set (0.01 sec)

([email protected]) 15:31:02 [(none)]>

五 MHA和Cetu联动切换

1、添加mail告警功能

  • 修改配置
[root@node05 software]# vim /etc/mail.rc 
#最后追加以下内容
set [email protected]             #此处设置发件人的信息
set smtp=smtp.163.com               #此处配置邮件服务地址，因为邮箱是163的，所以此处配置为smtp.163.com
set [email protected]    #此处配置发件人邮箱地址
set smtp-auth-password=xxxxxx        #切记此处配置abc是客户端授权码，不是发件人邮箱地址密码
set smtp-auth=login                  #邮件认证方式
  • 启动sendmail测试
[root@node05 ~]# systemctl start sendmail
[root@node05 ~]# mail -s "status" [email protected] < /etc/hosts
  • 修改sendmail脚本
[root@node05 ~]# cat /usr/local/share/perl5/MHA/sendMail.sh 
#!/bin/bash
## author : cch
## desc: sendmail


############ variable part ########################
conf=/masterha/app1/sample.cnf

if [ $# -ne 1 ];then
   mailsubject="mha--failover--`date +%Y%m%d%H%M`"
else
   mailsubject=$1
fi


############# main #########################
find_flag=`cat $conf|grep -v '^#'|grep "manager_workdir"|awk -F= '{print $2}'|wc -l`
if [  ${find_flag} -eq 1 ];then
   manager_workdir=`cat $conf|grep -v '^#'|grep "manager_workdir"|awk -F= '{print $2}'|sed 's/ //g'`
fi

#java SendMail service.netease.com ${manager_workdir}/to_list_mha.txt  "${mailsubject}" ${manager_workdir}/to_list_filename ${manager_workdir}/sendmail.txt

#修改邮箱地址
mail -s "${mailsubject}"  [email protected] < ${manager_workdir}/sendmail.txt

echo `date` >> ./sendMail.log
[root@node05 ~]#

猜你喜欢

转载自www.cnblogs.com/gczheng/p/9567284.html
今日推荐
Linus “吃狗粮”最积极！
开源日报 | Winamp播放器即将开源；生成式AI之战升级第二轮；Linus“吃狗粮”最积极；AI进入泡沫前期；吴泳铭为阿里云带来了什么？
NetBSD 禁止提交由 AI 生成的代码
Apache Doris 2.0.10 版本正式发布！
开源日报 | 大模型开战；大模型独角兽被曝卖身；周鸿祎建议谷歌开源所有产品；最大开源AI社区提供1000万美元共享GPU
开源日报 | Chrome内置Gemini的意义不在于Gemini；中国AI追随之路的五大误区；ECharts创始人“下海”养鱼；谷歌I/O开发者大会什么都有，只是没有惊喜
微软回应中国区AI团队“打包赴美”传闻
周排行
SVN服务端安装在阿里云
实战 | 相机标定
webpack核心概念
note20——》只要肯低头吃苦，人生就会有救
PAT甲级 1062 Talent and Virtue （25 分）排序
NG Toolset开发笔记--5GNR Resource Grid（26）
如何对待上司
oracle命令
第9章 STL迭代器
logstash使用es映射模板
每日归档
更多
2024-05-20(36)
2024-05-19(0)
2024-05-18(4)
2024-05-17(34)
2024-05-16(6)
2024-05-15(24)
2024-05-14(0)
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022