lvs+keepalived+mysql dual master replication to achieve load sharing and high availability

 
 

Keepalived+lvs+mysql master replication is a commonly used Mysql high-availability solution, where lvs provides read load balancing, and Keepalived realizes automatic failover through virtual VIP drift. Although Mysql is configured as a master replication, it is generally adopted Single-point write to ensure data consistency.
 

 

---

Host	IP	Features
VIP	192.168.30.250
keep-master	192.168.30.243	lvs+keepalived(主)
keep-slave	192.168.30.244	lvs+keepalived(备)
master-master	192.168.30.245	mysql (master)
master-slave	1929.168.30.246	mysql (master)

mysql dual master

mysql dual-master and master-slave configuration is similar, but the following steps are added

[root@mysql-master ~]# vim /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
log_slave_updates = 1    ##双主需要开启log_slave_updates功能
server-id = 10           ##不能相同
log-bin = mysql_bin      ##双主两个都是log-bin
auto_increment_offset = 1   ##表示自增长字段从1开始，他的取值范围是1 .. 65535
auto_increment_increment = 2 ##表示自增长字段每次递增2，取值范围是2 .. 65535

Restart the service
 
 
Create an authorized account on the master host and allow connection on the slave (192.168.30.246) host

mysql> grant replication slave on *.* to 'repl'@'192.168.30.246' identified by 'repl';
Query OK, 0 rows affected, 1 warning (0.00 sec)

 
View the current binlog status information of the master

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql_bin.000001 |      455 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

 

Set the master as your own master server on the slave server and enable the slave function

mysql> change master to
    -> master_host = '192.168.30.245',
    -> master_user = 'repl',
    -> master_password = 'repl',
    -> master_log_file = 'mysql_bin.000001',
    -> master_log_pos = 455 ;
Query OK, 0 rows affected, 2 warnings (0.02 sec)



mysql> start slave;
Query OK, 0 rows affected (0.00 sec)



mysql> show slave status\G;
*************************** 1. row ***************************

             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes

 
 

Modify the configuration file from the library

[root@mysql-slave ~]# cat /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
log_slave_updates = 1    ##双主需要开启log_slave_updates功能
server-id = 20           ##不能相同
log-bin = mysql_bin      ##双主两个都是log-bin
auto_increment_offset = 1   ##表示自增长字段从1开始，他的取值范围是1 .. 65535
auto_increment_increment = 2 ##表示自增长字段每次递增2，取值范围是2 .. 65535

 
Create an authorized account on the slave host and allow connection on the master (192.168.30.246) host

mysql> grant replication slave on *.* to 'repl'@'192.168.30.245' identified by 'repl';
Query OK, 0 rows affected, 1 warning (0.00 sec)

 
View slave's current binlog status information

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql_bin.000001 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

 
Set the slave as your own master server on the master server and enable the slave function

mysql> change master to
    -> master_host = '192.168.30.246',
    -> master_user = 'repl',
    -> master_password = 'repl',
    -> master_log_file = 'mysql_bin.000001',
    -> master_log_pos = 154 ;
Query OK, 0 rows affected, 2 warnings (0.00 sec)


mysql> show slave status\G;
*************************** 1. row ***************************
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes

 

 

keepalived

I use yum to install the
main keepalived

[root@keep-master ~]# yum -y install keepalived  ipvsadm

[root@keep-master ~]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   router_id lb01       
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 20 
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass zyy                 
    }
    virtual_ipaddress {
        192.168.30.250                ##VIP
    }
}

virtual_server 192.168.30.250 3306 {      # 定义虚拟服务器，地址与上面的virtual_ipaddress相同
    delay_loop 3                          # 健康检查时间间隔，3秒
    lb_algo rr                            # 负载均衡调度算法：rr|wrr|lc|wlc|sh|dh|lblc
    lb_kind DR                            # 负载均衡转发规则：NAT|DR|TUN
   # persistence_timeout 5                # 会话保持时间5秒，动态服务建议开启
    protocol TCP                          # 转发协议protocol，一般有tcp和udp两种

 #后端真实服务器，有几台就设置几个
    real_server 192.168.30.245 3306 {     
        weight 1
        TCP_CHECK {
            connect_port 3306
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.30.246 3306 {
        weight 1
        TCP_CHECK {
            connect_port 3306
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

 

Same from the library, change the priority and set it without preemption.

[root@keep-slave ~]# yum -y install keepalived  ipvsadm

[root@keep-slave ~]# vim /etc/keepalived/keepalived.conf 
! Configuration File for keepalived

global_defs {
   router_id lb02
}

vrrp_instance VI_1 {
    state BACKUP              ##BACKUP
    interface ens33
    virtual_router_id 20    
    priority 80             ##优先级要比master小
    nopreempt
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass zyy        
    }
    virtual_ipaddress {
        192.168.30.250
    }
}

virtual_server 192.168.30.250 3306 {
    delay_loop 3
    lb_algo rr
    lb_kind DR
  #  persistence_timeout 5
    protocol TCP

    real_server 192.168.30.245 3306 {
        weight 1
        TCP_CHECK {
            connect_port 3306
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.30.246 3306 {
        weight 1
        TCP_CHECK {
            connect_port 3306
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

 
Open the keepalived service

[root@keep-master ~]# systemctl start keepalived
[root@keep-slave ~]# systemctl start keepalived

 

View VIP


 
 

Check the status of the LVS cluster at this time, you can see that there are two RealServers, scheduling algorithms, weights and other information under the cluster. ActiveConn represents the number of active connections of the current RealServer.

[root@keep-master ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.30.250:3306 rr
  -> 192.168.30.245:3306          Route   1      0          0         
  -> 192.168.30.246:3306          Route   1      0          0       

 
 

Write the RealServer network configuration script
192.168.30.245 and 192.168.30.246 write the same things, here is a demonstration

[root@mysql-master ~]# cat /etc/init.d/realserver 
#!/bin/bash
VIP=192.168.30.250
. /etc/rc.d/init.d/functions

case "$1" in
# 禁用本地的ARP请求、绑定本地回环地址
start)
    /sbin/ifconfig lo down
    /sbin/ifconfig lo up
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    /sbin/sysctl -p >/dev/null 2>&1
    /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up # 在回环地址上绑定VIP，设定掩码，与Direct Server上自身的IP保持通信
    /sbin/route add -host $VIP dev lo:0
    echo "LVS-DR real server starts successfully.\n"
    ;;
stop)
    /sbin/ifconfig lo:0 down
    /sbin/route del $VIP >/dev/null 2>&1
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS-DR real server stopped.\n"
    ;;
status)
    isLoOn=`/sbin/ifconfig lo:0 | grep "$VIP"`
    isRoOn=`/bin/netstat -rn | grep "$VIP"`
    if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then
        echo "LVS-DR real server has run yet."
    else
        echo "LVS-DR real server is running."
    fi
    exit 3
    ;;
*)
    echo "Usage: $0 {start|stop|status}"
    exit 1
esac
exit 0

[root@mysql-master ~]# chmod 755 realserver
[root@mysql-master ~]# service realserver start

[root@mysql-slave ~]# chmod 755 realserver
[root@mysql-slave ~]# service realserver start

 
Set to start at boot

echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local

 

 

 

test

Add a test user

mysql> grant all on *.* to 'zyy'@'%' identified by 'zyy' ;
Query OK, 0 rows affected, 1 warning (0.00 sec)

1. Verify the LVS load balancing forwarding strategy

    MySQL客户端使用VIP连接数据库，并查看所连接的数据库服务器ID。可以看到，每次执行依次连接到192.168.30.246和192.168.30.245的MySQL，证明是轮询策略产生的结果。

 
 
 

2. Simulate the master failure of lvs

Stop keepalived on the master side

[root@keep-master ~]# systemctl stop keepalived

You can see that the VIP has drifted

 

Connect to MySQL at this time, and load balancing will not be affected.

 
 

3. Simulate that one of the databases is down

Close the slave's database

[root@mysql-slave ~]# service mysqld stop 
Shutting down MySQL............ SUCCESS! 


[root@mysql-slave ~]# ss -anlt
State      Recv-Q Send-Q Local Address:Port               Peer Address:Port              
LISTEN     0      128     *:22                  *:*                  
LISTEN     0      128    :::22                 :::*                 

At this time, only the mysql-master database is working, but it can ensure uninterrupted business