文章目录
前言
在生产环境中为什么要做主从复制、读写分离
在业务复杂的系统中，有这么一个情景，有一句sql语句需要锁表，导致暂时不能使用读的服务，那么就很影响运行中的业务，使用主从复制，让主库负责写，从库负责读，这样，即使主库出现了锁表的情景，通过读从库也可以保证业务的正常运作。
一、主从复制
1.1 MySQL主从复制原理
MySQL的主从复制是一个异步的复制过程（虽然一般情况下感觉是实时的），数据将从一个MySQL数据库（Master）复制到另一个MySQL数据库（Slave），在Master和Slave之间实现整个主从复制的过程是由三个线程参与完成的。其中两个线程（SQL线程和IO线程）在Slave端，另一个线程（I/O线程）在Master端。
要实现MySQL的主从复制，首先必须打开Master端的binlog记录功能，否则就无法实现。binlog: binary log，是主库中保存所有更新事件日志的二进制文件。因为整个复制过程实际上就是Slave从Master端获取binlog日志，然后在Slave上以相同顺序执行获取的binlog日志中的记录的各种SQL操作。
整个主从复制的过程：
（1）在Slave服务器上执行start slave命令开启主从复制开关，开始进行主从复制。
（2）此时，Slave服务器的IO线程会通过在master上已经授权的复制用户权限请求连接Master服务器，并请求从执行binlog日志文件中的指定位置（日志文件名和位置就是在配置主从复制服务时执行change master命令指定的）之后开始发送binlog日志内容。
（3）Master服务器接收来自Slave服务器的IO线程的请求后，其上负责复制的IO线程会根据Slave服务器的IO线程请求的信息分批读取指定binlog日志文件指定位置之后的binlog日志信息，然后返回给Slave端的IO线程。返回的信息中除了binlog日志内容外，还有在Master服务器端记录的IO线程。返回的信息中除了binlog中的下一个指定更新位置。
（4）当Slave服务器的IO线程获取到Master服务器上IO线程发送的日志内容、日志文件及位置点后，会将binlog日志内容依次写到Slave端自身的Relay Log（即中继日志）文件（Mysql-relay-bin.xxx）的最末端，并将新的binlog文件名和位置记录到master-info文件中，以便下一次读取master端新binlog日志时能告诉Master服务器从新binlog日志的指定文件及位置开始读取新的binlog日志内容
（5）Slave服务器端的SQL线程会实时检测本地Relay Log 中IO线程新增的日志内容，然后及时把Relay LOG 文件中的内容解析成sql语句，并在自身Slave服务器上按解析SQL语句的位置顺序执行应用这样sql语句，并在relay-log.info中记录当前应用中继日志的文件名和位置点
1.2 MySQL的复制类型
基于语句的复制
基于行的复制
混合类型的复制
1.3 MySQL复制的基本原则
每个Slave只有一个Master
每个Slave只能有一个唯一的服务ID
每个Master可以有多个Slave
二、读写分离
2.1 MySQL读写分离原理
读写分离就是只在主服务器上写，只在从服务器上读
主数据库处理事务性査询，而从数据库处理select査询
数据库复制被用来把事务性査询导致的变更同步到集群中的从数据库
客户机在读取MySQL数据库时遵循轮询规则
2.2 读写分离方案
基于程序代码内部实现
基于中间代理层实现
MySQL-Proxy
Amoeba
三、主从复制、读写分离实验
3.1 实验准备
1台MySQL主服务器master，2台MySQL从服务器slave01和slave02，1台amoeba服务器，1台客户机client，全部安装MySQL（具体操作可参考我之前的博客MySQL的安装）
3台MySQL服务器时间必须一致，才可实现主从复核
主服务器：master：192.168.100.9
从服务器1：slave1 :192.168.100.10
从服务器2：slave2 : 192.168.100.11
变形虫服务器：amoeba : 192.168.100.12
客户机一台
ID不能重复 maserID=9
slave1 id =10
slave2 id =11
Amoeba id=12
相关软件包
这里环境已配置好数据库
然后确认时间是否一致，可以date 命令查看
3.2 配置master服务器
3.2.1 修改主配置文件，开启日志服务，以及日志同步，及修改ID
[root@master ~]# vim /etc/my.cnf //添加三行
server-id = 9
log-bin=master-bin
log-slave-updates=true
3.2.2 重启服务，生成二进制日志
[root@master ~]# systemctl restart mysqld.service
3.2.3设置mysql 账户 root密码及登录查看状态
[root@master ~]# mysqladmin -u root password 'abc123'
[root@master ~]# mysql -uroot -p
创建账户
'192.168.100.网段的从服务器拥有复制权限，可以使用myslave身份123456密码复制所有的库和表'
mysql> grant replication slave on *.* to 'myslave'@'192.168.100.%' identified by'123456';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 3
Current database: *** NONE ***
Query OK, 0 rows affected, 1 warning (0.01 sec)
mysql> flush privileges; //刷新
Query OK, 0 rows affected (0.01 sec)
mysql> show master status; // 查看当前位置id，非常重要，之后不得有更改数据库操作'
+-------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+-------------------+----------+--------------+------------------+-------------------+
| master-bin.000001 | 604 | | | |
+-------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
3.2.4 搞完之后，不操作任何指令，等待从服务器同步
3.3 从服务器设置：
3.3.1 启用中继日志，指定中继日志的路径，设置ID
[root@slave1 ~]# vim /etc/my.cnf //添加三行
server-id = 10
relay-log=relay-log-bin
relay-log-index=slave-relay-bin.index //从服务器更新二进制日志'
3.3.2 重启使配置生效
[root@slave1 ~]# systemctl start mysqld
[root@slave1 ~]# systemctl restart mysqld
3.3.3 登录mysql
[root@slave1 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.17 Source distribution
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
3.3.4在从服务器中添加主服务器，登录用户，密码，日志文件，位置id’
mysql> change master to master_host='192.168.100.9',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=604;
Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave; //'开启主从同步'
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G; // '查看主从同步情况'
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.100.9
Master_User: myslave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: master-bin.000001
Read_Master_Log_Pos: 604
Relay_Log_File: relay-log-bin.000002
Relay_Log_Pos: 321
Relay_Master_Log_File: master-bin.000001
Slave_IO_Running: Yes // 'IO运行正常'
Slave_SQL_Running: Yes //QL运行正常'
Replicate_Do_DB:
3.4 Slave2配置
3.4.1启用中继日志，指定中继日志的路径，设置ID
[root@slave2 ~]# vim /etc/my.cnf //新增3行
server-id = 11
relay-log=relay-log-bin
relay-log-index=slave-relay-bin.index
3.4.2 重启使配置生效
[root@slave2 ~]# systemctl restart mysqld
[root@slave2 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.17 Source distribution
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
3.4.3在从服务器中添加主服务器，登录用户，密码，日志文件，位置id’
mysql> change master to master_host='192.168.100.9',master_user='myslave',master_password='123456',master_log_file='master-bin.000001',master_log_pos=604;
Query OK, 0 rows affected, 2 warnings (0.01 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G; // '查看主从同步情况'
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.100.9
Master_User: myslave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: master-bin.000001
Read_Master_Log_Pos: 604
Relay_Log_File: relay-log-bin.000002
Relay_Log_Pos: 321
Relay_Master_Log_File: master-bin.000001
Slave_IO_Running: Yes //运行正常
Slave_SQL_Running: Yes //运行正常
3.5 变形虫amoeba配置
3.5.1 安装JDK
[root@amoeba ~]# cd /opt
[root@amoeba opt]# ls
amoeba-mysql-binary-2.2.0.tar.gz jdk-6u14-linux-x64.bin mysql-5.7.17.tar.gz
boost_1_59_0.tar.gz mysql-5.7.17
[root@amoeba opt]# cp jdk-6u14-linux-x64.bin /usr/local/
[root@amoeba opt]# cd /usr/local/
[root@amoeba local]# chmod +x jdk-6u14-linux-x64.bin
[root@amoeba local]# ls
bin etc include lib libexec sbin src
boost games jdk-6u14-linux-x64.bin lib64 mysql share
[root@amoeba local]# ll
总用量 79932
drwxr-xr-x. 2 root root 6 4月 11 2018 bin
drwx------ 8 501 games 302 8月 12 2015 boost
drwxr-xr-x. 2 root root 6 4月 11 2018 etc
drwxr-xr-x. 2 root root 6 4月 11 2018 games
drwxr-xr-x. 2 root root 6 4月 11 2018 include
-rwxr-xr-x 1 root root 81849212 8月 27 09:19 jdk-6u14-linux-x64.bin
drwxr-xr-x. 2 root root 6 4月 11 2018 lib
drwxr-xr-x. 2 root root 6 4月 11 2018 lib64
drwxr-xr-x. 2 root root 6 4月 11 2018 libexec
drwxr-xr-x 12 mysql mysql 229 8月 27 08:01 mysql
drwxr-xr-x. 2 root root 6 4月 11 2018 sbin
drwxr-xr-x. 5 root root 49 8月 26 23:28 share
drwxr-xr-x. 2 root root 6 4月 11 2018 src
[root@amoeba local]# ./jdk-6u14-linux-x64.bin
一直空格到最后
Do you agree to the above license terms? [yes or no] --》
输入 yes
[root@amoeba local]#
[root@amoeba local]# mv jdk1.6.0_14/ jdk1.6
[root@amoeba local]# ls
bin etc include jdk-6u14-linux-x64.bin lib64 mysql share
boost games jdk1.6 lib libexec sbin src
3.5.2 配置环境变量
[root@amoeba local]# vim /etc/profile
尾部添加5行
export JAVA_HOME=/usr/local/jdk1.6
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin:$PATH:$HOME/bin
export AMOEBA_HOME=/usr/local/amoeba
export PATH=$PATH:$AMOEBA_HOME/bin
使配置生效
[root@amoeba local]# source /etc/profile
[root@amoeba local]# mkdir /usr/local/amoeba
[root@amoeba local]# cd /opt
[root@amoeba opt]# tar zxvf amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba/
[root@amoeba opt]# chmod -R 755 /usr/local/amoeba/
查看是否安装成功
[root@amoeba opt]# /usr/local/amoeba/bin/amoeba
amoeba start|stop
3.5.3 对3台服务器配置mysql 账户
三台服务器都添加mysql 账户（master ，slave 1 ，slave 2），用来给amoeba 访问
mysql> grant all on *.* to jue@'192.168.100.%' identified by '123123';
mysql> flush privileges;
3.5.4 修改amoeba 参数配置
[root@amoeba opt]# cd /usr/local/amoeba/conf
[root@amoeba conf]# ls
access_list.conf dbserver.dtd functionMap.xml rule.dtd
amoeba.dtd dbServers.xml log4j.dtd ruleFunctionMap.xml
amoeba.xml function.dtd log4j.xml rule.xml
[root@amoeba conf]# vim amoeba.xml
[root@amoeba conf]# vim dbServers.xml
[root@amoeba conf]# vim dbServers.xml
45 <dbServer name="master" parent="abstractServer">
46 <factoryConfig>
47 <!-- mysql ip -->
48 <property name="ipAddress">192.168.100.9</property>
49 </factoryConfig>
50 </dbServer>
51
52 <dbServer name="slave1" parent="abstractServer">
53 <factoryConfig>
54 <!-- mysql ip -->
55 <property name="ipAddress">192.168.100.10</property>
56 </factoryConfig>
57 </dbServer>
58
59 <dbServer name="slave2" parent="abstractServer">
60 <factoryConfig>
61 <!-- mysql ip -->
62 <property name="ipAddress">192.168.100.11</property>
63 </factoryConfig>
64 </dbServer>
65
66 <dbServer name="slaves" virtual="true">
67 <poolConfig class="com.meidusa.amoeba.server.MultipleServerPool">
68 <!-- Load balancing strategy: 1=ROUNDROBIN , 2=WEIGHTBASED , 3=HA-->
69 <property name="loadbalance">1</property>
70
71 <!-- Separated by commas,such as: server1,server2,server1 -->
72 <property name="poolNames">slave1,slave2</property>
73 </poolConfig>
74 </dbServer>
3.5.5 启动amoeba 服务
[root@amoeba conf]# /usr/local/amoeba/bin/amoeba start &
[1] 19874
[root@amoeba conf]# log4j:WARN log4j config load completed from file:/usr/local/amoeba/conf/log4j.xml
2020-08-27 10:55:39,181 INFO context.MysqlRuntimeContext - Amoeba for Mysql current versoin=5.1.45-mysql-amoeba-proxy-2.2.0
log4j:WARN ip access config load completed from file:/usr/local/amoeba/conf/access_list.conf
2020-08-27 10:55:39,291 INFO net.ServerableConnectionManager - Amoeba for Mysql listening on 0.0.0.0/0.0.0.0:8066.
2020-08-27 10:55:39,293 INFO net.ServerableConnectionManager - Amoeba Monitor Server listening on /127.0.0.1:62104.
3.5.6 确认启动成功
[root@amoeba ~]# netstat -ntap| grep 8066
tcp6 0 0 :::8066 :::* LISTEN 19874/java
tcp6 0 0 192.168.100.12:8066 192.168.100.48:47906 ESTABLISHED 19874/java
[root@amoeba ~]#
3.6 客户机访问amoeba 数据库测试
[root@shanan ~]# mysql -u amoeba -pabc123 -h 192.168.100.12 -P8066
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1382875200
Server version: 5.1.45-mysql-amoeba-proxy-2.2.0
Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
mysql> show databases;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 2022356744
Current database: *** NONE ***
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.05 sec)
3.7 客户端创建sha 数据库，测试主从复制功能
mysql> create database sha;
Query OK, 1 row affected (0.01 sec)
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sha |
| sys |
+--------------------+
5 rows in set (0.00 sec)
3.7.1 实验证明，主从复制同步成功
3.8 测试写分离
在slave1 和 2 上 执行此命令，停止同步
mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)
mysql> use sha;
Database changed
mysql> create table test1(id int(4),name varchar(50),info varchar(50));
Query OK, 0 rows affected (0.03 sec)
mysql> insert into test1 values(1,'zhangsan','this is master');
Query OK, 1 row affected (0.01 sec)
但是发现读不出
mysql> select * from test1;
ERROR 1146 (42S02): Table 'sha.test1' doesn't exist
mysql> select * from table test1;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax touse near 'table test1' at line 1
mysql> desc test1;
+-------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id | int(4) | YES | | NULL | |
| name | varchar(50) | YES | | NULL | |
| info | varchar(50) | YES | | NULL | |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)
mysql> select * from test1;
ERROR 1146 (42S02): Table 'sha.test1' doesn't exist
在slave1上查看，发现没有数据
mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)
mysql> use sha;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql>
在slave2上查看 ，发现没有数据
mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)
mysql> use sha;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql>
下面在master 上面 查看
3.8.1 实验证明 写入数据是写在master 上的，实现了写分离
3.9测试读分离
在slave1 上 创建表，并写入数据
mysql> create table test1(id int(4),name varchar(50),info varchar(50));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test1 values(2,'lisi','this is slave1');
Query OK, 1 row affected (0.02 sec)
mysql>
在slave2 上 创建表，并写入数据
mysql> create table test1(id int(4),name varchar(50),info varchar(50));
Query OK, 0 rows affected (0.01 sec)
mysql> insert into test1 values(3,'wangwu','this is slave2');
Query OK, 1 row affected (0.00 sec)
mysql>
在客户端访问数据库表测试
mysql> select * from test1;
+------+--------+----------------+
| id | name | info |
+------+--------+----------------+
| 3 | wangwu | this is slave2 |
+------+--------+----------------+
1 row in set (0.01 sec)
mysql> select * from test1;
+------+------+----------------+
| id | name | info |
+------+------+----------------+
| 2 | lisi | this is slave1 |
+------+------+----------------+
1 row in set (0.01 sec)
mysql> select * from test1;
+------+--------+----------------+
| id | name | info |
+------+--------+----------------+
| 3 | wangwu | this is slave2 |
+------+--------+----------------+
1 row in set (0.01 sec)
mysql> select * from test1;
+------+------+----------------+
| id | name | info |
+------+------+----------------+
| 2 | lisi | this is slave1 |
+------+------+----------------+
1 row in set (0.00 sec)
3.9.1 实验证明 一直处于轮询slave1 和 slave2 两个数据库 的状态
四. 同步失败的原因
4.1 .时间不一致，可执行以下操作同步时间
----------时间同步---
主服务器设置:192.168.100.9
yum install ntp -y
vim /etc/ntp.conf
server 127.127.100.0
/本地是时钟源/
fudge 127.127.100.0 stratum 8
/设置时间层级为8//
systemctl start ntpd
systemctl stop firewalld
setenforce o
节点服务器设置:192.168.100.10
yum install ntp ntpdate -y
systemctl start ntpd
systemctl stop firewalld
setenforce o
/usr/sbin/ntpdate 192.168.100.9//进行时间同步/
4.2 客户机访问amoeba ：失败
mysql> showdatabases;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 1382875200
Current database: *** NONE ***
此故障应该是 amoeba 服务器 amoeba 软件启动失败导致，建议修改配置文件
[root@amoeba conf]# vim dbServers.xml
[root@amoeba conf]# vim amoeba.xml
在确认成功启动后，成功解决问题
[root@amoeba conf]# /usr/local/amoeba/bin/amoeba start &
确认启动端口
[root@amoeba ~]# netstat -ntap| grep 8066
tcp6 0 0 :::8066 :::* LISTEN 19874/java
tcp6 0 0 192.168.100.12:8066 192.168.100.48:47906 ESTABLISHED 19874/java
客户机访问amoeba ：成功
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
4 rows in set (0.00 sec)