MySQL database master-slave replication and separation of read and write (distribution analysis)

Master-slave replication and separation of read and write

(1) Overview of read-write separation and master-slave replication

In the actual production environment, if the reading and writing of the database are all operated on the same database server, no matter in terms of security, high availability, or high concurrency, it cannot meet the actual needs. Therefore, in general, data is synchronized through Master-Slave, and then deployed and implemented by means of read-write separation to improve the concurrent load capacity of the database.

(2) Significance of reading and writing separation

Why do we need to separate read and write?
Because the database "write" (write 10000 data may take 3 minutes) operation is relatively time-consuming.
But the "read" of the database (reading 10,000 data may only take 5 seconds).
Therefore, the separation of reading and writing, the solution is that the writing of the database affects the efficiency of the query.

(3) Purpose of use of read-write separation

The database does not have to be separated from read and write. If the program uses a lot of databases, but there are few updates, it will be considered when there are many queries. Using database master-slave synchronization and read-write separation can share database pressure and improve performance.

(4) Common MySQL read-write separation is divided into two types

Based on the internal implementation of the program code

In the code, routing classification is performed according to select and insert. This type of method is currently the most widely used in the production environment.
The advantage is that the performance is better, because it is implemented in the program code, and there is no need to add additional equipment for hardware expenses; the disadvantage is that it requires developers to implement it, and the operation and maintenance personnel cannot start.
But not all applications are suitable for implementing read-write separation in the program code. Like some large and complex Java applications, if the read-write separation is implemented in the program code, the code changes will be relatively large.

Implementation based on the middle agent layer

The proxy is generally located between the client and the server. After receiving the client request, the proxy server passes the judgment and forwards it to the back-end database. There are the following representative programs:
MySQL-Proxy. MySQL-Proxy is an open source project of MySQL, which uses its own lua script to make SQL judgments.
Atlas. It is a data middle-tier project based on MySQL protocol developed and maintained by the infrastructure team of Qihoo 360's Web Platform Department. It is
based on the 0.8.2 version of mysql-proxy , which has been optimized and added some new features. 360 uses the mysql business run by Atlas internally, and it carries billions of read and write requests every day. Support things and stored procedures.
Amoeba. Developed by Chen Siru, the author once worked for Alibaba. The program is developed in the Java language, and Alibaba uses it in the production environment. But it does not support transactions and stored procedures.

(5) Replication types supported by MySQL

STATEMENT

• Statement-based replication. Execute sql statement on the server and execute the same statement on the slave server. MySQL uses statement-based replication by default, which has high execution efficiency.

ROW

• Row-based replication. Copy the changed content instead of executing the command on the slave server.

MIXED

• Mixed types of replication. By default, statement-based replication is adopted. Once it is found that statement-based replication cannot be accurately replicated, row-based replication will be adopted.

Experimental setup

Host	IP address	Required software package
Master	192.168.90.10	ntp 、 mysql
Amoeba	192.168.90.20	jdk-6u14-linux-x64.bin、amoeba
Slave1 (slave)	192.168.90.50	ntp 、ntpdate 、 mysql
Slave2 (slave)	192.168.90.60	ntp 、ntpdate 、 mysql
Client	192.168.90.70	NULL

Turn off the firewall of all hosts

systemctl stop firewalld
systemctl disable firewalld
setenforce 0

One, build MySQL master-slave replication

(1) MySQL master-slave server time synchronization

Main server settings

yum -y install ntp
vim /etc/ntp.conf

==在配置文件末端添加此代码==
server 127.127.90.0							设置本地是时钟源，注意修改网段
fudge 127.127.90.0 stratum 8				设置时间层级为8（限制在15内）

systemctl start ntpd

Setting from the server

yum -y install ntp ntpdate

service ntpd start
/usr/sbin/ntpdate 192.168.90.10			进行时间同步，指向Master服务器IP

crontab -e
*/30 * * * * /usr/sbin/ntpdate 192.168.90.10

(2) The mysql configuration of the main server

vim /etc/my.cnf

server-id = 1
log-bin=master-bin							添加，主服务器开启二进制日志
log-slave-updates=true						添加，允许从服务器更新二进制日志

systemctl restart mysqld

mysql -u root -p123123
GRANT REPLICATION SLAVE ON *.* TO 'myslave'@'192.168.90.%' IDENTIFIED BY '123123';		给从服务器授权
FLUSH PRIVILEGES;

show master status;

File 列显示日志名，Fosition 列显示偏移量

(3) mysql configuration from the server

vim /etc/my.cnf
server-id = 2						        修改，注意id与Master的不同，两个Slave的id也要不同
relay-log=relay-log-bin						添加，开启中继日志，从主服务器上同步日志文件记录到本地
relay-log-index=slave-relay-bin.index		添加，定义中继日志文件的位置和名称

systemctl restart mysqld

mysql -u root -p123123
change master to master_host='192.168.90.10' , master_user='myslave',master_password='123123',master_log_file='master-bin.000003',master_log_pos=154;
配置同步，注意 master_log_file 和 master_log_pos 的值要与Master查询的一致，这里的是例子，每个人的都不一样

start slave;						启动同步，如有报错执行 reset slave;
show slave status\G					查看 Slave 状态

确保 IO 和 SQL 线程都是 Yes，代表同步正常。
Slave_IO_Running: Yes				负责与主机的io通信
Slave_SQL_Running: Yes				负责自己的slave mysql进程

(4) Verify the effect of master-slave replication

Create a database on the master server, and you can view it on the slave server

mysql -uroot -p123123

create database bobo_ku;

Main server:

From the server:

Two, build MySQL read-write separation

(1) Amoeba server configuration Java environment

因为 Amoeba 基于是 jdk1.5 开发的，所以官方推荐使用 jdk1.5 或 1.6 版本，高版本不建议使用。
将jdk-6u14-linux-x64.bin 和 amoeba-mysql-binary-2.2.0.tar.gz.0 上传到/opt目录下。

cd /opt/
cp jdk-6u14-linux-x64.bin /usr/local/

cd /usr/local/
chmod +x jdk-6u14-linux-x64.bin 
./jdk-6u14-linux-x64.bin
按空格到最后一行
按yes，按enter

mv jdk1.6.0_14/ /usr/local/jdk1.6

==末行添加==
vim /etc/profile
export JAVA_HOME=/usr/local/jdk1.6
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/bin/:$PATH:$HOME/bin
export AMOEBA_HOME=/usr/local/amoeba
export PATH=$PATH:$AMOEBA_HOME/bin

source /etc/profile
java -version

==安装 Amoeba软件==
mkdir /usr/local/amoeba
tar zxvf /opt/amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba/
chmod -R 755 /usr/local/amoeba/
/usr/local/amoeba/bin/amoeba
如显示amoeba start|stop 说明安装成功

(2) Configure Amoeba read and write separation, two slaves read load balance

First, open permissions for Amoeba to access on the MySQL of Master, Slave1, and Slave2

grant all on *.* to test@'192.168.90.%' identified by '123.com';

amoeba server configuration amoeba service

cd /usr/local/amoeba/conf/

cp amoeba.xml amoeba.xml.bak
vim amoeba.xml	

==30修改==
<property name="user">amoeba</property>		

==32修改==
<property name="password">123123</property>	

==115修改==
<property name="defaultPool">master</property>

==117去掉注释并修改==
<property name="writePool">master</property>
<property name="readPool">slaves</property>

cp dbServers.xml dbServers.xml.bak

vim dbServers.xml								修改数据库配置文件

==23注释掉==
作用：默认进入test库 以防mysql中没有test库时，会报错
<!-- <property name="schema">test</property> -->

==26修改==
<property name="user">test</property>

==28-30去掉注释==
<property name="password">123.com</property>

==45修改，设置主服务器的名master==
<dbServer name="master"  parent="abstractServer">

==48修改，设置主服务器的地址==
<property name="ipAddress">192.168.90.10</property>

==52修改，设置从服务器的名slave1==
<dbServer name="slave1"  parent="abstractServer">

==55修改，设置从服务器1的地址==
<property name="ipAddress">192.168.90.50</property>

==58复制上面6行粘贴，设置从服务器2的名slave2和地址==
<dbServer name="slave2"  parent="abstractServer">
<property name="ipAddress">192.168.90.60</property>

==65修改==
<dbServer name="slaves" virtual="true">

==71修改==
<property name="poolNames">slave1,slave2</property>

/usr/local/amoeba/bin/amoeba start&					#启动Amoeba软件，按ctrl+c 返回
netstat -anpt | grep java							#查看8066端口是否开启，默认端口为TCP 8066

(3) Test results

Test on the client server:

使用yum快速安装MySQL虚拟客户端

yum install -y mysql mysql-server

mysql -u amoeba -p123123 -h 192.168.90.20 -P8066
通过amoeba服务器代理访问mysql ，在通过客户端连接mysql后写入的数据只有主服务会记录，然后同步给从服务器

On the main server

use bobo_ku;
create table test (id int(10),name varchar(10),address varchar(20));

Two slave servers

stop slave;											关闭同步
use bobo_ku;

On slave1

insert into test values('1','zhangsan','this_is_slave1');

On slave2

insert into test values('2','lisi','this_is_slave2');

On the main server

insert into test values('3','wangwu','this_is_master');

On the client server

use bobo_ku;
select * from test;		
客户端会分别向slave1和slave2读取数据，显示的只有在两个从服务器上添加的数据，没有在主服务器上添加的数据

insert into test values('4','qianqi','this_is_client');		只有主服务器上有此数据