concept
What is read-write separation?
The basic principle of read-write separation is to let the primary database handle transactional addition, modification, and deletion operations (INSERT, UPDATE, DELETE), while the slave database handles SELECT query operations. Database replication is used to synchronize changes caused by transactional operations to slave databases in the cluster.
Why read and write separation?
Because the "write" operation of the database (it may take 3 minutes to write 10,000 pieces of data) is time-consuming.
But the "reading" of the database (it may only take 5 seconds to read 10,000 pieces of data);
Therefore, the separation of reading and writing, the solution is that the writing of the database affects the efficiency of the query.
When to read and write separation?
The database does not necessarily have to be separated from reading and writing. If the program uses the database a lot, but there are few updates and many queries, it will be considered. Using database master-slave synchronization, and then through read-write separation can share the database pressure and improve performance.
Master-slave replication and read-write separation
In the actual production environment, the reading and writing of the database are all in the same database server, which cannot meet the actual needs. Whether in terms of security, high availability, or high concurrency, it is completely unable to meet actual needs.
Therefore, the data is synchronized through master-slave replication, and the concurrent load capacity of the database is improved through read-write separation. A bit similar to rsync (file synchronization tool), but the difference is that rsync backs up disk files, while mysql master-slave replication backs up data and statements in the database
Replication types supported by mysql
(1) STATEMENT : Statement-based replication. Execute the SQL statement on the server, and execute the same statement on the slave server. Mysql uses statement-based replication by default (before version 5.7), which has high execution efficiency. In the case of high concurrency, errors in the execution sequence and deadlocks of transactions may occur.
(2) ROW : Row-based replication. Copy the changed content instead of executing the command on the slave server. Accurate, but inefficient, and the saved file will be larger. (After version 5.7, ROW mode is used by default)
(3) MIXED : Copy of mixed types. By default, statement-based replication is used. Once it is found that statement-based replication cannot be accurately replicated, row-based replication will be used. Smarter, so use MIXED in most cases.
(3) At the same time, the Master node starts a dump thread for each I/O thread to notify and send binary events to it. After the I/O thread receives the bin-log content, it saves the content to the local relay of the slave node In the log (Relay log ), the Slave node will start the SQL thread to read binary events from the relay log, replay them locally, that is, parse them into SQL statements and execute them one by one, so that its data is consistent with that of the Master node. Finally, the I/O thread and the SQL thread will go to sleep and wait for the next wake-up
Remember two logs and three threads:
Two logs: binary log (bin log) , relay log (Relay log)
Three threads: I/O thread , dump thread , SQL thread
Notice:
- Relay logs are usually located in the OS cache, so the overhead of relay logs is small.
- The replication process has a very important limitation, that is, the replication is serialized on the Slave, that is to say, the parallel update operation on the Master cannot be operated in parallel on the Slave.
- For semi-synchronous replication, there will be an additional ack confirmation thread (ack collector thread), which is specially used to receive feedback information from the slave (collect the ack information returned by the slave node)
Solution to database master-slave data inconsistency
Method 1: After ignoring the error, continue to synchronize
This method is suitable for the situation where the master-slave database data is not much different, or the data is not required to be completely unified, and the data requirements are not strict.
Method 2: Re-do master-slave, complete synchronization
This method is suitable for the situation where the master-slave database data differs greatly, or the data is required to be completely unified
How to ensure data synchronization after mysql resumes from server hang?
- Physical method: rsync disk file synchronization. To use file recovery, the master node needs to stop the service.
- Master-slave replication: delete the original library of the slave node, and re-do the master-slave replication through the offset
Under what circumstances will semi-synchronous replication be reduced to asynchronous replication? When will synchronous replication resume?
-
When semi-synchronous replication times out (controlled by the rpl_semi_sync_master_timeout parameter, the default is 10000ms, that is, 10s), semi-synchronous replication will be temporarily closed and asynchronous replication will be used instead, that is, it will be automatically reduced to asynchronous work.
-
After the malster dump thread sends all the events of a transaction, if it receives a response from the slave library within rpl_ semi_sync_master_ timeout, the master-slave resumes semi-synchronous replication
MySQL master-slave replication delay reasons and optimization methods
Reasons for master-slave replication delay:
- The master server has high concurrency and forms a large number of transactions.
- network latency.
- Caused by master-slave hardware devices (cpu main frequency, memory IO, hard disk IO).
- It is synchronous replication, not asynchronous replication.
Optimization:
-
Optimize Mysql parameters from the library. For example, increase innodb_buffer_pool_size to allow more operations to be completed in Mysql memory and reduce disk operations.
-
Use a high-performance host from the library. Including powerful cpu, increased memory. Avoid using virtual cloud hosts and use physical hosts, which improves I/O performance.
-
Use SSD disk from library.
-
Network optimization to avoid synchronization across computer rooms
1. Asynchronous replication
MySQL's default replication is asynchronous. The master library will immediately return the results to the client after executing the transaction submitted by the client. It doesn't care whether the slave library has received and processed it, so there will be a problem, mainly crash At this time, the transactions that have been submitted by the master may not be transmitted to the slave. If at this time, the slave is forcibly promoted to master, the data on the new master may be incomplete.
2. Fully synchronous replication
It means that when the main library executes a transaction, all slave libraries execute the transaction before returning to the client. Because it is necessary to wait for all slave libraries to execute the transaction before returning, the performance of full synchronous replication will inevitably be seriously affected.
3. Semisynchronous replication
Between asynchronous replication and full synchronous replication, the master library does not return to the client immediately after executing the transaction submitted by the client, but waits for at least one slave library to receive and write to the relay log before returning to the client. Compared with asynchronous replication, semi-synchronous replication improves data security, and it also causes a certain degree of delay, which is at least a TCP/IP round-trip time. Therefore, semi-synchronous replication is best used in low-latency networks
Master-slave replication case demonstration (asynchronous replication)
lab environment:
Master server: 192.168.137.50, mysql5.7
Slave1 server: 192.168.137.60, mysql5.7
Slave2 server: 192.168.137.70, mysql5.7
Experiment ideas:
- Master and slave server time synchronization.
- Configure the master server, modify its configuration file, and create a synchronization account in the master database to authorize the use of the slave database.
- Configure the slave database, modify its configuration file, and configure synchronization in the slave database.
- Verify that the synchronization is successful
Experimental steps:
Turn off firewall and selinux on each server
systemctl disable --now firewalld
setenforce 0
MYSQL master-slave server time synchronization
Time synchronization:
- Each mysql server needs to set time synchronization to avoid confusion during data synchronization.
- If there is no external network, use ntp service. If it is connected to the external network, the network clock source can be used.
The following will demonstrate that the master node uses the local clock source and the slave node performs time synchronization
Master server uses local clock source
#安装时间同步工具（本地设置时钟源）
[root@yuji ~]# yum install -y ntp
#修改ntp配置文件，在末尾加入
[root@yuji ~]# vim /etc/ntp.conf
server 127.127.72.0 #设置本地时钟源，注意修改网段（72是网段）
fudge 127.127.72.0 stratum 8 #设置时间层级为8（限制在15以内）
#开启ntpd
[root@yuji ~]# systemctl start ntpd
Two slave servers synchronize the time of the master server
#安装时间同步工具
yum install -y ntp
#开启ntpd
systemctl start ntpd
#和主服务器进行时间同步
/usr/sbin/ntpdate 192.168.137.50
#设置定时任务，每30分钟同步一次
crontab -e
*/30 * * * * /usr/sbin/ntpdate 192.168.137.50
mysql configuration of master server
Modify the configuration file and create a synchronization account authorized to use from the database
[root@]# vim /etc/my.cnf
[mysqld]
......
server-id = 1 #指定服务ID号，master和两台slave都要不同
log-bin=mysql-bin #添加，主服务器开启二进制日志
binlog_format = MIXED #指定二进制日志(binlog)的记录格式为MIXED
log-slave-updates=true #添加，允许slave从master复制数据时可以写入到自己的二进制日志
expire_logs_days = 7 #设置二进制日志文件过期时间，默认值为0，表示logs不过期
max_binlog_size = 500M #设置二进制日志限制大小，如果超出给定值，日志就会发生滚动，默认值是1GB
#重启服务
[root@]# systemctl restart mysqld
[root@]# mysql -u root -pabc123
#给从服务器授权
grant replication slave on *.* to 'myslave'@'192.168.137.%' identified by '123123';
flush privileges; #刷新权限
show master status; #查看主服务器状态
//显示如下
+-------------------+----------+--------------+------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| mysql-bin.000006 | 603 | | |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)
#File 列显示日志名，Position 列显示偏移量
mysql configuration from the server
#修改配置文件
[root@s1 ~]# vim /etc/my.cnf
[mysqld]
......
server-id = 2 #修改，注意id与Master的不同，两个Slave的id也要不同
relay-log=relay-log-bin #添加，开启中继日志，从主服务器上同步日志文件记录到本地
relay-log-index=slave-relay-bin.index #添加，定义中继日志索引文件的位置和名称，一般和relay-log在同一目录
relay_log_recovery = 1 #选配项
#当 slave 从库宕机后，假如 relay-log 损坏了，导致一部分中继日志没有处理，则自动放弃所有未执行的 relay-log，并且重新从 master 上获取日志，这样就保证了 relay-log 的完整性。默认情况下该功能是关闭的，将 relay_log_recovery 的值设置为 1 时， 可在 slave 从库上开启该功能，建议开启。
#重启服务
[root@s1 ~]# systemctl restart mysqld
#登录数据库，进行同步设置
[root@s1 ~]# mysql -u root -p
CHANGE master to master_host='192.168.137.50',master_user='myslave',master_password='123123',master_log_file='mysql-bin.000001',master_log_pos=604;
#配置同步，注意 master_log_file 和 master_log_pos 的值要与Master查询的一致
start slave; #启动同步，如有报错执行 reset slave;
show slave status\G #查看 Slave 状态
##确保 IO 和 SQL 线程都是 Yes，代表同步正常。
Slave_IO_Running: Yes #负责与主机的IO通信
Slave_SQL_Running: Yes #负责自己的slave mysql进程
##一般 "Slave_IO_Running: No" 的可能原因：
1. 网络不通
2. my.cnf配置有问题（server-id重复）
3. 密码、file文件名、pos偏移量不对
4. 防火墙没有关闭
The Slave2 server is also configured in the same way. Note that the server-id in the configuration file is different from the previous two. I set it to "server-id = 3" here
Tips:
Common possible causes of "Slave_IO_Running: No"
- network issue
- There is a problem with my.cnf configuration (duplicate server-id)
- The password, file name, and pos offset are incorrect
- Firewall is not turned off
Verify the effect of master-slave replication
Create a new database on the master server, execute create database cxk666;
Go to the slave server to check whether the synchronization is successful, execute show databases;
Master-slave replication case demonstration (semi-synchronous replication)
lab environment:
Master server: 192.168.137.50, mysql5.7
Slave1 server: 192.168.137.60, mysql5.7
Slave2 server: 192.168.137.70, mysql5.7
Experimental operation:
systemctl disable --now firewalld
setenforce 0
-------2、主数据库配置---------
vim /etc/my.cnf #在 [mysqld] 区域添加下面内容
......
plugin-load=rpl_semi_sync_master=semisync_master.so #加载mysql半同步复制的插件
rpl_semi_sync_master_enabled=ON #或者设置为"1"，即开启半同步复制功能
rpl-semi-sync-master-timeout=1000 #超时时间为1000ms，即1s
systemctl restart mysqld
-------3、从数据库配置-----------
vim /etc/my.cnf
......
plugin-load=rpl_semi_sync_slave=semisync_slave.so
rpl_semi_sync_slave_enabled=ON
systemctl restart mysqld
-------4、查看半同步是否在运行---------------
#主数据库执行
show status like 'Rpl_semi_sync_master_status';
show variables like 'rpl_semi_sync_master_timeout';
#从数据库执行（此时可能还是OFF状态，需要在下一步重启IO线程后，从库半同步状态才会为ON）
show status like 'Rpl_semi_sync_slave_status';
#重启从数据库上的IO线程
STOP SLAVE IO_THREAD;
START SLAVE IO_THREAD;
#在主库查询半同步状态
show status like '%Rpl_semi%';
参数说明：
Rpl_semi_sync_master_clients #半同步复制客户端的个数
Rpl_semi_sync_master_net_avg_wait_time #平均等待时间（默认毫秒）
Rpl_semi_sync_master_net_wait_time #总共等待时间
Rpl_semi_sync_master_net_waits #等待次数
Rpl_semi_sync_master_no_times #关闭半同步复制的次数
Rpl_semi_sync_master_no_tx #表示没有成功接收slave提交的次数
Rpl_semi_sync_master_status #表示当前是异步模式还是半同步模式，on为半同步
Rpl_semi_sync_master_timefunc_failures #调用时间函数失败的次数
Rpl_semi_sync_master_tx_avg_wait_time #事物的平均传输时间
Rpl_semi_sync_master_tx_wait_time #事物的总共传输时间
Rpl_semi_sync_master_tx_waits #事物等待次数
Rpl_semi_sync_master_wait_pos_backtraverse #可以理解为"后来的先到了，而先来的还没有到的次数"
Rpl_semi_sync_master_wait_sessions #当前有多少个session因为slave的回复而造成等待
Rpl_semi_sync_master_yes_tx #成功接受到slave事物回复的次数
当半同步复制发生超时（由rpl_semi_sync_master_timeout参数控制，默认为10000ms，即10s），会暂时关闭半同步复制，转而使用异步复制，也就是会自动降为异步工作。
当 master dump 线程发送完一个事务的所有事件之后，如果在 rpl_semi_sync_master_timeout 内，收到了从库的响应， 则主从又重新恢复为半同步复制。
Notice:
1) In the architecture of one master and multiple slaves, if semi-synchronous replication is to be enabled, it is not required that all slaves are semi-synchronous.
2) MySQL 5.7 has greatly improved the performance of semi-synchronous replication.
- In the semi-synchronous replication of version 5.6, the dump thread undertakes two different and very frequent tasks: sending binlog to the slave, and waiting for the feedback information from the slave, and these two tasks are serial, and the dump thread must wait for the slave to return Only then will the next events transaction be transmitted. Dump thread has become the bottleneck of improving the performance of the whole semi-synchronization. In high-concurrency business scenarios, such a mechanism will affect the overall system throughput (TPS) of the database.
- In the semi-synchronous replication of version 5.7, an ack collector thread is independent, which is specially used to receive feedback information from slave. In this way, two threads on the master work independently, and can send binlog to slave and receive feedback from slave at the same time
Under what circumstances will semi-synchronous replication be reduced to asynchronous replication? When will synchronous replication resume?
- When semi-synchronous replication times out (controlled by the rpl_semi_sync_master_timeout parameter, the default is 10000ms, that is, 10s), semi-synchronous replication will be temporarily closed and asynchronous replication will be used instead, that is, it will be automatically reduced to asynchronous work.
- After the malster dump thread sends all the events of a transaction, if it receives a response from the slave library within the rpl_ semi_sync_master_ timeout, the master-slave resumes semi-synchronous replication.
The concept of read-write separation
The principle of MySQL read and write separation
- Read-write separation means only writing on the master server and reading only on the slave server.
- The basic principle is to let the master database handle transactional operations, while the slave database handles select queries.
- Database replication is used to synchronize changes caused by transactional operations on the primary database to the secondary databases in the cluster.
At present, the more common MysQL read and write separation is divided into the following two types
1) Based on the internal implementation of the program code
In the code, routing is classified according to select and insert. This method is also the most widely used in the production environment.
The advantage is that the performance is better, because it is implemented in the program code, and there is no need to add additional equipment for hardware expenses; the disadvantage is that it needs developers to implement, and the operation and maintenance personnel have no way to start.
However, not all applications are suitable for implementing read-write separation in the program code. Like some large-scale and complex Java applications, if the read-write separation is implemented in the program code, the code changes will be relatively large.
2) Implementation based on the intermediate proxy layer
The agent is generally located between the client and the server. After receiving the request from the client, the agent server forwards it to the back-end database after passing the judgment. The following representative procedures are shown.
(1) MySQL-Proxy. MySQL-Proxy is a MysQL open source project, and SQL judgment is performed through its own 1ua script.
(2) Atlas. It is a data middle layer project based on the MysQL protocol developed and maintained by the infrastructure team of the Web Platform Department of Qihoo 360. It is based on mysql-proxy version 0.8.2, optimized and added some new features. 360 internally uses atlas to run the mysql business, and the number of read and write requests carried by it every day reaches several thousand bars. Supports things as well as stored procedures.
(3) Amoeba. Developed by Chen Siru, the author once worked for Alibaba. The program is developed in the Java language, and Alibaba uses it in the production environment. But it does not support transactions and stored procedures.
(4) Mycat. It is a popular database middleware written based on Java language. It is a server that implements the MySq1 protocol. Its core function is to divide databases and tables. In conjunction with the master-slave mode of the database, read-write separation can also be achieved.
Since using MysQLProxy requires writing a large number of ua scripts, these Lua are not ready-made, but need to be written by yourself. This is very difficult for people who are not familiar with MysQLProxy built-in variables and MySQL Protocol.
Amoeba is a very easy to use, very portable software. Therefore, it is widely used in the proxy layer of the database in the production environment.
Read-write separation case (using Amoeba)
lab environment
Master server: 192.168.137.50, mysql5.7, centos7-2
Slave1 server: 192.168.137.60, mysql5.7, centos7-6
Amoeba server: 192.168.137.70, jdk1.6, Amoeba centos7-3
Client: 192.168.137.40, mysql5.7, centos7-4
Amoeba server configuration
#因为 Amoeba 基于是 jdk1.5 开发的，所以官方推荐使用 jdk1.5 或 1.6 版本，高版本不建议使用。
#先将jdk的二进制文件上传到/opt/目录下，之后复制到/usr/local/目录下
[root@Amo ~]# cd /opt/
[root@Amo opt]# cp jdk-6u14-linux-x64.bin /usr/local/
[root@Amo opt]# cd /usr/local/
[root@Amo local]# chmod +x jdk-6u14-linux-x64.bin #为二进制文件增加执行权限
[root@Amo local]# ./jdk-6u14-linux-x64.bin
##按yes，按enter
[root@Amo local]# mv jdk1.6.0_14/ /usr/local/jdk1.6 #将jdk目录重命名
#添加环境变量
[root@Amo local]# vim /etc/profile
export JAVA_HOME=/usr/local/jdk1.6
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
export AMOEBA_HOME=/usr/local/amoeba
export PATH=$PATH:$AMOEBA_HOME/bin
[root@Amo local]# source /etc/profile #刷新文件，使立即生效
[root@Amo local]# java -version #查看jdk版本
Install Amoeba software
-------1、安装 Amoeba软件---------
#创建Amoeba的解压目录
[root@Amo ~]# mkdir /usr/local/amoeba/
#将Amoeba安装包上传到/opt/目录，解压安装包
[root@Amo opt]# cd /opt/
[root@Amo opt]# tar zxvf amoeba-mysql-binary-2.2.0.tar.gz -C /usr/local/amoeba/
#增加目录权限
[root@Amo opt]# chmod -R 755 /usr/local/amoeba/
#开启Amoeba
[root@Amo opt]# /usr/local/amoeba/bin/amoeba #如显示amoeba start|stop说明安装成功
-------2、配置 Amoeba读写分离，两个 Slave 读负载均衡----------
#先在Master、Slave1、Slave2 的mysql上开放权限给 Amoeba 访问。注意：这里授权的用户名和密码，会在下一步写入数据库配置文件。
grant all on *.* to yuji@'192.168.137.%' identified by '123123';
#再回到amoeba服务器配置amoeba服务：
[root@Amo opt]# cd /usr/local/amoeba/conf/
#备份配置文件，修改amoeba配置文件
[root@Amo conf]# cp amoeba.xml amoeba.xml.bak
[root@Amo conf]# vim amoeba.xml
--30行--
<property name="user">amoeba</property> #30行和32行，授权客户端用于登录amoeba的账号和密码
--32行--
<property name="password">123456</property>
--115行--
<property name="defaultPool">master</property> #设置默认服务器池
--117-去掉注释-
<property name="writePool">master</property> #定义写的服务器池名称
<property name="readPool">slaves</property> #定义读的服务器池名称
#备份数据库配置文件，之后修改数据库配置文件dbServers.xml
[root@Amo conf]# cp dbServers.xml dbServers.xml.bak
[root@Amo conf]# vim dbServers.xml
--23行--注释掉 作用：默认进入test库，注释掉以防mysql中没有test库时，会报错
<!-- <property name="schema">test</property> -->
##26-30行，此用户就是之前在3台主从服务器上授权的用户，授权amoeba服务器用来登录mysql数据库的用户和密码。
--26行--修改
<property name="user">yuji</property>
--28-30行--去掉注释
<property name="password">123123</property>
--45行--修改，设置主服务器的名称master和地址
<dbServer name="master" parent="abstractServer">
--48行--修改，设置主服务器的地址
<property name="ipAddress">192.168.137.50</property>
--52行--修改，设置从服务器1的名称slave1
<dbServer name="slave1" parent="abstractServer">
--55行--修改，设置从服务器1的地址
<property name="ipAddress">192.168.137.60</property>
--58行--复制上面6行粘贴，设置从服务器2的名称slave2和地址
<dbServer name="slave2" parent="abstractServer">
<property name="ipAddress">192.168.137.70</property>
--65行--修改
<dbServer name="slaves" virtual="true">
--71行--修改
<property name="poolNames">slave1,slave2</property>
[root@Amo conf]# /usr/local/amoeba/bin/amoeba start & #后台启动Amoeba软件，按ctrl+c 返回
[root@Amo conf]# netstat -anpt | grep java #查看8066端口是否开启，默认端口为TCP 8066
The client installs the mariadb database
[root@ ~]# yum install -y mariadb-server mariadb #安装mariadb数据库
[root@ ~]# systemctl start mariadb.service #启动mariadb
#客户端通过amoeba服务器登录数据库，之后向库中写入数据：
mysql -u amoeba -pabc123 -h 192.168.137.50 -P8066
use yuji666;
create table class(id int,name char(10));
#通过amoeba服务器代理访问mysql ，再通过客户端连接mysql后写入的数据只有主服务会记录，然后同步给从--从服务器
Test read and write separation
//在两台slave服务器上，关闭同步：
stop slave; #关闭同步
use yuji666;
//在slave1上写入数据：
insert into class values('1','zhangsan');
//在slave2上写入数据：
insert into class values('2','lisi');
//在master服务器上写入数据：
insert into class values('3','wangwu');
//在客户端上查看数据：
use yuji666;
select * from class;
#客户端会分别向slave1和slave2读取数据（轮询），显示的只有在两个从服务器上添加的数据，没有在主服务器上添加的数据。说明读写是分离的，只从slave中读取数据。
insert into class values('4','qianqi',); //客户端插入数据，只有主服务器上有此数据
//在两个从服务器上执行 start slave; 即可实现同步主服务器中添加的数据
start slave; #开启同步
select * from class;
Key summary:
The working process of master-slave replication
The master node needs to enable the binary log, and the slave node needs to enable the relay log.
(1) The Master node records data changes into a binary log (bin log) . When the data on the Master changes (addition, deletion, modification), the change is written into the binary log.
(2) The slave node will detect whether the binary log of the master has changed within a certain time interval, and if it changes, it will start an I/O thread to request the binary event of the master. (request binary data)
(3) At the same time, the Master node starts a dump thread for each I/O thread , which is used to send binary events to it, and saves it to the local relay log (Relay log ) of the slave node. The Slave node will start the SQL thread from it After reading the binary events in the log, replay them locally, that is, parse them into SQL statements and execute them one by one, so that the data is consistent with that of the Master node. Finally, the I/O thread and the SQL thread will go to sleep and wait for the next wake-up.
Notice:
- Relay logs are usually located in the OS cache, so the overhead of relay logs is small.
- The replication process has a very important limitation, that is, the replication is serialized on the Slave, that is to say, the parallel update operation on the Master cannot be operated in parallel on the Slave.
- For semi-synchronous replication, there will be an additional ack confirmation thread (ack collector thread), which is specially used to receive feedback information from the slave (collect the ack information returned by the slave node).
2. Solution to database master-slave data inconsistency
Method 1: After ignoring the error, continue to synchronize
- This method is suitable for the situation where the master-slave database data is not much different, or the data is not required to be completely unified, and the data requirements are not strict.
Method 2: Re-do master-slave, complete synchronization
- This method is suitable for situations where the master-slave database data differ greatly, or the data is required to be completely unified.
3. How to ensure data synchronization after mysql resumes from the server?
- Physical method: rsync disk file synchronization. To use file recovery, the master node needs to stop the service.
- Master-slave replication: Delete the original library of the slave node, and re-do the master-slave replication through the offset.
4. Under what circumstances will semi-synchronous replication be reduced to asynchronous replication? When will synchronous replication resume?
- When semi-synchronous replication times out (controlled by the rpl_semi_sync_master_timeout parameter, the default is 10000ms, that is, 10s), semi-synchronous replication will be temporarily closed and asynchronous replication will be used instead, that is, it will be automatically reduced to asynchronous work.
- After the malster dump thread sends all the events of a transaction, if it receives a response from the slave library within the rpl_ semi_sync_master_ timeout, the master-slave resumes semi-synchronous replication.
5. MySQL master-slave replication delay reasons and optimization methods
Reasons for master-slave replication delay:
- The master server has high concurrency and forms a large number of transactions.
- network latency.
- Caused by master-slave hardware devices (cpu main frequency, memory IO, hard disk IO).
- It is synchronous replication, not asynchronous replication.
Optimization:
- Optimize Mysql parameters from the library. For example, increase innodb_buffer_pool_size to allow more operations to be completed in Mysql memory and reduce disk operations.
- Use a high-performance host from the library. Including powerful cpu, increased memory. Avoid using virtual cloud hosts and use physical hosts, which improves I/O performance.
- Use SSD disk from library.
- Network optimization to avoid synchronization across computer rooms.
6. Precautions
- Each master can have multiple slaves.
- Each slave can only have one master.
-
Each slave can only have a unique server ID (server-id).
-
The master must enable the binlog binary log function; usually for data security, the slave also enables the binlog function.