Preface
In many projects, especially Internet projects, the architecture of master-slave replication and read-write separation is adopted when using MySQL.
Why adopt a master-slave replication architecture with separation of reads and writes? How to achieve? What are the disadvantages? Let us start this learning journey with these questions!
Why use master-slave replication, read-write separation
Master-slave replication and read-write separation are generally used together. The purpose is very simple, is to improve the concurrent performance of the database . You think, assuming it is a stand-alone machine, reading and writing are done on one MySQL, the performance is definitely not high. If there are three MySQL, one mater is only responsible for write operations, and two salves are only responsible for read operations, can the performance be greatly improved?
Therefore, master-slave replication and read-write separation are for the database to support greater concurrency .
With the expansion of business volume, if it is a stand-alone deployment of MySQL, it will cause excessive I/O frequency. Using master-slave replication and separation of reads and writes can improve database availability .
Principle of master-slave replication
① When the Master node performs insert, update, and delete operations, it will be written to the binlog in order.
②The salve slave library connects to the master master library, and as many binlog dump threads are created as the master has as many slaves.
③When the binlog of the master node changes, the binlog dump thread will notify all the salve nodes and push the corresponding binlog content to the slave node.
④ After the I/O thread receives the binlog content, it writes the content to the local relay-log.
⑤The SQL thread reads the relay-log written by the I/O thread, and performs corresponding operations on the slave database according to the content of the relay-log.
How to achieve master-slave replication
I use three virtual machines (Linux) to demonstrate here, the IP is 104 (Master), 106 (Slave), 107 (Slave).
The expected effect is one master and two slaves, as shown in the following figure:
Master configuration
Use the command line to enter mysql:
mysql -u root -p
Then enter the password of the root user (if you forget the password, please check the reset password online~), and then create a user:
//192.168.0.106是slave从机的IP
GRANT REPLICATION SLAVE ON *.* to 'root'@'192.168.0.106' identified by 'Java@1234';
//192.168.0.107是slave从机的IP
GRANT REPLICATION SLAVE ON *.* to 'root'@'192.168.0.107' identified by 'Java@1234';
//刷新系统权限表的配置
FLUSH PRIVILEGES;The two users created are used when configuring the slave.
Next, find the mysql configuration file /etc/my.cnf and add the following configuration:
# 开启binlog
log-bin=mysql-binserver-id=104
# 需要同步的数据库，如果不配置则同步全部数据库binlog-do-db=test_db
# binlog日志保留的天数，清除超过10天的日志
# 防止日志文件过大，导致磁盘空间不足
expire-logs-days=10
After the configuration is complete, restart mysql:
service mysql restart
You can view the current binlog log information through the command line show master status\G; (useful later):
Slave configuration
Slave configuration is relatively simple. The slave must also be a MySQL server, so like the Master, find the /etc/my.cnf configuration file and add the following configuration:
# 不要和其他mysql服务id重复即可
server-id=106
Then use the command line to log in to the mysql server:
mysql -u root -p
Then enter the password to log in.
After entering mysql, enter the following command:
CHANGE MASTER TO
MASTER_HOST='192.168.0.104',//主机IP
MASTER_USER='root',//之前创建的用户账号
MASTER_PASSWORD='Java@1234',//之前创建的用户密码
MASTER_LOG_FILE='mysql-bin.000001',//master主机的binlog日志名称
MASTER_LOG_POS=862,//binlog日志偏移量
master_port=3306;//端口
Not finished yet, you need to start after setting:
# 启动slave服务
start slave;
How to verify whether the startup is successful after startup? Use the following command:
show slave status\G;
You can see the following information (extract some key information):
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.0.104
Master_User: root
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000001
Read_Master_Log_Pos: 619
Relay_Log_File: mysqld-relay-bin.000001
Relay_Log_Pos: 782
Relay_Master_Log_File: mysql-bin.000001 //binlog日志文件名称
Slave_IO_Running: Yes //Slave_IO线程、SQL线程都在运行
Slave_SQL_Running: Yes
Master_Server_Id: 104 //master主机的服务id
Master_UUID: 0ab6b3a6-e21d-11ea-aaa3-080027f8d623
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
Master_Retry_Count: 86400
Auto_Position: 0
The configuration of the other slave is the same, so I won't repeat it.
Test master-slave replication
Execute sql on the master host:
CREATE TABLE `tb_commodity_info` (
`id` varchar(32) NOT NULL,
`commodity_name` varchar(512) DEFAULT NULL COMMENT '商品名称',
`commodity_price` varchar(36) DEFAULT '0' COMMENT '商品价格',
`number` int(10) DEFAULT '0' COMMENT '商品数量',
`description` varchar(2048) DEFAULT '' COMMENT '商品描述',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='商品信息表';
Then we can see that the two slaves have synchronized and created a product information table:
Master-slave replication is complete! There is something for java technology lovers~
Read and write separation
After the master-slave replication is completed, we also need to realize read-write separation. The master is responsible for writing data, and the two slaves are responsible for reading data. How to achieve it?
There are many ways to achieve it. In the past, our company used AOP to judge by method name. The method name that starts with get, select, and query connects to slave, and the others connect to the master database.
But the code is a bit cumbersome to implement through AOP. Is there any ready-made framework? The answer is yes.
Apache ShardingSphere is an ecosystem composed of a set of open source distributed database middleware solutions. It consists of JDBC and Proxy.
ShardingSphere-JDBC is positioned as a lightweight Java framework and provides additional services in the JDBC layer of Java. It uses the client to directly connect to the database and provides services in the form of jar packages without additional deployment and dependencies. It can be understood as an enhanced version of the JDBC driver, fully compatible with JDBC and various ORM frameworks.
Read and write separation can be achieved using ShardingSphere-JDBC.
Let's demonstrate the code implementation of SpringBoot+Mybatis+Mybatis-plus+druid+ShardingSphere-JDBC.
Project configuration
Version description:
SpringBoot：2.0.1.RELEASE
druid：1.1.22
mybatis-spring-boot-starter:1.3.2
mybatis-plus-boot-starter：3.0.7
sharding-jdbc-spring-boot-starter:4.1.1
Add the maven configuration of sharding-jdbc:
<dependency>
<groupId>org.apache.shardingsphere</groupId>
<artifactId>sharding-jdbc-spring-boot-starter</artifactId>
<version>4.1.1</version>
</dependency>
Then add configuration in application.yml:
# 这是使用druid连接池的配置，其他的连接池配置可能有所不同
spring: shardingsphere: datasource: names: master,slave0,slave1 master: type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.jdbc.Driver url: jdbc:mysql://192.168.0.108:3306/test_db?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
username: yehongzhi password: YHZ@1234
slave0: type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.jdbc.Driver url: jdbc:mysql://192.168.0.109:3306/test_db?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
username: yehongzhi password: YHZ@1234
slave1: type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.jdbc.Driver url: jdbc:mysql://192.168.0.110:3306/test_db?useUnicode=true&characterEncoding=utf8&tinyInt1isBit=false&useSSL=false&serverTimezone=GMT
username: yehongzhi password: YHZ@1234
props: sql.show: true
masterslave: load-balance-algorithm-type: round_robin
sharding: master-slave-rules: master: master-data-source-name: master slave-data-source-names: slave0,slave1sharding.master-slave-rules is to indicate the master library and the slave library. Make sure not to write wrongly. Otherwise, writing data to the slave library will cause synchronization failure.
load-balance-algorithm-type is the routing strategy, and round_robin indicates the polling strategy.
Start the project, you can see the following information, which means the configuration is successful:
Write Controller interface:
/**
* 添加商品
*
* @param commodityName 商品名称
* @param commodityPrice 商品价格
* @param description 商品价格
* @param number 商品数量
* @return boolean 是否添加成功
* @author java技术爱好者
*/
@PostMapping("/insert")
public boolean insertCommodityInfo(@RequestParam(name = "commodityName") String commodityName,
@RequestParam(name = "commodityPrice") String commodityPrice,
@RequestParam(name = "description") String description,
@RequestParam(name = "number") Integer number) throws Exception {
return commodityInfoService.insertCommodityInfo(commodityName, commodityPrice, description, number);
}Ready, start testing!
test
Open POSTMAN and add products:
The console can see the following information:
The data query is done through the slave:
It's that simple!
Disadvantage
Although master-slave replication and separation of reads and writes can ensure the high availability of MySQL services and improve overall performance to a large extent, there are many problems:
- The slave synchronizes data from the master through the binlog log. If there is a network delay, the slave will experience data delay. Then it is possible that after the master writes the data, the slave reads the data may not be able to read it out immediately .
Some people may ask, are there any business issues?
In fact, this framework has already been thought of. When we look back at the previous screenshot, there is a sentence like this:
Recommended learning: As expected of Alibaba's senior architect, he has explained the problem of MySql master-slave replication delay so thoroughly, hurry up and collect it!