Advanced docker (Mysql master-slave replication principle and construction)

Advanced docker (Mysql master-slave replication principle and construction)

This article mainly records the establishment of mysql master-slave replication in docker

MySQL master-slave replication principle

**Main principle: **master saves data in binary log, I/O sends synchronization request to dump, dump sends data to I/O thread, I/O writes to local relay log, SQL thread reads The local relay log data is synchronized to its own database to complete the synchronization.

Types of replication supported by MySQL

1. Statement-based replication: A SQL statement executed on the master server executes the same statement on the slave server. MySQL uses statement-based replication by default, which is more efficient. But the binary log function must be enabled.
2. Replication based on rows: Copy the changed content instead of executing the command on the server.
3. Mixed type of replication: statement-based replication is used by default, and row-based replication is used when statement-based replication is found to be impossible.

Duplicated work process

1. Before each transaction updates data, the master records these changes in the binary log. After writing to the binary log, the Master notifies the storage engine to commit the transaction;
2. Slave copies Master's Binary log (binary log) to its Relay log (relay log). First, the Slave starts a working process - the I/O thread, the I/O thread opens a common connection on the Master, and then starts the Binlog dump process (binary log dump process). The Binlog dump process reads events from the Master's binary log. If it has kept up with the Master, it will sleep and wait for the Master to generate new events. The I/O thread writes these events to the relay log.
3. SQL slave thread (SQL slave thread) handles the last step of the process. The SQL thread reads events from the relay log, and replays the events in it to update the data of the Slave to make it consistent with the data in the Master. As long as the thread is consistent with the I/O thread, the relay log is usually located in the cache of the OS Medium, so the overhead of relaying logs is small.

Note: There is a very important limitation in the replication process, that is, the replication is serialized on the Slave, that is to say, the parallel operation on the Master cannot be performed in parallel on the Slave.

Install mysql master-slave replication

Create a new primary server container instance 3307

docker run -p 3307:3306 --name mysql-master \
-v /mydata/mysql-master/log:/var/log/mysql \
-v /mydata/mysql-master/data:/var/lib/mysql \
-v /mydata/mysql-master/conf:/etc/mysql \
-e MYSQL_ROOT_PASSWORD=root \
-d mysql:5.7

Create master configuration file

# 切换到/mydata/mysql-master/conf下
cd /mydata/mysql-master/conf
# 创建文件my.cnf
vim my.cnf
#### my.cnf内容如下
[mysqld]
## 设置server_id，同一局域网中需要唯一
server_id=101
## 指定不需要同步的数据库名称
binlog-ignore-db=mysql
## 开启二进制日志功能
log-bin=mall-mysql-bin
## 设置二进制日志使用内存大小（事务）
binlog_cache_size=1M
## 设置使用的二进制日志格式（mixed，statement，row）
binlog_format=mixed
## 二进制日志过期清理世间。默认值为0，表示不自动清理
expire_logs_days=7
## 跳过主从复制中遇到的所有错误或指定类型的错误，避免slave端复制中断
##如：1062错误是指一些主键重复，1032错误是因为主从数据库数据不一致
slave_skip_errors=1062

Restart the Master instance after modifying the configuration

docker restart mysql-master

Enter the mysql-master container

docker exec -it mysql-master /bin/bash
mysql -uroot -proot

Create a data synchronization user in the mysql-master container instance

-- 创建用户
CREATE USER 'slave'@'%' IDENTIFIED BY '123456';
-- 用户授权
GRANT REPLICATION SLAVE,REPLICATION CLIENT ON *.* TO 'slave'@'%';

Create a new slave server container instance 3308

docker run -p 3308:3306 --name mysql-slave \
-v /mydata/mysql-slave/log:/var/log/mysql \
-v /mydata/mysql-slave/data:/var/lib/mysql \
-v /mydata/mysql-slave/conf:/etc/mysql \
-e MYSQL_ROOT_PASSWORD=root \
-d mysql:5.7

Create a slave configuration file

# 切换到/mydata/mysql-slave/conf下
cd /mydata/mysql-slave/conf
# 创建文件my.cnf
vim my.cnf
#### my.cnf内容如下
[mysqld]
## 设置server_id，同一局域网中需要唯一
server_id=102
## 指定不需要同步的数据库名称
binlog-ignore-db=mysql
## 开启二进制日志功能，以备Slave作为其他数据库实例的Master时使用
log-bin=mall-mysql-slave1-bin
## 设置二进制日志使用内存大小（事务）
binlog_cache_size=1M
## 设置使用的二进制日志格式（mixed，statement，row）
binlog_format=mixed
## 二进制日志过期清理世间。默认值为0，表示不自动清理
expire_logs_days=7
## 跳过主从复制中遇到的所有错误或指定类型的错误，避免slave端复制中断
##如：1062错误是指一些主键重复，1032错误是因为主从数据库数据不一致
slave_skip_errors=1062
## relay_log配置中继日志
relay_log=mall-mysql-relay-bin
## log-slave-update表示slave将复制事件写进自己的二进制日志
log_slave_updates=1
## slave设置为只读（具有super权限的用户除外）
read_only=1

Restart the slave instance after modifying the configuration

docker restart mysql-slave

View the master-slave synchronization status in the master database

-- 这一步是在主机中的mysql下执行
show master status;

Enter the mysql-slave container

docker exec -it mysql-slave /bin/bash
mysql -uroot -proot

Configure master-slave replication in the slave database

- 注意以下操作是在从机中配置
-- master_host：主数据库的IP地址
-- master_port：主数据库的运行端口
-- master_user：在主数据库创建的用于同步数据的用户账号
-- master_password：在主数据库创建的用于同步数据的用户密码
-- master_log_file：指定从数据库要复制数据的日志文件，通过查看主数据的状态，获取File参数
-- master_log_pos：指定从数据库从那个位置开始复制数据，通过查看主数据库的状态，获取Position参数
-- master_connect_retry：连接失败重试的时间间隔，单位为秒
change master to master_host='106.52.224.231',master_user='slave',master_password='123456',master_port=3307,master_log_file='mall-mysql-bin.000001',master_log_pos=617,master_connect_retry=30;

View the master-slave replication synchronization status from the database

show slave status \G;
-- 主要是看Slave_IO_Running和Slave_SQL_Running

Enable master-slave replication in the slave database

start slave;

Check whether the slave database status is synchronized

show slave status \G;

Master-slave replication test

-- 主库操作
create database dbTest;
use dbTest;
create table student (id int,name varchar(20));
insert into student values(1,"zhangsan");
select * from student;

-- 从库操作
use dbTest;
select * from student;