Deploy GTID-based Mysql one-master two-slave cluster
Digression
We all know that Mysql's master-slave replication is a very important function, we will use it in most places in our daily life, such as hot standby, read-write separation, high availability, and so on. This article introduces the basic principles and basic deployment of mysql master-slave replication.
1. Environmental preparation
Three Mysql server
OS: centos 7
master: 192.168.81.128
slave 1: 192.168.81.129
slave 2: 192.168.81.130
Mysql instance
Three Mysql servers are installed and deployed respectively Mysql instance, the version is 5.7.30.
[root@server03 mysql3308]# ps -ef|grep mysql
root 3346 2908 0 15:23 pts/0 00:00:00 /bin/sh /d/mysqlbase/mysql3308/bin/mysqld_safe --defaults-file=/d/mysqldata/mysql3308/my.cnf.3308
mysql 4660 3346 0 15:23 pts/0 00:00:20 /d/mysqlbase/mysql3308/bin/mysqld --defaults-file=/d/mysqldata/mysql3308/my.cnf.3308 --basedir=/d/mysqlbase/mysql3308 --datadir=/d/mysqldata/mysql3308/mydata --plugin-dir=/d/mysqlbase/mysql3308/lib/plugin --user=mysql --log-error=/d/mysqldata/mysql3308/log/error.log --pid-file=/d/mysqldata/mysq3308/sock/mysql.pid --socket=/d/mysqldata/mysql3308/sock/mysql.sock --port=3308
For specific installation and deployment of Mysql instance, please refer to my other article:
https://blog.csdn.net/tah_001/article/details/107660943
Note: The server-id of the three instances cannot be the same and can be changed in the configuration file.
Second, the principle of Mysql master-slave replication
MySQL master-slave replication involves three threads, one running on the master node (log dump thread), and the other two (I/O thread, SQL thread) running on the slave node, as shown in the following diagram:
1、 主节点 binary log dump 线程
当从节点连接主节点时，主节点会创建一个log dump 线程，用于发送bin-log的内容。在读取bin-log中的操作时，此线程会对主节点上的bin-log加锁，当读取完成，甚至在发动给从节点之前，锁会被释放。
2、从节点I/O线程
当从节点上执行`start slave`命令之后，从节点会创建一个I/O线程用来连接主节点，请求主库中更新的bin-log。I/O线程接收到主节点binlog dump 进程发来的更新之后，保存在本地relay-log中。
3、从节点SQL线程
SQL线程负责读取relay log中的内容，解析成具体的操作并执行，最终保证主从数据的一致性。
4、对于每一个主从连接，都需要三个进程来完成。当主节点有多个从节点时，主节点会为每一个当前连接的从节点建一个binary log dump 进程，而每个从节点都有自己的I/O进程，SQL进程。从节点用两个线程将从主库拉取更新和执行分成独立的任务，这样在执行同步数据任务的时候，不会降低读操作的性能。比如，如果从节点没有运行，此时I/O进程可以很快从主节点获取更新，尽管SQL进程还没有执行。如果在SQL进程执行之前从节点服务停止，至少I/O进程已经从主节点拉取到了最新的变更并且保存在本地relay日志中，当服务再次起来之后，就可以完成数据的同步。
The basic process of Mysql master-slave replication is as follows:
the I/O process on the slave node connects to the master node and requests the log content from the specified location of the specified log file (or from the very beginning log); the master node receives the data from the slave node After the I/O request, the I/O process responsible for replication reads the log information after the specified location of the specified log according to the request information, and returns it to the slave node. In addition to the information contained in the log, the returned information also includes the bin-log file and bin-log position of the returned information; after receiving the content from the node's I/O process, the received log content is updated To the local relay log, and save the read binary log file name and location to the master-info file, so that the next time you read it, you can clearly tell the Master "Which bin-log do I need from? Please send me the content of the log after the position starts." After the SQL thread of Slave detects that the new content is added to the relay-log, it will parse the content of the relay-log into the actual operation performed on the Zhu node, and Execute in this database.
Mysql master-slave replication based on gtid:
A GTID-based replication method has been added since MySQL 5.6.5. GTID ensures that each transaction submitted on the main library has a unique ID in the cluster. This method strengthens the database's primary and backup consistency, fault recovery and fault tolerance.
In the original binary log-based replication, the slave database needs to tell the master database from which offset to perform incremental synchronization. If the specified error is specified, the data will be missed and the data will be inconsistent. With the help of GTID, in the event of a master/slave switchover, other MySQL slave libraries can automatically find the correct replication location on the new master library, which greatly simplifies the maintenance of the cluster under complex replication topologies and reduces the occurrence of artificial setting of replication locations. Risk of misoperation. In addition, GTID-based replication can ignore transactions that have already been executed, reducing the risk of data inconsistency.
gtid工作原理：
①当一个事务在主库端执行并提交时，产生GTID，一同记录到binlog日志中。
②binlog传输到slave,并存储到slave的relaylog后，读取这个GTID的这个值设置gtid_next变量，即告诉Slave，下一个要执行的GTID值。
③sql线程从relay log中获取GTID，然后对比slave端的binlog是否有该GTID。
④如果有记录，说明该GTID的事务已经执行，slave会忽略。
⑤如果没有记录，slave就会执行该GTID事务，并记录该GTID到自身的binlog，
在读取执行事务前会先检查其他session持有该GTID，确保不被重复执行。
⑥在解析过程中会判断是否有主键，如果没有就用二级索引，如果没有就用全部扫描。
Three, deployment
(1) Create a user for master-slave replication in the master
mysql> CREATE USER 'rep'@'%' IDENTIFIED BY '555666';
Query OK, 0 rows affected (0.48 sec)
mysql> GRANT REPLICATION SLAVE ON *.* TO 'rep'@'%' IDENTIFIED BY '555666';
Query OK, 0 rows affected, 1 warning (0.05 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.21 sec)
mysql> select user,host from mysql.user;
+---------------+-----------+
| user | host |
+---------------+-----------+
| rep | % |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+---------------+-----------+
4 rows in set (0.06 sec)
(2) Specify the master-slave replication relationship in all slaves
CHANGE MASTER TO
MASTER_HOST = '192.168.81.128',
MASTER_PORT = 3308,
MASTER_USER = 'rep',
MASTER_PASSWORD = '555666',
MASTER_AUTO_POSITION = 1;
开启slave的主从复制：
mysql> start slave;
Query OK, 0 rows affected (0.34 sec)
查看主从复制状态：
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.81.128
Master_User: rep
Master_Port: 3308
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 832
Relay_Log_File: mysql-relay-bin.000003
Relay_Log_Pos: 1045
Relay_Master_Log_File: mysql-bin.000002
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 832
Relay_Log_Space: 1429
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 92
Master_UUID: 372cdb11-dd77-11ea-8724-000c296c0825
Master_Info_File: mysql.slave_master_info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 372cdb11-dd77-11ea-8724-000c296c0825:1-3
Executed_Gtid_Set: 372cdb11-dd77-11ea-8724-000c296c0825:1-3
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
在master可以看到相应的dump进程：
mysql> show processlist;
+----+-----------------+----------------------+------+------------------+-------+---------------------------------------------------------------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+----+-----------------+----------------------+------+------------------+-------+---------------------------------------------------------------+------------------+
| 1 | event_scheduler | localhost | NULL | Daemon | 86180 | Waiting on empty queue | NULL |
| 5 | root | localhost | NULL | Query | 0 | starting | show processlist |
| 7 | rep | 192.168.81.129:51974 | NULL | Binlog Dump GTID | 345 | Master has sent all binlog to slave; waiting for more updates | NULL |
| 8 | rep | 192.168.81.130:60828 | NULL | Binlog Dump GTID | 20 | Master has sent all binlog to slave; waiting for more updates | NULL |
+----+-----------------+----------------------+------+------------------+-------+---------------------------------------------------------------+------------------+
4 rows in set (0.02 sec)
(3) Test master-slave replication
**master创建库表，并插入测试数据：**
mysql> create database tah001;
Query OK, 1 row affected (0.06 sec)
mysql> use tah001
Database changed
mysql> create table tb01(id int(10) primary key,name varchar(20));
Query OK, 0 rows affected (1.28 sec)
mysql> insert into tb01 values(1,'zhangsan'),(2,'lisi');
Query OK, 2 rows affected (0.80 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> select * from tb01;
+----+----------+
| id | name |
+----+----------+
| 1 | zhangsan |
| 2 | lisi |
+----+----------+
2 rows in set (0.00 sec)
**检查slave是否成功同步master测试数据：**
可以看到show slave status的Gtid已经发生了变化：
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: 372cdb11-dd77-11ea-8724-000c296c0825:1-6
Executed_Gtid_Set: 372cdb11-dd77-11ea-8724-000c296c0825:1-6
可以看到master的测试数据已经成功同步到slave：
mysql> use tah001
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from tb01;
+----+----------+
| id | name |
+----+----------+
| 1 | zhangsan |
| 2 | lisi |
+----+----------+
2 rows in set (0.00 sec)
So far, Mysql one-master two-slave cluster has been set up.
PS: I am a brand new example here and there is no data yet. If you need to add a slave to an already running instance, you need to import the master data to the slave and set gtid_purged:
stop slave;
reset master;
set global gtid_purged=
'372cdb11-dd77-11ea-8724-000c296c0825:1-6';
start slave;
Four, summary
Mysql one-master two-slave cluster has been installed and deployed, is it particularly simple and fast? The same method can be used to complete dual-master, multi-source replication, cascade replication and other configurations, we will share later.
It can be seen that it is much more convenient based on GTID compared to the traditional designated binlog site relationship.
Oh, not bad! ------Welcome to point out the mistakes and add better methods