MySQL study notes: GTID-based master-slave replication

Master-slave replication of GTID

background

Before the emergence of GTID, in a master-multiple-slave replication topology, if the master database went down, one of the multiple slave databases needed to be selected as the new master database. This process was relatively complicated. There is no straightforward way to find the binary log coordinates of the new master library corresponding to other slave libraries. The usual approach is to first find the last statement of the original master library copied from each slave library, and then find the binary log file containing the statement in the new master library. The first event position after the statement is the binary log connecting to the new master library. coordinate. The main difficulty is that there is no unique identifier that points out "the last statement of the original master database copied", so the concept of GTID later appeared in MySQL.

What is master-slave replication of GTID?

The full name of global transaction identifier GTID is Global Transaction Identifier, which is a unique identification of a transaction in the entire replication environment. It is a powerful feature added in MySQL 5.6. Its purpose is to enable automatic master-slave positioning and switching, unlike the previous need to specify files and locations . When using GTID replication, the GTID corresponding to the transaction is created when the transaction is submitted on the master database, and the slave database uses the GTID to identify and track each transaction when applying the relay log. GTID can be used to identify the location of replication when starting a new slave database or failing over to a new master database, greatly simplifying these tasks . Since GTID replication is entirely based on transactions, as long as all transactions submitted on the master database are also submitted on the slave database, the consistency between the two is guaranteed. GTID supports statement-based or row-based replication formats, but for best results, MySQL recommends using the row-based format. The GTID is always persisted on the master and slaves, which means the origin of any transaction applied to any slave can be determined by inspecting its binary log. Also, once a transaction with a given GTID is committed on a given library, that library will ignore any subsequent transactions with the same GTID. Therefore, transactions committed on the master database will only be applied once on the slave database, which also helps ensure consistency.

When the new master-slave switchover occurs, the new slave server knows which transactions have been done and which have not been done.

Every time a transaction is generated, the transaction is numbered.

Advantages: Replace traditional binlog+pos replication; use master_auto_position=1 to automatically match GTID breakpoints for replication

GTID format

The GTID is associated with every transaction committed on the master database. This identifier is unique not only to the library that initiated the transaction, but also across all libraries in a given replication topology. GTID is represented by a pair of coordinates separated by colons, for example: 8eed0f5b-6f9b-11e9-94a9-005056a57a4e:23. The first part
is the server_uuid of the main library, and the latter part is the sequence number determined in the order of submitted transactions on the main library. The submitted transaction The serial number starts from 1. The explicit GTID above means: The 23rd transaction committed on the server with 8eed0f5b-6f9b-11e9-94a9-005056a57a4e has this GTID.

working principle

1. When the master updates data, it will generate a GTID before the transaction and record it in the binlog together.
2. The I/O thread on the slave side writes the changed binlog into the local relay log.
3. The sql thread obtains the GTID from the relay log, and then compares the binlog on the slave side to see if there is a record.
4. If there is a record, it means that the transaction with this GTID has been executed, and the slave will ignore it.
5. If there is no record, the slave will execute the transaction of the GTID from the relay log and record it to the binlog.
6. During the parsing process, it will be judged whether there is a primary key. If not, use the secondary index. If not, use all scans.

Build semi-synchronous master-slave replication based on GTID

1. Modify the master configuration file

[root@master ~]# vim /etc/my.cnf
[mysqld_safe]

[client]
socket=/data/mysql/mysql.sock

[mysqld]
socket=/data/mysql/mysql.sock
port = 3306
open_files_limit = 8192
innodb_buffer_pool_size = 512M
character-set-server=utf8mb4
#skip-grant-tables
#error log
log-error = chen.err
#general log
general_log
#slow query log
slow_query_log = 1  
long_query_time = 0.001
#二进制日志 log bin
log_bin
server_id = 1
#日志超过3天自动过期
expire_logs_days = 15
#设置半同步超时时间
rpl_semi_sync_master_timeout=1000
#开启半同步主从复制,需要提前安装半同步插件
rpl_semi_sync_master_enabled=1
#开启GTID功能
gtid-mode=ON
enforce-gtid-consistency=ON
[mysql]
auto-rehash
prompt=\u@\d \R:\m  mysql>

2. Modify the slave configuration file

[root@slave ~]# vim /etc/my.cnf
[mysqld_safe]
 
[client]
socket=/data/mysql/mysql.sock
 
[mysqld]
socket=/data/mysql/mysql.sock 
port = 3306
open_files_limit = 8192
innodb_buffer_pool_size = 512M
character-set-server=utf8
log_bin
server_id = 2 
#开启从服务器半同步主从复制，需要提前安装半同步插件
rpl_semi_sync_slave_enabled=1
#开启GTID功能
gtid-mode=ON
enforce-gtid-consistency=ON
#开启从服务器将relay log内容也写入自己的二进制日志
log_slave_updates=ON
[mysql]
auto-rehash
prompt=\u@\d \R:\m  mysql> 

3. Refresh the mysqld service on the master and slave servers

[root@master ~]# service mysqld restart
Shutting down MySQL............ SUCCESS! 
Starting MySQL. SUCCESS!

[root@slave ~]# service mysqld restart
Shutting down MySQL............ SUCCESS! 
Starting MySQL. SUCCESS!

4. Create a new authorized user on the master server and copy the binary log to the slave

root@(none) 03:27  mysql>grant replication slave on *.* to 'chenlb'@'192.168.31.%' identified by 'Sanchuang1234#';
Query OK, 0 rows affected, 1 warning (0.02 sec)

192.168.31.% – The network segment where the slave server is located

5. The main server refreshes the binary log

root@(none) 03:17  mysql>reset master;

6. Modify master info information from the server and configure synchronization

#首先暂停slave
root@(none) 02:11  mysql>stop slave;
Query OK, 0 rows affected (0.01 sec)
root@(none) 02:11  mysql> change master to master_host='192.168.31.153', master_user='chenlb',master_password='Sanchuang1234#',master_port=3306,master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.01 sec)

7. Check whether the semi-synchronization and GTID functions are enabled

semi-synchronous function

main server

from server

GTID function

View show slave status from the server;

8.Effectiveness test

Data synchronization test

On the main server, build databases and tables to insert data.

root@tanxue 02:19  mysql>create database tanx;
Query OK, 1 row affected (1.01 sec)

root@tanxue 03:05  mysql>use tanx;
Database changed
root@tanx 03:05  mysql>create table t1(id int);
Query OK, 0 rows affected (0.01 sec)

root@tanx 03:05  mysql>insert into t1 values(1),(2)
    -> ;
Query OK, 2 rows affected (0.00 sec)
Records: 2  Duplicates: 0  Warnings: 0

root@tanx 03:05  mysql>show master status;
+-------------------+----------+--------------+------------------+------------------------------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                        |
+-------------------+----------+--------------+------------------+------------------------------------------+
| master-bin.000003 |     1326 |              |                  | f9a34f51-dc9e-11ed-ab5b-000c29751c59:1-6 |
+-------------------+----------+--------------+------------------+------------------------------------------+
1 row in set (0.00 sec)

Check from the server whether there are databases, tables and data

root@tanxue 03:06  mysql>show databases;
root@tanxue 03:06  mysql>use tanx;
root@tanx 03:06  mysql>show tables;
root@tanx 03:06  mysql>select * from t1;

Check whether the binary log format has a gtid field

View binary logs on the master server

Found the format of uuid+GTID

Compare GTID values

The main service checks the master status

View slave status from the server

Compare whether the two gtid values ​​are consistent. If they are consistent, it is successful!

At this point, GTID semi-synchronous master-slave replication has been successfully established!