MySQL case combat-MySQL database master-slave replication-advanced advanced features

Preface

This environment is based on Centos 7.8 system to build MySQL-5.7.14 for
specific construction, please refer to MySQL-5.7.14 environment construction

---

1. Delayed synchronization

Features

It is a special slave library that we manually configure. The delay between the slave library and the main library is N hours.
Why is there a delay slave:
solve the problem of database failure

• Physical damage: Master-slave replication is very good at solving physical damage. For example, the main library rm deletes database data and directly switches the application to the slave library.
• Logical damage: ordinary master-slave replication cannot solve logical damage, such as the main database drop database dbname

Physical failure


logical failure

failure recovery ideas

Failure recovery ideas:

• 1 master and 1 slave, the slave library has a delay of 5 minutes, and the master library accidentally deletes 1 library
• Erroneous deletion detected within 5 minutes
• Stop the SQL thread from the library
• Intercept relaylog
• Starting point: When the SQL thread is stopped, the last application position of the relay
• End point: delete the previous position (GTID) by mistake
• Restore intercepted logs to the slave library
• Remove from the library identity and replace the main library work

Case implementation

Configure delayed synchronization from the library

mysql> stop slave;
Query OK, 0 rows affected (0.00 sec)

mysql> change master to master_delay=300;
Query OK, 0 rows affected (0.00 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

The main library creates a relay database, a relay table, and inserts two pieces of data

mysql> create database relay;
Query OK, 1 row affected (0.00 sec)

mysql> use relay;
Database changed
mysql> create table tb1(id int);
Query OK, 0 rows affected (0.00 sec)

mysql> insert into tb1 values(1),(2);
Query OK, 2 rows affected (0.17 sec)
Records: 2  Duplicates: 0  Warnings: 0

mysql> select * from tb1;
+------+
| id   |
+------+
|    1 |
|    2 |
+------+
2 rows in set (0.00 sec)

View the delayed synchronization status from the library

View the database information from the library

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
6 rows in set (0.00 sec)

Slave library

# 停止sql线程
mysql> stop slave sql_thread;
Query OK, 0 rows affected (0.00 sec)

View the location information of the

relay log from the library Extract the relay log information from the library

[root@mysql-source_code ~]# mkdir /backup/mysqld/ -p
[root@mysql-source_code ~]# mysqlbinlog --start-position=385 --stop-position=908 mysql-source_code-relay-bin.000002 -r /backup/mysqld/relay.sql
[root@mysql-source_code ~]# ll /backup/mysqld/relay.sql 
-rw-r----- 1 root root 2707 Feb  6 11:35 /backup/mysqld/relay.sql

Recover from the library relay log

mysql> source /backup/mysqld/relay.sql

View the status of recovering data from the library

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| db1                |
| db2                |
| mysql              |
| performance_schema |
| relay              |
| sys                |
+--------------------+
7 rows in set (0.00 sec)

mysql> use relay
Database changed
mysql> select * from tb1;
+------+
| id   |
+------+
|    1 |
|    2 |
+------+
2 rows in set (0.00 sec)

Remove from library status

mysql> stop slave;
Query OK, 0 rows affected (0.01 sec)

mysql> reset slave all;
Query OK, 0 rows affected (0.01 sec)

Two, filter replication

Features

Filtering replication can be started from two aspects:

• Configure the dump thread of the main library to send only the db binary that needs to be synchronized
• Configure the sql thread from the library to only play back the db binary files we need to synchronize

Specific principles


Filtering and copying application scenarios:

case implementation

1. Limit the main library

Modify the master configuration file

[root@mysql-yum ~]# vim /etc/my.cnf
binlog_ignore_db=d2
[root@mysql-yum ~]# systemctl restart mysqld

Create a database from the main library

mysql> create database d1;
Query OK, 1 row affected (0.00 sec)

mysql> create database d2;
Query OK, 1 row affected (0.00 sec)

mysql> create database d3;
Query OK, 1 row affected (0.00 sec)

mysql> show databases like 'd_';
+---------------+
| Database (d_) |
+---------------+
| d1            |
| d2            |
| d3            |
+---------------+
3 rows in set (0.00 sec)

View from library

mysql> show databases like 'd_';
+---------------+
| Database (d_) |
+---------------+
| d1            |
| d3            |
+---------------+
2 rows in set (0.00 sec)

2. Limit slave library

Modify the configuration file from the library

[root@mysql-source_code ~]# vim /etc/my.cnf
[mysqld3307]
replicate_ignore_db=mysql_1
[root@mysql-source_code ~]# mysqld_multi stop 3307
[root@mysql-source_code ~]# mysqld_multi start 3307

Create a database from the main library

mysql> create database mysql_1;
Query OK, 1 row affected (0.00 sec)

mysql> create database mysql_2;
Query OK, 1 row affected (0.00 sec)

mysql> create database mysql_3;
Query OK, 1 row affected (0.00 sec)


mysql> show databases like 'mysql__';
+--------------------+
| Database (mysql__) |
+--------------------+
| mysql_1            |
| mysql_2            |
| mysql_3            |
+--------------------+
3 rows in set (0.00 sec)

View from library

mysql> show databases like 'mysql__';
+--------------------+
| Database (mysql__) |
+--------------------+
| mysql_2            |
| mysql_3            |
+--------------------+
2 rows in set (0.00 sec)

3. GTID copy (emphasis on mastering)

Regardless of the cascading situation or the one-master-multiple-slave situation, you can find something automatically through GTID, instead of finding something through File_name and File_position as before.

Specific principle

• When the master updates the data, it will generate the GTID before the transaction and record it in the binlog.
• The i/o thread on the slave side writes the changed binlog to the local relay log.
• The sql thread obtains the GTID from the relay log, and then compares whether there is a record in the binlog on the slave side.
• If there is a record, it means that the GTID transaction has been executed and the slave will ignore it.
• If there is no record, the slave will execute the GTID transaction from the relay log and record it to the binlog

Case implementation
Configure the main library configuration file

[root@mysql-yum ~]# vim /etc/my.cn
[mysqld]
log_bin=mysql-bin
server_id=1
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1
[root@mysql-yum ~]# systemctl restart mysqld

Create a synchronization account for the main library

mysql> grant replication slave on *.* to zwq@'192.168.5.%' identified by '123456';
Query OK, 0 rows affected, 1 warning (0.00 sec

Configure Slave Configuration File

[root@localhost ~]# vim /etc/my.cnf
[mysqld_multi]
mysqld = /usr/local/mysql/bin/mysqld_safe
mysqladmin = /usr/local/mysql/bin/mysqladmin
user = zwq
pass = 123456

[mysqld3306]
socket = /mysql/3306/mysql.sock
port = 3306
pid-file = /mysql/3306/mysql.pid
datadir = /mysql/3306/data
basedir = /usr/local/mysql
server_id=2
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1

[mysqld3307]
socket = /mysql/3307/mysql.sock
port = 3307
pid-file = /mysql/3307/mysql.pid
datadir = /mysql/3307/data
basedir = /usr/local/mysql
server_id=3
replicate_ignore_db=mysql_1
gtid-mode=on
enforce-gtid-consistency=true
log-slave-updates=1

[root@mysql-source_code ~]# mysqld_multi stop 3306,3307
[root@mysql-source_code ~]# netstat -lnutp | grep 330
[root@mysql-source_code ~]# mysqld_multi start 3306,3307
[root@mysql-source_code ~]# netstat -lnutp | grep 330
tcp6       0      0 :::3306                 :::*                    LISTEN      3932/mysqld         
tcp6       0      0 :::3307                 :::*                    LISTEN      3949/mysqld  

View the binary file information of the master library

Configure the slave server

--- 3306
mysql> change master to
    -> master_host='192.168.5.11',
    -> master_user='zwq',
    -> master_password='123456',
    -> master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.06 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)


--- 3307
mysql> change master to
    -> master_host='192.168.5.11',
    -> master_user='zwq',
    -> master_password='123456',
    -> master_auto_position=1;
Query OK, 0 rows affected, 2 warnings (0.11 sec)

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

View slave server status
3306

3307

test:

Create database

mysql> create databases s1;
mysql> show databases like 's_';
+---------------+
| Database (s_) |
+---------------+
| s1            |
+---------------+
1 row in set (0.00 sec)

View from library

--- 3306
mysql> show databases like 's_';
+---------------+
| Database (s_) |
+---------------+
| s1            |
+---------------+
1 row in set (0.01 sec)

--- 3307
mysql> show databases like 's_';
+---------------+
| Database (s_) |
+---------------+
| s1            |
+---------------+
1 row in set (0.00 sec)

Fourth, semi-synchronous replication (understand)

Introduction to Semi-synchronization: Solving the problem of master-slave data consistency.
By default, MySQL replication is asynchronous, which means that the master server and its slave servers are independent. Asynchronous replication can provide the best performance, because the master server does not need to wait to verify whether the updated data has been copied to the slave server after writing the updated data to its binary log (Binlog) file, and can freely process other incoming data. Transaction processing request. But this also brings a high risk. If a failure occurs on the master server or the slave server, it will cause the inconsistency of the master and slave data, and even cause data loss during recovery.
A semi-synchronous replication function has been introduced from MySQL 5.5, which can ensure data consistency and redundancy between the master server and at least one slave server in the access chain. In this configuration structure, a master server and many of its slave servers are configured, so in a replication topology, at least one slave server must confirm that the update has been received and written before the parent master server performs transaction processing The relay log (Relay Log). When a timeout occurs, the source master server must temporarily switch to asynchronous replication mode and re-replicate until at least one slave server set to semi-synchronous replication mode receives information in time.
After 5.5 semi-synchronous replication, MySQL 5.6 has optimized and improved it. Two of these are more important:

• After the master-slave switch, in the traditional way, you need to find the binlog and POS points, and then change the master point. In mysql5.6, you no longer need to know the binlog and POS points, you only need to know the IP and port of the master. The account password is enough, because the synchronous replication is automatic, mysql automatically finds some synchronization through the internal mechanism GTID.
• Multi-threaded replication. In the previous version, synchronous replication was single-threaded and could only be executed one by one. In MySQL 5.6, multi-threaded replication between multiple libraries can be achieved, but the tables in one library are multi-threaded. it is invalid.

Configuration ideas

加载插件
主:
INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
从:
INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
查看是否加载成功:
show plugins;
启动:
主:
SET GLOBAL rpl_semi_sync_master_enabled = 1;
从:
SET GLOBAL rpl_semi_sync_slave_enabled = 1;
重启从库上的IO线程
STOP SLAVE IO_THREAD;
START SLAVE IO_THREAD;
查看是否在运行
主:
show status like 'Rpl_semi_sync_master_status';
从:
show status like 'Rpl_semi_sync_slave_status';