MySQL master-slave replication: semi-synchronous, asynchronous

outline

• foreword

• How to extend MySQL?

• MySQL Replication WorkFlow

• MySQL master-slave replication mode

• Practical drill

  • MySQL asynchronous replication implementation

  • MySQL semi-synchronous replication implementation

• Thinking in Experiments

• Summarize

foreword

In this article we introduce MySQL Replicationthe relevant content, we first introduce MySQL CLusterthe implementation principle and how to build a step by stepMySQL Replication Cluster

To understand this article, you need to know:  MySQL基本操作,MySQL日志类型及其作用

How to extend MySQL?

Everyone should know before; when the performance of a single server is insufficient, there are two ways to scale

• Scale Up: Vertical expansion, which refers to improving the performance (hardware) of a single server to improve the performance of the service 

• Scale Out: Horizontal expansion refers to adding servers to share the load of a single server through load balancing, thereby improving service performance

We can achieve load balancing of some services through LVS, HAProxy, Nginx and other software, but there MySQLwill be some problems if we use them, as follows

• MySQLHow to Ensure Data Consistency of Multiple Servers

• How to ensure the integrity of data caused by multiple servers submitting transactions at the same time

• How to ensure that when a server goes down, its transactions can be submitted or ROLLBACK normally

• …

There are many more issues that we need to consider than extending WEBservices. After all, as we all know,  数据无价in many important scenarios, the  data cannot be lost at all, but it MySQLis not very good in this regard, and the Oracledatabase is particularly powerful in this regard. , so many banks use Oracledatabases to store customer account information, etc. So, don't we use MySQLit? This is obviously impossible. Let's first introduce MySQL Replication Clusterseveral methods of synchronizing data.

• Synchronous  Replication

• Asynchronous  Replication

• Semisynchronous  Replication

  Synchronous replication : It means that the client connects to MySQL主服务器write a piece of data and MySQL主服务器synchronizes to it MySQL从服务器. It needs to wait for the response from the server to send a synchronization completion before returning to the client OK. The process of waiting for synchronization is blocked. If there are N slave servers, Very inefficient.
  Asynchronous replication : It means that the client connects to MySQL主服务器write a piece of data, and after the main library writes the Binlog log, it can successfully return the client OK, without waiting for the Binlog log to be transmitted to the slave library. MySQL主服务器The written data will be sent to the server for a period MySQL从服务器of time. During this period, the data of the slave server may be inconsistent with the master service. Once the master database goes down, the log may be lost.
  Semi-synchronous replication : It means that the client connects to MySQL主服务器write a piece of data, MySQL主服务器only synchronously replicates the data to one of the slave servers, and semi-synchronously replicates it to other slave servers to achieve the effect of complete synchronization of one of the slave servers.

MySQL Replication WorkFlow

Master/Slave workflow

After the master-slave relationship is established, if the master server has data modified,  BINLOGit will be updated, and the slave service will synchronize to the local by IO-Threadreading the master server BINLOGto the local Relay Log, and then SQL-Threadreplaying it.

MySQL master-slave replication mode

In different business models, we can use different MySQLmaster-slave replication architectures, which are generally divided into the following two types

• Master/Slave:  refers to the one-master-multiple-slave mode, which can effectively share read requests, but write requests cannot complete load sharing, the slave nodes may have inconsistent data, and if Masterthe client is down, the client cannot write. It works, but there are still many problems

• Master/Master:  refers to the multi-master mode. In this mode, multiple MySQLservers are all Master, which means that they can all perform read and write operations, but there are more serious problems. When multiple clients write data at the same time Serious problems such as data conflicts may occur due to replication delays

Commonly used architecture diagrams

We can use keepalived to achieve high availability of Master, and use semi-synchronous mode to achieve full synchronization of data

The above method takes up too much bandwidth of the Master. We can make one Slave act as the Master and synchronize data for other Slaves

Practical drill

 

MySQL asynchronous replication implementation

Experimental topology

Environment deployment

We need to install on each server MySQL, here is the rpm package installation, the version is 5.1

[root@node1 ~]# yum install mysql-server -y
[root@node2 ~]# yum install mysql-server -y
[root@node3 ~]# yum install mysql-server -y

configuration file

node1配置文件(master)

    [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    user=mysql
    # Disabling symbolic-links is recommended to prevent assorted security risks
    symbolic-links=0
    innodb_file_per_table = 1
    log_bin=master-log    #开启二进制日志
    log_bin_index=1
    server_id=1           #设置serverid

    [mysqld_safe]
    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid

node2配置文件(slave1)

    [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    user=mysql
    # Disabling symbolic-links is recommended to prevent assorted security risks
    symbolic-links=0
    relay-log=relay-log         #开启relay日志
    innodb_file_per_table = 1
    read-only = 1               #设置只读
    server_id = 2               #设置serverid

    [mysqld_safe]
    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid

node3配置文件(slave2)

    [mysqld]
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    user=mysql
    # Disabling symbolic-links is recommended to prevent assorted security risks
    symbolic-links=0
    relay-log=relay-log
    innodb_file_per_table = 1
    read-only = 1
    server_id = 3

    [mysqld_safe]
    log-error=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid


##启动mysql
##注意: serverid一定不能相同

Configure Master

Slave node synchronization needs to be done by a user with specific permissions, so we need to create a user and give REPLICATION SLAVE, REPLICATION CLIENTpermissions

mysql> GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'rpuser'@'%' IDENTIFIED BY 'passwd';
Query OK, 0 rows affected (0.00 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> SHOW MASTER STATUS;   我们要记录下pos的数值
+-------------------+----------+--------------+------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-log.000005 |      523 |              |                  |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Note the statement here:  GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'rpuser'@'%' IDENTIFIED BY 'passwd';

Let's analyze it step by step:

GRANT

REPLICATION SLAVE privilege:
The REPLICATION SLAVE privilege should be granted to accounts that are used by slave servers to connect to the current server as their master. Without this privilege, the slave cannot request updates that have been made to databases on the master server. 
REPLICATION SLAVE is a necessary and basic permission, which directly grants the slave server the right to perform replicate operations after connecting to the master with this account.

REPLICATION CLIENT permission:

The REPLICATION CLIENT privilege enables the use of SHOW MASTER STATUS and SHOW SLAVE STATUS. 
REPLICATION CLIENT enables the user to use the SHOW MASTER STATUS and SHOW SLAVE STATUS commands, that is to say, this privilege is used to grant the account the power to monitor the Replication status.

Generally speaking, we will create an account dedicated to Replication on the primary server alone. This account must have the REPLICATION SLAVE permission. In addition, there is no need to add unnecessary permissions to ensure that the user has a single responsibility.

ON *.*: permissions are loaded on the object, *.* means all;

TO: to whom the authority is given;

'rpuser'@'%' INDENTIFIED BY 'passwd': The user whose user name is rpuser, and the user's server ip is %, the wildcard is used here, that is, there is no limit to ip, if it is @'192.168.1.%', then Indicates all IPs between 192.168.1.1 and 192.168.1.255, using 'passwd' as the password for verification.

The advantage of doing this: On the one hand, it is very convenient to use the same account to monitor and manage Replication, and it is not necessary to distinguish between slave and master. On the other hand, the configuration of repl account on slave and master is the same, so if we switch slave and master, No changes to the account are required.

Configure Slave(1)

mysql> CHANGE MASTER TO
    -> MASTER_HOST='172.16.1.2',
    -> MASTER_USER='rpuser',
    -> MASTER_PASSWORD='passwd',
    -> MASTER_LOG_FILE='master-log.000005',
    -> MASTER_LOG_POS=523;
Query OK, 0 rows affected (0.01 sec

mysql> SHOW SLAVE STATUS\G;  #查看相应信息
#########省略##################
Master_Log_File: master-log.000005
Read_Master_Log_Pos: 523
Relay_Log_File: relay-log.000001
Relay_Log_Pos: 4
Relay_Master_Log_File: master-log.000005
Slave_IO_Running: No    #IO-thread没有启动
Slave_SQL_Running: No   #SQL-thread没有启动
#########省略##################

mysql> START SLAVE;  #启动sql-thread和io-thred

mysql> SHOW SLAVE STATUS\G;  #查看相应信息 
Query OK, 0 rows affected (0.00 sec)
#########省略##################
Master_Log_File: master-log.000005
Read_Master_Log_Pos: 523
Relay_Log_File: relay-log.000002
Relay_Log_Pos: 252
Relay_Master_Log_File: master-log.000005
Slave_IO_Running: Yes     #IO-thread启动
Slave_SQL_Running: Yes    #SQL-thread启动
#########省略##################

Note a few configurations here:

    -> MASTER_HOST='172.16.1.2',
    -> MASTER_USER='rpuser',
    -> MASTER_PASSWORD='passwd',

Must match the command configuration executed on MASTER.

 

configure slave(2)

The process is the same as the configuration slave(1), no demonstration

Test master-slave replication

##在主服务器上创建数据库和表

[root@node1 ~]# mysql

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| test               |
+--------------------+
3 rows in set (0.00 sec)

mysql> CREATE DATABASE replication;
Query OK, 1 row affected (0.00 sec)

mysql> USE replication;
Database changed
mysql> CREATE TABLE t1 (id int unsigned auto_increment primary key, name char(30));
Query OK, 0 rows affected (0.01 sec)

mysql> DESC t1;
+-------+------------------+------+-----+---------+----------------+
| Field | Type             | Null | Key | Default | Extra          |
+-------+------------------+------+-----+---------+----------------+
| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | char(30)         | YES  |     | NULL    |                |
+-------+------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)

mysql> SHOW MASTER STATUS;
+-------------------+----------+--------------+------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-log.000005 |      765 |              |                  |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)

##在slave服务器测试

[root@node2 ~]# mysql

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| replication        |
| test               |
+--------------------+
4 rows in set (0.00 sec)

mysql> use replication;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> DESC t1;
+-------+------------------+------+-----+---------+----------------+
| Field | Type             | Null | Key | Default | Extra          |
+-------+------------------+------+-----+---------+----------------+
| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name  | char(30)         | YES  |     | NULL    |                |
+-------+------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)

mysql> SHOW SLAVE STATUS\G;

#########省略##################
Master_Log_File: master-log.000005
Read_Master_Log_Pos: 765   #已经同步
Relay_Log_File: relay-log.000002
Relay_Log_Pos: 494
Relay_Master_Log_File: master-log.000005
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
#########省略##################

 

MySQL semi-synchronous replication implementation

Since MySQLsemi-synchronous replication is MySQLonly provided in the form of a plug-in after 5.5, we MySQLneed to replace it with version 5.5 here.MariaDB

Since many steps are repeated with the above, I will not write it out, first configure it as M/S and then follow me below

Experimental topology

configure master

半同步的插件在/usr/lib64/mysql/plugin下, master用的semisync_master.so，slave用的semisync_slave.so，

MariaDB [(none)]> INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET GLOBAL rpl_semi_sync_master_enabled = 1;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET GLOBAL rpl_semi_sync_master_timeout = 2000;   #设置超时时间为2S
Query OK, 0 rows affected (0.00 sec)

Configure slave

MariaDB [(none)]> INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET GLOBAL rpl_semi_sync_slave_enabled=1;
Query OK, 0 rows affected (0.00 sec)

Under normal circumstances, these two PLUGINs should be standard installed on the master and slave. Who knows whether the standby database will be switched to the main database one day.

After installing PLUGIN, we can define the configuration of the main library and the standby library according to the topology, mainly including the following configuration items;
1.rpl_semi_sync_master_enabled  
  — Control whether semisync is enabled on the main library, turn it on or off, and take effect immediately
2.rpl_semi_sync_slave_enabled     
  — Controls whether semisync is enabled in the standby database.
When the main database opens semisync, at least one linked standby database must have semisync enabled, otherwise the main library thread will wait every time until it times out; therefore, if you want to close semisync, you must close it first. Main library configuration, and then close the standby library configuration
3.rpl_semi_sync_master_timeout    
  - Controls the waiting time of the client on the main library. After waiting for this long, the client returns, and the synchronous replication degenerates into the native asynchronous replication. The
  unit is milliseconds, and the default value is 10000 , that is, 10 seconds  
. 4. rpl_semi_sync_master_wait_no_slave
is turned on by default, which means that when the standby database starts up and keeps up with the main database, it will automatically switch to the synchronization mode. If it is turned off, even if the standby database is up and keeps up, semi-synchronization will not be enabled;
5 .rpl_semi_sync_master_trace_level and rpl_semi_sync_slave_trace_level
— the level of output monitoring information, click on the document for details, different levels may output more detailed information, used for DEBUG
running state variables are also richer, not to go into details, there are a lot of introductions on the Internet, official documents also Very detailed

 

verify

因为在我的环境中，即使是半同步复制，也是直接就完成，看不出效果，所以我们故意将slave节点关闭
[root@node2 ~]# service mysql stop



##master创建数据库
MariaDB [(none)]> CREATE DATABASE TEST3;
Query OK, 1 row affected (2.00 sec)        #等待2s, 超时不再等待，直接创建

MariaDB [(none)]> CREATE DATABASE TEST4;
Query OK, 1 row affected (0.00 sec)


##完成，这个可能不是特别直观，但是由于我这边环境实在做不出效果，望大家理解

Thinking in Experiments

• If the data of the master and slave servers are quite different, it is best to use the binary log of the master server to replay it on the slave server first, and then synchronize

Summarize

In fact, I plan to write about SSL replication in this article, but I didn't write it because of the time constraints. Generally speaking, I am not particularly satisfied. There are many places that have not been explained clearly. I hope everyone will understand.

The level of the author is very low. If there are any mistakes, please point them out in time. If you think this article is well written, please give a like~(≧▽≦)/~ 
Author: AnyISaIln QQ: 1449472454 
Thanks: MageEdu