MySQL --13 master-slave replication

I. Introduction master-slave replication

MySQL's replication is a feature that allows the server to replicate changes from one instance to another.

1) the master server and all data changes to the structure of binary log.
2) requesting the slave server from the primary server and binary log its application content locally.
3) IO: main library request, obtain a new event on the first performance ever, and place the relaylog
4) SQL: sql statement from relaylog will translate to execute from the library

II. Master-Slave Replication Works

Master-slave replication premise

1) Two or more database Example
2) Turn on the main library binary log
3) primary database user must copy
4) server_id main library from different libraries and
5) required before enabling a copy function, to obtain from the library the data (primary backup, and then the recording binlog position) before the main gallery
6) from the library in the first opening from the master copy, when the main reservoir must be known: IP, Port, User, password, logfile, POS
. 7) from the library to open the related threads: IO, SQL
8) libraries need to copy the records relevant information from the user, should also have been recorded in the last request from the main library to which the binary log
9) request from the library over the binlog, first to survive, and execute binlog, saved information is executed

From the main involves copying files to and threads

Main Library:

1) main library binlog: Modified record event occurred in the main library of
2) dump thread: to transfer from the library (TP) binary log thread

From the library:

1) relay-log (relay logs): master repository stores all over binlog event TP
2) master.info: user information storage replication, the last request to the primary database binlog position of the point
3) IO thread: receiving hair main library to the binlog log, also requested the main library from the library thread
4) SQL thread: the implementation of the main library TP over logs

Master-slave replication schematics

Premise condition: call the shots from the previous copy, you must ensure data consistency from the main library and between the library

1) through a change master to tell from the statement ip Library main library, Port, the User, password, File, POS
2) turn copy the necessary IO thread and the SQL thread from the library by the start slave command
3) from the library by holding the IO thread change master to the user's password information, connect the main library, verify the legitimacy
4) after a successful connection from the library, the library will ask the Lord according to the binlog pos, there is no one than this update is
received 5) from the main library after library requests, more binlog what information, if there will be new data dump from thread to thread the library IO
6) received the main library sent binlog events from the library by IO thread, stored in the TCP / IP cache, and returns the ACK update master.info
7) the contents of TCP / IP to the cache memory in the relay-log
8) SQL thread reads relay-log.info, read relay-log has been performed the last location point, and proceed to the subsequent relay-log after logging, the implementation is complete, the update relay-log.info

Third, the main building from the actual copy

Main library operations:

1) modify the configuration file

#编辑mysql配置文件
[root@db01 ~]# vim /etc/my.cnf
#在mysqld标签下配置
[mysqld]
#主库server-id为1，从库不等于1
server_id =1
#开启binlog日志
log_bin=mysql-bin

2) Create a master-slave replication user

#登录数据库
[root@db01 ~]# mysql -uroot -poldboy123
#创建rep用户
mysql> grant replication slave on *.* to rep@'10.0.0.%' identified by '123';

From the library operations:

1) modify the configuration file

#修改db02配置文件
[root@db02 ~]# vim /etc/my.cnf
#在mysqld标签下配置
[mysqld]
#主库server-id为1，从库不等于1， 但从库之间的server_id可以相等
server_id =5
#重启mysql
[root@db02 ~]# /etc/init.d/mysqld restart

#记录主库binlog及位置点
[root@db01 ~]# mysql -uroot -poldboy123
mysql> show master status;
|  mysql-bin.000002 |      317 
#登陆数据库
[root@db02 ~]# mysql -uroot -poldboy123
#执行change master to 语句
mysql> change master to
-> master_host='10.0.0.51',
-> master_user='rep',
-> master_password='123',
-> master_log_file='mysql-bin.000002',
-> master_log_pos=317,
-> master_port=3306;

mysql> start slave;
mysql> show slave status\G
             Slave_IO_Running: Yes
             Slave_SQL_Running: Yes

IV. From the master copy base Troubleshooting

IO thread error Solutions

# IO线程报错：
解决思路：
1.网络
[root@db02 ~]# ping 10.0.0.51
1）硬件层，路由，交换机，网络设备
2）网线
3）安全组规则
4）插错网线口

2.端口
[root@db02 ~]# telnet 10.0.0.51 3306
#关闭防火墙
systemctl stop firewalld
#防火墙添加允许mysql端口
firewalld-cmd --add-service=mysql 
firewalld-cmd --add-port=3306/tcp

3.用户名
mysql> grant replication slave on *.* to rep@'%' identified by '123';

4.密码，先登录测试
[root@db03 data]# mysql -urep -p123 -h10.0.0.51

如果报错  #rep@'db03'，需在参数，跳过反向解析
vim /etc/my.cnf
skip_name_resolve

#搭建主从时，用户名、密码、主机域、端口一定要一致。
 change master to
 master_host='10.0.0.51',#1
 master_user='rep',#2
 master_password='123',#3
 master_log_file='mysql-bin.000003',
 master_log_pos=169853,
 master_port=3306;
 

SQL thread error

Processing method:

#临时停止同步
mysql> stop slave;
#将同步指针向下移动一个（可重复操作）
mysql> set global sql_slave_skip_counter=1;
#开启同步
mysql> start slave;

Processing Method Two:

#编辑配置文件
[root@db01 ~]# vim /etc/my.cnf
#在[mysqld]标签下添加以下参数，把线程号添加到配置文件
slave-skip-errors=1032,1062,1007

But the method, Method II are all the risks, just skip the error, can not solve the problem fundamentally

Processing Method three:

1) Re database backup, restore from the library to
a 2) from the library to read only

#在命令行临时设置
set global read_only=1;
#在配置文件中永久生效
read_only=1

Note: If this is all logged-on user permissions, including the super super powers, can still operate

1. all rights, even if the profile is set up read-only, or can be operated.
2. All without permission. He even specified to select, insert, delete, create rights are not operating, only read-only.

E.g:

#设置配置文件永久生效
[root@db03 ~]# vim /etc/my.cnf
read_only=1
#重启
[root@db03 ~]# /etc/init.d/mysqld  restart
Shutting down MySQL.. SUCCESS! 
Starting MySQL. SUCCESS!
#查看
mysql> show variables like 'read_only';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| read_only     | ON    |
+---------------+-------+
1 row in set (0.00 sec)

1. all rights

[root@db03 ~]# mysql
mysql> grant all on *.* to rea@'%' identified by '123';
Query OK, 0 rows affected (0.00 sec)

[root@db03 ~]# mysql -urea -p123 -h 10.0.0.53
mysql> create database aaa;
Query OK, 1 row affected (0.01 sec)

2. not all permissions

mysql> grant select,create,delete,insert on *.* to rea1@'%' identified by '123';
Query OK, 0 rows affected (0.00 sec)

[root@db03 ~]# mysql -urea1 -p123 -h10.0.0.53
mysql> create database bbb;
ERROR 1290 (HY000): The MySQL server is running with the --read-only option so it cannot execute this statement
mysql> drop database test;
ERROR 1290 (HY000): The MySQL server is running with the --read-only option so it cannot execute this statement
...

V. delay from the library

The main deficiencies from ordinary copy may

1) Logical corruption how to do?
2) can not guarantee the operation of the main library, the library could do
3) High Availability? Automatic failover?
4) Copy filter

Companies generally will be delayed 3-6 hours

Delay configuration method from the library

#停止主从
mysql>stop slave;
#设置延时为180秒
mysql> change master to master_delay= 180;
#开启主从
mysql>start slave;
#查看状态
mysql> show slave status\G
SQL_Delay: 60

#或者做主从的时候直接指定延时
 change master to
 master_host='10.0.0.51',
 ...
 master_delay=3600;

3.延时从库停止方法
#停止主从
mysql> stop slave;
#设置延时为0
mysql> change master to master_delay = 0;
#开启主从
mysql> start slave;

Business Case:

Order of the total data 500G, needed to restore a normal backup 1.5-2 h
1) disposed 3,600 seconds delay

mysql>change master to master_delay = 3600;

2) the main library

drop database db;

3) how the use of delay from the library, to recover data?

Tip:
1, from the library is stored in the datadir relaylog under
2, mysqlbinlog relaylog can intercept the content
3, show relay log events in ' db01-relay-bin.000001';

Deal with the idea:

1) Stop SQL thread

2) to intercept point before the erroneous erasing relaylog

• relay-log.info get to the point position to the last run, as the starting point for recovery
• Relay-log Analysis of the contents of documents, access to accidentally deleted position before

Analog fault at:

1) Close sql

2) analog data

3) open from the library delay 3600s

4) destroy, delete the simulation library failure. (These steps is completed within 5 minutes.)

5) from the library, shut down SQL thread

6) taken relay-log

Recovery relay.sql

Method One: Cancel from the library identity, recover data from the database when the main library

Method two: the master library library pass data from the lead out, to recover data

Simulation environment

1. Main Library

#一直写数据
[root@db01 ~]# sh mysqldump.sh
#全备
[root@db01 ~]# mysqldump -uroot -p1 -A --triggers -R --master-data=2 --single-transaction |gzip >/backup/full.gz
Warning: Using a password on the command line interface can be insecure.
#查看
[root@db01 ~]# ll /backup/
total 376
-rw-r--r-- 1 root root 384381 Nov 17 09:22 full.gz
#查看mysql-bin和起点
[root@db01 ~]# zcat /backup/full.gz |head -25
-- CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=517;

#scp到对端
[root@db01 ~]# scp /backup/dbdb.sql  172.16.1.54:/tmp
[root@db01 ~]# mysql -uroot -p1
mysql> grant replication slave on *.* to rep@'%' identified by '123';

1. From the library

#准备初始化环境
[root@db04 ~]# /etc/init.d/mysqld  stop
[root@db04 ~]# rm -fr /application/mysql/data/*
[root@db04 ~]# ./mysql_install_db  --user=mysql --basedir=/opt/mysql --datadir=/opt/mysql/data/
[root@db04 ~]# /etc/init.d/mysqld  start

#导库
[root@db04 ~]# zcat /tmp/dbdb.sql  |mysql
#主从
[root@db04 ~]# mysql -uroot -p1
mysql> change master to
        master_host='10.0.0.51',
        master_user='db',
        master_password='123',
        master_log_file='mysql-bin.000001',
        master_log_pos=517,
        master_port=3306,   
        master_delay=3600;

1. Delete fault simulation library main library

[root@db01 ~]# mysql -uroot -p1
mysql> select count(*) from db1.t1;
+----------+
| count(*) |
+----------+
|      888 |
+----------+
1 row in set (0.00 sec)

mysql> drop database db1;
Query OK, 1 row affected (0.02 sec)

1. From the library to close sql thread

[root@db04 ~]# mysql
           SQL_Delay: 3600

mysql> select count(*) from db1.t1;
+----------+
| count(*) |
+----------+
|      655 |
+----------+
1 row in set (0.00 sec)

mysql> stop slave sql_thread;
Query OK, 0 rows affected (0.01 sec)
             Slave_IO_Running: Yes
            Slave_SQL_Running: No

5. accidentally deleted from the primary library library derived copied to the main library

[root@db04 data]# mysqldump -B db1>/tmp/db_quan.sql
[root@db04 data]# scp /tmp/db_quan.sql  172.16.1.51:/backup/

6. intercept the data is not synchronized to the operation of the library before puncturing.

#获取起点
[root@db04 data]# cat relay-log.info 
7
./db04-relay-bin.000002
283
mysql-bin.000003
44040
3600
0
1


#获取终点
方法一：
[root@db04 data]# mysql
#查看relaylog事件
mysql> show relaylog events in 'db04-relay-bin.000002';
| db04-relay-bin.000002 | 45118 | Query       |         1 |      161795 | drop database db1  
#或者在命令行执行
[root@db04 ~]# mysql -e "show relaylog events in 'db04-relay-bin.000002'";

#方法二：
[root@db04 data]# mysqlbinlog  -d db1 --base64-output=decode-rows -vvv db04-relay-bin.000002 |grep -i -B 5 'drop database'
#191117 13:38:56 server id 1  end_log_pos 88875 CRC32 0xcfd701dd    Xid = 15750
COMMIT/*!*/;
# at 45118
#191117 13:38:58 server id 1  end_log_pos 88964 CRC32 0x905241e7    Query   thread_id=2671  exec_time=0 error_code=0
SET TIMESTAMP=1573969138/*!*/;
drop database db1

#截取这段数据导出并发送给主库
[root@db04 data]# mysqlbinlog  --start-position=283 --stop-position=45118 /opt/mysql/data/db04-relay-bin.000002 >/tmp/db_zeng.sql
[root@db04 data]# scp /tmp/db_zeng.sql  172.16.1.51:/backup/

7. master library and view content into two data integrity

[root@db01 data]# mysql -uroot  -p1 </backup/db_quan.sql 
[root@db01 data]# mysql -uroot  -p1 </backup/db_zeng.sql 
mysql> select count(*) from db1.t1;
+----------+
| count(*) |
+----------+
|      900 |
+----------+
1 row in set (0.00 sec)

8. Turn sql thread from the library to see if synchronization will delay off

[root@db04 data]# mysql
mysql> start slave sql_thread;
mysql> stop slave;
mysql> change master to master_delay=0;
mysql> start slave;
#查看
mysql> select count(*) from db1.t1;
+----------+
| count(*) |
+----------+
|      900 |
+----------+
1 row in set (0.00 sec)

9. Turn delay again

mysql> stop slave;
mysql> change master to master_delay=3600;
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)