change master to master_delay时自动将relay log清掉了,会有问题吗

版权声明:本文原创,转载请注明出处。 https://blog.csdn.net/weixin_39004901/article/details/88862500

在修改延迟从库的master_delay数值的时候,仔细多看了一眼,发现设置新的master_delay之后,MySQL自动将从库的所有relay log清空,并重新生成,序号从000001开始,也就是把relay log全部删除了,并重新生成relay log。这样对于还没应用的relay log,会不会造成从库的数据丢失呢?下面用我的测试环境验证一下。

下面的这个从库准备修改master_delay,修改前这个从库的SQL_THREAD应用到了slave-relay-bin.000002的1316982,而IO_THREAD抽取到主库mysql-bin.000074的2688524,这是我的测试环境,我停止主库一切操作,防止主库生产新的binlog,当前这个从库是master_delay=3600:

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.204.128
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000074
          Read_Master_Log_Pos: 2688524
               Relay_Log_File: slave-relay-bin.000002
                Relay_Log_Pos: 1316982
        Relay_Master_Log_File: mysql-bin.000074
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql,information_schema,performance_schema,sys
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2638066
              Relay_Log_Space: 1367647
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 12
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 128
                  Master_UUID: f4859d1f-f82f-11e8-afbe-000c2999b048
             Master_Info_File: /mysqlData/data/master.info
                    SQL_Delay: 3600
          SQL_Remaining_Delay: 3588
      Slave_SQL_Running_State: Waiting until MASTER_DELAY seconds after master executed event
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: f4859d1f-f82f-11e8-afbe-000c2999b048:101574-101575
            Executed_Gtid_Set: 2b7ac8e6-f834-11e8-97d4-000c29d2f92c:1-11,
f4859d1f-f82f-11e8-afbe-000c2999b048:1-101574
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

查看当前relay log的最新位置是1367409,可以知道当前的延迟从库还没应用最新的relay log:

[root@slave data]# ll
total 189840
-rw-r----- 1 mysql mysql 56 Dec 5 10:19 auto.cnf
-rw------- 1 mysql mysql 1679 Dec 5 10:19 ca-key.pem
-rw-r--r-- 1 mysql mysql 1107 Dec 5 10:19 ca.pem
-rw-r--r-- 1 mysql mysql 1107 Dec 5 10:19 client-cert.pem
-rw------- 1 mysql mysql 1679 Dec 5 10:19 client-key.pem
-rw-r----- 1 mysql mysql 545 Mar 19 09:50 ib_buffer_pool
-rw-r----- 1 mysql mysql 79691776 Mar 28 09:10 ibdata1
-rw-r----- 1 mysql mysql 50331648 Mar 28 09:10 ib_logfile0
-rw-r----- 1 mysql mysql 50331648 Mar 28 09:10 ib_logfile1
-rw-r----- 1 mysql mysql 12582912 Mar 28 08:49 ibtmp1
-rw-r----- 1 mysql mysql 133 Mar 28 09:23 master.info
drwxr-x--- 2 mysql mysql 4096 Dec 5 10:19 mysql
drwxr-x--- 2 mysql mysql 4096 Dec 5 10:19 performance_schema
-rw------- 1 mysql mysql 1675 Dec 5 10:19 private_key.pem
-rw-r--r-- 1 mysql mysql 451 Dec 5 10:19 public_key.pem
-rw-r----- 1 mysql mysql 70 Mar 28 09:12 relay-log.info
drwxr-x--- 2 mysql mysql 4096 Mar 28 08:55 sam
-rw-r--r-- 1 mysql mysql 1107 Dec 5 10:19 server-cert.pem
-rw------- 1 mysql mysql 1675 Dec 5 10:19 server-key.pem
-rw-r----- 1 mysql mysql 207 Mar 28 09:06 slave-relay-bin.000001
-rw-r----- 1 mysql mysql 1367440 Mar 28 09:13 slave-relay-bin.000002
-rw-r----- 1 mysql mysql 50 Mar 28 09:06 slave-relay-bin.index
drwxr-x--- 2 mysql mysql 12288 Dec 5 10:19 sys
drwxr-x--- 2 mysql mysql 4096 Dec 20 17:31 test1
drwxr-x--- 2 mysql mysql 4096 Feb 28 10:07 Test2
[root@slave data]# mysqlbinlog --no-defaults --base64-output=decode-rows -vv slave-relay-bin.000002|tail -10
### WHERE
### @1=1 /* INT meta=0 nullable=1 is_null=0 */
# at 1367409
#190328 9:13:31 server id 128 end_log_pos 2688524 CRC32 0x17b1a6bc Xid = 76
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

那么下面修改一下master_delay数值,看一下relay log的变化:

mysql> stop slave;
Query OK, 0 rows affected (0.01 sec)

mysql> change master to master_delay=7200;
Query OK, 0 rows affected (0.01 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: 
                  Master_Host: 192.168.204.128
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000074
          Read_Master_Log_Pos: 2638066
               Relay_Log_File: slave-relay-bin.000001
                Relay_Log_Pos: 4
        Relay_Master_Log_File: mysql-bin.000074
             Slave_IO_Running: No
            Slave_SQL_Running: No
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql,information_schema,performance_schema,sys
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2638066
              Relay_Log_Space: 194
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 128
                  Master_UUID: f4859d1f-f82f-11e8-afbe-000c2999b048
             Master_Info_File: /mysqlData/data/master.info
                    SQL_Delay: 7200
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: 
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: 
            Executed_Gtid_Set: 2b7ac8e6-f834-11e8-97d4-000c29d2f92c:1-11,
f4859d1f-f82f-11e8-afbe-000c2999b048:1-101574
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

可以看到SQL_THREAD已经从slave-relay-bin.000001的4pos重新开始应用主库的binlog,经检查也可以发现,relay log目录下,之前的relay log都被清除了,只有一个新生成的slave-relay-bin.000001。这里有值得怀疑的地方,在修改master_delay之前,从库是还没完全应用relay log的,设置relay log之后就把所有relay log删除了,会不会造成数据丢失呢?

先看一下新生成的relay log内容:

[root@slave data]# mysqlbinlog --no-defaults --base64-output=decode-rows -vv slave-relay-bin.000001
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#190328 9:26:39 server id 129 end_log_pos 123 CRC32 0xf46eec25 Start: binlog v 4, server v 5.7.23-log created 190328 9:26:39 at startup
# This Format_description_event appears in a relay log and was generated by the slave thread.
# at 123
#190328 9:26:39 server id 129 end_log_pos 194 CRC32 0x3dd638f1 Previous-GTIDs
# f4859d1f-f82f-11e8-afbe-000c2999b048:101574-101575
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

注意信息:

#190328 9:26:39 server id 129 end_log_pos 194 CRC32 0x3dd638f1 Previous-GTIDs
# f4859d1f-f82f-11e8-afbe-000c2999b048:101574-101575

再看一下修改master_delay之前,从库的Executed_Gtid_Set:

Executed_Gtid_Set: 2b7ac8e6-f834-11e8-97d4-000c29d2f92c:1-11,
f4859d1f-f82f-11e8-afbe-000c2999b048:1-101574

也就是说,从库已经应用到f4859d1f-f82f-11e8-afbe-000c2999b048:101574的位置,而从库已经应用到f4859d1f-f82f-11e8-afbe-000c2999b048:101575往后的relay log已经被清掉了。正好跟新生成的relay log里记录的信息对应。

我们再把从库开启来看看什么情况:

mysql> start slave;
Query OK, 0 rows affected (0.00 sec)

mysql> show slave status\G
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: 192.168.204.128
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysql-bin.000074
          Read_Master_Log_Pos: 2688524
               Relay_Log_File: slave-relay-bin.000002
                Relay_Log_Pos: 414
        Relay_Master_Log_File: mysql-bin.000074
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB: 
          Replicate_Ignore_DB: mysql,information_schema,performance_schema,sys
           Replicate_Do_Table: 
       Replicate_Ignore_Table: 
      Replicate_Wild_Do_Table: 
  Replicate_Wild_Ignore_Table: 
                   Last_Errno: 0
                   Last_Error: 
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 2638066
              Relay_Log_Space: 51119
              Until_Condition: None
               Until_Log_File: 
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File: 
           Master_SSL_CA_Path: 
              Master_SSL_Cert: 
            Master_SSL_Cipher: 
               Master_SSL_Key: 
        Seconds_Behind_Master: 1278
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error: 
               Last_SQL_Errno: 0
               Last_SQL_Error: 
  Replicate_Ignore_Server_Ids: 
             Master_Server_Id: 128
                  Master_UUID: f4859d1f-f82f-11e8-afbe-000c2999b048
             Master_Info_File: /mysqlData/data/master.info
                    SQL_Delay: 7200
          SQL_Remaining_Delay: 5922
      Slave_SQL_Running_State: Waiting until MASTER_DELAY seconds after master executed event
           Master_Retry_Count: 86400
                  Master_Bind: 
      Last_IO_Error_Timestamp: 
     Last_SQL_Error_Timestamp: 
               Master_SSL_Crl: 
           Master_SSL_Crlpath: 
           Retrieved_Gtid_Set: f4859d1f-f82f-11e8-afbe-000c2999b048:101575
            Executed_Gtid_Set: 2b7ac8e6-f834-11e8-97d4-000c29d2f92c:1-11,
f4859d1f-f82f-11e8-afbe-000c2999b048:1-101574
                Auto_Position: 1
         Replicate_Rewrite_DB: 
                 Channel_Name: 
           Master_TLS_Version: 
1 row in set (0.00 sec)

开启以后,MySQL将SQL_THREAD的复制起点修改为slave-relay-bin.000002的414pos,去看看这个点的binlog内容吧:

[root@slave data]# mysqlbinlog --no-defaults --base64-output=decode-rows -vv slave-relay-bin.000002|head -30
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#190328 9:34:44 server id 129 end_log_pos 123 CRC32 0xc3f6b22b Start: binlog v 4, server v 5.7.23-log created 190328 9:34:44
# This Format_description_event appears in a relay log and was generated by the slave thread.
# at 123
#190328 9:34:44 server id 129 end_log_pos 154 CRC32 0xd9b6b59b Previous-GTIDs
# [empty]
# at 154
#700101 8:00:00 server id 128 end_log_pos 0 CRC32 0x77cf191a Rotate to mysql-bin.000074 pos: 4
# at 201
#190328 8:53:41 server id 128 end_log_pos 123 CRC32 0xc6fe537b Start: binlog v 4, server v 5.7.23-log created 190328 8:53:41
# at 320
#190328 9:34:44 server id 0 end_log_pos 367 CRC32 0x046d2208 Rotate to mysql-bin.000074 pos: 194
# at 367
#190328 9:34:44 server id 0 end_log_pos 414 CRC32 0x35d0b369 Rotate to mysql-bin.000074 pos: 2638066
# at 414
#190328 9:13:31 server id 128 end_log_pos 2638131 CRC32 0xc1bdf334 GTID last_committed=22 sequence_number=23 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= 'f4859d1f-f82f-11e8-afbe-000c2999b048:101575'/*!*/;
# at 479
#190328 9:13:31 server id 128 end_log_pos 2638202 CRC32 0x9f896abd Query thread_id=2 exec_time=0 error_code=0
SET TIMESTAMP=1553735611/*!*/;
SET @@session.pseudo_thread_id=2/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1436549152/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=45/*!*/;

可以看到,在slave-relay-bin.000002的414pos开始,重新拉取了,f4859d1f-f82f-11e8-afbe-000c2999b048:101575及往后的binlog。终于真相大白,原来清掉relay log以后,MySQL会重新拉取还没应用过的binlog,避免数据丢失。

那么还是存在一个问题:每次修改延迟数据,都要清掉relay log,再重新拉取binlog,造成主库和从库之间重复的数据流量,消耗了不必要的资源。

这个问题能不能解决呢?当然可以。造成上述问题的原因很简单,就是操作不规范。
修改master_delay,根本不需要stop slave将整个从库停止,而只需要stop slave sql_thread即可,当采用后者的方式,MySQL就不会清除并重新生成relay log了。其实MySQL也已经明确地指示了:

mysql> change master to master_delay=86400;
ERROR 3085 (HY000): This operation cannot be performed with a running slave sql thread; run STOP SLAVE SQL_THREAD FOR CHANNEL '' first.

猜你喜欢

转载自blog.csdn.net/weixin_39004901/article/details/88862500