Scene 1, MGR startup log
mysql01.com
operating
1.
[[email protected]][(none)]> CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='password' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)
----
2.
[[email protected]][(none)]> SET GLOBAL group_replication_bootstrap_group=ON;
Query OK, 0 rows affected (0.00 sec)
[[email protected]][(none)]> START GROUP_REPLICATION;
Query OK, 0 rows affected (2.10 sec)
[[email protected]][(none)]> SET GLOBAL group_replication_bootstrap_group=OFF;
Query OK, 0 rows affected (0.00 sec)
----
performance_schema.replication_group_members
1.
----
2.
----
3.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | ONLINE |
| mysql03.com | ONLINE |
| mysql02.com | ONLINE |
+-------------+--------------+
3 rows in set (0.01 sec)
----
Journal
1.
2019-08-25T23:08:47.829585+08:00 265 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.
----
2.
2019-08-25T23:13:03.221490+08:00 265 [Note] Plugin group_replication reported: 'Group communication SSL configuration: group_replication_ssl_mode: "DISABLED"'
2019-08-25T23:13:03.221655+08:00 265 [Note] Plugin group_replication reported: '[GCS] Added automatically IP ranges 10.0.2.5/24,127.0.0.1/8 to the whitelist'
2019-08-25T23:13:03.221848+08:00 265 [Warning] Plugin group_replication reported: '[GCS] Automatically adding IPv4 localhost address to the whitelist. It is mandatory that it is added.'
2019-08-25T23:13:03.221911+08:00 265 [Note] Plugin group_replication reported: '[GCS] SSL was not enabled'
2019-08-25T23:13:03.221936+08:00 265 [Note] Plugin group_replication reported: 'Initialized group communication with configuration: group_replication_group_name: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; group_replication_local_address: "10.0.2.5:33081"; group_replication_group_seeds: "10.0.2.5:33081,10.0.2.6:33081,10.0.2.7:33081"; group_replication_bootstrap_group: true; group_replication_poll_spin_loops: 0; group_replication_compression_threshold: 100; group_replication_ip_whitelist: "AUTOMATIC"'
2019-08-25T23:13:03.221967+08:00 265 [Note] Plugin group_replication reported: '[GCS] Configured number of attempts to join: 0'
2019-08-25T23:13:03.221999+08:00 265 [Note] Plugin group_replication reported: '[GCS] Configured time between attempts to join: 5 seconds'
2019-08-25T23:13:03.222029+08:00 265 [Note] Plugin group_replication reported: 'Member configuration: member_id: 1; member_uuid: "27a60549-a643-11e9-bc30-080027f22add"; single-primary mode: "true"; group_replication_auto_increment_increment: 7; '
2019-08-25T23:13:03.223137+08:00 268 [Note] Plugin group_replication reported: 'Detected previous RESET MASTER invocation or an issue exists in the group replication applier relay log. Purging existing applier logs.'
2019-08-25T23:13:03.251864+08:00 268 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_applier' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:13:03.292261+08:00 271 [Note] Slave SQL thread for channel 'group_replication_applier' initialized, starting replication in log 'FIRST' at position 0, relay log '/home/mysql/mysql3308/log/relay-group_replication_applier.000001' position: 4
2019-08-25T23:13:03.292357+08:00 265 [Note] Plugin group_replication reported: 'Group Replication applier module successfully initialized!'
2019-08-25T23:13:03.292385+08:00 265 [Note] Plugin group_replication reported: 'auto_increment_increment is set to 7'
2019-08-25T23:13:03.292394+08:00 265 [Note] Plugin group_replication reported: 'auto_increment_offset is set to 1'
2019-08-25T23:13:03.304284+08:00 0 [Note] Plugin group_replication reported: 'XCom protocol version: 3'
2019-08-25T23:13:03.304538+08:00 0 [Note] Plugin group_replication reported: 'XCom initialized and ready to accept incoming connections on port 33081'
2019-08-25T23:13:04.312822+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308 on view 15667459843091222:1.'
2019-08-25T23:13:04.313004+08:00 278 [Note] Plugin group_replication reported: 'Only one server alive. Declaring this server as online within the replication group'
2019-08-25T23:13:04.616934+08:00 0 [Note] Plugin group_replication reported: 'This server was declared online within the replication group'
2019-08-25T23:13:04.617068+08:00 0 [Note] Plugin group_replication reported: 'A new primary with address mysql01.com:3308 was elected, enabling conflict detection until the new primary applies all relay logs.'
2019-08-25T23:13:04.617145+08:00 280 [Note] Plugin group_replication reported: 'This server is working as primary member.'
----
2019-08-25T23:22:01.774233+08:00 0 [Note] Plugin group_replication reported: 'Members joined the group: mysql02.com:3308'
2019-08-25T23:22:01.774439+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:2.'
2019-08-25T23:22:01.857379+08:00 283 [Note] Start binlog_dump to master_thread_id(283) slave_server(2), pos(, 4)
2019-08-25T23:22:01.950354+08:00 0 [Note] Plugin group_replication reported: 'The member with address mysql02.com:3308 was declared online within the replication group'
----
2019-08-25T23:27:58.806437+08:00 0 [Note] Plugin group_replication reported: 'Members joined the group: mysql03.com:3308'
2019-08-25T23:27:58.806698+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql03.com:3308, mysql02.com:3308 on view 15667459843091222:3.'
2019-08-25T23:27:58.976004+08:00 0 [Note] Plugin group_replication reported: 'The member with address mysql03.com:3308 was declared online within the replication group'
mysql02.com
operating
1.
[[email protected]][(none)]> CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='password' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)
----
2.
[[email protected]][(none)]> START GROUP_REPLICATION;
Query OK, 0 rows affected (5.81 sec)
----
performance_schema.replication_group_members
1.
----
2.
----
3.
----
Journal
1.
2019-08-25T23:08:51.985464+08:00 154 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.
----
2.
2019-08-25T23:21:57.054975+08:00 154 [Note] Plugin group_replication reported: 'Group communication SSL configuration: group_replication_ssl_mode: "DISABLED"'
2019-08-25T23:21:57.060734+08:00 154 [Note] Plugin group_replication reported: '[GCS] Added automatically IP ranges 10.0.2.6/24,127.0.0.1/8 to the whitelist'
2019-08-25T23:21:57.060869+08:00 154 [Warning] Plugin group_replication reported: '[GCS] Automatically adding IPv4 localhost address to the whitelist. It is mandatory that it is added.'
2019-08-25T23:21:57.060939+08:00 154 [Note] Plugin group_replication reported: '[GCS] SSL was not enabled'
2019-08-25T23:21:57.060979+08:00 154 [Note] Plugin group_replication reported: 'Initialized group communication with configuration: group_replication_group_name: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; group_replication_local_address: "10.0.2.6:33081"; group_replication_group_seeds: "10.0.2.5:33081,10.0.2.6:33081,10.0.2.7:33081"; group_replication_bootstrap_group: false; group_replication_poll_spin_loops: 0; group_replication_compression_threshold: 1000000; group_replication_ip_whitelist: "AUTOMATIC"'
2019-08-25T23:21:57.061028+08:00 154 [Note] Plugin group_replication reported: '[GCS] Configured number of attempts to join: 0'
2019-08-25T23:21:57.061057+08:00 154 [Note] Plugin group_replication reported: '[GCS] Configured time between attempts to join: 5 seconds'
2019-08-25T23:21:57.061100+08:00 154 [Note] Plugin group_replication reported: 'Member configuration: member_id: 2; member_uuid: "b48091b5-bc4c-11e9-84d6-0800270c3d91"; single-primary mode: "true"; group_replication_auto_increment_increment: 7; '
2019-08-25T23:21:57.061229+08:00 159 [Note] Plugin group_replication reported: 'Detected previous RESET MASTER invocation or an issue exists in the group replication applier relay log. Purging existing applier logs.'
2019-08-25T23:21:57.073723+08:00 159 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_applier' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:21:57.109719+08:00 162 [Note] Slave SQL thread for channel 'group_replication_applier' initialized, starting replication in log 'FIRST' at position 0, relay log '/home/mysql/mysql3308/log/relay-group_replication_applier.000001' position: 4
2019-08-25T23:21:57.109840+08:00 154 [Note] Plugin group_replication reported: 'Group Replication applier module successfully initialized!'
2019-08-25T23:21:57.109878+08:00 154 [Note] Plugin group_replication reported: 'auto_increment_increment is set to 7'
2019-08-25T23:21:57.109890+08:00 154 [Note] Plugin group_replication reported: 'auto_increment_offset is set to 2'
2019-08-25T23:21:57.126217+08:00 0 [Note] Plugin group_replication reported: 'XCom protocol version: 3'
2019-08-25T23:21:57.126443+08:00 0 [Note] Plugin group_replication reported: 'XCom initialized and ready to accept incoming connections on port 33081'
2019-08-25T23:22:01.847850+08:00 154 [Note] Plugin group_replication reported: 'This server is working as secondary member with primary member address mysql01.com:3308.'
2019-08-25T23:22:01.848680+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:2.'
2019-08-25T23:22:01.848983+08:00 169 [Note] Plugin group_replication reported: 'Establishing group recovery connection with a possible donor. Attempt 1/10'
2019-08-25T23:22:01.885579+08:00 169 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='mysql01.com', master_port= 3308, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:22:01.903258+08:00 169 [Note] Plugin group_replication reported: 'Establishing connection to a group replication recovery donor 27a60549-a643-11e9-bc30-080027f22add at mysql01.com port: 3308.'
2019-08-25T23:22:01.903547+08:00 171 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
2019-08-25T23:22:01.906394+08:00 171 [Note] Slave I/O thread for channel 'group_replication_recovery': connected to master '[email protected]:3308',replication started in log 'FIRST' at position 4
2019-08-25T23:22:01.924322+08:00 172 [Note] Slave SQL thread for channel 'group_replication_recovery' initialized, starting replication in log 'FIRST' at position 0, relay log '/home/mysql/mysql3308/log/relay-group_replication_recovery.000001' position: 4
2019-08-25T23:22:01.978434+08:00 169 [Note] Plugin group_replication reported: 'Terminating existing group replication donor connection and purging the corresponding logs.'
2019-08-25T23:22:01.978714+08:00 172 [Warning] Slave SQL for channel 'group_replication_recovery': Coordinator thread of multi-threaded slave is being stopped in the middle of assigning a group of events; deferring to exit until the group completion ... , Error_code: 0
2019-08-25T23:22:01.983731+08:00 172 [Note] Slave SQL thread for channel 'group_replication_recovery' exiting, replication stopped in log 'bin.000001' at position 758
2019-08-25T23:22:01.986027+08:00 171 [Note] Slave I/O thread killed while reading event for channel 'group_replication_recovery'
2019-08-25T23:22:01.986121+08:00 171 [Note] Slave I/O thread exiting for channel 'group_replication_recovery', read up to log 'bin.000001', position 758
2019-08-25T23:22:02.008099+08:00 169 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='mysql01.com', master_port= 3308, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:22:02.023603+08:00 0 [Note] Plugin group_replication reported: 'This server was declared online within the replication group'
----
2019-08-25T23:27:58.879670+08:00 0 [Note] Plugin group_replication reported: 'Members joined the group: mysql03.com:3308'
2019-08-25T23:27:58.879924+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql03.com:3308, mysql02.com:3308 on view 15667459843091222:3.'
2019-08-25T23:27:58.962822+08:00 181 [Note] Start binlog_dump to master_thread_id(181) slave_server(3), pos(, 4)
2019-08-25T23:27:59.049385+08:00 0 [Note] Plugin group_replication reported: 'The member with address mysql03.com:3308 was declared online within the replication group'
mysql03.com
operating
1.
[[email protected]][(none)]> CHANGE MASTER TO MASTER_USER='rpl_user', MASTER_PASSWORD='password' FOR CHANNEL 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.02 sec)
----
performance_schema.replication_group_members
1.
----
2.
----
3.
----
Journal
1.
2019-08-25T23:08:55.531641+08:00 82 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''.
----
2.
2019-08-25T23:27:56.759988+08:00 82 [Note] Plugin group_replication reported: 'Group communication SSL configuration: group_replication_ssl_mode: "DISABLED"'
2019-08-25T23:27:56.767834+08:00 82 [Note] Plugin group_replication reported: '[GCS] Added automatically IP ranges 10.0.2.7/24,127.0.0.1/8 to the whitelist'
2019-08-25T23:27:56.768028+08:00 82 [Warning] Plugin group_replication reported: '[GCS] Automatically adding IPv4 localhost address to the whitelist. It is mandatory that it is added.'
2019-08-25T23:27:56.768120+08:00 82 [Note] Plugin group_replication reported: '[GCS] SSL was not enabled'
2019-08-25T23:27:56.768177+08:00 82 [Note] Plugin group_replication reported: 'Initialized group communication with configuration: group_replication_group_name: "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; group_replication_local_address: "10.0.2.7:33081"; group_replication_group_seeds: "10.0.2.5:33081,10.0.2.6:33081,10.0.2.7:33081"; group_replication_bootstrap_group: false; group_replication_poll_spin_loops: 0; group_replication_compression_threshold: 1000000; group_replication_ip_whitelist: "AUTOMATIC"'
2019-08-25T23:27:56.768472+08:00 82 [Note] Plugin group_replication reported: '[GCS] Configured number of attempts to join: 0'
2019-08-25T23:27:56.768622+08:00 82 [Note] Plugin group_replication reported: '[GCS] Configured time between attempts to join: 5 seconds'
2019-08-25T23:27:56.768876+08:00 82 [Note] Plugin group_replication reported: 'Member configuration: member_id: 3; member_uuid: "77731d39-bc4c-11e9-956d-080027aa34ac"; single-primary mode: "true"; group_replication_auto_increment_increment: 7; '
2019-08-25T23:27:56.769046+08:00 84 [Note] Plugin group_replication reported: 'Detected previous RESET MASTER invocation or an issue exists in the group replication applier relay log. Purging existing applier logs.'
2019-08-25T23:27:56.788105+08:00 84 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_applier' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:27:56.827896+08:00 87 [Note] Slave SQL thread for channel 'group_replication_applier' initialized, starting replication in log 'FIRST' at position 0, relay log '/home/mysql/mysql3308/log/relay-group_replication_applier.000001' position: 4
2019-08-25T23:27:56.827994+08:00 82 [Note] Plugin group_replication reported: 'Group Replication applier module successfully initialized!'
2019-08-25T23:27:56.828023+08:00 82 [Note] Plugin group_replication reported: 'auto_increment_increment is set to 7'
2019-08-25T23:27:56.828032+08:00 82 [Note] Plugin group_replication reported: 'auto_increment_offset is set to 3'
2019-08-25T23:27:56.842870+08:00 0 [Note] Plugin group_replication reported: 'XCom protocol version: 3'
2019-08-25T23:27:56.843086+08:00 0 [Note] Plugin group_replication reported: 'XCom initialized and ready to accept incoming connections on port 33081'
2019-08-25T23:27:58.880407+08:00 82 [Note] Plugin group_replication reported: 'This server is working as secondary member with primary member address mysql01.com:3308.'
2019-08-25T23:27:58.880857+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql03.com:3308, mysql02.com:3308 on view 15667459843091222:3.'
2019-08-25T23:27:58.880981+08:00 94 [Note] Plugin group_replication reported: 'Establishing group recovery connection with a possible donor. Attempt 1/10'
2019-08-25T23:27:58.920355+08:00 94 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='', master_port= 3306, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='mysql02.com', master_port= 3308, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:27:58.935906+08:00 94 [Note] Plugin group_replication reported: 'Establishing connection to a group replication recovery donor b48091b5-bc4c-11e9-84d6-0800270c3d91 at mysql02.com port: 3308.'
2019-08-25T23:27:58.936692+08:00 96 [Warning] Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
2019-08-25T23:27:58.938211+08:00 96 [Note] Slave I/O thread for channel 'group_replication_recovery': connected to master '[email protected]:3308',replication started in log 'FIRST' at position 4
2019-08-25T23:27:58.956728+08:00 97 [Note] Slave SQL thread for channel 'group_replication_recovery' initialized, starting replication in log 'FIRST' at position 0, relay log '/home/mysql/mysql3308/log/relay-group_replication_recovery.000001' position: 4
2019-08-25T23:27:59.009077+08:00 94 [Note] Plugin group_replication reported: 'Terminating existing group replication donor connection and purging the corresponding logs.'
2019-08-25T23:27:59.009274+08:00 97 [Warning] Slave SQL for channel 'group_replication_recovery': Coordinator thread of multi-threaded slave is being stopped in the middle of assigning a group of events; deferring to exit until the group completion ... , Error_code: 0
2019-08-25T23:27:59.012125+08:00 97 [Note] Slave SQL thread for channel 'group_replication_recovery' exiting, replication stopped in log 'bin.000001' at position 1082
2019-08-25T23:27:59.013836+08:00 96 [Note] Slave I/O thread killed while reading event for channel 'group_replication_recovery'
2019-08-25T23:27:59.013914+08:00 96 [Note] Slave I/O thread exiting for channel 'group_replication_recovery', read up to log 'bin.000001', position 1082
2019-08-25T23:27:59.035895+08:00 94 [Note] 'CHANGE MASTER TO FOR CHANNEL 'group_replication_recovery' executed'. Previous state master_host='mysql02.com', master_port= 3308, master_log_file='', master_log_pos= 4, master_bind=''. New state master_host='<NULL>', master_port= 0, master_log_file='', master_log_pos= 4, master_bind=''.
2019-08-25T23:27:59.049625+08:00 0 [Note] Plugin group_replication reported: 'This server was declared online within the replication group'
----
Scene 2, MGR network partition log
mysql01.com
operating
1.
----
performance_schema.replication_group_members
1.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | ONLINE |
| mysql02.com | ONLINE |
+-------------+--------------+
2 rows in set (0.00 sec)
----
Journal
1.
2019-08-25T23:47:43.353479+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql03.com:3308 has become unreachable.'
2019-08-25T23:47:43.535791+08:00 0 [Note] Plugin group_replication reported: '[GCS] Removing members that have failed while processing new view.'
2019-08-25T23:47:44.357277+08:00 0 [Warning] Plugin group_replication reported: 'Members removed from the group: mysql03.com:3308'
2019-08-25T23:47:44.357476+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:4.'
----
mysql02.com
operating
1.
----
performance_schema.replication_group_members
1.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | ONLINE |
| mysql02.com | ONLINE |
+-------------+--------------+
2 rows in set (0.00 sec)
----
Journal
1.
2019-08-25T23:47:44.430556+08:00 0 [Warning] Plugin group_replication reported: 'Members removed from the group: mysql03.com:3308'
2019-08-25T23:47:44.430768+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:4.'
----
mysql03.com
operating
1.
[root@mysql03 ~]# iptables -A INPUT -p tcp -s 10.0.2.5 -j DROP
----
performance_schema.replication_group_members
1.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql03.com | ERROR |
+-------------+--------------+
1 row in set (0.00 sec)
----
Journal
1.
2019-08-25T23:47:42.690451+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql01.com:3308 has become unreachable.'
2019-08-25T23:47:43.813812+08:00 0 [ERROR] Plugin group_replication reported: 'Member was expelled from the group due to network failures, changing member status to ERROR.'
2019-08-25T23:47:43.814155+08:00 0 [Note] Plugin group_replication reported: 'Going to wait for view modification'
2019-08-25T23:47:47.295592+08:00 0 [ERROR] Plugin group_replication reported: 'Member was expelled from the group due to network failures, changing member status to ERROR.'
2019-08-25T23:47:47.296077+08:00 0 [Note] Plugin group_replication reported: 'Going to wait for view modification'
Scene 3, MGR network unreachable log
mysql01.com
operating
1.
----
performance_schema.replication_group_members
1.
----
Journal
1.
2019-08-25T23:57:13.764756+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql03.com:3308 has become unreachable.'
2019-08-25T23:57:13.802046+08:00 0 [Note] Plugin group_replication reported: '[GCS] Removing members that have failed while processing new view.'
2019-08-25T23:57:14.767258+08:00 0 [Warning] Plugin group_replication reported: 'Members removed from the group: mysql03.com:3308'
2019-08-25T23:57:14.767524+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:6.'
mysql02.com
operating
1.
----
performance_schema.replication_group_members
1.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | ONLINE |
| mysql02.com | ONLINE |
+-------------+--------------+
2 rows in set (0.00 sec)
Journal
1.2019-08-25T23:57:13.839467+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql03.com:3308 has become unreachable.'
2019-08-25T23:57:14.840650+08:00 0 [Warning] Plugin group_replication reported: 'Members removed from the group: mysql03.com:3308'
2019-08-25T23:57:14.840855+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed to mysql01.com:3308, mysql02.com:3308 on view 15667459843091222:6.'
----
mysql03.com
operating
1.
[root@mysql03 ~]# iptables -A INPUT -p tcp -s 10.0.2.5 -j DROP
[root@mysql03 ~]# iptables -A INPUT -p tcp -s 10.0.2.6 -j DROP
----
2.
[root@mysql03 ~]# iptables -F
----
3.
[[email protected]][(none)]> set global group_replication_unreachable_majority_timeout = 10;
Query OK, 0 rows affected (0.00 sec)
----
4.
[root@mysql03 ~]# iptables -A INPUT -p tcp -s 10.0.2.5 -j DROP
[root@mysql03 ~]# iptables -A INPUT -p tcp -s 10.0.2.6 -j DROP
----
performance_schema.replication_group_members
1.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | UNREACHABLE |
| mysql03.com | ONLINE |
| mysql02.com | UNREACHABLE |
+-------------+--------------+
3 rows in set (0.00 sec)
----
2.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql03.com | ERROR |
+-------------+--------------+
1 row in set (0.00 sec)
3.
----
4.
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | UNREACHABLE |
| mysql03.com | ONLINE |
| mysql02.com | UNREACHABLE |
+-------------+--------------+
3 rows in set (0.00 sec)
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql01.com | UNREACHABLE |
| mysql03.com | ERROR |
| mysql02.com | UNREACHABLE |
+-------------+--------------+
3 rows in set (0.00 sec)
[[email protected]][(none)]> select MEMBER_HOST,MEMBER_STATE from performance_schema.replication_group_members;
+-------------+--------------+
| MEMBER_HOST | MEMBER_STATE |
+-------------+--------------+
| mysql03.com | ERROR |
+-------------+--------------+
1 row in set (0.00 sec)
Journal
1.
2019-08-25T23:57:13.942230+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql01.com:3308 has become unreachable.'
2019-08-25T23:57:13.942315+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql02.com:3308 has become unreachable.'
2019-08-25T23:57:13.942331+08:00 0 [ERROR] Plugin group_replication reported: 'This server is not able to reach a majority of members in the group. This server will now block all updates. The server will remain blocked until contact with the majority is restored. It is possible to use group_replication_force_members to force a new group membership.'
----
2.
2019-08-26T00:14:00.999806+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql02.com:3308 is reachable again.'
2019-08-26T00:14:00.999883+08:00 0 [Warning] Plugin group_replication reported: 'The member has resumed contact with a majority of the members in the group. Regular operation is restored and transactions are unblocked.'
2019-08-26T00:14:02.000766+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql01.com:3308 is reachable again.'
2019-08-26T00:14:02.162728+08:00 0 [ERROR] Plugin group_replication reported: 'Member was expelled from the group due to network failures, changing member status to ERROR.'
2019-08-26T00:14:02.162958+08:00 0 [Note] Plugin group_replication reported: 'Going to wait for view modification'
2019-08-26T00:14:05.906240+08:00 0 [ERROR] Plugin group_replication reported: 'Member was expelled from the group due to network failures, changing member status to ERROR.'
2019-08-26T00:14:05.906497+08:00 0 [Note] Plugin group_replication reported: 'Going to wait for view modification'
----
3.
----
4.
2019-08-26T00:24:04.608437+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql01.com:3308 has become unreachable.'
2019-08-26T00:24:04.608497+08:00 0 [Warning] Plugin group_replication reported: 'Member with address mysql02.com:3308 has become unreachable.'
2019-08-26T00:24:04.608508+08:00 0 [ERROR] Plugin group_replication reported: 'This server is not able to reach a majority of members in the group. This server will now block all updates. The server will remain blocked for the next 10 seconds. Unless contact with the majority is restored, after this time the member will error out and leave the group. It is possible to use group_replication_force_members to force a new group membership.'
----
2019-08-26T00:24:14.611490+08:00 0 [ERROR] Plugin group_replication reported: 'This member could not reach a majority of the members for more than 10 seconds. The member will now leave the group as instructed by the group_replication_unreachable_majority_timeout option.'
2019-08-26T00:24:14.611613+08:00 0 [ERROR] Plugin group_replication reported: 'The server was automatically set into read only mode after an error was detected.'
----
2019-08-26T00:24:44.613532+08:00 0 [ERROR] Plugin group_replication reported: '[GCS] Timeout while waiting for the group communication engine to exit!'
2019-08-26T00:24:44.613633+08:00 0 [ERROR] Plugin group_replication reported: '[GCS] The member has failed to gracefully leave the group.'
2019-08-26T00:24:44.614044+08:00 0 [Warning] Plugin group_replication reported: 'read failed'
2019-08-26T00:24:44.641654+08:00 0 [Note] Plugin group_replication reported: 'Group membership changed: This member has left the group.'
----