mycat data synchronization

mysql5.7 supports two transaction synchronization

From the master copy in two ways: based on the log (binlog), based GTID (global transaction identifier). The log-based (the binlog) manner.

In previous versions of mysql, separate read and write implementations are generally based on a master log from the replication achieved, it will generate a problem, that is, after the master goes down, because the slave synchronization delay problems can cause master and slave content different, and even more different from each other slave. So in order to solve this problem, then mysql5.7.6 be added after version control based GTID affairs, specifically, is each transaction by a unique identifier gtid, when the slave master was executed successfully written to disk to complete the transaction, if master suddenly goes down, then it is automatically rolled back. Data consistency is guaranteed.

Common methods of operation and based on the master copy from the log can not be much worse, mainly two switches open

enforce_gtid_consistency = ON

gtid_mode = ON

Then tell us about this particular process to build a master-slave synchronization.

Master：

First of all you want to modify mysql configuration file, the configuration file path I have here is /etc/mysql/mysql.conf.d/mysqld.cnf, based docker, different versions of the location may be different, most of the windows called my.cnf , Download: https: //hub.docker.com/r/alexzhuo/mysql/.

Here only interception that period to be modified

before fixing:

#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
修改后

ID. 1 =-Server
log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
actually removed two annotations, binlog opened, must be opened from the master binlog only basis to synchronize between.

Which is the server-id mysql cluster, each node should have its own id, can not be repeated, usually master is set to 1, slave set to 2,3,4,5 If this line is commented out, then the slave node is not started up.

About binlog_do_db, it is the development of operations which libraries are written in the binlog, which is synchronized library, which library binlog_ignore_db is not written to the binlog, that is what libraries are not synchronized. Under normal circumstances can comment out two lines, that is to synchronize all of the libraries. In the slave node will have such a configuration, but the configuration is not the same field, the slave terminal can be controlled to reduce the pressure of the master, but also can be accurate to a table are synchronized, it is generally set in the slave node, visible The following figure:

Then look at the synchronous format

Can choose according to their needs, the default is row.

Then we need to create a user on the master node, then give it authorized designed to make copying tasks, but it does not select or modify any one table. Statement is as follows:

the User the Create 'dba' @ '%' IDENTIFIED by '123456';
. * ON * Grant Slave Replication to dba;
then we can export all the contents out of the current database, and then import the new database. Of course, the new slave database is also a docker. Export statements are as follows

mysqldump --single-transaction --master-data = 2 --triggers --routines --all-databases -uroot -p> all.sql
exported to text files.
If you do not server-id and log_bin comment on, then when exporting the database will be reported the following error

@ 701cc1949c81 the root: / Home / MySQL # --single the mysqldump-Transaction Data = --master-2-Databases -uroot---triggers --routines --all -p> all.sql
the Enter password:
the mysqldump: Error: ON Binlogging server not active
then only need to follow the above talked about open binlog mysql can then restart.
Then we can re-open a new docker, and then import the library, of course, the first thing you want to transfer all.sql just exported to the new docker container import statement is:

mysql -uroot -p <all.sql
then as from the database, we have to pay attention to where the two configuration

1 is a server-id, but also open and can not be repeated and the master, as with the above method, /etc/mysql/mysql.conf.d/mysqld.cnf modify this file.

before fixing

#server-id = 1
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
修改后

2 = server-id
#log_bin = /var/log/mysql/mysql-bin.log
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
the master is different, only modify a server-id that can be , because the slave does not need to turn on binlog.

2, we have to modify the UUID database, otherwise the slave is started when the error. Since the master and slave are generated by the same image of a docker, so UUID default is the same, you need to modify the configuration file.

View the current mysql UUID command

MySQL> Show Variables like '%% server_uuid';
+ --------------- + ---------------------- + ----------------
| variable_name | Value |
+ --------------- + ----------- + ---------------------------
| server_uuid | b790fa18-404a-11E7-b542-0242ac110002 |
+ -------- + ------- + --------------------------------------
solve this problem the method is to modify /var/lib/mysql/auto.cnf this file UUID, and as long as the other master, slave can be different, and then restart the database. If the master and slave is not the same, then the master slave can not be performed after the Start I / O, and will be given as follows

After these are done, we can let the slave synchronization ready to start, and the following instructions

change master to master_host = 'Master IP address',
-> MASTER_USER = 'DBA',
-> master_password = '123456',
-> MASTER_LOG_FILE = 'the binlog file name',
-> MASTER_LOG_POS = number;
first row to know the master IP address and port, if the port is 3306 then you can only fill in the IP address for the docker container, you can find the following command IP

docker inspect <容器ID>
......
"Networks": {
"bridge": {
"IPAMConfig": null,
"Links": null,
"Aliases": null,
"NetworkID": "cada7bb270b98235a23771daa68c16a0393bf7acdbf9beea824381c0565e716b",
"EndpointID": "195ec54ec8fa862dc45d0ac35db9ece1474276b355eec97bf5186bc72dbe6e7d",
"Gateway": "172.17.0.1",
"IPAddress": "172.17.0.3",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:11:00:03",
"Driveropts ": null so that you can see the IP of a container, and if your vessel deployed on two machines, then you can directly do port mapping.}
}

master_log_file this parameter is generally binlog.00001 (new database) or binlog.0000X (X is a natural number, and will gradually increase over time). To get this binlog file name, we can go to find all.sql the file you just exported, and there are related records, as follows

-
- to the Position Start Replication Point-in-Time or from Recovery
-

- the MASTER the TO of MASTER_LOG_FILE the CHANGE = 'MySQL-bin.000001', MASTER_LOG_POS = 154;
or after you manually synchronize the master or slave data (such as sql or directly introduced into the copy of the database files), can be seen at the end of the command master as follows

mysql> show master status \G
*************************** 1. row ***************************
File: mysql-bin.000001
Position: 154
Binlog_Do_DB:
Binlog_Ignore_DB:
1 row in set (0.00 sec)

This document MASTER_LOG_FILE correspond master_log_file this parameter, MASTER_LOG_POS correspond master_log_pos this parameter. So I became a configuration commands:

Master to MASTER_HOST = Change '172.17.0.3',
-> MASTER_USER = 'DBA',
-> master_password = '123456',
-> MASTER_LOG_FILE = 'MySQL-bin.000001',
-> MASTER_LOG_POS = 154;
If successful, can check whether the correct configuration, is:

MySQL> Show Slave Status \ G
*************************** 1. Row ************* **************
Slave_IO_State:
MASTER_HOST: 172.17.0.3
MASTER_USER: DBA
MASTER_PORT: 3306
Connect_Retry: 60
Master_Log_File: MySQL-bin.000001
Read_Master_Log_Pos: 154
Relay_Log_File: 6b901656c610-Relay-bin.000001
RELAY_LOG_POS : 4
Relay_Master_Log_File: MySQL-bin.000001
Slave_IO_Running: no
Slave_SQL_Running: no
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
......
there are the two most important parameters Slave_IO_Running and Slave_SQL_Running, because the current is not turned slave nodes, two a is NO, the following command can be opened by

start slave;
later on, if the above two fields are yes Description slave node synchronization is successful. If there is NO, then the error will be displayed in Last_IO_Error field, as follows

mysql> show slave status \G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 172.17.0.3
Master_User: dba
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000001
Read_Master_Log_Pos: 154
Relay_Log_File: 6b901656c610-relay-bin.000001
Relay_Log_Pos: 4
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 154
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1593
Last_IO_Error: Fatal error: The slave I/O thread stops because master and slave have equal MySQL server UUIDs; these UUIDs must be different for replication to work.
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID:
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp: 180527 08:16:18
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
当然这个问题上面已经提供了解决方案。

When Slave_IO_Running and Slave_SQL_Running is yes, we can re-master node performs building a database, build tables, insert, update operation, and then on again to see whether slave synchronization of data.

The above has been completed based on the traditional log from the master copy, to open the following new features mysql 5.7, and transaction-based replication, its major operating on two points

1, in the master and slave are open enforce_gtid_consistency

2, in the master and slave are open gtid_mode

We have to perform the following operations on the master and slave

First, check to see if has opened gtid, if the return is empty then that also have turned off

MySQL> Show the Variables like 'grid_mode';
Empty set (0.00 sec)
We then execute the following command to open the
set @@ global.enforce_gtid_consistency = The warn;
set @@ global.enforce_gtid_consistency = ON;
Why set a statement to be executed it twice because the parameters are not allowed to set off from on, and to off-warn-on transition process. After setting, we look to see whether the error errorlog

tail -f /var/log/mysql/error.log
2018-05-27T07:58:22.327301Z 0 [Warning] 'proxies_priv' entry '@ root@localhost' ignored in --skip-name-resolve mode.
2018-05-27T07:58:22.328919Z 0 [Warning] 'tables_priv' entry 'sys_config mysql.sys@localhost' ignored in --skip-name-resolve mode.
2018-05-27T07:58:22.340490Z 0 [Note] Event Scheduler: Loaded 0 events
2018-05-27T07:58:22.341126Z 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.7.18-0ubuntu0.16.04.1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu)
2018-05-27T07:58:22.341233Z 0 [Note] Executing 'SELECT * FROM INFORMATION_SCHEMA.TABLES;' to get a list of tables using the deprecated partition engine. You may use the startup option '--disable-partition-engine-check' to skip this check.
2018-05-27T07: 58: 22.341274Z 0 [Note] Beginning of non-natively Partitioned List of Tables
2018-05-27T07: 58: 22.366089Z 0 [Note] End of List of non-natively Partitioned Tables
2018-05- 27T08:. 19: 47.238163Z. 9 [Note] to the Start binlog_dump master_thread_id (. 9) slave_server (2), POS (MySQL-bin.000001, 154)
2018-05-27T08: 37 [: 15.629583Z. 8 [Note] from the Changed OFF ENFORCE_GTID_CONSISTENCY to WARN.
no error can look successful, then continue following the operation

Global Variables like Show 'enforce_gtid%';
+ -------------------------- + ------- +
| variable_name | the Value |
+ -------------------------- + ------- +
| enforce_gtid_consistency | ON |
+ -------- + ------- + ------------------
SET @@ global.gtid_mode = off_permissive;
SET = @@ global.gtid_mode on_permissive;
SET @@ Global. gtid_mode = on;
Similarly, global.gtid_mode this parameter must also be off-off_permissive-on_permissive-on process changes, after setting up we also need to look at whether the error log, and then check whether open.

Variables like Show 'gtid_mode';
+ ------- + --------------- +
| variable_name | the Value |
+ ----------------------- + ------- + ---
| gtid_mode | oN |
+ --------------- + ------- +
are all executed in the master and slave nodes after the above command, master node without rebooting. However slave nodes need to restart the slave mode, the following

MySQL> STOP Slave;
Query the OK, 0 rows affected (0.05 sec)

MySQL> Change Master to master_auto_position =. 1;
Query the OK, 0 rows affected (0.31 sec)

MySQL> Start Slave;
Query the OK, 0 rows affected (0.03 sec)
so to complete the restart slave nodes. Then we look at the immediate execution command parameters

mysql> show slave status \G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 172.17.0.3
Master_User: dba
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 154
Relay_Log_File: 6b901656c610-relay-bin.000002
Relay_Log_Pos: 367
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 154
Relay_Log_Space: 581
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: b790fa18-404a-11e7-b542-0242ac110002
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has the Read All Relay log; Waiting for More the Updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 Row in the SET (0.00 sec)
based on the log and found nothing different, just Master_UUID became /var/lib/mysql/auto.cnf this file on the master node to write a UUID. But when we re-master on the implementation of some other insert or write operation, Executed_Gtid_Set this parameter will have value, but also for a value Master_UUID associated. Gtid copy is based on whether a sign of success. as follows

Note that if after accordance with the above configuration, manually enforce_gtid_consistency and gtid_mode set to ON, if you restart the mysql container or service, you will find there is no automatic synchronization, Slave_IO_Running and Slave_SQL_Running is NO, will complain when the start slave start again, as follows

we /etc/mysql/mysql.conf.d/mysqld.cnf
......
enforce_gtid_consistency = ON
gtid_mode = ON

When you save a good master and slave two docker, migrate to other machines or downtime restart, after replacing the master or IP, you will find the slave end often get up. Given as follows

MySQL> Start slave;
ERROR 1872 (HY000): Slave to the initialize failed The Relay log info from at The Repository Structure
This is because after the master or slave restart, the original is not on the binlog on, we need to manually reset the slave node, without master moving, good gtid it again without setting up, slave end of the run the following command.

MySQL> Slave RESET;
MySQL> Slave Start;
Note that since the above is provided when the open mode gtid

change master to master_auto_position=1;

Statement, so if the change master to re-execute the statement at this time, then the inside

-> master_log_file = 'binlog file name'

-> master_log_pos = number;

Above do not succeed, because the next gtid mode, it will automatically sync these two parameters, if you really want to set the number of rows binlog position manually, you need to change master to master_auto_position = 0;

Also reset slave and start slave mode after due under gtid not need to fill binlog file name and line number, if you change the master node IP, then you can simply re-set the IP master, pay attention;

change master to master_host='Master的IP地址',
-> master_user='dba',
-> master_password='123456'
reset slave;
start slave;

One step directly open GTID mode

Programs of the above, is the first open generic log copy mode, and then based replication mode GTID how one step directly open gtid pattern here. The method is very simple, because in gtid mode, you can automatically locate the file name and line number master binlog, so eliminating the trouble of going to see the master server. As long as the master and slave data are first identical (by introducing mysqldump), while no operation master (3306 require that all connections are turned off). Then you can directly use the change master command. Detailed process is as follows

1, Master and configurations are also the same as above. Need to open binlog, and open enforce_gtid_consistency and gtid_mode (if they think that every time you start master must manually open too much trouble, then put both written my.cnf, the above mentioned)

2, the configuration file my.cnf Slave added

= the ON enforce_gtid_consistency
gtid_mode = the ON
and forget modification serverid

server-id = 2
after the start, set the IP address of the master by change master statement, without providing binlog position and the number of lines, just to ensure that the current master of library and library contents can be consistent

change master to master_host = 'Master of the IP address',
-> MASTER_USER = 'dba',
-> master_password = '123456'
Start Slave;
then check whether is yes to
MySQL> Show Slave Status \ G
--- ------------------
Disclaimer: This article is CSDN blogger "lvshaorong 'original article, follow the CC 4.0 by-sa copyright agreement, reproduced, please attach the original source and link this statement.
Original link: https: //blog.csdn.net/lvshaorong/article/details/80471878

mycat data synchronization

Guess you like