MySQL operation and maintenance 28-MySQL replication

1. MySQL replication mode

1. MySQL has three replication modes as follows

copy mode parameter value	schema name	illustrate	advantage	shortcoming
statement	SQL statement-based replication	That is, logical replication, which realizes data replication by executing the statements executed on the main database	Simple and easy, the log is smaller, and the readability of the log is good for troubleshooting	Some operations will not be correctly copied to the slave library, such as operations with LIMIT clauses but without ORDER BY, such as support for stored procedures and triggers. In addition, more records need to be locked from the library
row	row-based replication	Physical replication, copying the modification of the data itself to the slave library	All changes are replicated, more secure and accurate, less table locks	The log is larger and less readable, and the structure of the master-slave table is required to be completely consistent
mixed	blend mode	Mainly based on logical replication, in some special cases such as using UUID(), it is changed to physical replication	It has both the efficiency of the statement mode and the accuracy of the row mode, and it is recommended to use the mixed mode

2. The copy mode supports dynamic modification at runtime, the command is as follows. Note that the slave library can adjust itself to adapt to the replication mode of the master library, so you only need to modify the replication mode of the master library:

mysql> SET GLOBAL binlog_format = 'STATEMENT';
mysql> SET SESSION binlog_format = 'STATEMENT';

2. Compatibility of MySQL replication

1. The version of the main library and the slave library should be kept as consistent as possible to avoid other problems in the future.
2. When the versions of the master and slave databases are inconsistent, it is recommended that the version of the slave database be higher than that of the master database. In most cases, MySQL supports this type of replication. Copying from a higher version of the main library to a lower version of the slave library may be feasible, but the official support is not guaranteed.
3. Do not cross the large version number between the master and slave libraries, as that may cause problems, such as copying from MySQL4.1 to MySQL5.1. If you must do this, in order to reduce the risk, you can consider inserting a MySQL5.0 version slave library in the middle to avoid some unexpected problems.
4. Before upgrading MySQL in the production environment, first upgrade the version of the slave library to verify and ensure that the replication function is normal.

3. Two types of logs related to MySQL replication

3.1. Relay logs

1. By default, relay logs use filenames of the form host_name-relay-bin.nnnnnn, where host_name is the slave server hostname and nnnnnn is the sequence number. Create consecutive relay log files with consecutive sequence numbers, starting with 000001. Relay logs currently in use from the server track index file. The relay log index file name defaults to host_name-relay-bin.index. These files are created by default in the server's data directory. The default filenames can be overridden with the --relay-log and --relay-log-index server options. It is strongly recommended to specify the default file name, that is, the log file name does not have a host name prefix, which will facilitate migration operations and troubleshooting.
2. Relay logs have the same format as binary logs and can be read with mysqlbinlog. It is automatically deleted as soon as the SQL thread has executed all the events in the relay log and the relay log is no longer needed. There is no mechanism to delete the relay log directly, since the SQL thread can take care of the completion.

3.2, MySQL replication status log

1. The slave library will create two additional small files in the data directory, the default names are master.info and relay-log.info, which contain the information displayed by the output of the SHOW SLAVE STATUS statement. The state files are saved on the hard disk, so the state files are not lost when the slave server is shut down. The next time the slave is started, these files are read to determine how much of the binary log it has read from the master, and to what extent it has processed its own relay logs. The master.info file is updated by the I/O thread. The relay-log.info file is updated by the SQL thread.
2. When backing up slave data, you should also back up these two small files and the relay log file. They can be used to continue replication after restoring data from a slave server.
3. If the relay log is lost but there is still a relay-log.info file, you can check the file to determine the extent of the binary log in the master server that the SQL thread has executed. For example, the following command shows: The current slave library is just executing to the master server The library log file mysql -bin.006800 is executed at 401289356. Then we can execute the CHANGE MASTER command with the Master_Log_File and Master_LOG_POS options to tell the slave server that it needs to re-read the binary log from this point.

shell> cat relay-log.info 
/usr/local/mysql/log/relay-bin.015161
401289501
mysql-bin.006800
401289356

4. Implementation example of MySQL master-slave replication

4.1. Master-slave library installation

1. Install the main library and the slave library, and enable binary logs on the main library. Note that the server-id of the master and slave libraries must be set to be different. It is recommended that the server-id be set to the last 8 digits of the IP plus the port (port), similar to as follows.

[mysqld]
log-bin=mysql-bin
server-id=1713306     # server-id可以跟服务器的IP一致

4.2. Main library configuration

2. Record the File and Position of the current binary log file of the main library, as follows:

mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      154 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+

3. Create a special account for replication in the main library, grant permissions, and allow access from the library. The command is as follows:

GRANT replication slave,replication client ON *.* TO replic_user IDENTIFIED BY 'xxxxxxxxxxx';

4.3. Configuration of the slave library

4. Edit the configuration file of the slave library. An example of the configuration file of the slave library is shown below: only the server-id must be set, and other options are optional. These options are explained as follows:
   • log_bin=mysql-bin: It is recommended that the master and slave configure the same name, otherwise in the future configuration, the problem will be much more complicated.
   • log_slave_updates=1: log_slave_updates determines whether to write these SQL operations into the binary log of the slave library itself when performing replication. If it is 1, it will be written, if it is 0, it will not be written. Set this value to 1 if this library may be set as the main library in the future. The downside of setting this to 1 is larger I/O writes.
   • read_only=1: Only users with SUPER authority can modify data.

server_id = 1723306
log_slave_updates = 1
log_bin = mysql-bin
relay_log = /path_to_mysql_log/mysql-relay-bin
read_only = 1

5. Execute the following statement on the slave library, where MASTER_LOG_FILE and MASTER_LOG_POS are the File and Position values ​​of the master library recorded in the previous second step. Do not use the method of specifying master_host and master_port in the configuration file. These configurations only take effect when MySQL is started for the first time:

mysql >
CHANGE MASTER TO
MASTER_HOST='192.168.124.171',
MASTER_PORT=3306,
MASTER_USER='replic_user',
MASTER_PASSWORD='xxxxxxxxxxx',
MASTER_LOG_FILE='mysql-bin.000001',
MASTER_LOG_POS=154;

4.4. Start replication and confirm success

6. Execute the following command on the slave library to start the slave.

mysql> start slave;

7. Check whether the replication is normal on the slave library: the values ​​of Slave_IO_Running and Slave_SQL_Running should be YES, and Seconds_Behind_Master should not be NULL.

mysql> SHOW SLAVE STATUS \G;
……
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
……
Seconds_Behind_Master: 0
……

5. Monitoring of MySQL replication

You can monitor the status of MySQL replication in the slave library through SHOW SLAVE STATUS. The output parameters are as follows:

6. Operation and maintenance of MySQL monitoring

6.1. Modify the main library configuration in the slave library

• You can use the CHANGE MASTER TO command to dynamically modify the information connected to the master library, such as modifying the password of the replication user. Note that you only need to specify the changed parameters when modifying, and do not need to specify the changed parameters.
• The following parameters can be modified:
  • MASTER_HOST and MASTER_PORT specify the IP and PORT of the main library.
  • MASTER_LOG_FILE and MASTER_LOG_POS specify the name and location of the binary log of the main library.
  • MASTER_USER and MASTER_PASSWORD specify the account and password of the replication user, and this account will be used to connect to the main library, so the main library needs to give this account the REPLICATION SLAVE permission to copy data.
  • CHANGE MASTER TO deletes all relay log files and starts a new one, unless you specify RELAY_LOG_FILE or RELAY_LOG_POS. In this case, relay logs will be kept.
  • The CHANGE MASTER TO command will update the contents of the master.info and relay-log.info files.
• The following is a complete example of modification, first stop the copy from the library, then modify, and then start the copy:

mysql> STOP SLAVE; 
mysql> CHANGE MASTER TO MASTER_PASSWORD='new3cret';
mysql> START SLAVE; 

6.2. Start and stop copying from the library

1. START/STOP SLAVE without options will start/stop two slave library threads (ie I/O thread and SQL thread) at the same time. STARTSLAVE requires SUPER permission.
2. A thread can be started or stopped individually, as shown in the following example:
   2.1. Start an I/O thread:

mysql> START SLAVE IO_THREAD

2.2. Start the SQL thread:

mysql> START SLAVE SQL_THREAD

3. START SALVE can specify that the slave replicate until a given point in the master's binary log is reached by adding an UNTIL clause. If the SQL_THREAD option is specified in the statement, it will only start the SQL thread. Otherwise, it starts two slave library threads at the same time. Normally, we only manipulate the SQL thread.

mysql> START SLAVE [SQL_THREAD] UNTIL MASTER_LOG_FILE = 'log_name', MASTER_LOG_POS = log_pos

6.3. Display the slave library in the main library

You can run the following command to display all slave libraries of the current master library:

mysql>  SHOW SLAVE HOSTS;

6.4. How to change the master library from which the slave library is synchronized

Suppose there are 4 MySQL instances A, B, C, and D. A is the master database, and B, C, and D are slave databases. Now it is necessary to adjust the architecture so that B is A's slave library, and C and D are B's slave library. We can adjust according to the following steps.

1. Close the slave libraries of B, C, and D respectively:

mysql> STOP  SLAVE;

2. To view the log location information on the main library A, the command is as follows.

mysql>  show master status;
+-------------------------+----------------+---------------------+-------------------------+
| File                           | Position      | Binlog_Do_DB  | Binlog_Ignore_DB |
+-------------------------+----------------+---------------------+-------------------------+
| mysql-bin.000003 | 307827324 |                             |                                  |
+-------------------------+----------------+---------------------+-------------------------+

3. Run the following commands on all slave libraries B, C, and D, and automatically disconnect the SQL thread when they are synchronized to the specified location. At this time, the data on B, C, and D should be consistent, and they are all synchronized to a log point in A library.

mysql>  START SLAVE SQL_THREAD UNTIL MASTER_LOG_FILE = ' mysql-bin.000003', MASTER_LOG_POS = 307827324;

4. Run SHOW MASTER STATUS on the B library to record the log location information log file name, log_file_positioniion.

5. Run the following command on slave libraries C and D to point to the new master library B.

mysql > STOP SLAVE;
mysql > RESET SLAVE;
mysql >
CHANGE MASTER TO
MASTER_HOST='11.11.11.11',
MASTER_PORT=3306,
MASTER_USER='replic_user',
MASTER_PASSWORD='password',
MASTER_LOG_FILE='log file name',
MASTER_LOG_POS=log_file_positiion;

6. Finally, run the START SLAVE command on the B library, so that A can be adjusted as the master library, B as the slave library of A, and C and D as the slave library of B.

7. Other types of copying

7.1, MySQL master-master replication

7.1.1. Basic configuration

1. The steps to configure master-master replication are very simple. Perform the steps of configuring master-slave replication for each library separately, that is, first configure according to the master-slave node, and then in turn, configure the slave node as the master node and the master node as the slave node. That's it.
2. It must be ensured that only one library is written at any time. The master-master replication is for fault redundancy rather than multi-point writing, so it should generally be configured as Active-Standby (active-standby) instead of Active-Active (active-active). Generally speaking, configuring master-master replication will make maintenance more complicated, and may also bring hidden dangers, requiring more complete monitoring measures and automation methods.

7.1.2, auto-increment key conflict resolution

1. Configure master-master replication, the main problem that needs to be solved is auto-increment key/primary key conflict: the server changes auto_increment_increment to control the interval between auto-increment column values. auto_increment_offset is used to determine the starting point of the self-increment column value. Assuming that there are two hosts A and B, and they are masters and slaves of each other, then the mysqld section of my.cnf can be configured as follows.

• A host

auto_increment_increment=3
auto_increment_offset=1

• B host

auto_increment_increment=3
auto_increment_offset=2

2. We also need to pay attention to that, in addition to the self-increment fields cannot conflict with each other, the key values ​​​​of all tables cannot conflict with each other, and the operation at the same time needs to ensure that the same key value will not be inserted.

7.2, MySQL cascade replication

If it needs to be configured in the form of A→B→C→D→E, the arrow means copy to, then follow the steps below. Note that the more nodes there are, the worse the robustness will be. It is recommended not to exceed 4-5 nodes.

1. First, open the log_slave_updates option of each instance, and the first and last two instances do not need to be opened.
2. Make sure that the server-id of each host is different.
3. Configure each pair of master and slave, A→B, B→C, C→D, D→E.

7.3, MySQL cross-IDC replication

1. There is no difference between the deployment of cross-IDC replication architecture and the deployment of chain replication (cascade replication) in a single computer room. However, due to the instability of the network, the replication may be unstable, the maintenance cost is high, and the external network may be required IP can only be copied, which reduces security.
2. For cross-IDC replication, it is recommended to use a common master-slave architecture instead of a chained replication architecture. A simple master-slave architecture is more robust.
3. Try to only write in the central main library, and other computer rooms are only used for reading. This can not only simplify the architecture, but also avoid the problem of maintaining consistency caused by multi-point writing. If it is an MM architecture, a computer room should also be used as a standby (Standby), only for disaster recovery.
4. When the amount of data is large, the network may become a bottleneck. It is recommended to use the mixed log replication mode. You can set slave_compressed_protocol=1 in the slave library to compress the transmitted data, and this option can be set dynamically.
5. Due to the cross-IDC master-slave replication, the cost of rebuilding is relatively high. When you know exactly what kind of error occurs in the database, you can ignore this error. You can use "slave-skip-errors=error_code1,error_code2...|al l", but do not abuse it , otherwise it is easy to lead to inconsistency between the master and the slave without knowing it.
6. Due to cross-IDC replication, the network may be unstable, and the application should handle the impact of network latency on user experience.