I believe everyone is familiar with MySQL master-slave synchronization. With the gradual increase of system application access, the read and write access pressure of a single database also increases. When the read and write access reaches a certain bottleneck, the read and write efficiency of the database will be improved. Sudden drop, or even unavailability; in order to solve such problems, MySQL cluster is usually used. When the main library goes down, the cluster will automatically upgrade a slave library to the master library and continue to provide external services; then the relationship between the master library and the slave library How is the data synchronized? This article conducts the following analysis for MySQL 5.7 version. Let's explore with the author how the mysql master and slave are synchronized.
MySQL master-slave replication principle In order to reduce the pressure on the master database, read and write should be separated at the system application level. Write operations go to the master database, and read operations go to the slave database. The following figure shows the principle diagram of master-slave replication given by MySQL official website. In the figure, you can easily understand the process of read-write separation and master-slave synchronization, which disperses the access pressure of the database, improves the performance and availability of the entire system, and reduces the failure rate of database downtime caused by large access volumes.
Introduction to binlog MySQL master-slave synchronization is based on the master-slave replication of binlog files. In order to better understand the master-slave synchronization process, here is a brief introduction to the binlog log file. The binlog log is used to log all statements that have updated data or have potentially updated data (eg, a DELETE that does not match any row). Statements are saved as "events", which describe data changes, and are saved to disk in binary form. We can view the contents of the file through the viewing tool mysqlbinlog provided by mysql, such as mysqlbinlog mysql-bin.00001 | more, here pay attention to the suffix 00001 of the binlog file, the size and number of binlog files will continue to increase, when MySQL stops or When restarting, a new binlog file will be generated, and the suffix name will increase by the serial number, such as mysql-bin.00002, mysql-bin.00003, and when the binlog file size exceeds the max_binlog_size system variable configuration, a new binlog file will also be generated. 1. Binlog log format (1) statement: The sql advantage of recording each piece of changed data : the binlog file is smaller, saves I/O, and has high performance. Disadvantages: Not all data changes will be written to the binlog file, especially using some special functions in MySQL (such as LOAD_FILE(), UUID(), etc.) and some indeterminate statement operations, resulting in master-slave data that cannot be replicated question. (2) row: do not record sql, only record the details of changes in each row of data Advantages: The details of changes in each row of data are recorded in detail, which also means that there will be no problems that cannot be replicated due to the use of some special functions or other circumstances.
Disadvantages: Since the row format records the change details of each row of data, a large amount of binlog log content will be generated, the performance is poor, and the probability of master-slave synchronization delay will increase.
(3) Mixed: The general statement modification uses the statment format to save the binlog. For example, for some functions, the statement cannot complete the master-slave replication operation, and the binlog is saved in the row format. MySQL will treat the records differently according to each specific sql statement executed. The log form, that is, choose one between Statement and Row.
2. Binlog log content The content viewed by the mysqlbinlog command is as follows:
Binlog content viewed by event type:
3. binlog事件类型
MySQL binlog记录的所有操作实际上都有对应的事件类型的，譬如STATEMENT格式中的DML操作对应的是QUERY_EVENT类型，ROW格式下的DML操作对应的是ROWS_EVENT类型，如果想了解更多请参考官方文档，有关binlog日志内容不在这里过多赘述，简单介绍一下是为了更好的理解主从复制的细节，下面我们进入正题。
MySQL主从复制原理
mysql主从复制需要三个线程，master（binlog dump thread）、slave（I/O thread 、SQL thread）。
master
（1）binlog dump线程：当主库中有数据更新时，那么主库就会根据按照设置的binlog格式，将此次更新的事件类型写入到主库的binlog文件中，此时主库会创建log dump线程通知slave有数据更新，当I/O线程请求日志内容时，会将此时的binlog名称和当前更新的位置同时传给slave的I/O线程。
slave
（2）I/O线程：该线程会连接到master，向log dump线程请求一份指定binlog文件位置的副本，并将请求回来的binlog存到本地的relay log中，relay log和binlog日志一样也是记录了数据更新的事件，它也是按照递增后缀名的方式，产生多个relay log（ host_name-relay-bin.000001）文件，slave会使用一个index文件（ host_name-relay-bin.index）来追踪当前正在使用的relay log文件。
(3) SQL thread: After the thread detects that the relay log has been updated, it will read and perform the redo operation locally, and re-execute the events that occurred in the main library locally to ensure the synchronization of master-slave data. In addition, if all events in a relay log file are executed, the SQL thread will automatically delete the relay log file.
Below is a schematic diagram of the entire replication process:
Master-slave synchronization delay The master-slave replication of mysql is a single-threaded operation. The master library generates binlogs for all DDL and DML. Binlogs are written sequentially, so the efficiency is very high. It is relatively high, however, the SQL thread of the slave implements the DDL and DML operations of the main library on the slave. The IO operations of DML and DDL are random, not sequential, and the cost is much higher. There may also be lock contention caused by other queries on the slave. Since SQL is also single-threaded, a DDL is stuck and needs to be executed for a long time. For a long event, subsequent DDL threads will wait for the DDL to complete before executing, which leads to delays. When the TPS concurrency of the main library is high, the amount of DDL generated exceeds the range that a sql thread of the slave can bear, and the delay occurs. In addition, there may be lock waiting caused by the large query statement of the slave. Since the master-slave synchronization delay exists objectively, we can only design from our own architecture, and try to make the DDL of the master library execute quickly. Several common solutions are listed below:
- The implementation of the persistence layer of the business adopts the sub-database architecture, and the mysql service can be expanded in parallel to disperse the pressure;
- The infrastructure of the service adds the cache layer of memcache or Redis between the business and mysql. Reduce the read pressure of mysql;
- Use a better hardware device than the main library as a slave;
- sync_binlog is set to 0 on the slave side;
- --logs-slave-updates Updates received by the slave from the master are not logged in its binary log;
- Disable slave binlog.
http://www.iteye.com/news/32664