1. Reason:
One day, I saw that the alarm of the master-slave replication delay was a bit frequent, and I wondered if it could be completely solved.
Generally, master-slave replication involves three threads, all of which are single-threaded: Binlog Dump (master) ----->IO Thread (slave) -----> SQL Thread (slave). Replication delays generally occur in two places
1) The SQL thread is too busy (may require a large amount of application data, which may conflict with locks and resources with some operations of the slave library itself; the main library can write concurrently, but the SQL thread cannot; the main reason)
2) Network jitter causes IO thread replication delay (secondary cause).
Second, the solution:
MySQL has the concept of multiple SQL Threads since 5.6, which can restore data concurrently, that is, parallel replication technology.
In MySQL 5.6, set the parameter slave_parallel_workers = 4(>1), you can have 4 SQL Threads (coordinator threads) for parallel replication, and its status is: Waiting for an evant from Coordinator.
But its parallelism is only based on Schema, that is, based on library. If there are multiple schemas in the database instance, this setting can greatly improve the speed of slave replication. Usually, a single database with multiple tables is a more common situation.
That library-based concurrency is useless. The core idea is that the data of the concurrent submission of tables under different schemas will not affect each other, that is, the slave node can assign a thread similar to SQL to each of the different schemas in the relay log.
To replay the transactions submitted by the main library in the relay log, keep the data consistent with the main library.
In MySQL 5.7, group-commit-based parallel replication (Enhanced Multi-threaded Slaves) was introduced, setting the parameter slave_parallel_workers>0 and global.slave_parallel_type='LOGICAL_CLOCK',
It can support slave_parallel_workers worker threads to concurrently execute transactions submitted by the main library in the relay log under one schema. Its core idea: transactions submitted by a group can be played back in parallel (with binary log group commit);
Transactions with the same last_committed (different sequence_num) in the relay log of the slave machine can be executed concurrently.
Among them, the variable slave-parallel-type can have two values: the default value of DATABASE, the parallel replication method based on the library; LOGICAL_CLOCK: the parallel replication method based on group submission
MySQL 5.7 enables Enhanced Multi-Threaded Slave configuration:
# slave
slave-parallel-type=LOGICAL_CLOCK
slave-parallel-workers=16
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log_recovery=ON
So far, MySQL has completely solved the problem of replication delay, congratulations!
3. Reference documents
Official documentation: https://dev.mysql.com/doc/refman/5.7/en/replication-options-slave.html
Inside君的文章：http://www.ttlsa.com/mysql/mysql-5-7-enhanced-multi-thread-salve/