The problem of synchronization delay of mysql database from library

Execute show slave status on the slave server; you can view many synchronization parameters, the parameters we need to pay special attention to are as follows:
Master_Log_File: The name of the master server binary log file currently being read by the I/O thread in SLAVE
Read_Master_Log_Pos: In the current In the primary server binary log, the position that the I/O thread in SLAVE has read
Relay_Log_File: The name of the relay log file that the SQL thread is currently reading and executing
Relay_Log_Pos: In the current relay log, the SQL thread has read Location of fetch and execution Relay_Master_Log_File
: Name of the master server binary log file containing most recent events executed by the SQL thread Slave_IO_Running
: Whether the I/O thread was started and successfully connected to the master server The time gap, in seconds, between the slave SQL thread and the slave I/O thread.

1. Show slave status shows that the parameter Seconds_Behind_Master
is not 0, and this value may be very large
. 2. Show slave status shows that the parameters Relay_Master_Log_File and Master_Log_File show that the number of bin-log is very different, indicating that bin-log is in The slave database is not synchronized in time, so the recently executed bin-log is very different from the bin-log read by the current IO thread.
3. There are a large number of mysql-relay-log logs in the MySQL slave database data directory. After the log synchronization is completed It will be automatically deleted by the system, and there are a large number of logs, indicating that the master-slave synchronization delay is very serious

a, MySQL database master-slave synchronization delay principle
MySQL master-slave synchronization principle:
the master database writes binlog sequentially for write operations, the slave database single thread goes to the master database to sequentially read the "binlog of write operations", and the binlog obtained from the database is executed as it is locally (random write) to ensure that the master and slave data are logically consistent.
The master-slave replication of mysql is a single-threaded operation. The main library generates binlog for all DDL and DML. The binlog is written sequentially, so the efficiency is very high. Here, the Slave_SQL_Running thread of the slave implements the DDL and DML operations of the main library on the slave. The IO operations of DML and DDL are random, not sequential, and the cost is much higher. Other queries on the slave may also generate lock contention. Since Slave_SQL_Running is also single-threaded, a DDL card master needs to be executed for 10 minutes. Then all subsequent DDLs will wait for this DDL to be executed before continuing, which leads to a delay.
Some friends will ask: "The same DDL on the main library also needs to be executed for 10 minutes, why is the slave delayed?" The answer is that the master can be concurrent, but the Slave_SQL_Running thread cannot.

b. How is the master-slave synchronization delay of MySQL database generated?
When the TPS concurrency of the main database is high, the amount of DDL generated exceeds the range that a sql thread of the slave can bear, and then the delay occurs. Of course, there may be lock waiting with the large query statement of the slave.
The primary reason: the database is under too much pressure to read and write in the business, the CPU computing load is heavy, the network card load is heavy, and the random IO of the hard disk is too high
. Secondary reasons: the performance impact of reading and writing binlog, and network transmission delay.

c, MySQL database master-slave synchronization delay solution.

In terms of architecture
1. The implementation of the persistence layer of the business adopts a sub-database architecture, and the mysql service can be expanded in parallel to disperse the pressure.
2. Separate reading and writing of a single library, one master and multiple slaves, master writing and slave reading, dispersing pressure. In this way, the pressure of the slave library is higher than that of the master library, which protects the master library.
3. The service infrastructure adds the cache layer of memcache or redis between the business and mysql. Reduce mysql read pressure.
4. The mysql of different businesses are physically placed on different machines to spread the pressure.
5. Use a better hardware device than the main library as slave

To sum up, mysql pressure is small, and the delay will naturally become smaller.

hardware

1. Use a good server, for example, the performance of 4u is obviously better than that of 2u, and the performance of 2u is obviously better than that of 1u.
2. Use ssd or disk array or san for storage to improve the performance of random write.
3. The master and slave are guaranteed to be under the same switch and in a 10 Gigabit environment.
To sum up, the hardware is strong, and the delay will naturally become smaller. In short, the solution to reducing latency is to spend money and time.

mysql master-slave synchronization acceleration

1. sync_binlog is set to 0 on the slave side
2. --logs-slave-updates Updates received from the master server are not recorded in its binary log.
3. Directly disable binlog on the
slave side 4. On the slave side, if the storage engine used is innodb, innodb_flush_log_at_trx_commit =2

从文件系统本身属性角度优化
master端
修改linux、Unix文件系统中文件的etime属性, 由于每当读文件时OS都会将读取操作发生的时间回写到磁盘上,对于读操作频繁的数据库文件来说这是没必要的,只会增加磁盘系统的负担影响I/O性能。可以通过设置文件系统的mount属性,组织操作系统写atime信息,在linux上的操作为:
打开/etc/fstab,加上noatime参数
/dev/sdb1 /data reiserfs noatime 1 2
然后重新mount文件系统
#mount -oremount /data

PS:
主库是写,对数据安全性较高,比如sync_binlog=1,innodb_flush_log_at_trx_commit = 1 之类的设置是需要的
而slave则不需要这么高的数据安全,完全可以讲sync_binlog设置为0或者关闭binlog,innodb_flushlog也可以设置为0来提高sql的执行效率
1、sync_binlog=1 o
MySQL提供一个sync_binlog参数来控制数据库的binlog刷到磁盘上去。
默认,sync_binlog=0,表示MySQL不控制binlog的刷新,由文件系统自己控制它的缓存的刷新。这时候的性能是最好的,但是风险也是最大的。一旦系统Crash,在binlog_cache中的所有binlog信息都会被丢失。
如果sync_binlog>0,表示每sync_binlog次事务提交,MySQL调用文件系统的刷新操作将缓存刷下去。最安全的就是sync_binlog=1了,表示每次事务提交,MySQL都会把binlog刷下去,是最安全但是性能损耗最大的设置。这样的话,在数据库所在的主机操作系统损坏或者突然掉电的情况下,系统才有可能丢失1个事务的数据。
但是binlog虽然是顺序IO,但是设置sync_binlog=1,多个事务同时提交,同样很大的影响MySQL和IO性能。
虽然可以通过group commit的补丁缓解,但是刷新的频率过高对IO的影响也非常大。对于高并发事务的系统来说,
“sync_binlog”设置为0和设置为1的系统写入性能差距可能高达5倍甚至更多。
所以很多MySQL DBA设置的sync_binlog并不是最安全的1,而是2或者是0。这样牺牲一定的一致性,可以获得更高的并发和性能。
默认情况下,并不是每次写入时都将binlog与硬盘同步。因此如果操作系统或机器(不仅仅是MySQL服务器)崩溃,有可能binlog中最后的语句丢失了。要想防止这种情况,你可以使用sync_binlog全局变量(1是最安全的值,但也是最慢的),使binlog在每N次binlog写入后与硬盘同步。即使sync_binlog设置为1,出现崩溃时,也有可能表内容和binlog内容之间存在不一致性。

2、innodb_flush_log_at_trx_commit (这个很管用)
抱怨Innodb比MyISAM慢 100倍?那么你大概是忘了调整这个值。默认值1的意思是每一次事务提交或事务外的指令都需要把日志写入(flush)硬盘,这是很费时的。特别是使用电池供电缓存(Battery backed up cache)时。设成2对于很多运用,特别是从MyISAM表转过来的是可以的,它的意思是不写入硬盘而是写入系统缓存。
日志仍然会每秒flush到硬 盘,所以你一般不会丢失超过1-2秒的更新。设成0会更快一点,但安全方面比较差,即使MySQL挂了也可能会丢失事务的数据。而值2只会在整个操作系统 挂了时才可能丢数据。

3. The ls(1) command can be used to list the atime, ctime and mtime of a file.
The access time of the atime file is changed when the file is read or executed The create time of the ctime file is changed when the file is written, the owner, permissions or link settings are changed The modified time of the mtime file
is changed when the file is written ls -lc filename List file's ctime ls
-lu filename List file's atime ls -l filename List file's mtime stat filename List atime, mtime, ctime atime is not necessarily accessing the file It was modified later because: when using the ext3 file system, if the noatime parameter is used when mounting, the atime information will not be updated. These three time stamps are placed in the inode. If mtime and atime are modified, the inode will be changed. Since the inode is changed, the ctime will be changed. The reason why noatime is used in the mount option is that the file system does not want to do it. Too many modifications to improve read performance







Guess you like

Origin http://10.200.1.11:23101/article/api/json?id=326711304&siteId=291194637