[Mysql] is too large sql file import is too slow to solve the problem

MySQL disk write policy control and data security are two key parameters:  innodb_flush_log_at_trx_commit  and  sync_binlog

Parameters: innodb_flush_log_at_trx_commit

If set to 0, log buffer once per second to write the log file, the log file and simultaneously flush (flushed to disk) operation. In this mode, when the transaction commits, do not take the initiative to trigger actions written to disk;

If set to 1, in this mode, each time a transaction is committed, mysql will be the log buffer data is written in the log file, the log file and simultaneously flush (flushed to disk) operation;

If set to 2, in this mode, each time a transaction is committed, mysql will be the log buffer data is written to the log file, but at the same time the log file will not be flush (flushed to disk) operations per second will the implementation of a log file flush (flushed to disk) operation.

Note: Due to the process of scheduling problems, we can not guarantee "executed every second flush (flushed to disk) operation" 100% "is performed once every second."

 

Parameters: sync_binlog

If N = 0, as the operating system refresh mechanism as other documents, mysql will not sync to disk to go, but rely on the operating system to refresh the binary log.

If N> 0, mysql N times at each write binary log binary log, it calls fdatasync () function to synchronize the binary log binary log to disk to go.

Note: If autocommit is enabled, then each sentence statement there will be a write operation; otherwise, each transaction corresponds to a write operation.

Figure:

 

performance:

Test Scenario 1:

innodb_flush_log_at_trx_commit=2
sync_binlog=1000

Test Scenario 2:

innodb_flush_log_at_trx_commit=1
sync_binlog=1000

Test Scenario 3:

innodb_flush_log_at_trx_commit=1
sync_binlog=1

Test Scenario 4:

innodb_flush_log_at_trx_commit=1
sync_binlog=1000

Test Scenario 5:

innodb_flush_log_at_trx_commit=2
sync_binlog=1000

 

innodb_flush_log_at_trx_commit sync_binlog                          TPS                                
1000 2 41000
1000 1 33000
1 1 26000
1000 1 33000

 

 

 

 

This shows that:

Ⅰ, when innodb_flush_log_at_trx_commit = 1 and sync_binlog = 1, the worst performance of a write operation;

Ⅱ, when innodb_flush_log_at_trx_commit = 2 and sync_binlog = 2, the writing operation for maximum performance;

 

Safety:

When innodb_flush_log_at_trx_commit is set to 0, mysqld process will lead to the collapse of the last second of all transaction data is lost.

When the case innodb_flush_log_at_trx_commit and sync_binlog are most secure when 1, mysqld process crash or crash the server, binary log only up to a transaction may be lost.

When innodb_flush_log_at_trx_commit set to 2, only in case of power failure or server crash, all transactions last second before the data may be lost.

Guess you like

Origin www.cnblogs.com/ttkl/p/12023640.html