For some systems with a large amount of data, the problems faced by the database are not only low query efficiency, but also long data storage time. Especially like a reporting system , the time spent on data import may be as long as several hours or a dozen hours a day. Therefore, it makes sense to optimize database insert performance. The most effective way is: merge + transaction + ordered data for the fastest INSERT operation.
After some performance tests on MySQL innodb, we found some methods to improve the efficiency of insert for your reference.
1. One SQL statement inserts multiple pieces of data.
Commonly used insert statements are:
| 1234 | INSERT INTO `insert_table` (`datetime`, `uid`, `content`, `type`) VALUES ('0', 'userid_0', 'content_0', 0);INSERT INTO `insert_table` (`datetime`, `uid`, `content`, `type`) VALUES ('1', 'userid_1', 'content_1', 1); |
changed to:
| 1 2 | INSERTINTO`insert_table`(`datetime`,`uid`,`content`,`type`) VALUES('0','userid_0','content_0',0),('1','userid_1','content_1',1); |
The modified insertion operation can improve the insertion efficiency of the program. The main reason for the high execution efficiency of the second SQL here is that the amount of logs after merging (MySQL's binlog and innodb transactions allow logs) is reduced, which reduces the amount and frequency of log flushing, thereby improving efficiency. By combining SQL statements, the number of SQL statement parsing can also be reduced, and the IO of network transmission can be reduced.
Here are some test comparison data, which are to import a single piece of data and convert it into a SQL statement for import, and test 100, 1,000, and 10,000 data records respectively. 
Reference: http://www.2cto.com/database/201606/520871.html