When insert...select... is used to insert records, if the select table is of innodb type, no matter what type of table the insert table is, the records of the select table will be locked. For those applications migrated from oracle, special attention is required, because oracle does not have similar problems, so insert...select... operations are very common in oracle applications. For example, sometimes statistical analysis is performed on a large number of records, and then the intermediate results of the statistics are inserted into another table. Because this operation is performed very little, the corresponding index may not be set. If the corresponding adjustment is not made after migrating to the mysql database, during this operation, the locking of all records caused by the full table scan of the table that needs to be selected will actually cause very serious problems to other operations of the application. influence.
The main reason is that mysql is different from oracle in implementing the replication mechanism. If the select table is not locked, the result set inserted from the database during recovery may be different, resulting in inconsistency of master-slave data. If master-slave replication is not used, closing binlog cannot avoid locking of select records. Some documents mention that this phenomenon can be avoided by setting innodb_locks_unsafe_for_binlog. When this parameter is set to true, it will not lock the select result set. Locked, but such a setting may bring very serious hidden dangers. If you use this binlog to restore the slave database or perform disaster recovery of the master database, the execution effect of the master database may be different from that of the master database.
The main reason is that mysql is different from oracle in implementing the replication mechanism. If the select table is not locked, the result set inserted from the database during recovery may be different, resulting in inconsistency of master-slave data. If master-slave replication is not used, closing binlog cannot avoid locking of select records. Some documents mention that this phenomenon can be avoided by setting innodb_locks_unsafe_for_binlog. When this parameter is set to true, it will not lock the select result set. Locked, but such a setting may bring very serious hidden dangers. If you use this binlog to restore the slave database or perform disaster recovery of the master database, the execution effect of the master database may be different from that of the master database.
Therefore, it is recommended to set this parameter to avoid the lock caused by insert...select.... If you need to perform insert...select operations that may scan a large amount of data, it is recommended to use select...into outfile and load data infile. The combination is implemented, so that the record will not be locked.
Do a test to prove that insert...select... blocks the dml operation:
The test environment is master-slave replication, so it is not suitable to set the parameter innodb_locks_unsafe_for_binlog
Prepare data:
mysql> select count(*) from test; +----------+ | count(*) | +----------+ | 262142 | +----------+ 1 row in set (0.04 sec) mysql> create table test_tmp as select * from test where 1=0; Query OK, 0 rows affected (0.02 sec) Records: 0 Duplicates: 0 Warnings: 0
session 1:
mysql> insert into test_tmp select * from test; Query OK, 262142 rows affected (8.76 sec) Records: 262142 Duplicates: 0 Warnings: 0
session 2:
Do the dml operation when session1 is not over:
mysql> delete from test limit 1; Query OK, 1 row affected (8.27 sec) #Running for 8.27 seconds, most of which is waiting for the lock to be released
session 3:
mysql> show processlist; +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+ | 2 | repl | 172.17.61.132:49246 | NULL | Binlog Dump | 19675 | Master has sent all binlog to slave; waiting for more updates | NULL | | 24 | root | localhost | l5m | Query | 6 | Sending data | insert into test_tmp select * from test | | 25 | root | localhost | NULL | Query | 0 | starting | show processlist | | 26 | root | localhost | l5m | Query | 5 | updating | delete from test limit 1 | +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+ 4 rows in set (0.00 sec)
Test again if select...into outfile will lock the table?
mysql> select * from test into outfile '/u01/backup/test.sql'; Query OK, 262141 rows affected (0.23 sec)
The same amount of data is exported with outfile. Wow, it is too fast to test. More data testing is required:
mysql> insert into test(name,create_time) select name,create_time from test; Query OK, 262140 rows affected (12.13 sec) Records: 262140 Duplicates: 0 Warnings: 0 mysql> insert into test(name,create_time) select name,create_time from test; Query OK, 524280 rows affected (10.38 sec) Records: 524280 Duplicates: 0 Warnings: 0 mysql> insert into test(name,create_time) select name,create_time from test; Query OK, 1048560 rows affected (28.17 sec) Records: 1048560 Duplicates: 0 Warnings: 0 mysql> insert into test(name,create_time) select name,create_time from test; Query OK, 2097120 rows affected (1 min 1.68 sec) Records: 2097120 Duplicates: 0 Warnings: 0
session 1:
mysql> select * from test into outfile '/u01/backup/test1.sql'; Query OK, 4194240 rows affected (6.67 sec)
session 2:
mysql> delete from test limit 1; Query OK, 1 row affected (0.11 sec)
The delete operation is done immediately and does not wait for select...outfile to complete
session 3:
mysql> show processlist; +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+ | Id | User | Host | db | Command | Time | State | Info | +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+ | 2 | repl | 172.17.61.132:49246 | NULL | Binlog Dump | 20425 | Master has sent all binlog to slave; waiting for more updates | NULL | | 24 | root | localhost | l5m | Query | 2 | Sending data | select * from test into outfile '/u01/backup/test1.sql' | | 25 | root | localhost | NULL | Query | 0 | starting | show processlist | | 26 | root | localhost | l5m | Sleep | 1 | | NULL | +----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+ 4 rows in set (0.12 sec)
Did not see any waiting or blocking.
So select...outfile does not block the dml operation, and the combination of select...into outfile and load data infile can be used instead of insert...select to complete the insert operation.