mysql- insert select带来的锁问题

    当使用 insert...select...进行记录的插入时,如果select的表是innodb类型的,不论insert的表是什么类型的表,都会对select的表的纪录进行锁定。对于那些从oracle迁移过来的应用,需要特别的注意,因为oracle并不存在类似的问题,所以在oracle的应用中insert...select...操作非常的常见。例如:有时候会对比较多的纪录进行统计分析,然后将统计的中间结果插入到另外一个表,这样的操作因为进行的非常少,所以可能并没有设置相应的索引。如果迁移到mysql数据库后不进行相应的调整,那么在进行这个操作期间,对需要select的表实际上是进行的全表扫描导致的所有记录的锁定,将会对应用的其他操作造成非常严重的影响。

究其主要原因,是因为 mysql 在实现复制的机制时和 oracle 是不同的,如果不进行 select 表的锁定,则可能造成从数据库在恢复期间插入结果集的不同,造成主从数据的不一致。如果不采用主从复制,关闭 binlog 并不能避免对 select 纪录的锁定,某些文档中提到可以通过设置innodb_locks_unsafe_for_binlog来避免这个现象,当这个参数设置为true的时候,将不会对select的结果集加锁,但是这样的设置将可能带来非常严重的隐患。如果使用这个binlog进行从数据库的恢复,或者进行主数据库的灾难恢复,都将可能和主数据库的执行效果不同。

因此,推荐通过设置这个参数来避免 insert...select...导致的锁,如果需要进行可能会扫描大量数据的 insert...select 操作,推荐使用select...into outfile 和 load data infile 的组合来实现,这样是不会对纪录进行锁定的。


做一个测试来证明insert...select...会阻塞dml操作:

测试环境是主从复制的,所以不适合设置参数innodb_locks_unsafe_for_binlog

准备数据:

mysql> select count(*) from test;
+----------+
| count(*) |
+----------+
|   262142 |
+----------+
1 row in set (0.04 sec)

mysql> create table test_tmp as select * from test where 1=0;
Query OK, 0 rows affected (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 0

session 1:

mysql> insert into test_tmp select * from test;
Query OK, 262142 rows affected (8.76 sec)
Records: 262142  Duplicates: 0  Warnings: 0

session 2:

在session1还没有结束的时候来做dml操作:

mysql> delete from test limit 1;
Query OK, 1 row affected (8.27 sec) #运行了8.27秒,其中大部分时间都是等待锁释放

session 3:

mysql> show processlist;
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+
| Id | User | Host                | db   | Command     | Time  | State                                                         | Info                                    |
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+
|  2 | repl | 172.17.61.132:49246 | NULL | Binlog Dump | 19675 | Master has sent all binlog to slave; waiting for more updates | NULL                                    |
| 24 | root | localhost           | l5m  | Query       |     6 | Sending data                                                  | insert into test_tmp select * from test |
| 25 | root | localhost           | NULL | Query       |     0 | starting                                                      | show processlist                        |
| 26 | root | localhost           | l5m  | Query       |     5 | updating                                                      | delete from test limit 1                |
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+-----------------------------------------+
4 rows in set (0.00 sec)

再测试一下select...into outfile 会不会对表加锁?

mysql> select * from test into outfile '/u01/backup/test.sql';
Query OK, 262141 rows affected (0.23 sec)

同样多的数据用outfile导出,哇太快没法测,需要多些数据测试:

mysql> insert into test(name,create_time) select name,create_time from test;
Query OK, 262140 rows affected (12.13 sec)
Records: 262140  Duplicates: 0  Warnings: 0

mysql> insert into test(name,create_time) select name,create_time from test;
Query OK, 524280 rows affected (10.38 sec)
Records: 524280  Duplicates: 0  Warnings: 0

mysql> insert into test(name,create_time) select name,create_time from test;
Query OK, 1048560 rows affected (28.17 sec)
Records: 1048560  Duplicates: 0  Warnings: 0

mysql> insert into test(name,create_time) select name,create_time from test;
Query OK, 2097120 rows affected (1 min 1.68 sec)
Records: 2097120  Duplicates: 0  Warnings: 0

session 1:

mysql>  select * from test into outfile '/u01/backup/test1.sql';
Query OK, 4194240 rows affected (6.67 sec)

session 2:

mysql> delete from test limit 1;
Query OK, 1 row affected (0.11 sec)

删除操作是立即完成的,并没有等待select ...outfile完成

session 3:

mysql> show processlist;
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+
| Id | User | Host                | db   | Command     | Time  | State                                                         | Info                                                    |
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+
|  2 | repl | 172.17.61.132:49246 | NULL | Binlog Dump | 20425 | Master has sent all binlog to slave; waiting for more updates | NULL                                                    |
| 24 | root | localhost           | l5m  | Query       |     2 | Sending data                                                  | select * from test into outfile '/u01/backup/test1.sql' |
| 25 | root | localhost           | NULL | Query       |     0 | starting                                                      | show processlist                                        |
| 26 | root | localhost           | l5m  | Sleep       |     1 |                                                               | NULL                                                    |
+----+------+---------------------+------+-------------+-------+---------------------------------------------------------------+---------------------------------------------------------+
4 rows in set (0.12 sec)

没有看到任何的等待或者阻塞。

所以select ...outfile是不阻塞dml操作的,可以用select...into outfile 和 load data infile 的组合来代替insert...select完成插入操作。

猜你喜欢

转载自blog.csdn.net/jolly10/article/details/80095443
今日推荐