1 Scene restoration
1.1 Version information
MySQL版本：5.6.36-82.1-log
Mybatis-Plus的starter版本：3.3.2
存储引擎：InnoDB
1.2 Deadlock phenomenon
Student A used the com.baomidou.mybatisplus provided by Mybatis-Plus inthe production environment. extension.service.IService#saveOrUpdate(T, com.baomidou.mybatisplus.core.conditions.Wrapper) method (hereinafter referred to as B method ), in the concurrent scenario, the database reported the following error
2 Why is gap lock deadlock?
As shown in the figure above, the database reports a deadlock. There are thousands of deadlock scenarios. Why is it determined that method B is a deadlock caused by gap lock?
2.1 What is deadlock?
Two transactions wait for each other's locks, causing each other to block, resulting in a deadlock.
2.2 What is a gap lock?
-
Gap lock is a type of MySQL row lock. Different from Record lock, gap lock locks a gap.
-
The locking rules are as follows:
MySQL will look to the left for the first value that is smaller than the current index value, and to the right for the first value that is larger than the current index value (if there is none, it will be positive infinity), and lock this interval to prevent other transactions in this interval. Insert data.
2.3 Why does MySQL introduce gap locks?
Combined with Record lock to form Next-key lock, they work together to avoid phantom reads under the isolation level of repeatable read.
2.4 Gap lock deadlock analysis
In theory, an open source framework has been polished for many years and the methods provided should not cause such serious errors. However, the theory is only theoretical. The fact is that a deadlock occurred, so we started a round of in-depth investigation. First, let’s start with the source code of this method. The source code is as follows:
default boolean saveOrUpdate(T entity, Wrapper<T> updateWrapper) {
return this.update(entity, updateWrapper) || this.saveOrUpdate(entity);
}
Judging from the source code, this method does not follow the routine. The normal logic should be to execute the query first, modify it if it exists, and add it if it does not exist. However, this method performs the modification immediately. We wondered whether MySQL added any locks during modifications that caused the deadlock, so we found the DBA and obtained the latest deadlock log, that is, executed show engine innodb status , we found two key pieces of information:
*** (1) TRANSACTION:
...省略日志
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 347 n bits 80 index `PRIMARY` of table `database_name`.`table_name` trx id 71C lock_mode X locks gap before rec insert intention waiting
*** (2) TRANSACTION:
...省略日志
*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 0 page no 347 n bits 80 index `PRIMARY` of table `database_name`.`table_name` trx id 71D lock_mode X locks gap before rec insert intention waiting
A simple translation means that when transaction one acquires the insertion intention lock, it needs to wait for the gap lock (added by transaction two) to be released. At the same time, when transaction two acquires the insertion intention lock, it also waits for the gap lock to be released (transaction one added), **( This article does not discuss the locks added by MySQL during modification and insertion. We add gap locks during modifications and obtain insertion intention locks during insertion as known conditions) ** Then let's go back to method B. In a concurrent scenario, is it very big? The probability is that transaction one and transaction two are waiting for each other's gap lock, resulting in a deadlock.
Now that we have the theory, let's verify this scenario with real data.
2.5 Verify gap lock deadlock
- Prepare the following table structure (hereinafter referred to as verification one)
create table t_gap_lock(
id int auto_increment primary key comment '主键ID',
name varchar(64) not null comment '名称',
age int not null comment '年龄'
) comment '间隙锁测试表';
- Prepare the following table data
mysql> select * from t_gap_lock;
+----+------+-----+
| id | name | age |
+----+------+-----+
| 1 | 张三 | 18 |
| 5 | 李四 | 19 |
| 6 | 王五 | 20 |
| 9 | 赵六 | 21 |
| 12 | 孙七 | 22 |
+----+------+-----+
- We start transaction one and execute the following statement,Note that we have not submitted the transaction at this time
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 4;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
- At the same time, we open transaction two and execute the following statement,We also do not submit the transaction for transaction two
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 7;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
- Next we execute the following statement in transaction one
mysql> insert into t_gap_lock(id, name, age) value (7,'间隙锁7',27);
- We will find that transaction one is blocked, and then we execute the following statement to see the transaction currently being locked.
mysql> SELECT * FROM INFORMATION_SCHEMA.INNODB_LOCKS \G;
*************************** 1. row ***************************
lock_id: 749:0:360:3
lock_trx_id: 749
lock_mode: X,GAP
lock_type: RECORD
lock_table: `test`.`t_gap_lock`
lock_index: `PRIMARY`
lock_space: 0
lock_page: 360
lock_rec: 3
lock_data: 5
*************************** 2. row ***************************
lock_id: 74A:0:360:3
lock_trx_id: 74A
lock_mode: X,GAP
lock_type: RECORD
lock_table: `test`.`t_gap_lock`
lock_index: `PRIMARY`
lock_space: 0
lock_page: 360
lock_rec: 3
lock_data: 5
2 rows in set (0.00 sec)
According to lock_type and lock_mode, we can clearly see that the lock type is row lock and the lock mode is gap lock.
- At the same time, we execute the following statement in transaction two
insert into t_gap_lock(id, name, age) value (4,'间隙锁4',24);
- As soon as the above statement is executed, the database immediately reports a deadlock and rolls back transaction 2 (*** WE ROLL BACK TRANSACTION (2) can be seen in the deadlock log)
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
At this point, careful students will find that, eh, you deliberately created a gap and let two transactions insert data into each other's gaps. This is too deliberate. This scenario basically does not happen in production environments. Yes, how can such a scenario exist in a production environment? The above data is just to let everyone intuitively see the deadlock process of gap lock. Next, let’s have another set of data, which we call verification two.
- We still verify the table structure and data, and we perform such an operation. First, we start transaction one and perform the following operations, but the transaction is still not submitted.
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 4;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
- At the same time, we start transaction two and perform the same operation as transaction one. We will be surprised to find that it is also successful.
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 4;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
- So we performed the following operations in transaction one, and we were surprised to find that transaction one was blocked.
insert into t_gap_lock(id, name, age) value (4,'间隙锁4',24);
- While transaction one was blocked, we executed the same statement in transaction two, and we found that the database immediately reported a deadlock.
insert into t_gap_lock(id, name, age) value (4,'间隙锁4',24);
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
Verification 2 completely reproduces the online deadlock process, that is, transaction 1 executes the update statement first, transaction 2 also executes the update statement at the same time, and then executes the primary key query statement as soon as the transaction finds that it has not been updated. It is found that Indeed, there is not, so the insert statement is executed, but the insertion must first acquire the insertion intention lock. When acquiring the insertion intention lock, it is found that the gap has been locked by transaction two, so the transaction starts waiting for transaction two to release the gap lock. In the same way, Transaction 2 also performs the above operation, which eventually causes transaction 1 and transaction 2 to wait for each other to release the gap lock, eventually leading to a deadlock.
Verification 2 also illustrates a problem, that is, gap lock locking is non-mutually exclusive, that is, after transaction locks gap A, transaction 2 can still lock gap A.
3 How to solve it?
3.1 Turn off the gap lock (not recommended)
-
Lower the isolation level, for example to read-committed.
-
Directly modify my.cnf, change the switch, innodb_locks_unsafe_for_binlog to 1, and the default is 0 to enable it.
PS: The above method is only applicable to current business scenarios where the problem of phantom reading is not really concerned about.
3.2 Customize saveOrUpdate method (recommended)
RecommendationWrite your ownA saveOrUpdate method. Of course, you can also directly use the saveOrUpdate method provided by Mybatis-Plus. However, according to the source code, there will be There are many additional reflection operations, and transactions are also added. As we all know, MySQL single table operations do not require transactions at all, which will increase additional overhead.
@Transactional(
rollbackFor = {Exception.class}
)
public boolean saveOrUpdate(T entity) {
if (null == entity) {
return false;
} else {
Class<?> cls = entity.getClass();
TableInfo tableInfo = TableInfoHelper.getTableInfo(cls);
Assert.notNull(tableInfo, "error: can not execute. because can not find cache of TableInfo for entity!", new Object[0]);
String keyProperty = tableInfo.getKeyProperty();
Assert.notEmpty(keyProperty, "error: can not execute. because can not find column for id from entity!", new Object[0]);
Object idVal = ReflectionKit.getFieldValue(entity, tableInfo.getKeyProperty());
return !StringUtils.checkValNull(idVal) && !Objects.isNull(this.getById((Serializable)idVal)) ? this.updateById(entity) : this.save(entity);
}
}
4 Expand
4.1 What happens if two transactions modify a row that exists?
In verification two, the two transactions modified non-existing rows, and both of them successfully added gap locks. If the two transactions modified existing rows, would MySQL still add gap locks? Or reduce the gap lock from locking the gap to locking a row? With doubts, we perform the following data verification. We still use the table and data from verification 1.
- First we open the transaction and execute the following statement
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 1;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
- We open transaction two again, execute the same statement, and find that transaction two has been blocked.
mysql> begin;
Query OK, 0 rows affected (0.00 sec)
mysql> update t_gap_lock t set t.age = 25 where t.id = 1;
- At this time, we execute the following statement to see the transactions currently being locked.
mysql> SELECT * FROM INFORMATION_SCHEMA.INNODB_LOCKS \G;
*************************** 1. row ***************************
lock_id: 75C:0:360:2
lock_trx_id: 75C
lock_mode: X
lock_type: RECORD
lock_table: `test`.`t_gap_lock`
lock_index: `PRIMARY`
lock_space: 0
lock_page: 360
lock_rec: 2
lock_data: 1
*************************** 2. row ***************************
lock_id: 75B:0:360:2
lock_trx_id: 75B
lock_mode: X
lock_type: RECORD
lock_table: `test`.`t_gap_lock`
lock_index: `PRIMARY`
lock_space: 0
lock_page: 360
lock_rec: 2
lock_data: 1
2 rows in set (0.00 sec)
According to lock_type and lock_mode, we see that the lock added by transactions one and two becomes Record Lock, and no gap lock is added. Based on the above data, it is verified that MySQL will add Record Lock to the row when modifying the existing data, which is different from gap lock. What is important is that the lock is mutually exclusive, that is, different transactions cannot add Record Lock to the same row of records at the same time.
5 Conclusion
Although this method provided by Mybatis-Plus may cause deadlock, it is still undeniable that it is a very excellent enhanced framework. The lambda writing method it provides greatly improves our development efficiency in daily work, so everything is smooth. To use duality, we should adhere to a dialectical attitude, try familiar methods, and use unfamiliar methods with caution.
The above is the whole process of our gap lock deadlock analysis in the production environment. If you feel that this article has given you a little understanding of gap locks and gap lock deadlocks, don’t forget to click three times in one click and support Zhuanzhuan technology. Zhuanzhuan Technology will bring more production practice and exploration to everyone in the future.
Zhuanzhuan is a technical learning and exchange platform for R&D centers and industry partners, regularly sharing frontline practical experience and cutting-edge technical topics in the industry.
Follow the public accounts "Zhuanzhuan Technology" (comprehensive), "Zhuanzhuan FE" (focused on FE), "Zhuanzhuan QA" (focused on QA), and more useful practices, Welcome to communicate and share~