Interview - How to solve the MySQL deadlock problem?

There is a high probability that we will encounter deadlock problems when using MySQL, which is really a very headache. This article will introduce deadlocks, analyze and discuss common deadlock cases, and give some suggestions on how to avoid deadlocks as much as possible.

1. What is deadlock

Deadlock is a common problem in concurrent systems, and it also occurs in the concurrent read and write request scenarios of the database MySQL. When there are two or more transactions, both parties are waiting for the other party to release the locks they already hold, or because the lock sequence is inconsistent, causing a cycle of waiting for lock resources, a "deadlock" will occur. The common error message is Deadlock found when trying to get lock....

For example, transaction A holds X1 lock and applies for X2 lock, and transaction B holds X2 lock and applies for X1 lock. A and B transactions hold locks and apply for the locks held by each other to enter a circular wait, resulting in a deadlock.

As shown in the figure above, the resource requests of the four cars on the right have a loop phenomenon, that is, an infinite loop, which leads to a deadlock.

From the definition of deadlock, several elements of MySQL deadlock are:

1. two or more transactions

2. Each transaction already holds a lock and applies for a new lock

3. Lock resources can only be held by the same transaction at the same time or are incompatible

4. Transactions wait for each other circularly because of holding locks and applying for locks

2. InnoDB lock type

In order to analyze deadlocks, it is necessary for us to have an understanding of InnoDB's lock types.

The MySQL InnoDB engine implements the standard行级别锁：共享锁( S lock ) 和排他锁 ( X lock )

1. Different transactions can add S lock to the same row record at the same time.

2. If a transaction adds an X lock to a row record, other transactions cannot add an S lock or an X lock, resulting in lock waiting.

If transaction T1 holds the S lock of row r, when another transaction T2 requests the lock of r, it will do the following:

1. T2's request for S lock is immediately granted, and as a result, both T1 and T2 hold the S lock of row r

2. T2 request for X lock cannot be granted immediately

If T1 holds the X lock of r, then T2's request for the X and S locks of r cannot be granted immediately, and T2 must wait for T1 to release the X lock, because the X lock is not compatible with any lock. The compatibility of shared locks and exclusive locks is as follows:

2.1, gap lock (gap lock) 

Gap locks lock a gap to prevent insertion. Assuming that the index column has three values ​​of 2, 4, and 8, if you lock 4, you will also lock the two gaps (2,4) and (4,8) at the same time. Other transactions cannot insert records with index values ​​between these two gaps. However, gap locks have an exception:

1. If the index column is a unique index, only this record (only row lock) will be locked, and the gap will not be locked.

2. For a joint index and it is a unique index, if the where condition only includes a part of the joint index, then a gap lock will still be added.

2.2、next-key lock

The next-key lock is actually a combination of row lock + gap lock in front of this record. Assuming there are index values ​​10, 11, 13 and 20, then possible next-key locks include:

(negative infinity,10],(10,11],(11,13],(13,20],(20,positive infinity)

Under the RR isolation level, InnoDB uses next-key lock mainly to prevent 幻读problems.

2.3. Intention lock (Intention lock)

In order to support multi-granularity locking, InnoDB allows row locks and table locks to exist at the same time. In order to support locking operations at different granularities, InnoDB supports an additional lock method called Intention Lock. Intention lock divides locked objects into multiple levels. Intention lock means that transactions want to lock at a finer granularity. There are two types of intent locks:

1. Intentional shared lock ( IS ): the transaction intends to add shared locks to certain rows in the table

2. Intentional exclusive lock (IX): The transaction intends to exclusive lock some rows in the table

Since the InnoDB storage engine supports row-level locks, intent locks do not actually block any requests other than full table scans. The compatibility between table-level intent locks and row-level locks is as follows:

2.4, insert intention lock (Insert Intention lock) 

The insertion intention lock is a gap lock set before inserting a row of records. This lock releases a signal of an insertion mode, that is, when multiple transactions are inserted in the same index gap, they do not need to be inserted at the same position in the gap. wait for each other. Assuming that a column has index values ​​2 and 6, as long as the insertion positions of the two transactions are different (for example, transaction A inserts 3, and transaction B inserts 4), then they can be inserted at the same time.

2.5. Lock mode compatibility matrix

The horizontal is the lock that is held, and the vertical is the lock that is being requested:

3. Read the deadlock log 

Before analyzing specific cases, let's first understand how to read the deadlock log, and use the information in the deadlock log as much as possible to help us solve the deadlock problem.

The database scenario of the following test cases is as follows:MySQL 5.7 事务隔离级别为 RR

The table structure and data are as follows:

The test case is as follows:

 You can view the latest deadlock log by executing show engine innodb status.

3.1. The log analysis is as follows:

1.***** (1) TRANSACTION: TRANSACTION 2322, ACTIVE 6 sec starting index read

 The transaction number is 2322, active for 6 seconds, and starting index read indicates that the transaction status is reading data according to the index. Other common states are:

mysql tables in use 1 Indicates that the current transaction uses a table.

locked 1 Indicates that there is a table lock on the table, LOCK_IX for DML statements

LOCK WAIT 2 lock struct(s), heap size 1136, 1 row lock(s)

LOCK WAIT Indicates waiting for a lock, 2 lock struct(s) indicating that the length of the trx->trx_locks lock list is 2, and each linked list node represents a lock structure held by the transaction, including table locks, record locks, and self-increasing locks. In this use case, 2locks means IX lock and lock_mode X (Next-key lock)

1 row lock(s) Indicates the number of row record locks/gap locks held by the current transaction.

MySQL thread id 37, OS thread handle 140445500716800, query id 1234 127.0.0.1 root updating

MySQL thread id 37 Indicates that the ID of the thread executing the transaction is 37 (that is, show processlist; the displayed ID)

delete from student where stuno=5 Indicates the sql that is being executed by transaction 1. The more uncomfortable thing is  show engine innodb status that the complete sql cannot be viewed, and it usually shows the sql that is currently waiting for the lock.

 ***** (1) WAITING FOR THIS LOCK TO BE GRANTED:

RECORD LOCKS space id 11 page no 5 n bits 72 index idx_stuno of table cw****.****student trx id 2322 lock_mode X waiting

RECORD LOCKS means record locks. This item means that transaction 1 is waiting for the X lock of idx_stuno on the table student, which is actually Next-Key Lock in this case.

The log of transaction 2 is similar to the above analysis:

2.***** (2) HOLDS THE LOCK(S):

RECORD LOCKS space id 11 page no 5 n bits 72 index idx_stuno of table cw****.****student trx id 2321 lock_mode X

It shows that the insert into student(stuno,score) values(2,10) of transaction 2 holds Lock mode X with a=5

 LOCK_gap, but we can't see the delete from student where stuno=5;

This is also the root cause of the problem that it is difficult for DBAs to analyze deadlocks based on logs alone.

3.***** (2) WAITING FOR THIS LOCK TO BE GRANTED:

RECORD LOCKS space id 11 page no 5 n bits 72 index idx_stuno of table cw****.****student trx id 2321 lock_mode X locks gap before rec insert intention waiting

Indicates that the insert statement of transaction 2 is waiting to insert the intent lock lock_mode X locks gap before rec insert intention waiting ( LOCK_X + LOCK_REC_gap )

4. Classic case analysis

4.1. Transaction concurrent insert unique key conflict

The table structure and data are as follows:

The test case is as follows:

 The log analysis is as follows:

1. Transaction T2 insert into t7(id,a) values ​​(26,10) The statement insert is successful, holding a=10 排他行锁( Xlocks rec but no gap )

2. Transaction T1 insert into t7(id,a) values ​​(30,10), because the first insert of T2 has already inserted a=10 record, transaction T1 insert a=10 will cause a unique key conflict, need to apply for the only conflict Index plus S Next-key Lock (that is, lock mode S waiting) This is a 间隙锁gap area that will apply to lock (,10], (10,20].

3. Transaction T2 insert into t7(id,a) values ​​(40,9) The value of a=9 inserted by this statement is applied for in transaction T1,  gap 锁4-10之间so the second insert statement of transaction T2 needs to wait for  S-Next-key Lock 锁the release of transaction T1, in the log Lock_mode X locks gap before rec insert intention waiting is displayed in .

 4.2. The concurrent deadlock problem of update first and then insert

 The table structure is as follows, without data:

The test case is as follows:

 Deadlock analysis:
You can see that the records of the two transactions update do not exist, and they are obtained successively 间隙锁( gap 锁). The gap locks are compatible, so the update link will not be blocked. Both hold gap locks and then compete for inserts 意向锁. When there are other sessions holding the gap lock, the current session cannot apply for the insertion intention lock, resulting in a deadlock.

5. How to avoid deadlock as much as possible

1. Reasonably design the index, put the highly differentiated columns in front of the composite index, so that business SQL can pass through the index as much as possible 定位更少的行，减少锁竞争.

2. Adjust the execution sequence of business logic SQL to avoid update/delete SQL that holds locks for a long time in front of the transaction.

3. Avoid 大事务, try to split large transactions into multiple small transactions for processing, and the probability of lock conflicts in small transactions is also smaller.

4. to 固定的顺序access tables and rows. For example, for two transactions that update data, the order of transaction A updating data is 1, 2; the order of transaction B updating data is 2, 1. This is more likely to cause a deadlock.

5. In a system with relatively high concurrency, do not explicitly lock, especially in a transaction. Such as the select ... for update statement, if it is in a transaction （运行了 start transaction 或设置了autocommit 等于0）, then the found records will be locked.

6. Try 主键/索引to find records by going, range search increases the possibility of lock conflicts, and don't use the database to do some extra amount calculation work. For example, some programs use statements like "select ... where ... order by rand();". Since such statements do not use indexes, the data in the entire table will be locked.

7. Optimize SQL and table design to reduce situations where too many resources are used at the same time. For example, 减少连接的表, convert complex SQL  分解into multiple simple SQL.