Mysql deadlock analysis Case (a)

Overview

Xiao Bian encounter problems Deadlock at work, this is terrible for programmers, especially for a never encountered this problem is really small series is shivering, but the total solution to the problem, so we should step by step, step by step investigation and analysis, this is the growth, ha ha.

problem

To be able to do business from small series, and to a complete description of the problem. Xiao Bian simulation terrible problem as follows:

Mysql version 8.0

-- 创建表结构
CREATE TABLE t(
    `id` int NOT NULL AUTO_INCREMENT,
    `s_id` int(11) NOT NULL COMMENT 'student_id 学生Id', 
    `name` varchar(40) COLLATE utf8_bin DEFAULT NULL COMMENT '姓名',
  PRIMARY KEY (`id`),
  KEY `idx_s_id` (`s_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_bin ;

-- 初始化数据
INSERT INTO t(s_id,name)  VALUES(1,'name1'),(2,'name2'),(3,'name3'),(4,'name4'),(5,'name5'),(6,'name6'),(7,'name7'),(8,'name8'),(9,'name9'),(10,'name10');

session1	session2
start transaction	-
update t set name='cName11' where s_id=11； ## ==s_id=11不存在==	start transaction
-	update t set name='cName12' where s_id=12； ## s_id=12不存在
INSERT INTO t(s_id,name) VALUES(11,'name11') ## ==waitting	-
-	INSERT INTO t(s_id,name) VALUES(12,'name12')
-	ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction

Well, after been to the steps above, there has been a terrible Deadlock, I do use the command View SHOW ENGINE InnoDB status \G;

mysql> SHOW ENGINE InnoDB status \G;
*************************** 1. row ***************************
  Type: InnoDB
  Name:
Status:
=====================================
2019-07-29 10:59:40 0x7f47a0117700 INNODB MONITOR OUTPUT
=====================================
Per second averages calculated from the last 16 seconds
-----------------
BACKGROUND THREAD
-----------------
srv_master_thread loops: 57 srv_active, 0 srv_shutdown, 27956 srv_idle
srv_master_thread log flush and writes: 0
----------
SEMAPHORES
----------
OS WAIT ARRAY INFO: reservation count 58
OS WAIT ARRAY INFO: signal count 63
RW-shared spins 19, rounds 30, OS waits 11
RW-excl spins 13, rounds 51, OS waits 0
RW-sx spins 1, rounds 1, OS waits 0
Spin rounds per wait: 1.58 RW-shared, 3.92 RW-excl, 1.00 RW-sx
------------------------
LATEST DETECTED DEADLOCK
------------------------
2019-07-29 10:56:58 0x7f47a0117700
*** (1) TRANSACTION:
TRANSACTION 98735, ACTIVE 28 sec inserting
mysql tables in use 1, locked 1
LOCK WAIT 3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1
MySQL thread id 9, OS thread handle 139945604609792, query id 1491 localhost root update
INSERT INTO t(s_id,name) VALUES(11,'name11')
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98735 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

*** (2) TRANSACTION:
TRANSACTION 98734, ACTIVE 40 sec inserting
mysql tables in use 1, locked 1
3 lock struct(s), heap size 1136, 2 row lock(s), undo log entries 1
MySQL thread id 8, OS thread handle 139945604904704, query id 1492 localhost root update
INSERT INTO t(s_id,name) VALUES(12,'name12')
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98734 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98734 lock_mode X insert intention waiting
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

*** WE ROLL BACK TRANSACTION (2)
------------
TRANSACTIONS
------------
Trx id counter 98736
Purge done for trx's n:o < 98721 undo n:o < 15 state: running but idle
History list length 74
LIST OF TRANSACTIONS FOR EACH SESSION:
---TRANSACTION 421420599174496, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421420599173592, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421420599172688, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 421420599170880, not started
0 lock struct(s), heap size 1136, 0 row lock(s)
---TRANSACTION 98735, ACTIVE 190 sec
4 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 1
MySQL thread id 9, OS thread handle 139945604609792, query id 1491 localhost root
TABLE LOCK table `jackpot1`.`t` trx id 98735 lock mode IX
RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98735 lock_mode X
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98735 lock_mode X insert intention
Record lock, heap no 1 PHYSICAL RECORD: n_fields 1; compact format; info bits 0
 0: len 8; hex 73757072656d756d; asc supremum;;

RECORD LOCKS space id 136 page no 5 n bits 80 index idx_s_id of table `jackpot1`.`t` trx id 98735 lock_mode X locks gap before rec
Record lock, heap no 12 PHYSICAL RECORD: n_fields 2; compact format; info bits 0
 0: len 4; hex 8000000b; asc     ;;
 1: len 4; hex 8000000b; asc     ;;

--------
FILE I/O
--------
I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
I/O thread 1 state: waiting for completed aio requests (log thread)
I/O thread 2 state: waiting for completed aio requests (read thread)
I/O thread 3 state: waiting for completed aio requests (read thread)
I/O thread 4 state: waiting for completed aio requests (read thread)
I/O thread 5 state: waiting for completed aio requests (read thread)
I/O thread 6 state: waiting for completed aio requests (write thread)
I/O thread 7 state: waiting for completed aio requests (write thread)
I/O thread 8 state: waiting for completed aio requests (write thread)
I/O thread 9 state: waiting for completed aio requests (write thread)
Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
 ibuf aio reads:, log i/o's:, sync i/o's:
Pending flushes (fsync) log: 0; buffer pool: 0
18651 OS file reads, 1447 OS file writes, 760 OS fsyncs
0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
-------------------------------------
INSERT BUFFER AND ADAPTIVE HASH INDEX
-------------------------------------
Ibuf: size 1, free list len 105, seg size 107, 2 merges
merged operations:
 insert 18, delete mark 0, delete 0
discarded operations:
 insert 0, delete mark 0, delete 0
Hash table size 34679, node heap has 2 buffer(s)
Hash table size 34679, node heap has 2 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 1 buffer(s)
Hash table size 34679, node heap has 0 buffer(s)
Hash table size 34679, node heap has 2 buffer(s)
Hash table size 34679, node heap has 5 buffer(s)
0.00 hash searches/s, 0.00 non-hash searches/s
---
LOG
---
Log sequence number          1130751648
Log buffer assigned up to    1130751648
Log buffer completed up to   1130751648
Log written up to            1130751648
Log flushed up to            1130751648
Added dirty pages up to      1130751648
Pages flushed up to          1130751648
Last checkpoint at           1130751648
427 log i/o's done, 0.00 log i/o's/second
----------------------
BUFFER POOL AND MEMORY
----------------------
Total large memory allocated 137363456
Dictionary memory allocated 597484
Buffer pool size   8192
Free buffers       1024
Database pages     7156
Old database pages 2621
Modified db pages  0
Pending reads      0
Pending writes: LRU 0, flush list 0, single page 0
Pages made young 21371, not young 75108
0.00 youngs/s, 0.00 non-youngs/s
Pages read 18577, created 194, written 796
0.00 reads/s, 0.00 creates/s, 0.00 writes/s
Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000
Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
LRU len: 7156, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
--------------
ROW OPERATIONS
--------------
0 queries inside InnoDB, 0 queries in queue
0 read views open inside InnoDB 
Process ID=1, Main thread ID=139945114007296 , state=sleeping
Number of rows inserted 226, updated 459, deleted 71, read 7457495
0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
----------------------------
END OF INNODB MONITOR OUTPUT
============================

1 row in set (0.00 sec)

ERROR:
No query specified

We keywords LATEST DETECTED DEADLOCKbegan to see discovery

• (1) TRANSACTION: (1) WAITING FOR THIS LOCK TO BE GRANTED
• (2) TRANSACTION: (2) HOLDS THE LOCK(S): (2) WAITING FOR THIS LOCK TO BE GRANTED:
  As it can be seen in the execution INSERTtime of the occurrence of Deadlock statement, but still unable to determine why a problem, then we table performance_schema.data_locksby looking at the

-- 第一次执行INSERT时执行，打印lock的情况
mysql> SELECT `ENGINE`,THREAD_ID,EVENT_ID,OBJECT_SCHEMA,OBJECT_NAME,INDEX_NAME,OBJECT_INSTANCE_BEGIN,LOCK_TYPE,LOCK_MODE,LOCK_STATUS,LOCK_DATA FROM performance_schema.data_locks ;
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+-----------+-------------+------------------------+
| ENGINE | THREAD_ID | EVENT_ID | OBJECT_SCHEMA | OBJECT_NAME | INDEX_NAME | OBJECT_INSTANCE_BEGIN | LOCK_TYPE | LOCK_MODE | LOCK_STATUS | LOCK_DATA              |
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+-----------+-------------+------------------------+
| INNODB |        48 |      119 | jackpot1      | t           | NULL       |       139945536711608 | TABLE     | IX        | GRANTED     | NULL                   |
| INNODB |        48 |      119 | jackpot1      | t           | idx_s_id   |       139945536708664 | RECORD    | X         | GRANTED     | supremum pseudo-record |
| INNODB |        47 |      150 | jackpot1      | t           | NULL       |       139945536705784 | TABLE     | IX        | GRANTED     | NULL                   |
| INNODB |        47 |      150 | jackpot1      | t           | idx_s_id   |       139945536702744 | RECORD    | X         | GRANTED     | supremum pseudo-record |
| INNODB |        47 |      151 | jackpot1      | t           | PRIMARY    |       139945536703776 | RECORD    | X         | GRANTED     | supremum pseudo-record |
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+-----------+-------------+------------------------+
5 rows in set (0.00 sec)

-- 第二次执行INSERT时执行，打印lock的情况
mysql> SELECT `ENGINE`,THREAD_ID,EVENT_ID,OBJECT_SCHEMA,OBJECT_NAME,INDEX_NAME,OBJECT_INSTANCE_BEGIN,LOCK_TYPE,LOCK_MODE,LOCK_STATUS,LOCK_DATA FROM performance_schema.data_locks ;
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+--------------------+-------------+------------------------+
| ENGINE | THREAD_ID | EVENT_ID | OBJECT_SCHEMA | OBJECT_NAME | INDEX_NAME | OBJECT_INSTANCE_BEGIN | LOCK_TYPE | LOCK_MODE          | LOCK_STATUS | LOCK_DATA              |
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+--------------------+-------------+------------------------+
| INNODB |        47 |      150 | jackpot1      | t           | NULL       |       139945536705784 | TABLE     | IX                 | GRANTED     | NULL                   |
| INNODB |        47 |      150 | jackpot1      | t           | idx_s_id   |       139945536702744 | RECORD    | X                  | GRANTED     | supremum pseudo-record |
| INNODB |        47 |      151 | jackpot1      | t           | PRIMARY    |       139945536703776 | RECORD    | X                  | GRANTED     | supremum pseudo-record |
| INNODB |        47 |      152 | jackpot1      | t           | PRIMARY    |       139945536704120 | RECORD    | X,GAP              | GRANTED     | 14                     |
| INNODB |        47 |      152 | jackpot1      | t           | idx_s_id   |       139945536704464 | RECORD    | X,INSERT_INTENTION | GRANTED     | supremum pseudo-record |
| INNODB |        47 |      152 | jackpot1      | t           | idx_s_id   |       139945536704808 | RECORD    | X,GAP              | GRANTED     | 12, 14                 |
+--------+-----------+----------+---------------+-------------+------------+-----------------------+-----------+--------------------+-------------+------------------------+
6 rows in set (0.00 sec)

• Analysis of the first performance

  We found EVENT_ID 119 and the first table-level Implementation IX Lock 150 to perform the update produced and row-level implementation of Next-Key Lock, a field where the LOCK_DATAcontent supremum pseudo-recordis very important, I passed in Mysql official document queries have meaning on

For the last interval, the next-key lock locks the gap above the largest
value in the index and the “supremum” pseudo-record having a value higher
than any value actually in the index. The supremum is not a real index
record, so, in effect, this next-key lock locks only the gap following the
largest index value.

Google translate what we

对于最后一个间隔，下一个键锁定将间隙锁定在索引中最大值之上，而“ supremum ”
伪记录的值高于索引中实际的任何值。supremum不是真正的索引记录，因此，实际上，
此下一键锁定仅锁定最大索引值之后的间隙。

That is the value of the pseudo supremun recording than any actual value of the index, it means, in the rear end of the maximum value of the index, i.e. the nature of Next-Key Lock
as follows according to the definition of the Next-Key Locks

A next-key lock is a combination of a record lock on the index record and a gap lock on the gap before the index record.

InnoDB performs row-level locking in such a way that when it searches or scans a table index, it sets shared or exclusive locks on the index records it encounters. Thus, the row-level locks are actually index-record locks. A next-key lock on an index record also affects the “gap” before that index record. That is, a next-key lock is an index-record lock plus a gap lock on the gap preceding the index record. If one session has a shared or exclusive lock on record R in an index, another session cannot insert a new index record in the gap immediately before R in the index order.

Note here saying, we want to draw focus, because the analysis behind the need to use
If one session has a shared or exclusive lock on record R in an index, another session cannot insert a new index record in the gap immediately before R in the index order.
means that

如果一个会话R在索引中具有共享或独占锁定记录 ，则另一个会话不能R在索引顺序之前的间隙中插入新索引记录 。  

At this point we also understand why the first INSERT why in a wait state, because INSERT time to wait for the release of Session2 update X locks; if you wait too long Mysql have self-protection tips overtime to give upERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction

• Second performance analysis
  we found that the implementation of the second INSERT directly Deadlock, where we also understand why, because Session1 INSERT waiting for an X-lock release Session2 update, and Session2 INSERT waiting Session1 update X-lock release, which constitutes a typical deadlock. And there will be Mysql ROLLBACK mechanism to roll back to any one of Session, which means the release of one of the update hold X lock, so that another Session completed successfully.

Oh, and here's record EVENT_ID 152 LOCK_DATAfield contents 12, 14and what does it mean?
A: non-unique index value corresponding to the index is a unique feature of the lock GAP Mysql official documents to be investigated, i.e., 12 denotes s_id, 14 denotes id,

Well, this time talking about this Deadlock problem has been solved, we have to summarize it.

• 1 search engine has encountered a problem requires a combination of experience and a step by step analysis, patience not to give up.
• 2 projects in the usual try not to update records that do not exist