When developing multi-user, database-driven applications, the biggest difficulty is: on the one hand, to maximize the use of concurrent database access, on the one hand, to ensure that each user can read and modify data in a consistent manner, for this reason Locking mechanism.
6.1 What is a lock
The lock mechanism is used to manage concurrent access to shared resources. InnoDB not only locks table data at the row level, but also locks in multiple other places within the database, allowing concurrent access to many different resources. For example, operating the LRU list in the buffer pool, deleting and adding elements on the mobile LRU list, etc.
The implementation of InnoDB locks is very similar to the Oracle database, providing consistent non-locking reads, row-level lock support, row-level locks have no associated additional overhead, and can get concurrency and consistency.
6.2 Lock and latch
Latches are generally called latches (lightweight), because they require a very short lock time, and if the duration is long, the performance will be poor. In InnoDB, latch can be divided into mutex and rwlock (read-write lock). The purpose is to ensure the accuracy of concurrent threads operating critical resources, and there is usually no deadlock verification.
lock generally locks objects in the database, such as tables, pages, and rows. And the lock object is only released after the transaction commit or rollback.
| lock | latch | |
| Object | Business | Thread |
| protection | Database memory | Memory data structure |
| duration | The whole transaction process | Critical resource |
| mode | Row lock, table lock, intention lock | Read-write lock, mutex lock |
| Deadlock | Deadlock detection and processing through wait for graph and time out mechanisms | No deadlock detection and processing mechanism, only through the order of application lock to ensure that no deadlock occurs |
| Exist in | lock manager hash table | Object of each data structure |
6.3 Locks in the InnoDB storage engine
6.3.1 Types of locks
InnoDB implements the following two standard row-level locks:
- Shared lock, S Lock, allows transactions to read a row of data
- Exclusive lock, X Lock, allows transactions to delete or update a row of data
Both S and X locks are row locks, and compatibility refers to the compatibility of records on the same row.
InnoDB supports multi-granularity locking, which supports row-level locks and table-level locks at the same time, that is, intent locks, intent locks divide the locked object into multiple levels, and intent locks mean that transactions want more fine-grained Lock it.
If you need to add an X lock to the record r on the page, you need to intent lock IX on the database A, table, and page, and finally X lock on r. If any of these parts cause a wait, then the operation needs to wait for the coarse-grained lock to complete. InnoDB supports intent locks which are relatively simple. Intent locks are table-level locks, mainly to reveal the type of locks that will be requested in the next row in a transaction.
- Intent shared lock (IS Lock), the transaction wants to obtain a shared lock of a few rows in the table
- Intent exclusive lock (IX Lock), the transaction wants to obtain an exclusive lock of a few rows in the table
InnoDB supports row-level locks, so intent locks will not block any requests other than full table scans.
| IS | IX | S | X | |
| IS | compatible | Arrow Rong | compatible | Not compatible |
| IX | compatible | compatible | Not compatible | Not compatible |
| S | compatible | Not compatible | compatible | Not compatible |
| X | Not compatible | Not compatible | Not compatible | Not compatible |
6.3.2 Consistent non-locking read
Consistent non-locking read (consistent non-locking read) refers to InnoDB through multi-version concurrency control (multi versioning Concurrency control) to read the current row of data in the database at the time of execution. If at this time, the read row is performing delete or update, then the read operation will not wait for the lock on the row to be released. On the contrary, InnoDB will read a snapshot data of the row (this implementation is done by undo segment ), And the undo segment is used to roll back data in a transaction, so there is no additional overhead. In addition, reading snapshot data does not need to be locked, because no transaction needs to modify historical data .
Consistent non-locking reads are InnoDB's default read mode (Repeatable isolation level) , which means that reads do not occupy and wait for locks on the table.
A row records more than one snapshot data. This technology is generally called row multi-version concurrency control technology. At the transaction isolation level of READ COMMITED and REPEATABLE READ, InnoDB uses non-locking consistent reads.
Under the READ COMMITED transaction isolation level, for snapshot data, the latest snapshot data of the locked row is always read. From the perspective of database theory, it violates the isolation I in ACID. REPEATABLE READ reads the row data version at the beginning of the transaction.
| time | Session A | Session B |
| 1 | BEGIN | |
| 2 | select * from t where id=1 | |
| 3 | BEGIN | |
| 4 | update t set id=3 where id=1 At this time, the transaction is not submitted, X lock is added, the query of session A5, use MVCC query results |
|
| 5 | select * from t where id=1 | |
| 6 | COMMIT； | |
| 7 | select * from t where id=1 | |
| 8 | COMMIT； |
In the READ COMMITED isolation level, 1, 5 returns data with id = 1, 7 returns null, because after session B is submitted, the data with id = 1 is updated to 3, and the data at this time is the latest. Under REPEATABLE READ, 1, 5, 7 always return the data with id = 1, because the data before the start of the transaction is always read.
6.3.3 Consistent lock read
In some cases, users need to explicitly lock the database read operation to ensure data consistency. This requires the database to support locking statements, even for select queries. InnoDB supports two consistent lock read operations for select statements:
- select …… for update
- select …… lock in share mode
select …… for update adds X lock to the read row, no other transaction can add any lock to the row. select …… share in mode Add S lock to the read row record, other transactions can add S lock to the locked row, but can not add X lock, X lock can only block.
select …… for update and select …… share in mode must be in the transaction, transaction commit, lock release.
6.3.4 Self-growth and lock
In InnoDB's memory structure, there is a self-increment counter for each table that contains self-increment values. Execute the following statement to get the counter value:
select MAX(auto_inc_col) from t for update;
The insert operation will add 1 to the self-incrementing column according to this self-incrementing counter value. This implementation is called AUTO-INC Locking. This kind of lock actually uses a special table lock mechanism. In order to improve the insertion performance, the lock is not released after the transaction is completed, but after the SQL statement for the insertion of the self-increment value is completed. Release immediately.
Although AUTO-INC Locking improves concurrency to some extent, the performance of concurrent inserts for columns with self-increasing values is poor, and the transaction must wait for the completion of the previous insert (although it does not need to wait for the completion of the transaction). Insert ... The insertion of a large amount of select data will affect the performance of the insert, because another transaction will be blocked.
Starting with MySQL 5.1.22, InnoDB provides a self-growth implementation mechanism for lightweight mutexes . Starting from this version, InnoDB provides a parameter innodb_autoinc_lock_mode to control the self-increment mode, which defaults to 1.
| innodb_autoinc_lock_mode | Explanation |
| 0 | The implementation before MySQL5.1.22, that is, through AUTO-INC lock mode, because of the new self-growth mode, this value should not be the user's preference |
| 1 | The default value, for "simple inserts", this value will use the mutex to accumulate the counters in memory. For "bulk inserts", the traditional AUTO-INC locking method is still used. In this configuration, if the rollback operation is not considered, the growth of the self-added value is continuous. And thinking in this way, the statement-based way of replication can still work very well. Note: If you have used AUTO-INC locking, and you need to perform "simple inserts" again, you still need to wait for the release of "AUTO-INC locking" |
| 2 | In this mode, all "insert-likes" self-increasing values are generated through mutual exclusion, not AUTO-INC locking. Obviously, this is the highest performance method, however, it will bring some problems because With the existence of concurrency, the value of self-increase may not be continuous at each insertion. In addition, it is important that there will be problems based on Statement-based Replication. Therefore, using this mode, row-base replication should be used at all times, so as to ensure maximum concurrency and consistency of replication master-slave data. |
| Insert type | Explanation |
| insert-like | 所有的插入语句，包括 insert、replace、insert……select、replace……select、load data等 |
| simple inserts | 能在插入前就确定插入行数的语句，包括insert，replace等，不包含 insert …… on duplicate key update这类SQL |
| bulk inserts | 在插入前不能确定插入行数的语，如 insert……select，replace……select、load data等 |
| mixed-mode inserts | 插入中有一部分的值是自增长的，一部分是可以确定的，如insert into t (e1,e2) values (1,'a'),(NULL,'b'),(3,'c')，也可以是 insert …… on duplicate key update这类SQL |
另外，在 InnoDB中，自增长的列必须是索引，同时是索引的第一个列，否则MySQL会抛出异常。
6.3.5 外键和锁
外键主要用于引用完整性的约束检查，在InnoDB中，对于一个外键列，如果没有显示的对这个列添加索引，InnoDB会自动对其加索引，避免表锁。
对于外键的插入或更新，需要先查询父表，但是对于父表的查询不是使用一致性非锁定读（MVCC），因为这样会发生数据不一致的问题，使用的是 select ……lock in share mode，主动为付表加S锁，如果这时父表上已有X锁，则阻塞。
| 时间 | 会话A | 会话B |
| 1 | BEGIN | |
| 2 | delete from parent where id=5 | |
| 3 | BEGIN | |
| 4 | insert into child select 2,5 #第二列是外键，执行时被阻塞waiting |
6.4 锁的算法
6.4.1 行锁的三种算法
InnoDB 有三种行锁算法，分别是
- Record Lock：单个记录上的锁
- Gap Lock：间隙锁，锁定一个范围，但不包含记录本身
- Next-Key Lock：Gap Lock + Record Lock，锁定一个范围，并锁定记录本身
Record Lock 总是锁住索引记录，如果 InnoDB在建立时没有设置任何一个索引，那么 InnoDB会使用隐士的主键来锁定。
InnoDB对于行的查询都是采用 Next-Key Lock，其目的是为了解决幻读 Phantom Problem，是谓词锁的一种改进。
当查询的索引含有唯一属性时，InnoDB 会对Next-Key Lock优化，降级为 Record Key，以此提高应用的并发性。
如前面介绍的。next-key lock 降级为record lock是在查询的列是唯一索引的情况下，若是辅助索引，则情况不同：
create table z (a int, b int , Primary key(a), key(b)) insert into z select 1,1; insert into z select 3,1; insert into z select 5,3; insert into z select 7,6; insert into z select 10,8;
其中b是辅助索引，此时执行 select * from z where b=3 for update;
此时，sql 语句通过索引列b进行查询，因此其使用传统的 next-key lock 进行加锁，并且由于有两个索引，其需要分别进行锁定。对于聚集索引，仅对列a=5的索引加record lock。而辅助索引，加的是next-key lock，锁定的是（1，3）范围，特别注意的是，InnoDB还会对辅助索引下一个键值加上 gap lock，即还有一个（3，6）范围的锁。因此，一下SQL都会被阻塞：
select * from z where a=5 lock in share mode; insert into z select 4,2; insert into z select 6,5;
从上面可以看出，gap lock的作用是为了阻止多个事务将记录插入到同一个范围内，而这会导致幻读问题的产生。用户可以通过以下两种方式来关闭 Gap Lock：
- 将事务隔离级别改为 READ COMMITED
- 将参数 innodb_locks_unsafe_for_binlog设为1
在上述的配置下，除了外键约束和唯一性检查依然需要的Gap Lock，其余情况仅使用 Record Lock进行锁定，需要牢记的是，上述配置破坏了事务的隔离性，并且对 replication可能会导致不一致。且从性能上看，READ COMMITED也不会优于默认的 READ REPEATABLE；
在 InnoDB中，对Insert的操作，其会检查插入记录的下一条记录是否被锁定，若已锁定，则不允许查询。对于上面的例子，会话A已经锁定了表中b=3的记录，即已锁定了（1，3）的范围，这时如果在其他会话中进行如下的插入同样会导致阻塞
insert into z select 2，2；
因为检测到辅助索引列b上插入2的记录时，发现3已经被索引，而将插入修改为如下值，可以立即执行：
insert into z select 2，0；
最后，对于唯一键值的锁定，next-key lock降级为record ke仅存在于查询所有的唯一索引列。若唯一索引由多个列组成，而查询仅是查找多个唯一索引列中的一个，那么查询其实是range类型，而不是point查询，故InnoDB依然采用 next-key lock进行锁定。
6.4.2 解决 Phantom Problem
幻读指的是在同一个事务下，连续执行两次相同的SQL语句可能返回不一样的结果，第二次的SQL语句可能会返回之前不存在的行。
InnoDB采用 next-key lock 的算法解决了 Phantom Problem，对 select * from t where id > 2 for update，锁住的不单是5这个值，而是对（2，+∞）这个范围加了X锁。因此，对这个范围的插入是不允许的，从而避免幻读。
| 时间 | 会话A | 会话B |
| 1 | set session tx_isolation = 'READ-COMMITED' |
|
| 2 | BEGIN | |
| 3 | select * from t where a>2 for update; ***********1 row ************* a:4 |
|
| 4 | BEGIN | |
| 5 | insert into t select 4 | |
| 6 | COMMIT; | |
| 7 | select * from t where a>2 for update; ***********1 row ************* a:4 ***********2 row ************* a:5 |
REPEATABLE READ 采用的是 next-key locking加锁。而 READCOMMITED 采用的是 record lock .
此外，用户可以通过 InnoDB的 next-key lock在应用层面实现唯一性的检查：
select * from table where col=xxx lock in share mode; if not found any row : #unique for insert value insert into table values(……)；
如果用户通过一个索引查询一个值，并对该行加上了S lock，那么即使查询的值不存在，其锁定的也是一个范围，因此若没有返回任何行，那么新插入的值一定是唯一的。
那，如果在第一步select lock in share mode时，有多个事务并发操作，那么这种唯一性检查是否会有问题，其实不会，因为会发生死锁。只有一个事务会成功，其他的事务会抛出死锁错误。
6.5 锁问题
6.5.1 脏读
脏数据是指未提交的数据，如果读到了脏数据，即一个事务可以读到另一个事务未提交的数据，则显然违反了数据库的隔离性。
脏读指的是在不同事务下，当前事务可以读到另外事务未提交的数据，即脏数据。
| 时间 | 会话A | 会话B |
| 1 | set @@tx_isolation = 'read-ncommited' |
|
| 2 | set @@tx_isolation = 'read-ncommited' |
|
| 3 | BEGIN |
|
| 4 | select * from t ; **********1 row ************* a:1 |
|
| 5 | insert into t select 2; | |
| 6 | select * from t ; **********1 row ************* a:1 **********2 row ************* a:2 |
脏读发生条件是需要事务的隔离级别为 read uncommited；目前大部分数据库至少设置为 read COMMITED；
6.5.2 不可重复读
不可重复读和脏读的区别是：脏读是读到未提交的数据，而不可重复读读到的是已提交的数据，但是违反了事务一致性的要求。
| 时间 | 会话A | 会话B |
| 1 | et @@tx_isolation = 'read-commited' |
|
| 2 | et @@tx_isolation = 'read-commited' |
|
| 3 | BEGIN | BEGIN |
| 4 | select * from t ; **********1 row ************* a:1 |
|
| 5 | insert into t select 2； | |
| 6 | COMMITED | |
| 7 | select * from t ; **********1 row ************* a:1 **********2 row ************* a:2 |
一般来说，不可重复读是可接受的，因为读到的是已提交的数据，本身没有带来很大问题。在 InnoDB中使用 next-key lock避免不可重复读问题，即 幻读（Phantom Problem）。在 Next-Key lock算法下，对索引的扫描，不仅是锁住扫描到的索引，还有这些索引覆盖的范围，因此在这个范围内插入是不允许的。这样则避免了另外的事务在这个范围内的插入导致不可重读的问题。
6.5.3 丢失更新
丢失更新就是一个事务的更新操作被另一个事务的更新操作覆盖，从而导致数据不一致。
- 事务1将行记录r更新为v1，但是事务未提交
- 事务2将行记录r更新为v2，事务未提交
- 事务1提交
- 事务2提交
当前数据库的任何隔离级别下，以上情况都不会导致数据库理论意义上的丢失更新问题，因为，对于行的DML操作，需要对行货其他粗粒度级别的对象加锁，步骤2，事务2并不能对记录进行更新，被阻塞，直到事务1提交。
但在生产应用中，还有一个逻辑意义的丢失更新问题，而导致该问题的不是因为数据库本身的问题，简单来说，下面情况会发生丢失更新：
- 事务T1查询一行数据，放入本地内存，返回给User1
- 事务T2查询一行数据，放入本地内存，返回给User2
- User1修改后，更新数据库提交
- User2修改后，更新数据库提交
显然，这个过程中，User1的修改操作”丢失“了。在银行操作中，尤为恐怖。要避免丢失，需要让事务串行化。
| 时间 | 会话A | 会话B |
| 1 | BEGIN | |
| 2 | select cash into @cash from account where user=pUser for update #加X锁 |
|
| 3 | select cash into @cash from account where user=pUser for update #等待，直到m提交后，锁释放 |
|
| …… | …… | |
| m | update account set cash = @cash - 9000 where user = pUser |
|
| m+1 | commit | |
| m+2 | update account set cash = @cash -1 where user = pUser |
|
| m+3 | commit |
6.6 阻塞
因为不同锁之间的兼容性关系，在有些时刻一个事务中的锁需要等待另一个事务中的锁释放它所占用的资源，这就是阻塞。
阻塞并不是一件坏事，其是为了确保事务可以并发且正常地运行。
需要牢记的是，默认情况下，InnoDB存储引擎不会回滚超时引发的错误异常，其实，InnoDB在大部分情况下都不会对异常进行回滚。
| 时间 | 会话A | 会话B |
| 1 | select * from t； **********3 row ************* a:1 a:2 a:4 |
|
| 2 | BEGIN | |
| 3 | select * from t where a < 4 for update; **********2 row ************* a:1 a:2 #对（2，4）上X锁 |
|
| 4 | BEGIN | |
| 5 | insert into t select 5; | |
| 6 | insert into t select 3; #等待超时，需要确定超时后，5的插入是否需要回滚或提交，否则这是十分危险的状态 |
6.7 死锁
死锁是指两个或两个以上的事务在执行过程中，因争夺资源而造成的一种相互等待的现象。解决死锁的办法：
- 超时回滚
- wait-for graph 等待图的方式进行死锁检测，采用深度有限的算法，选择回滚undo量最小的事务。
死锁的概率与以下因素有关：
- 事务的数量n，数量越多死锁概率越大
- 事务操作的数量r，数量越多，死锁概率越大
- 操作数据的集合R，越小，死锁概率越大
6.8 锁升级
锁升级就是将当前锁的粒度降低，例如把行锁升级为页锁，把页锁升级为表锁。
InnoDB不存在锁升级，因为其不是根据每个记录来产生行锁的。相反，其根据每个事务访问的每个页对锁进行管理，采用的是位图的方式。因此不管一个事务锁住页中的一个记录还是多个记录，开销都是一样的。