table of Contents
MVCC implemented in InnoDB MySql in
1, RR-level transaction isolation level really can not solve the phantom read it? The answer is
A block or two locks?
Because there are a large number of concurrent access, in order to prevent deadlocks, the general application of the method is recommended once the blockade is at the beginning stage of the process, already know in advance what data will be used, then all locked after the run method, and then unlock all . This way you can effectively avoid deadlocks, but is not applicable in the database, since the beginning of the transaction, the database does not know what data will be used.
Database follows the two-phase locking protocol, the transaction will be divided into two phases, phase lock and unlock stage (so called two-phase locking)
- Phase lock: in this phase can be locked operation. Before any data read operation to apply for and obtain the S-lock (shared lock, other transactions can continue to add a shared lock, but can not add exclusive lock), during the write operation to apply for and obtain before X-lock (exclusive lock, other transactions can not acquire any locks). Lock is unsuccessful, the transaction enters a waiting state until the lock before continuing the successful execution.
- Unlock Stage: When the transaction to release a blockade affairs into the unlocked stage, can only unlock operation at this stage can no longer be locked operation.
| Affairs | Lock / unlock processing |
|---|---|
| begin； | |
| insert into test ….. | Plus corresponding lock insert |
| update test set… | Add update corresponding lock |
| delete from test …. | Plus delete the corresponding lock |
| commit; | When the transaction commits, while releasing insert, update, delete the corresponding lock |
Although this method can not avoid deadlock, but the two-phase locking protocol ensures scheduling of concurrent transactions is serialization (serialization is very important, especially in data recovery and backup) of.
Mysql, Oracle, PostgreSql mature database, for performance reasons, and are based MVCC (multi-version concurrency control) with the theoretical basis of optimistic locking to avoid dirty reads, non-repeatable read, phantom read three questions;
(
1 difference, non-repeatable read and phantom read
Many people tend to confuse non-repeatable read and phantom read, indeed somewhat similar to both. However, non-repeatable reads focuses on update and delete, and phantom reads focus is insert.
If the locking mechanism used to achieve these two levels of isolation, in repeatable read, the first read sql data, the data will be locked, other transactions can not modify these data, we can achieve repeatable read . However, this method can not lock the data insert, so when Transaction A previously read data, or modify all the data, transaction B can still submit insert data, transaction A then you will find more than a baffling not before data, which is read magic, can not be avoided by the row lock. Need Serializable isolation level, read with read lock, write, write locks, read and write locks mutex, doing so can effectively prevent phantom reads, non-repeatable read, dirty reads and other issues, but will greatly reduce concurrency database ability.
2, pessimistic locking and optimistic locking difference
- Pessimistic locking: refers to the data by outsiders (including the current else the system, as well as transaction processing from external systems) to modify a conservative approach, the entire data processing, data is locked. Pessimistic locks for locking mechanism relies on the database lock mechanism provided in (only the database layer provided in order to truly ensure exclusive access to data, or even achieve a locking mechanism in this system, still can not avoid modifying the external system data ) . In order to guarantee transaction isolation, need consistency locked read. When reading data lock, other transactions can not modify these data. Modify, delete data must be locked when, other transactions can not read the data.
- Optimistic locking: relatively pessimistic locking, the lock mechanism optimistic taken a more relaxed; pessimistic locking lock mechanism dependent on implementation of the database, in order to ensure the greatest degree of exclusivity. But followed by a large number of database performance overhead, especially for long transactions, such costs often can not afford; optimistic locking in a certain extent, solved the problem. Use a data version (Version) recording mechanism to achieve. Data version: To add a version identifier data, is generally achieved by adding a "version" field to table. Read data is read out together with this version number, the time after the update of the version number plus one; At this time, the current version will be submitted to the database table corresponding to the recorded data are compared, the version number is greater than if the database table, filed data current version number is to be updated, otherwise considered stale data.
）
MySQL locks the species
Many types of locks in MySQL, there are common table and row locks, but also new entrants Metadata Lock and so on, the whole table lock is a table lock, although can be divided into read and write locks, but that is locked live an entire table, will lead to decline in concurrent capacity, it is generally used when doing ddl treatment.
Row lock is locked rows, this locking method is more complex, but since the lock only limited data, other data not for limitation, the strong concurrency, MySQL row locks are generally used to handle concurrent transactions. That is where the main row lock discussed.
Read Committed (read submission)
In the RC level, the read data is not locked, but write, modify, and delete data is the need locked. Results are as follows
MySQL> show create table class_teacher \G\
Table: class_teacher
Create Table: CREATE TABLE `class_teacher` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`class_name` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
`teacher_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_teacher_id` (`teacher_id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
1 row in set (0.02 sec)
MySQL> select * from class_teacher;
+----+--------------+------------+
| id | class_name | teacher_id |
+----+--------------+------------+
| 1 | 初三一班 | 1 |
| 3 | 初二一班 | 2 |
| 4 | 初二二班 | 2 |
+----+--------------+------------+
Because MySQL's InnoDB default is to use the RR level, so we need to turn the session into a RC level, and mode settings binlog
SET session transaction isolation level read committed;
SET SESSION binlog_format = 'ROW';（或者是MIXED）
| A transaction | Transaction B |
|---|---|
| begin; | begin; |
| update class_teacher set class_name=‘初三二班’ where teacher_id=1; | update class_teacher set class_name=‘初三三班’ where teacher_id=1; |
| commit; |
In order to prevent conflicts of concurrent modification process, the transaction A MySQL to teacher_id = 1 data line lock, and remain not commit (to release the lock), then the transaction B also has been unable to get the line lock, wait until the timeout.
At this point we have to note, teacher_id is indexed, if there is no index class_name it? update class_teacher set teacher_id = 3 where class_name = 'junior class'; then add all the data rows of the row lock an entire table will MySQL. Here it sounds a bit weird, but when the process is running sql, MySQL does not know which rows are class_name = 'junior class' of (not indexed thing), if a condition can not be quickly filtered through indexing, storage engine level to It will return all records after locking, then filtered by MySQL Server layer.
But in the actual use, MySQL has made some improvements, the filter conditions in MySQL Server, found not satisfied, calls unlock_row method, the record does not satisfy the conditions of the release of the lock (lock against the constraints Sec protocol). In doing so, ensure that the final will meet the locks held on the recording conditions, but still locking operation of each record can not be omitted. Visible Even MySQL, for efficiency also would violate norms. (See "High Performance MySQL" Chinese third edition p181)
This situation also applies to MySQL's default isolation level RR. So for a large amount of data tables do the bulk modified, if you can not use the corresponding index, when the MySQL Server filtering data particularly slow, although there will be some lines of data is not modified, but they are still locked a phenomenon.
MVCC implemented in InnoDB MySql in
InnoDB for each row of data add two extra hidden values to achieve MVCC, these two values when a record line data is created (or updated), a record which rows are being deleted. In practice, the version number is stored in the transaction, not to open a new transaction, the version number is incremented affairs. In repeatable read transaction isolation level Rapeatabel Read below:
- SELECT When creating a version number read <= current transaction version number, delete the version number is null or> current transaction version number (here, mainly on account of the open transaction process once, read many)
- INSERT, the saved version of the current transaction is to create a version number for the line
- DELETE, the preservation of the current transaction version number is the version number of deleted rows
- On UPDATE, insert a new record, save the current transaction to create a version of the line version, the version number of the current transaction while preserving the original line to be deleted.
Although MVCC to open up additional storage space for each row of records, there will be more of an internal inspection, maintenance work, but you can reduce the use of locks, most do not read lock;
1, RR-level transaction isolation level really can not solve the phantom read it? The answer is
Whether we learn from text books databases, or seen from the network, meaning large transactions above are four isolation levels listed in this module, RR level is repeatable read, but can not solve the phantom read , and can be resolved only at the level Serializable phantom reads. So I added a transaction C to show results. Add a teacher_id = the data commit 1, RR level should be a phantom read phenomenon in C transaction, the transaction at the time of the query A teacher_id = 1 data transaction will read C newly added data. But after tests found that MySQL does not exist in this case, C submitted after the transaction, the transaction A is still not reading this data. MySQL can be seen in the RR level, the problem is solved read magic of reading. See the figure below
Reading problems are solved, according to the definition of MVCC, there will be a conflict complicated by submitting data, how to solve it during the conflict? We look at the RR level process for InnoDB in write data.
The difference between (1), "read" and "read" the
Some people may be wondering, transaction isolation level are in fact the definition of reading data, but in here, it split became read and write two modules to explain. This is mainly because MySQL is read, and transaction isolation level is read, is not the same.
We shall see, in RR level by MVCC mechanism, though so data becomes repeatable read, but we read data may be historical data, the data is not timely, not the database current data! This is in some particularly sensitive to aging of data services, it is likely to go wrong.
对于这种读取历史数据的方式，我们叫它快照读 (snapshot read)，而读取数据库当前版本数据的方式，叫当前读 (current read)。很显然，在MVCC中：
- 快照读：就是select
- select * from table ….;
- 当前读：特殊的读操作，插入/更新/删除操作，属于当前读，处理的都是当前的数据，需要加锁。
- select * from table where ? lock in share mode;
- select * from table where ? for update;
- insert;
- update ;
- delete;
事务的隔离级别实际上都是定义了当前读的级别，MySQL为了减少锁处理（包括等待其它锁）的时间，提升并发能力，引入了快照读的概念，使得select不用加锁。而update、insert这些“当前读”，就需要另外的模块来解决了。
（2）、写（”当前读”）
事务的隔离级别中虽然只定义了读数据的要求，实际上这也可以说是写数据的要求。上文的“读”，实际是讲的快照读；而这里说的“写”就是当前读了。
为了解决当前读中的幻读问题，MySQL事务使用了Next-Key锁。
2、Mysql如何做到RR事务隔离级别下不会出现幻读（Next-Key锁=GAP锁+行锁）
行锁（是mvcc版的乐观锁）防止别的事务修改或删除，GAP锁防止别的事务新增，行锁和GAP锁结合形成的的Next-Key锁共同解决了RR级别在写数据时的幻读问题。
（1）、Next-Key锁
Next-Key锁是行锁和GAP（间隙锁）的合并，行锁上文已经介绍了，接下来说下GAP间隙锁。
行锁可以防止不同事务版本的数据修改提交时造成数据冲突的情况。但如何避免别的事务插入数据就成了问题。我们可以看看RR级别和RC级别的对比
RC级别：
| 事务A | 事务B | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| begin; | begin; |
|||||||||
| select id,class_name,teacher_id from class_teacher where teacher_id=30;
|
||||||||||
| update class_teacher set class_name='初三四班' where teacher_id=30; |
|
|||||||||
|
|
insert into class_teacher values (null,'初三二班',30); commit; |
|||||||||
| select id,class_name,teacher_id from class_teacher where teacher_id=30;
|
RR级别：
| 事务A | 事务B | ||||||
|---|---|---|---|---|---|---|---|
| begin; | begin; |
||||||
| select id,class_name,teacher_id from class_teacher where teacher_id=30;
|
|||||||
| update class_teacher set class_name='初三四班' where teacher_id=30; |
|
||||||
|
|
insert into class_teacher values (null,'初三二班',30); waiting.... |
||||||
| select id,class_name,teacher_id from class_teacher where teacher_id=30;
|
|||||||
| commit; | 事务Acommit后，事务B的insert执行。 |
通过对比我们可以发现，在RC级别中，事务A修改了所有teacher_id=30的数据，但是当事务Binsert进新数据后，事务A发现莫名其妙多了一行teacher_id=30的数据，而且没有被之前的update语句所修改，这就是“当前读”的幻读。
RR级别中，事务A在update后加锁，事务B无法插入新数据，这样事务A在update前后读的数据保持一致，避免了幻读。这个锁，就是Gap锁。
MySQL是这么实现的：
在class_teacher这张表中，teacher_id是个索引，那么它就会维护一套B+树的数据关系，为了简化，我们用链表结构来表达（实际上是个树形结构，但原理相同）
如图所示，InnoDB使用的是聚集索引，teacher_id身为二级索引，就要维护一个索引字段和主键id的树状结构（这里用链表形式表现），并保持顺序排列。
Innodb将这段数据分成几个个区间
- (negative infinity, 5],
- (5,30],
- (30,positive infinity)；
update class_teacher set class_name=‘初三四班’ where teacher_id=30;不仅用行锁，锁住了相应的数据行；同时也在两边的区间，（5,30]和（30，positive infinity），都加入了gap锁。这样事务B就无法在这个两个区间insert进新数据。
受限于这种实现方式，Innodb很多时候会锁住不需要锁的区间。如下所示：
| 事务A | 事务B | 事务C | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| begin; | begin; | begin; | |||||||||
| select id,class_name,teacher_id from class_teacher;
|
|||||||||||
| update class_teacher set class_name='初一一班' where teacher_id=20; | |||||||||||
| insert into class_teacher values (null,'初三五班',10); waiting ..... |
insert into class_teacher values (null,'初三五班',40); | ||||||||||
| commit; | 事务A commit之后，这条语句才插入成功 | commit; | |||||||||
| commit; |
update的teacher_id=20是在(5，30]区间，即使没有修改任何数据，Innodb也会在这个区间加gap锁，而其它区间不会影响，事务C正常插入。
如果使用的是没有索引的字段，比如update class_teacher set teacher_id=7 where class_name=‘初三八班（即使没有匹配到任何数据）’,那么会给全表加入gap锁。同时，它不能像上文中行锁一样经过MySQL Server过滤自动解除不满足条件的锁，因为没有索引，则这些字段也就没有排序，也就没有区间。除非该事务提交，否则其它事务无法插入任何数据。
Row locks to prevent other transactions from modifying or deleting, GAP lock prevents other transactions new, Next-Key Lock line locks and lock GAP combine to form the magic of reading together to solve the problem of RR level in writing data.
Serializable
This level is very simple, shared locks to read, write, add exclusive lock, reading and writing are mutually exclusive. Use pessimistic locking theory, easy to implement data more secure, but concurrent capacity is very poor. If your business is small or no concurrent particular concurrent, but also requires timely and reliable data, you can use this mode.
Here Tucao one, do not see the lock will not select say, Serializable at this level, still locked!
Reprinted Source: https://tech.meituan.com/2014/08/20/innodb-lock.html
Need to focus article: Mysql in "select ... for update" exclusive lock analysis
