The implementation principle of MVCC is also a very high-frequency interview question. When I was finishing this article, I felt that the information on the Internet was written in various ways on this knowledge point, as if everyone's understanding was not consistent.
Here is a summary of what I understand. Personally, I think this is an article with high gold content (haha), "so please read it carefully" , it will definitely benefit you.
If you don't understand anything in the article, or think that what I said is wrong, please leave a message to communicate with each other.
"Foreword"
Some basic concepts I will not elaborate here. For example, what is a transaction? ACID of a transaction? Four isolation levels?
I have written an article before about the problems of transaction concurrency: a detailed explanation of dirty reads, non-repeatable reads, and phantom reads
If you are still not clear about the difference between non-repeatable reading and phantom reading, it is highly recommended to read the above article. Because many people confuse non-repeatable reading with phantom reading.
Therefore, it is thought that MVCC can solve phantom reading. In fact, what MVCC solves is not phantom reading, but non-repeatable reading. The following will use practical examples to prove this.
1. What is MVCC
"Multi-versioning" : Refers to a technique that improves concurrency. In the earliest database system, only read and read can be concurrent, and both read and write, write and read, and write and write must be blocked. After the introduction of multiple versions, "only writes block each other" , and the other three operations can be parallelized, which greatly improves the concurrency of InnoDB.
In the internal implementation, InnoDB saves multiple versions of each data through the undo log, and can retrieve the historical version of the data for users to read. The data version read by each transaction may be different. In the same transaction, the user can only see the changes committed before the transaction created the snapshot and the changes made by the transaction itself.
MVCC only works under two isolation levels, Read Committed and Repeatable Read. The other two isolation levels are incompatible with MVCC.
Because of uncommitted reads , the most recent data row is read instead of the data row that matches the current transaction version. Serializable (Serializable) will lock all the data read.
The implementation principle of MVCC mainly relies on "two hidden fields in each row of records, undo log, ReadView"
2. Some concepts related to MVCC
Here we first understand some concepts related to MVCC. After these concepts are understood, we will demonstrate the specific workflow of MVCC through practical examples.
1. Transaction version number
❝Every time a transaction is opened, a self-increasing transaction ID is obtained from the database, and the execution order of the transaction can be judged from the transaction ID. This is the transaction version number.
❞
That is, whenever begin, the first thing to do is to obtain a self-increasing transaction ID from the database, which is also the transaction ID of the current transaction.
2. Hidden fields
For the InnoDB storage engine, each row of records has two hidden columns "trx_id" and "roll_pointer" . If there is a primary key or a non-NULL UNIQUE key in the data table, the row_id will not be created, otherwise InnoDB will automatically generate a monotonically increasing hidden primary key row_id.
| column name | Is it necessary | describe |
|---|---|---|
| row_id | no | Monotonically increasing row ID, not required, occupies 6 bytes. This has nothing to do with MVCC |
| trx_id | Yes | Record the transaction ID of the transaction that operates the row of data |
| roll_pointer | Yes | Rollback pointer, pointing to the undo log information of the current record line |
The record operation here refers to insert|update|delete. For the delete operation only, InnoDB considers it an update operation, but it will update an additional delete bit, representing the row as deleted, not really deleted.
3、undo log
"The undo log can be understood as a rollback log, which stores the old version of data" . Before the table record is modified, the original data will be copied to the undo log. If the transaction is rolled back, the data can be restored through the undo log. Or if the current record line is not visible, you can follow the undo log chain to find the record line version that satisfies its visibility condition.
When insert/update/delete (essentially update, just update a special delete bit field) operation, undo log will be generated.
In InnoDB, undo logs are divided into the following two categories:
1) "insert undo log" : The undo log generated when a transaction inserts a new record is only needed when the transaction is rolled back, and can be discarded immediately after the transaction is committed.
2) "update undo log" : The undo log generated when the transaction performs delete and update operations on the record is not only required for transaction rollback, but also for snapshot reading. Only when the snapshot used by the database does not involve the log record, the corresponding The rollback journal will be deleted.
What is the use of undo log?
❝1. When the transaction is rolled back, atomicity and consistency are guaranteed. 2. If the current record row is not visible, you can follow the undo log chain to find the record row version that satisfies its visibility conditions (for MVCC snapshot read).
❞
4. Version chain
When multiple transactions operate a row of data in parallel, the modification of the row of data by different transactions will generate multiple versions, and then through the rollback pointer (roll_pointer), a linked list is formed. This linked list is called a "version chain" . as follows:
5. Snapshot read and current read
"Snapshot Read" : Read the visible version of the record data (there is an old version). Without locking, ordinary select statements are snapshot reads , such as:
select * from user where id = 1;
"Current read" : The latest version of the record data is read, and the current read is the one that is explicitly locked.
select * from user where id = 1 for update;
select * from user where id = 1 lock in share mode;
6、ReadView
❝ReadView is a record snapshot generated by a transaction during snapshot reading, which can help us solve the visibility problem
❞
If a transaction wants to query a row record, which version of the row record needs to be read? ReadView is to solve this problem. ReadView keeps a "list of all active transactions when the current transaction is open" .
How does ReadView ensure visibility judgment? Let's take a look at several important properties of ReadView
-
"trx_ids" : IDs of active (uncommitted) read and write transactions in the current system, whose data structure is a List. ( Key note : The active transactions in trx_ids here do not include the current transaction itself and committed transactions, which is very important)
-
"low_limit_id" : The largest transaction ID that has appeared so far + 1, that is, the next transaction ID to be allocated.
-
"up_limit_id" : The smallest transaction ID in the active transaction list trx_ids, if trx_ids is empty, the up_limit_id is low_limit_id.
-
"creator_trx_id" : Indicates the transaction id of the transaction that generated the ReadView
How to determine whether a record is visible when accessing a record, the specific rules are as follows:
-
If the version is accessed
事务ID = creator_trx_id, it means that the current transaction accesses the record that it has modified, then the version is visible to the current transaction ; -
If the version is accessed
事务ID < up_limit_id, it means that the transaction that generated this version has been committed before the current transaction generates ReadView, then this version is visible to the current transaction . -
If the
事务ID > low_limit_idvalue of the version is accessed, it means that the transaction that generates this version is opened after the current transaction generates ReadView, so this version is not visible to the current transaction . -
If it is between the accessed versions
事务ID在 up_limit_id和low_limit_id, it is necessary to determine whether the transaction ID of the version is in the trx_ids list. If it is, it means that the transaction that generated the version is still active when the ReadView is created, and the version cannot be accessed; if not, Indicates that the transaction that generated the version when the ReadView was created has been committed and the version can be accessed.
Draw a picture to understand
A question to consider here is that 何时创建ReadView？
As mentioned above, ReadView is to solve the problem of which version of the row record a transaction needs to read. So what does it mean? The ReadView is only created when it is selected. But there are differences at different isolation levels:
❝Under the RC isolation level, each select will create the latest ReadView;
Under the RR isolation level, ReadView is only created when the first select request in the current transaction;
❞
What about insert/update/delete operations?
These operations do not create a ReadView . However, when these operations are started and the transaction is not committed, then its transaction ID will exist in other ReadView records that exist in the query transaction, that is, in trx_ids.
Three, MVCC realization principle analysis
1. How to query a record
-
Get the transaction ID of the transaction itself, ie trx_id . (This is not obtained when selecting, but when the transaction is started, that is, when it begins)
-
Get ReadView (this is only generated when select)
-
If the data is queried in the database table, then go to the transaction version number in ReadView for comparison.
-
If it does not meet the visibility rules of ReadView, that is , historical snapshots in Undo log are required until data that meets the rules is returned;
InnoDB implements MVCC through ReadView+ Undo Log implementation, Undo Log saves historical snapshots, and ReadView visibility rules help determine whether the current version of the data is visible.
2. How does MVCC implement read committed and repeatable read?
In fact, the other processes are the same. The only difference between read committed and repeatable read is: under the RC isolation level, each select will create the latest ReadView; under the RR isolation level, when the first in the transaction ReadView is created only after a select request.
After reading the following example, you should understand.
4. Classic interview questions: Can MVCC solve the phantom reading problem?
I have checked a lot of information on this issue, some say it can be solved, some say it can't be solved, and some people say it can solve some hallucination scenes. The partial solution here refers to the ability to solve the phantom read problem of snapshot reading, but not the phantom read problem of current read.
For details, see the following article
Interview question: Can MVCC solve phantom readings? https://blog.csdn.net/qq_35590091/article/details/107734005
Let me start with my conclusion:
❝MVCC can solve the non-repeatable read problem, but it cannot solve the phantom read problem, neither snapshot read nor current read. The RR level solves the phantom read by the lock mechanism, not the MVCC mechanism.
❞
Since so many people on the Internet say that MVCC can solve the phantom reading problem of snapshot reading, here is an example to illustrate that MVCC cannot solve the phantom reading problem of snapshot reading.
Suppose there is a user table, the id of this table is the primary key. There are 4 pieces of data in the table at the beginning.
Here is research at RR level (repeatable read).
1. Transaction A, query whether there is a record with id=5, if not, insert it. This is the normal business logic we expect.
2. At this time, transaction B adds a new record with id=5, and submits the transaction.
3. Transaction A, when querying id=5, it is found that there is still no record.
The above article is an example to illustrate that transaction A reads the same for the first time and the second time, so it is considered that the phantom reading is solved. I don't think this is to solve the phantom reading, but to solve the impossible repeated reading. It guarantees that the results read the first time and the second time are the same.
Has the phantom read been resolved? Obviously not, because at this time if transaction A performs an insert operation
INSERT INTO `user` (`id`, `name`, `pwd`) VALUES (5, '田七', 'fff');
Finally, transaction A commits the transaction and finds that an error is reported. This is very strange. When I checked, there was no such record, but when I inserted it, it told me that the primary key conflicted, which was like an illusion. This is the phantom reading problem.
Therefore, MVCC cannot be solved, and a lock is required to solve it.
The premise that transaction A can be inserted normally is that other transactions cannot insert id=5 and submit successfully. To solve this problem is also very simple, that is, transaction A first obtains the exclusive lock of id=5.
We can add an exclusive lock when transaction A queries for the first time
select * from `user` where id = 5 for update
Then the insertion action of transaction B will always belong to the blocking state until transaction A is successfully inserted and committed. Then in the end, transaction B reports a primary key conflict and rolls back. However, transaction A will not fail to insert because there is no such record when querying. It also solves the phantom reading problem.
Therefore, it is the lock mechanism, not the MVCC mechanism, that solves the phantom read problem at the RR level .
If you disagree with my point of view, you can leave your point of view, and we can discuss and exchange together.
Reprinted: Read it once to understand: Detailed explanation of MVCC principle