Principles of affairs
Transaction Basics
A transaction is a set of operations, which is an indivisible unit of work. A transaction will submit or revoke operation requests to the system as a whole, that is, these operations will either succeed at the same time or fail at the same time.
The four major characteristics of transactions:
- Atomicity: A transaction is an indivisible minimum unit of operations, either all succeed or all fail.
- Consistency: When a transaction is completed, all data must be in a consistent state.
- Isolation: The isolation mechanism provided by the database system ensures that transactions run in an independent environment that is not affected by external concurrent operations.
- Durability: Once a transaction is committed or rolled back, its changes to the data in the database are permanent.
When we study the principles of transactions, we study how MySQL's InnoDB engine ensures these four characteristics of transactions.
As for these four major features, they are actually divided into two parts. The atomicity, consistency, and persistence are actually guaranteed by two logs in InnoDB, one is the redo log and the other is the undo log. Isolation is guaranteed through database locks and MVCC. We study the principles of transactions, mainly to study redolog, undolog and MVCC.
redo log
The redo log records the physical modification of the data page when the transaction is committed, and is used to achieve transaction durability. The log file consists of two parts: the redo log buffer and the redo log file. The former is in memory and the latter is on disk. When the transaction is committed, all modification information will be stored in the log file, which is used to refresh dirty pages to disk and perform data processing when an error occurs. Reuse.
If there is no redolog, what problems may exist? Let's analyze it together
We know that in the memory structure of the InnoDB engine, the main memory area is the buffer pool, and many data pages are cached in the buffer pool. When we perform multiple addition, deletion, and modification operations in a transaction, the InnoDB engine will first operate on the data in the Buffer Pool. If If there is no corresponding data in the buffer, the data from the disk will be loaded through the background thread, stored in the buffer, and then the data in the buffer pool will be modified. The modified data page is called a dirty page. Dirty pages will be flushed to the disk through a background thread at a certain opportunity to ensure that the buffer and disk data are consistent. The dirty page data in the buffer is not refreshed in real time, but the data in the buffer is refreshed to the disk after a period of time. If an error occurs in the process of refreshing to the disk, the user is prompted that the transaction was submitted successfully, but the data is not persisted. When this happens, a problem arises, as the durability of the transaction is not guaranteed.
InnoDB provides a log redo log. Next, let's analyze how to solve this problem through redolog.
With redolog, when the data in the buffer is added, deleted or modified, will first record the changes in the data page of the operation in the redo log buffer . When the transaction is committed, will flush the data in the redo log buffer to the redo log disk file. After a period of time, if an error occurs when flushing the dirty pages of the buffer to the disk, you can use the redo log for data recovery, thus ensuring the durability of the transaction. If the dirty page is successfully flushed to the disk or the data involved has been written to the disk, the redolog will have no effect at this time and can be deleted, so the two existing redolog files are written in a loop.
undo log
The rollback log is used to record information before the data is modified. It has two functions: providing rollback (guaranteeing the atomicity of transactions) and MVCC (multi-version concurrency control).
undo log is different from the physical log recorded by redo log. It is a logical log. It can be thought that when a record is deleted, a corresponding insert record will be recorded in the undo log, and vice versa, when a record is updated, a corresponding update record will be recorded in the undo log. The opposite operation of the current operation will be recorded to facilitate rollback. When rollback is executed, the corresponding content can be read from the logical record in the undo log and rolled back.
Destruction and storage of undo log
Undo log destruction: The undo log is generated when a transaction is executed. When the transaction is committed, the undo log will not be deleted immediately because these logs may also be used for MVCC.
Undo log storage: The undo log is managed and recorded in segments and is stored in the rollback segment introduced earlier. It contains 1024 undo log segments internally.
MVCC
basic concept
currently reading
What is read is the latest version of the record. When reading, it must be ensured that other concurrent transactions cannot modify the current record, and the read record will be locked. For our daily operations, such as: select ... lock in share mode (shared lock), select ... for update, update, insert, delete (exclusive lock) are all current reads.
In the test, we can see that even under the default RR isolation level, transaction A can still read the latest submitted content of transaction B, because the lock in share mode shared lock is added after the query statement. At this time Is the current read operation. Of course, when we add an exclusive lock, it is also the current read operation.
Snapshot read
Simple select (without locking) is snapshot reading. Snapshot reading reads the visible version of the recorded data, which may be historical data. Without locking, it is a non-blocking read.
- Read Committed: Each time you select, a snapshot read is generated.
- Repeatable Read: The first select statement after starting a transaction is where the snapshot is read.
- Serializable: Snapshot reads will degenerate into current reads.
In the test, we saw that even if transaction B submitted data, it could not be queried in transaction A. The reason is that ordinary select is a snapshot read, and under the current default RR isolation level, the first select statement after starting a transaction is where the snapshot is read. The same select statement executed later will obtain data from the snapshot, which may It is not the latest data, thus ensuring repeatable reading.
MVCC
The full name is Multi-Version Concurrency Control, multi-version concurrency control. It refers to maintaining multiple versions of a data so that there is no conflict in read and write operations. Snapshot reading provides a non-blocking read function for MySQL to implement MVCC. The specific implementation of MVCC also needs to rely on three implicit fields in the database record, undo log, and readView.
Hidden fields
When we create a table, we can explicitly see the fields we declared when we look at the table structure. In fact, in addition to the fields we declared, InnoDB will automatically add three hidden fields to us and their meanings are:
The first two fields will definitely be added. Whether to add the last field DB_ROW_ID depends on whether the current table has a primary key. If there is a primary key, the hidden field will not be added. Field.
| Hidden fields | meaning |
|---|---|
| DB_TRX_ID | The most recently modified transaction ID is the transaction ID of the record that was inserted into this record or the last time the record was modified. |
| DB_ROLL_PTR | The rollback pointer points to the previous version of this record and is used in conjunction with undo log to point to the previous version. |
| DB_ROW_ID | Hidden primary key. If the table structure does not specify a primary key, this hidden field will be generated. |
undolog
introduce
Rollback log, a log generated during insert, update, and delete to facilitate data rollback. When inserting, the undo log generated is only needed during rollback and can be deleted immediately after the transaction is committed. When updating and deleting, the undo log generated is not only needed during rollback, but also during snapshot reading, and will not be deleted immediately.
version chain
DB_TRX_ID: Representsthe most recently modified transaction ID, the record is inserted into this record or the last time it was modified The recorded transaction ID is incremented by itself.
DB_ROLL_PTR: Since this data was just inserted and has not been updated, the value of this field is null.
Then, four concurrent transactions are accessing this table at the same time. Note:The undolog log does not record specific data. It needs to execute the log based on the current record to get the previous version of the data. Here, for the convenience of understanding, it points to the data a>
first step
When transaction 2 executes the first modification statement, the undo log will be recorded. Record the appearance of the data before the change; and then update the record , and record the transaction ID and rollback pointer of this operation. The rollback pointer is used to specify which version to roll back to if a rollback occurs.
The second step
When transaction 3 executes the first modification statement, the undo log will also be recorded to record the data before the change; then update the record, and record the transaction of this operation ID, rollback pointer, the rollback pointer is used to specify which version to roll back to if a rollback occurs.
third step
When transaction 4 executes the first modification statement, the undo log will also be recorded to record the appearance before the data change; then the record is updated, and the transaction ID and rollback pointer of this operation are recorded. The rollback pointer is used to specify what happens if Roll back, which version to roll back to.
Finally, we found that modifications to the same record by different transactions or the same transaction will cause the record's undolog to generate a record version linked list. The head of the linked list is the latest old record, and the tail of the linked list is the oldest old record.
readview
ReadView is the basis for MVCC to extract data when snapshot read SQL is executed. It records and maintains the currently active transaction (uncommitted) ID of the system.
ReadView contains four core fields:
| Field | meaning |
|---|---|
| m_ids | The collection of currently active transaction IDs |
| my_trx_id | Minimum active transaction ID |
| max_trx_id | Pre-allocated transaction ID, current maximum transaction ID + 1 (because transaction ID is auto-incrementing) |
| creator_trx_id | Transaction ID of the ReadView creator |
The access rules for version chain data are specified in readview (trx_id represents the transaction ID corresponding to the current undolog version chain):trx_id represents the transaction ID corresponding to the current undolog version chain a>
| condition | Is it accessible? | illustrate |
|---|---|---|
| trx_id == creator_trx_id | This version can be accessed | Established, indicating that the data was changed by the current transaction. |
| trx_id < min_trx_id | This version can be accessed | Established, indicating that the data has been submitted. |
| trx_id > max_trx_id | This version is not accessible | Established, indicating that the transaction was started after ReadView was generated. |
| min_trx_id <= trx_id <= max_trx_id | If trx_id is not in m_ids, this version can be accessed | Established, indicating that the data has been submitted. |
Different isolation levels have different timings for generating ReadView:
- READ COMMITTED: A ReadView is generated every time a snapshot read is performed in a transaction.
- REPEATABLE READ: A ReadView is generated only when a snapshot read is performed for the first time in a transaction, and the ReadView is reused subsequently.
Principle analysis
RC isolation level
Under the RC isolation level, ReadView is generated every time a snapshot read is performed in a transaction.
In transaction 5, the record with ID 30 was queried twice. Since the isolation level is Read Committed, a ReadView will be generated for each snapshot read. The ReadView generated twice is as follows. When snapshot reading obtains data, it needs to match the data in the undolog version chain according to the generated ReadView and ReadView's version chain access rules, and finally decide The data returned by this snapshot read.
Let’s first look at the specific reading process of the first snapshot read:
That is to say, as long as you follow the access rules of readview, as long as there is one that satisfies the rules, you can return the corresponding snapshot. According to this rule, mvcc implements the transaction isolation level of read committed.
- First match
this record, the trx_id corresponding to this record is 4, that is, 4 is brought into the matching rule on the right. ①Not satisfied ②Not satisfied ③Not satisfied ④Neither satisfied , If neither is satisfied, continue to match the next item in the undo log version chain.
- then matches the second one
. The corresponding trx_id of this record is 3, that is, 3 is brought into the matching rule on the right. ①Not satisfied ②Not satisfied ③Not satisfied ④Neither satisfied nor satisfied, then continue to match the next item in the undo log version chain.
- then matches the third record
. The corresponding trx_id of this record is 2, that is, 2 is brought into the matching rule on the right. ①Not satisfied ②Satisfied Terminate matching, this snapshot read, the data returned is the data recorded in the version chain.
RR isolation level
Under the RR isolation level, ReadView is generated only when a snapshot read is performed for the first time in a transaction, and the ReadView is reused subsequently. RR is repeatable read. In a transaction, if the same select statement is executed twice, the query results will be the same.
We see that under the RR isolation level, the ReadView is only generated when the first snapshot is read in the transaction, and the ReadView is reused in subsequent times. Since the ReadView is the same, the version chain matching rules of the ReadView are also the same, so the final snapshot read The result returned is also the same
Summarize
The implementation principle of MVCC is realized through the hidden fields of the InnoDB table, UndoLog version chain, and ReadView. MVCC + lock achieves transaction isolation. The consistency is guaranteed by redolog and undolog.
