A transaction is to ensure that a set of database operations either all succeed or all fail. In MySQL, transaction support is implemented at the engine layer. As you know now, MySQL is a system that supports multiple engines, but not all engines support transactions. For example, MySQL's native MyISAM engine does not support transactions, which is one of the important reasons why MyISAM was replaced by InnoDB.
When multiple transactions are executed on the database at the same time, there may be problems with dirty read, non-repeatable read, and phantom read, so there is an "isolation level" concept. SQL standard transaction isolation levels include: read uncommitted (read uncommitted), read committed (read committed), repeatable read (repeatable read) and serialization (serializable).
- Read uncommitted means that when a transaction is not yet committed, the changes it makes can be seen by other transactions.
- Read commit means that after a transaction is committed, the changes it makes will be seen by other transactions.
- Repeatable read means that the data seen during the execution of a transaction is always consistent with the data seen when the transaction is started. That is, uncommitted changes are also invisible to other transactions.
- Serialization means that for the same row of records, "write" will add "write lock", and "read" will add "read lock". When a read-write lock conflict occurs, the later-accessed transaction must wait for the completion of the previous transaction before it can continue.
In implementation, a view will be created in the database, and the logical result of the view will prevail when accessing. In the "repeatable read" isolation level, this view is created when the transaction is started, and this view is used throughout the existence of the transaction. In the "read commit" isolation level, this view is created when each SQL statement starts to execute. It should be noted here that the "read uncommitted" isolation level directly returns the latest value on the record, without the concept of view; while the "serialization" isolation level directly uses locking to avoid parallel access.
After talking about the isolation level of transactions, let's take a look at how transaction isolation is implemented. Here we expand on the description of " repeatable reading ".
In MySQL, actually every record will record a rollback operation when it is updated. The latest value on the record, through the rollback operation, can get the value of the previous state.
Assuming that a value is sequentially changed from 1 to 2, 3, and 4, there will be similar records in the rollback log. There can be multiple versions of the same record in the system, which is the multi-version concurrency control (MVCC) of the database .
If there is another transaction that is changing 4 to 5, the transaction corresponding to read-view A, B, and C will not conflict. When will the rollback log be deleted? The system will judge that when there are no transactions that need to use these rollback logs, the rollback logs will be deleted. When is it no longer needed? That is when there is no read-view earlier than this rollback log in the system.
MVCC uses multi-version management of data when concurrently accessing the database to avoid concurrent blocking of reading data due to the blockage of write locks. In layman's terms, MVCC saves the historical version of the data and processes whether the data is displayed according to the comparison version number, so as to achieve the effect of ensuring transaction isolation without locking when reading data.
In InnoDB, each row has 2 hidden columns DATA_TRX_ID and DATA_ROLL_PTR (if the primary key is not defined, there is also a hidden primary key column):
- DATA_TRX_ID represents the transaction ID that recently modified the row of data, the transaction may not necessarily be submitted.
- DATA_ROLL_PTR represents a pointer to the rollback segment of the row. All old versions on the row are organized in the form of a linked list in undo, and this value officially points to the history linked list of the row in undo.
The key to the entire MVCC is achieved through the two hidden columns DATA_TRX_ID and DATA_ROLL_PTR.
Long transactions mean that there will be very old transaction views in the system. Since these transactions may access any data in the database at any time, before the transaction is committed, the rollback records that it may use in the database must be retained, which will cause a large amount of storage space to be occupied. In MySQL 5.5 and earlier versions, the rollback log is placed in the ibdata file together with the data dictionary. Even if the long transaction is finally committed and the rollback segment is cleaned up, the file will not become smaller. For example, the data is only 20GB, and the rollback segment has a library of 200GB. In the end, it has to clean up the rollback segment and rebuild the entire library. Long transactions should be avoided as much as possible.
How the transaction is started:
- Explicitly start the transaction statement, begin or start transaction. The supporting commit statement is commit, and the rollback statement is rollback.
- set autocommit=0, this command will turn off the automatic submission of this thread. This means that if you only execute a select statement, the transaction will start and it will not be committed automatically. This transaction continues until you actively execute a commit or rollback statement, or disconnect.
Some client connection frameworks will execute a set autocommit=0 command first after a successful connection by default . This leads to the next query in the transaction, if it is a long connection, it will lead to unexpected long transactions. In addition, for a business that requires frequent use of transactions, the second method does not need to actively execute a "begin" at the beginning of each transaction, which reduces the number of statement interactions. If you also have this concern, I suggest that you use the commit work and chain syntax to submit the transaction and automatically start the next transaction. At the same time, the advantage is that it is clear whether each statement is in a transaction from the perspective of program development.
You can query long transactions that last longer than 60s in the innodb_trx table of the information_schema library:
select * from information_schema.innodb_trx where TIME_TO_SEC(timediff(now(),trx_started))>60In MySQL, there are two concepts of "view":
- One is view. It is a virtual table defined with a query statement, which executes the query statement and generates the result when it is called. The syntax for creating a view is create view..., and its query method is the same as that of a table.
- The other is the consistent read view used by InnoDB to implement MVCC, that is, consistent read view, which is used to support the implementation of RC (Read Committed) and RR (Repeatable Read) isolation levels. It has no physical structure and is used to define "what data can I see" during transaction execution.
The implementation logic of MVCC has been explained above, and the read view is disassembled here to explain the difference between query and update in another way.
First of all, I said before that the transaction display start mode, begin/start transaction, after the execution of the first statement that operates the InnoDB table, the transaction is really started, and then the consistency view is generated. In addition, we can also execute, start transaction with consistent snapshot , immediately start the transaction and generate a consistent view.
Under the repeatable read isolation level, the transaction "takes a snapshot" when it starts. Note that this snapshot is based on the entire library . The realization principle of this snapshot:
At the beginning of the transaction, apply for a unique transaction ID from InnoDB's transaction system (in strict increments in the order of application). Each row of data also has multiple versions. Each time the transaction updates the data, a new data version is generated, and the transaction id is assigned to the transaction ID of this data version, which is recorded as row trx_id. At the same time, the old data version should be retained, and in the new data version, there is information that can be directly obtained. In other words, a row of records in the data table may actually have multiple versions (row), and each version has its own row trx_id.
The following figure shows the state after a record is continuously updated by multiple transactions:
In the dashed box are 4 versions of the same row of data. The latest version is V4, and the value of k is 22. It was updated by the transaction with transaction id 25, so its row trx_id is also 25. U1, U2, and U3 are undo log (rollback log), which is the location pointed to by DATA_ROLL_PTR mentioned above. V1, V2, and V3 do not actually exist physically, but are calculated based on the current version and undo log each time it is needed. For example, when V2 is needed, it is calculated by V4 by executing U3 and U2 in turn.
In terms of implementation, InnoDB constructs an array for each transaction to store all transaction IDs that are currently "active" at the moment the transaction starts. "Active" means that it has been activated but has not yet been submitted. The minimum value of the transaction ID in the array is recorded as the low water mark, and the maximum value of the transaction ID that has been created in the current system plus 1 is recorded as the high water mark. This view array and the high water mark form a consistent view of the current transaction.
The visibility rule of the data version is obtained based on the comparison between the row trx_id of the data and the consistency view. InnoDB takes advantage of the feature of " all data has multiple versions " and realizes the ability to "create snapshots in seconds". This view array divides all row trx_ids into several different situations:
Query logic
For the start moment of the current transaction, a data version of row trx_id has the following possibilities:
- If it falls in the green part, it means that this version is a committed transaction or generated by the current transaction itself, and this data is visible;
- If it falls in the red part, it means that this version is generated by a transaction started in the future and is definitely not visible;
- If it falls in the yellow part, it includes two situations: if row trx_id is in the array, it means that this version is generated by a transaction that has not yet been committed and is not visible; if row trx_id is not in the array, it means that this version has been committed The transaction generated (commit in the process of read view generation), visible.
Simply put, for a data version, for a transaction view, in addition to its own updates are always visible, there are three situations:
- The version has not been submitted and is not visible;
- The version has been submitted, but it was submitted after the view was created and is not visible;
- The version has been submitted, and it was submitted before the view was created and is visible.
Update update logic
Update data is read first and then written, and this read can only read the current value, which is called "current read". In fact, in addition to the update statement, if the select statement is locked, it is also currently read.
That is to add lock in share mode or for update, respectively add read lock (S lock, shared lock) and write lock (X lock, exclusive lock)
The core of repeatable reads is consistent read; while transactions update data, they can only use current reads. If the row lock of the current record is occupied by other transactions, you need to enter the lock wait.
The logic of read submission is similar to that of repeatable read. The main difference between them is:
- Under the repeatable read isolation level, you only need to create a consistent view at the beginning of the transaction, and then other queries in the transaction will share this consistent view;
- Under the read commit isolation level, a new view will be recalculated before each statement is executed.
Note: Under the read-commit isolation level, start transaction with consistent snapshot is meaningless and is equivalent to ordinary start transaction.
Here is a set of short answer questions:
Why does the database have transactions?
In order to ensure the final consistency of the data.
What are the characteristics of the transaction?
Atomicity, isolation, consistency, and durability.
What problems does the concurrency of transactions cause?
Concurrency of transactions will cause dirty reads, repeated reads, and phantom reads
How to solve the problem of transaction concurrency?
Set transaction isolation level, read uncommitted, read commit, repeated read, serialization.
In what way does the database ensure transaction isolation?
The isolation of transactions is achieved by locking.
What are the problems caused by frequent locking?
There is no way to modify it when reading data. There is no way to read data when modifying data, which greatly reduces database performance.
How does the database solve the performance problem after locking?
MVCC multi-version control realizes that the read data does not need to be locked, and the read data can be modified at the same time. It can be read at the same time when the data is modified.
The above is a summary of personal study in Lin Xiaobin’s column "45 Lectures on MySQL Practice" .