Introduction: Whenever there are multiple queries that need to modify data at the same time, concurrency control problems will arise. MySQL implements concurrency control through multi-version concurrency control and locking.
Whenever there are multiple queries that need to modify data at the same time, concurrency control problems will arise. MySQL implements concurrency control through multi-version concurrency control and locking.
When multiple users execute transactions concurrently to access the same database, consistency problems such as dirty writes, dirty reads, non-repeatable reads, and phantom reads may occur. The situations where concurrent transactions access the same record can be divided into the following three types.
-
read-read
That is, multiple concurrent transactions read the same record one after another. This situation is allowed because the read operation does not modify the contents of the record.
-
write-write
That is, multiple concurrent transactions modify the same record one after another. In this case, the "dirty write" phenomenon will occur, and any isolation level does not allow this phenomenon to occur. At this time, it is necessary to use the lock mechanism to queue these uncommitted concurrent transactions and execute them sequentially.
When a transaction wants to modify this record, it needs to lock it first, and the transaction can continue to operate if the lock is successful; if the lock fails, the transaction needs to wait.
-
read-write or write-read
That is, one transaction is performing a read operation and the other transaction is performing a write operation. In this case, dirty reads, non-repeatable reads, and phantom reads may occur. MySQL employs the following two solutions.
(1) Read operations use multi-version concurrency control (MVCC), and write operations are locked.
(2) Both read and write operations are locked.
Using the MVCC method, read operations and write operations do not conflict with each other, and the performance is higher. In the locking mode, read and write operations need to be queued for execution, which affects performance.
In general, MVCC can be used to solve the problem of concurrent execution of read and write operations, but some business scenarios require that the latest version of the record must be read every time, and the old version of the record is not allowed to be read. In this case, only Locking can be used.
01、MVCC
MVCC (Multi-Version Concurrency Control, Multi-Version Concurrency Control) is a lock-free concurrency control mechanism used to resolve read-write conflicts. It is used in the database to control concurrently executed transactions, so that transactions can be isolated. Its essence is to replace locking when performing read operations and reduce the burden caused by locking. The write operation uses the latest version of the record, and the read operation uses the historical version of the record, so that the read-write and write-read operations of different transactions can be executed concurrently, improving the concurrent performance of the database.
MVCC is controlled by saving a snapshot of the data at a certain point in time (Read View). The same data record can have multiple different versions, and the clustered index record and the roll_pointer attribute of the undo log are concatenated into a record version chain, and the visibility of a certain version of the record can be judged through the generated snapshot. By adding corresponding constraints when querying, the corresponding version of data that the user wants is obtained.
MVCC applies only to the READ COMMITTED (committed read) and REPEATABLE READ (repeatable read) levels in the MySQL isolation level. MVCC is actually the process of accessing the version chain of records when performing ordinary read operations using these two isolation levels of transactions.
1. The relationship between MVCC and the four isolation levels
1) READ UNCOMMITTED (read uncommitted)
Due to the presence of dirty reads, that is, the data rows of uncommitted transactions can be read, so MVCC is not applicable.
2) SERIALIZABLE (serialization)
Since InnoDB will lock the tables involved, it is not a row-level lock, and there is no row version control problem, so MVCC is not applicable.
3) READ COMMITTED (commit reading)
A snapshot is generated every time data is read, and the old snapshot is updated to ensure that the content committed by other transactions can be read.
4) REPEATABLE READ (repeatable read)
A snapshot is only generated when the data is read for the first time, and will not be updated in the future. All subsequent read operations will reuse this snapshot to ensure the consistency of each read operation.
It can be seen that although the isolation level of REPEATABLE READ (repeatable read) is higher than that of READ COMMITTED (committed read), the overhead is relatively small because snapshots are not updated frequently.
2. Two ways to read data records
1) Current read
Read the latest version of the current data, and after reading the data, the data will be locked to prevent other transactions from changing it. When performing a write operation, a "current read" is required to read the latest version of the data record.
2) Snapshot read
In fact, it is to read the snapshot in MVCC, which can read all version information of the data, including the information of the old version. That is to say, what "snapshot read" reads is not necessarily the latest version of the data, but may be the previous historical version.
Under the READ COMMITTED isolation level, "snapshot read" and "current read" have the same results, and both read the latest version of data that has been submitted.
Under the REPEATABLE READ (repeatable read) isolation level, "current read" is the latest version of data that has been submitted by other transactions, and "snapshot read" is the version read before the current transaction. The timing of creating a snapshot determines the read version.
In MySQL, MVCC is implemented by three implicit columns in records, undo logs and snapshots, etc.
02. Lock mechanism
MySQL supports different storage engines, and the locking mechanisms of different storage engines are also different. For example: MyISAM and MEMORY storage engines only support table-level locks; InnoDB storage engines support both row-level locks and table-level locks, but row-level locks are used by default.
1. Classification of locks
1) Classified by lock granularity
Theoretically, each time only the data currently being operated is locked, the maximum concurrency will be obtained, but managing locks is very resource-intensive. Therefore, the database system needs to strike a balance between high concurrent response and system performance, thus creating the concept of "lock granularity".
Lock granularity, that is, the locked data range, can measure the relationship between the overhead of managing locks and concurrency performance. The larger the lock granularity, the larger the locking range, the lower the overhead of managing locks, and the worse the concurrency. According to the lock granularity from large to small, it can be divided into the following three types of locks.
(1) Table-level Locking (Table-level Locking): also known as table locks, used to lock a table. Depending on the type of lock, other users cannot insert records into the table, and are even restricted from reading data from it. There are two types of table-level locks: read locks and write locks.
Table-level locks are characterized by low overhead and fast locking; no deadlocks; large locking granularity, the highest probability of lock conflicts, and the lowest concurrency.
(2) Page-level Locking: It is used to lock a page where data records are located. It is a unique locking level in MySQL, which is not common in other database management software. Page is the basic unit of interaction between disk and memory, and the page size is generally 16KB.
The characteristics of page-level locks are that the overhead and locking time are between table locks and row locks; deadlocks may occur; the locking granularity is between table locks and row locks, and the concurrency is average.
(3) Row-level Locking (Row-level Locking): also known as row lock, used to lock a row (that is, a record). In this case, only the rows used by the thread are locked, and other rows in the table are available to other threads, so row-level locks can support concurrent processing to the greatest extent. Row-level locking is not a locking mechanism provided by MySQL, but implemented by the storage engine itself. The locking mechanism of InnoDB is row-level locking.
Row-level locks are characterized by high overhead and slow locking; deadlocks may occur; the locking granularity is the smallest, the probability of lock conflicts is the lowest, and the concurrency is the highest. There are three types of row-level locks: exclusive locks, shared locks, and intent locks.
Table-level locks are implemented by the database server, and row-level locks are implemented by the storage engine. The smaller the scope of data locking, the better the concurrency of the database.
2) Classified by the type of database operation
Transactions can read or write to the database. According to the type of database operation, it can be divided into the following four types of locks.
(1) Shared lock (Shared Lock): also called read lock, referred to as S lock. When a transaction wants to read a record, it needs to acquire the shared lock of the record first. The read operation will not modify the record data. Multiple read operations can be performed at the same time without affecting each other. Multiple transactions can add shared locks to the same record at the same time. With shared locks, exclusive locks cannot be acquired.
(2) Exclusive lock: also called exclusive lock, write lock, or X lock for short. When a transaction wants to modify a record, it needs to acquire an exclusive lock on the record first. It blocks other exclusive and shared locks until the current transaction's write operation is complete. After the current transaction commits, the exclusive lock will be released.
(3) Intention Shared Lock: IS lock for short. When a transaction is about to add a shared lock on a record, it needs to add an intent shared lock at the table level first.
(4) Intention Exclusive Lock: IX lock for short. When a transaction is about to add an exclusive lock on a record, it needs to add an intent exclusive lock at the table level first.
hint
Intentional shared locks and intentional exclusive locks are table-level locks. MySQL designs them to quickly determine whether the records in the table are locked when adding table-level shared locks and exclusive locks later, so as to avoid viewing through traversal There are no locked records in the table. Table 12-1 shows the compatibility relationship of these four locks at the table level.
■ Table 12-1 Compatibility relationship of table-level locks
2. Manage locks in the InnoDB storage engine
1) Shared locks and exclusive locks at the table level
(1) The basic syntax for setting table-level locks is as follows.
The syntax is explained below.
tbl_name [[AS] alias] is "table name [as alias]".
READ is to add a shared lock to the table.
WRITE is to add an exclusive lock to the table.
(2) After the lock operation on the data table is completed, it needs to be unlocked. The basic syntax format is as follows.
hint
Table-level shared locks and exclusive locks in the InnoDB storage engine are only used in some special cases (such as system crash recovery). When executing SELECT, INSERT, UPDATE, DELETE and other statements on a table, the InnoDB storage engine is No table-level shared or exclusive locks will be added to this table. Of course, users can manually add table-level locks on tables using the InnoDB storage engine using manual lock table statements such as LOCK TABLES as needed, but this should be avoided as much as possible, because this will not only provide no additional protection, but will reduce concurrency capabilities.
In addition, when executing DDL statements such as ALTER TABLE, DROP TABLE, etc. on a certain table, other transactions will block when concurrently executing DML statements such as SELECT, INSERT, DELETE, UPDATE, etc. on this table. Similarly, when a transaction executes a DML statement on a table, other transactions that execute DDL statements on this table will also be blocked.
This process is implemented by using MetaData Lock (MDL) at the server layer. Generally, table-level shared locks and exclusive locks provided by storage engines are not used.
Metadata lock is not introduced here due to limited space, please refer to other materials by yourself.
[Example 12-11] Lock the account table accounts in the bank database in the form of a shared lock.
(1) Add a shared lock to the account table accounts.
(2) Query the data of accounts table accounts.
You can query all the records in the account table accounts, indicating that after adding a shared lock to the table, you can perform normal read operations on the table.
(3) Delete a record in the table.
At this time, there is an error in deleting records, and the error message is prompted: [Err] 1099 - Table 'accounts' was locked with a READ lock and can't be updated, indicating that after adding a shared lock, the table cannot be deleted.
(4) Keep the existing session window open, and query the data of accounts table accounts in a new session window. All the records in the account table accounts can be queried in the new session window, indicating that the shared lock is compatible with other sessions.
(5) Go ahead and add an insert statement in this new session window.
From the execution results, we can see that the statement has been in the waiting state of "processing", and no result is displayed. Because the account table accounts has a shared lock, other users cannot write to it.
(6) Enter the unlock statement in the previous session window.
As can be seen from the execution results, after the shared lock of the account table accounts is released, the insert statement is executed successfully immediately, and the newly added records can be queried by the query statement, but the execution time of the insert statement is the sum of the waiting time and the execution time of the statement.
2) Row-level shared locks and exclusive locks
Before users perform write operations such as INSERT, UPDATE, and DELETE on tables of the InnoDB storage engine, the storage engine will automatically add row-level exclusive locks for related records. When the statement is executed, the storage engine automatically unlocks it.
But for ordinary SELECT statements, the InnoDB storage engine does not automatically lock. To ensure that the data queried in the current transaction will not be updated or deleted by other transactions, and to avoid consistency issues such as dirty reads, non-repeatable reads, and phantom reads, it is necessary to explicitly add row-level shared locks and exclusive locks to query operations Lock.
(1) Set the row-level shared lock in the query statement, the basic syntax format is as follows.
The syntax is explained below.
FOR SHARE means to add a row-level shared lock when querying, followed by NOWAIT | SKIP LOCKED, these two parameters are new features of MySQL 8.0.
NOWAIT is an optional option that causes FOR SHARE or FOR UPDATE queries to execute immediately, returning an error if a row lock cannot be acquired because of a lock held by another transaction.
SKIP LOCKED is an optional option, which means that the FOR SHARE or FOR UPDATE query is executed immediately, and rows locked by another transaction are not included in the result set.
LOCK IN SHARE MODE also means to add a row-level shared lock when querying, which is the same function as FOR SHARE.
(2) Set the row-level exclusive lock in the query statement, the basic syntax format is as follows.
Grammar description: FOR UPDATE means to add row-level exclusive lock during query, followed by NOWAIT | SKIP LOCKED parameter, the meaning is the same as above.
hint
The life cycle of the above-mentioned row-level lock is very short. You can extend the life cycle of the row-level lock by manually opening the transaction. The life cycle of the row-level lock in the transaction starts from locking and does not end until the transaction is committed or rolled back.
[Example 12-12] Add a row-level lock to the account table accounts in the bank database.
(1) Open two session windows and switch to the bank database.
(2) In session window 1, add an exclusive lock to the row whose id value is 1 in the accounts table.
(3) In session window 2, enter the following code.
After executing the above code, the information of account A can be queried normally.
(4) Continue to input the following SQL statement in session window 2.
After executing the above code, it has been processing, and there is no result displayed, and it enters the exclusive lock waiting state. After waiting for a period of time, the lock waiting timeout prompt of "[Err] 1205 -Lock wait timeout exceeded; try restarting transaction" will be displayed.
If you enter the "ROLLBACK;" command in session window 1 while session window 2 is waiting, session window 2 will execute successfully immediately.
(5) Enter the "ROLLBACK;" statement or "COMMIT;" statement in the session window 2 to end the transaction.
hint
Row-level locks are only applicable to the InnoDB storage engine. If the table is not an InnoDB storage engine, you can use the "ALTER TABLE tablename ENGINE = storage engine name;" statement to change it. Of course, in an actual production environment with a huge amount of data, it is best not to change the storage engine casually.
You can use the following statement to view the storage engine of the table, and the execution result is shown in Figure 12-2.
■ Figure 12-2 View the storage engine of the table
3. Row-level lock types commonly used in InnoDB
There are three types of row-level locks commonly used by the InnoDB storage engine as follows.
(1) Record Lock: Record lock, which only locks the record itself, and is realized by locking the index row. Even if a table does not define any indexes, record locks will lock index records. If the table is created without any indexes, the InnoDB storage engine will use the implicit primary key for locking.
(2) Gap Lock: gap lock, which is used to lock the gap between records and prevent other transactions from inserting new records into the gap. It is set for the REPEATABLE READ (repeatable read) transaction isolation level, which can prevent phantom reading to the greatest extent. occur.
(3) Next-key Lock: A combination of Record Lock and Gap Lock, which not only protects the record itself, but also prevents other transactions from inserting new records into the gap. The InnoDB storage engine uses this locking method for row queries.
4. View transaction locking
In the InnoDB storage engine, you can use the following methods to check the locking status of transactions.
(1) Use the following statement under the MySQL console.
The amount of information output by this command is very large, divided into multiple sections of output, each section corresponds to different parts of the InnoDB storage engine, allowing users to understand the running status of the InnoDB storage engine, which is of great value to developers and maintenance personnel, especially When doing deadlock analysis and performance tuning.
The TRANSACTIONS part is the statistical information about the transaction, but this statement alone cannot show which transaction has added which locks to which records. You can set the system variable innodb_status_output_locks to ON first, and then run this command. The SQL statement runs as follows.
Taking the transaction in Example 12-12 as an example, the output of the TRANSACTIONS part is as follows.
In this way, which transaction adds which locks to which records is displayed clearly. Part of the output analysis is as follows.
① RECORD LOCKS space id 27 page no 4 n bits 72 index PRIMARY of table 'bank1'.'accounts' trx id 3859 lock_mode X locks rec but not gap waiting
The output here represents a lock structure, the space id is 27, the page no is 4, the n_bits attribute value is 72, the corresponding index is the clustered index PRIMARY, and the stored lock type is an X-type record lock (exclusive lock). The output after this statement is the detailed information of the locked record.
② TABLE LOCK table 'bank1'.'accounts' trx id 3859 lock modeIX
The output here indicates that the transaction with id 3859 has added a table-level intent exclusive lock to the accounts table in the bank database.
(2) You can check the currently executing transaction information through the INNODB_TRX table in the system database information_schema, part of the output is shown in Figure 12-3.
■ Figure 12-3 Partial output of INNODB_TRX table
From the above output, you can see the transaction id, status, start time, isolation level and other information. Among them, trx_tables_locked indicates how many table-level locks are added to the transaction, trx_rows_locked indicates how many row-level locks are added, and trx_lock_structs indicates how many in-memory lock structures are generated by the transaction.
5. Deadlock
Deadlock (Dead Lock) means that two or more processes need to use the same data. During the execution process, they are always in a state of waiting for the other party to release resources. If there is no external force, they will always be in a waiting state, so that A deadlock has occurred.
When MySQL detects a deadlock, it will select a smaller transaction to roll back, and prompt an error message: [Err] 1213 -Deadlock found when trying to get lock; try restarting transaction.
Tip: The so-called smaller transactions refer to transactions with fewer records affected during transaction execution.
In MySQL 8.0, if the lock cannot be obtained, adding NOWAIT, SKIP LOCKED parameters will skip the lock waiting, or skip the lock.
You can use the "SHOW ENGINE INNODB STATUS \G" statement to view the latest deadlock information. If deadlocks occur frequently, you can set the global system variable innodb_print_all_deadlocks to ON, and record the information each time a deadlock occurs in the MySQL error log, so that you can analyze more deadlocks by viewing the error log.
