Mysql transaction isolation level and lock mechanism
Overview
Databases generally execute multiple transactions concurrently. Multiple transactions may concurrently perform CRUD operations on the same data, which sometimes leads to problems such as dirty reads, dirty writes, non-repeatable reads, and phantom reads.
In order to solve the problem of multi-transaction concurrency, Mysql database has designed a transaction isolation mechanism, a lock mechanism,
and an MVCC multi-version concurrency control isolation mechanism, using a set of mechanisms to solve the problem of multi-transaction concurrency.
Transaction and its ACID attributes
Transaction has the following four attributes, usually referred to as the ACID attribute of the transaction.
Atomicity (Atomicity) : A transaction is an atomic operation unit. The modification of data is either all executed or not executed.
Consistency (Consistent) : At the beginning and completion of the transaction, the data must be in a consistent state. This means that all relevant data rules
must be applied to transaction modifications to maintain data integrity.
Isolation : The database system provides a certain isolation mechanism to ensure that transactions are executed in an "independent" environment that is not affected by external concurrent operations. This means that the intermediate state in the transaction process is invisible to the outside, and vice versa.
Durable : After the transaction is completed, its modification to the data is permanent and can be maintained even if a system failure occurs.
Problems caused by concurrent transaction processing
Lost Update or Dirty Write
When two or more transactions select the same row of data, and then update the row based on the initially selected value, because each transaction does not know the existence of other transactions, loss will occur Update issue-The last update covers updates made by other firms.
Dirty Reads
A transaction is modifying a record. Before the transaction is completed and submitted, the data of this record is in an inconsistent state; at this time, another transaction also reads the same record, if not With control, the second transaction reads these "dirty" data and performs further
processing based on this , resulting in uncommitted data dependencies. This phenomenon is vividly called "dirty reading".
In a word: Transaction A reads the data that has been modified but not yet committed by Transaction B, and performs operations on this data. At this time, if the B transaction rolls back, the data read by A is invalid and does not meet the consistency requirements.
Can not be re-read (Non-Repeatable Reads)
a transaction at a certain time after the read data, read previously read data again, but read data is found to have occurred changed
variable, or some records have been deleted! This phenomenon is called "non-repeatable reading".
One sentence: the same query statement in transaction A reads inconsistent results at different times, and does not meet the isolation
phantom reads (Phantom Reads).
A transaction reads previously retrieved data under the same query conditions, but it is found that other transactions insert With new data that meets its query conditions, this phenomenon is called "phantom reading".
In a word: Transaction A reads the new data submitted by transaction B, which does not meet the isolation
transaction isolation level.
"Dirty read", "non-repeatable read" and "phantom read" are actually database read consistency issues, which must be The database provides a certain transaction isolation mechanism to solve it.
The stricter the transaction isolation of the database, the smaller the side effects of concurrency, but the greater the cost, because transaction isolation essentially makes transactions "serialized" to a certain extent, which is obviously contradictory to "concurrency". At the same time, different applications have different requirements for read consistency and transaction isolation. For example, multiple applications are not sensitive to "non-repeatable read" and "phantom read", and may be more concerned about the ability of concurrent data access.
The transaction isolation level of the current database: show variables like'tx_isolation';
Set the transaction isolation level: set tx_isolation='REPEATABLE-READ';
Mysql default transaction isolation level is repeatable read. When developing programs with Spring, if the isolation level is not set The isolation level set by Mysql is used by default. If Spring is set, the set isolation level will be used
Lock details
A lock is a mechanism for a computer to coordinate multiple processes or threads to access a resource concurrently. In a database, in addition to contention for traditional computing resources (such as CPU, RAM, I/O, etc.), data is also a resource that needs to be shared by users. How to ensure the consistency and effectiveness of concurrent data access is a problem that all databases must solve, and lock conflicts are also an important factor affecting the performance of concurrent database access.
Lock classification
- In terms of performance, it is divided into optimistic locking (implemented by version comparison) and pessimistic locking
- From the types of database operations, it is divided into read locks and write locks (both belong to pessimistic locks).
Read locks (shared locks, S locks (Shared)): For the same data, multiple read operations can be performed at the same time without each other Impact
Write lock (exclusive lock, X lock (eXclusive)): Before the current write operation is completed, it will block other write locks and read locks - From the granularity of data operations, it is divided into table locks and row locks
Table lock Locks the
entire table for each operation. Low overhead, fast locking; no deadlock; large locking granularity, the highest probability of lock conflicts, and the lowest concurrency; it is generally used in scenarios where the entire table is migrated.
Basic operation
- Manually increase the table lock
lock table table name read(write), table name 2 read(write); - View the locks added on the table
show open tables; - Delete table lock
unlock tables;
analysis
- Add read lock The
current session and other sessions can read the table. Inserting or updating the locked table in the current session will report an error, while inserting or updating other sessions will wait - Adding a write lock The
current session has no problem with adding , deleting, modifying, and checking the table, and all operations on the table by other sessions are blocked
Conclusion
1. Reading a MyISAM table (adding a read lock) will not block other processes from reading requests to the same table, but it will block writing requests to the same table. Only when the read lock is released, the write operations of other processes will be executed.
2. The write operation to the MylSAM table (write lock) will block other processes from reading and writing the same table. Only when the write lock is released, the read and write operations of other processes will be performed
Row lock Locks a
row of data for each operation. High overhead and slow locking; deadlock will occur; the smallest locking granularity, the lowest probability of lock conflicts, and the highest concurrency.
There are two biggest differences between InnoDB and MYISAM:
- InnoDB supports transactions (TRANSACTION)
- InnoDB supports row-level locks
Scenario
One session opens the transaction update and does not commit, another session updates the same record will block, update different records will not block
to sum up:
MyISAM will automatically add read locks to all the tables involved before executing the query statement SELECT, and automatically add write locks to the tables involved in the update, insert, and delete operations.
InnoDB will not lock when executing the query statement SELECT (non-serial isolation level). However, update, insert, and delete operations will add row locks.
In short, read locks will block writes, but not reads. The write lock will block both reading and writing.
Read uncommitted : set tx_isolation='read-uncommitted'
B's transaction has not yet been committed, A can query the updated data of B. Once B's transaction is rolled back for some reason, all operations will be cancelled. The data queried by A is actually dirty data
Read committed : set tx_isolation='read-committed'
A queries all records. Before A's transaction is submitted, open B and update the table. B's transaction has not been submitted yet. A cannot query B's updated data, which solves the dirty Read problem, B’s transaction is submitted, A executes the same query as the previous step, and the result is inconsistent with the previous step, that is, a non-repeatable read problem occurs
Repeatable read : set tx_isolation='repeatable-read'
A queries all records. Before A's transaction is submitted, open B, update the table and submit. All records in the query table of A are consistent with the result of the previous query, no non-repeatable The problem of reading. Reopen B, insert a new piece of data and submit it. Inquire all the records in the table A, no new data is found, so it feels that there is no phantom reading, but the newly added data can be updated successfully by executing update in A. The new data of B can be found by querying again, and there is a phantom reading problem
Serialization : set tx_isolation='serializable'
A query all records, open B, update the same record will be blocked waiting, update different records can be successful, indicating that innodb queries in serial mode will also be locked.
If A executes a range query, then all rows in the range including the gap interval range where each row record is located will be locked. At this time, if B inserts data in this range, it will be blocked, so phantom reads are avoided.
This isolation level has extremely low concurrency and is rarely used in development.
Gap Lock Gap Lock
locks the gap between two values. Mysql default level is repeatable-read, there is no way to solve the problem of phantom reading, gap lock can solve the problem of phantom reading in some cases.
Range: All line records included in the range (including gap line records) and the gap where the line records are located will be locked (the right side does not include the left side).
The gap lock will only take effect under the repeatable read isolation level.
Next-key Locks
Next-Key Locks is a combination of row lock and gap lock (not including the left side).
Non-indexed row locks will be upgraded to table locks. The
locks are mainly added to the index. If the non-indexed fields are updated, the row locks will become table locks.
InnoDB row locks are locks for indexes, not for records. And the index cannot be invalidated, otherwise it will be upgraded from row lock to table lock.
To lock a row, you can also use lock in share mode (shared lock) and for update (exclusive lock), for example: select * from test_innodb_lock where a = 2 for update; so other sessions can only read this row of data, and the modification will be Block until the session of the locked row is submitted
Conclusion
Because the Innodb storage engine implements row-level locking, although the performance loss caused by the implementation of the locking mechanism may be higher than that of table-level locking, it is far superior to MYISAM tables in terms of overall concurrent processing capabilities. Level locked. When the system concurrency is high, the overall performance of Innodb will have obvious advantages compared with MYISAM.
However, Innodb's row-level locking also has its fragile side. When we use it improperly, Innodb's overall performance may be worse.
Row lock analysis Analyze
the contention of row locks on the system by checking the InnoDB_row_lock state variable
1 show status like 'innodb_row_lock%';
The description of each state quantity is as follows:
Innodb_row_lock_current_waits: the number of locks currently waiting for
Innodb_row_lock_time: the total length of lock time from the start of the system to the present
Innodb_row_lock_time_avg: the average time spent on each wait
Innodb_row_lock_time_max: the longest waiting time from the start of the system to now Time
Innodb_row_lock_waits: the total number of waits since the system was started up to now.
For these 5 state variables, the most important ones are:
Innodb_row_lock_time_avg (average waiting time)
Innodb_row_lock_waits (total number of waits)
Innodb_row_lock_time (total waiting time)
especially when the number of waits is high , And every time the waiting time is not small, we need to analyze why there are so many waiting in the system,
and then proceed to formulate an optimization plan based on the analysis results.
View the related data sheet of INFORMATION_SCHEMA system library lock
1 ‐‐ 查看事务
2 select * from INFORMATION_SCHEMA.INNODB_TRX;
3 ‐‐ 查看锁
4 select * from INFORMATION_SCHEMA.INNODB_LOCKS;
5 ‐‐ 查看锁等待
6 select * from INFORMATION_SCHEMA.INNODB_LOCK_WAITS;
7
8 ‐‐ 释放锁，trx_mysql_thread_id可以从INNODB_TRX表里查看到
9 kill trx_mysql_thread_id
10
11 ‐‐ 查看锁等待详细信息
12 show engine innodb status\G;
Deadlock
set tx_isolation='repeatable-read';
Session_1 execution: select * from table where id=1 for update;
Session_2 execution: select * from table where id=2 for update;
Session_1 execution: select * from table where id=2 for update ;
Session_2 execution: select * from table where id=1 for update;
view recent deadlock log information: show engine innodb status\G; in
most cases, mysql can automatically detect deadlock and roll back the transaction that caused the deadlock, but some Situation MySQL cannot automatically detect deadlock
Lock optimization suggestions
- As far as possible, all data retrieval is completed through the index, to avoid the upgrade of non-indexed row locks to table locks
- Design the index reasonably to minimize the scope of the lock
- Minimize the scope of search conditions to avoid gap locks
- Try to control the size of the transaction, reduce the amount of locked resources and the length of time, and try to execute the SQL involving transaction locking at the end of the transaction
- Low-level transaction isolation possible