Locking is a key feature that distinguishes the database system from the file system. The lock mechanism is used to manage concurrent access to shared resources.
The InnoDB engine locks the table data to provide data integrity and consistency. In addition, it also uses locks in multiple places within the database to ensure concurrent access to a variety of different resources, such as adding, deleting, and modifying LRU The elements in the list.
The implementation of the lock mechanism used by different databases and engines may be completely different. For MyISAM, the lock is a table lock. There is no problem with concurrent reads, but the performance of concurrent inserts is poor. If the insert is at the bottom, MyISAM can still have certain concurrent operations. For SQL server, it was all page locks before the 2005 version. Compared with MyISAM's table lock performance, the performance of table locks has been improved. In the 2005 version, optimistic concurrency and pessimistic concurrency were supported. Under optimistic concurrency, row-level locks began to be supported. MySQL is completely different.
The implementation of InnoDB engine locks is very similar to that of Oracle. It provides consistent read and row-level lock support. Row-level locks have no related overhead and can simultaneously obtain concurrency and consistency.
Two row-level locks are implemented in InnoDB:
1. Shared lock (S Lock): allows transactions to read a row of data.
2. Exclusive lock (X Lock): Allow transactions to delete or update a row of data.
Lock compatibility: A transaction has already acquired the shared lock of row r, and another transaction can immediately acquire the shared lock of row r, because the read does not change the data of row r. At this time, if a transaction wants to obtain an exclusive lock on row r, it must wait for the transaction to release the shared lock on row r. This situation is called lock incompatibility.
The InnoDB engine supports multi-granularity locking, allowing row-level locks and table-level locks to exist at the same time. InnoDB supports an additional locking method, called intent locks. Intent locks are table-level locks. The design purpose is to indicate that a transaction is holding a lock on a row or is preparing to hold a lock. There are two intention locks. :
1. Intentional shared lock (IS Lock): A transaction wants to acquire a shared lock on certain rows in a table.
2. Intent exclusive lock (IX Lock): A transaction wants to acquire an exclusive lock on certain rows in a table.
Before the transaction requests the S lock and X lock, it needs to obtain the corresponding IS, IX locks, and intention locks.
View the information of the current lock request:
Before InnoDB Plugin, you can only view the current database request through the SHOW FULL PROCESSLIST, SHOW ENGINE INNODB STATUSwait command, and then judge the lock situation in the current transaction. In the new version, three tables, innodb_trx, innodb_locks, and innodb_lock_waits, have been added under the information_schema architecture to make it easier to monitor current transactions and analyze possible lock issues. First look at the innodb_trx table fields:
1.trx_id: the unique transaction ID inside the InnoDB engine.
2.trx_state: current transaction state.
3.trx_started: the start time of the transaction.
4. trx_requested_lock_id: the lock ID of the waiting transaction. If the trx_state state is LOCK WAIT, this value represents the ID of the lock resource occupied by the transaction before the current transaction is waiting. If trx_state is not LOCK WAIT, the value is NULL.
5.trx_wait_started: The time the transaction waits to start.
6.trx_weight: The weight of the transaction, reflecting the number of rows modified and locked by a transaction. When a deadlock needs to be rolled back, the InnoDB engine will choose the smallest value for rollback.
7.trx_mysql_thread_id: thread id in MySQL.
8.trx_query: SQL statement run by the transaction, this value is sometimes displayed as NULL in actual use.
The innodb_trx table can only display the currently running transactions and cannot determine the lock status. Use the innodb_locks table to view the lock:
1.lock_id: lock ID.
2. lock_trx_id: transaction ID.
3.lock_mode: the mode of the lock.
4. lock_type: the type of lock, table lock or row lock.
5.lock_table: The table to be locked.
6.lock_index: The index of the lock.
7.lock_space: table space ID.
8. lock_page: the number of pages that are locked, this value is NULL when the table is locked.
9.lock_rec: The number of rows locked, this value is NULL when the table is locked.
10.lock_data: the primary key value of the locked row, this value is NULL when the table is locked. This value is not a reliable value. When performing a range search, lock_data may only return the primary key value of the first row. If the current resource is locked, and the page in the locked buffer pool is replaced, the value will be displayed as NULL when viewing the innodb_locks table.
After detecting the lock status on each table, you can judge the lock waiting status. If the transaction volume is too large, it is not easy to judge. At this time, you can check the waiting status through the innodb_lock_waits table, which consists of the following fields:
1. requesting_trx_id: The transaction ID that requested the lock resource.
2.requesting_lock_id: ID of the requested lock.
3. blockint_trx_id: the transaction ID that currently occupies this lock.
4. blocking_lock_id: The ID of the currently occupied lock.
You can execute the following joint query to visually see the detailed information:
SELECT r.trx_id waiting_trx_id, r.trx_mysql_thread_id waiting_thread, r.trx_query waiting_query, b.trx_id blocking_trx_id, b.trx_mysql_thread_id blocking_thread, b.trx_query blocking_query
FROM information_schema.innodb_lock_waits w
INNER JOIN information_schema.innodb_trx b
ON b.trx_id = w.blocking_trx_id
INNER JOIN information_schema.innodb_trx r
ON r.trx_id = w.requesting_trx_id;
INNER JOIN is equal to JOIN, meaning that the common part of the two tables is selected. The results of the operation are as follows:
Consistent non-locking row reading means that InnoDB reads the row data in the current execution time database by way of row multi-version control. If the read row is performing DELETE, UPDATE operations, at this time the read operation will not wait for the release of the row lock, but will read a snapshot data of the row: the
reason why it is called a non-locking read is because it is not needed Wait for the release of the X lock on the accessed row. The snapshot data is the data of the previous version of the row, which is implemented through the Undo segment, and the Undo segment was originally used to roll back the data in the transaction, so the snapshot data itself has no overhead. Reading the snapshot data does not need to be locked, because there is no need to modify the historical data.
Non-locking read improves the concurrency of data read. Under the default settings of the InnoDB engine, this is the default read method, but under different transaction isolation levels, the read method is different, not every transaction isolation level Downstream reads are all consistent reads. Even if they all use consistent reads, the definition of snapshot data is different.
Snapshot data is the historical version before the current row data. There may be multiple historical versions, and a row may have more than one snapshot data. This is called row multi-version technology. The concurrency control brought by this is called multi-version concurrency control ( Multi Version Concurrency Control, MVCC).
Under Read Committed and Repeatable Read (InnoDB engine default transaction isolation level), InnoDB uses non-locking consistent reads, but they have different definitions for snapshots. Under Read Committed transaction isolation levels, non-consistent reads are always read locked. The latest snapshot data of the row, in the Repeatable Read transaction isolation level, non-consistent read always reads the row data version at the beginning of the transaction.
If there are two transactions, A transaction is as follows:
At this time transaction A has started and read the row data with id 1, but the transaction has not ended. At this time, make the following changes in transaction B: the
same transaction B has not been committed At this time, an X lock is added to the row with id 1. If the row with id 1 is read in transaction A at this time, if the transaction isolation level of the InnoDB engine is now Read Committed or Repeatable Read, non-locking will be used Consistent reading of the snapshot content, the result is as follows (assuming that there are only these two transactions during this period, there will be only one version of the snapshot data
at this time ): if transaction B commits the transaction
at this time : if transaction A reads again at this time For data with id 1, the results are different under different transaction isolation levels:
1. If it is Read Committed transaction isolation level:
it will always read the latest version of the row, if the row is locked, read the latest snapshot, Therefore, the row data with id 1 cannot be read:
2. If it is Repeatable Read transaction isolation level:
it will always read the row data at the beginning of the transaction, so A should be able to read the row with id 1 at this time :
Show the above example from a time perspective:
For the Read Committed transaction isolation level, it violates the isolation in ACID.
By default, the SELECT operation of the InnoDB engine uses consistent non-locking read, but in some cases it is necessary to lock the read operation:
1. SELECT… FOR UPDATE: add an X lock to the read row record, other transactions want to be Any locks on these lines will be blocked.
2. SELECT… LOCK IN SHARE MODE: Add an S lock to the read row record, other transactions can add S lock to the locked record, but it will block the X lock transaction.
When the above two statements are used in a transaction, the transaction is committed and the lock is released.
For consistent non-locking read, even if the read row has been used SELECT... FOR UPDATE, it can be read,
The InnoDB engine has an auto-increment counter for each table that contains an auto-increment value to get the value of the counter:
SELECT MAX(auto_inc_col)
FROM tableName
FOR UPDATE;
The insert operation adds 1 to this counter value and assigns it to the self-incrementing column. The implementation method is AUTO-INC Locking. This implementation method uses a special table locking mechanism. In order to improve the insert performance, the lock is not released after a transaction ends. Instead, it is released immediately after the SQL inserted into the self-increasing value is completed. This method has poor performance for concurrent inserts with self-growth columns, and needs to wait for the completion of the previous insert. For inserts with large amounts of data such as INSERT… SELECT, other insert operations will be blocked.
Starting from MySQL 5.1.22, the InnoDB engine provides a lightweight mutex auto-increment implementation mechanism, which greatly improves the performance of auto-increment value insertion. Starting from this version, the InnoDB engine provides the parameter innodb_autoinc_lock_mode, the default value is 1. , The following is the self-increasing insertion classification:
1. INSERT-like refers to all insert statements, such as INSERT, REPLACE, INSERT… SELECT, REPLACE… SELECT, LOAD DATA, etc.
2. Simple inserts refers to statements that can confirm the number of rows inserted before inserting, such as INSERT, REPLACE, etc., but not including INSERT… ON DUPLICATE KEY UPDATE (MySQL specific statement, when the primary key of the insert record conflicts in the table, execute Update ) This type of SQL statement.
3. Bulk inserts refer to statements that cannot determine the number of rows inserted before inserting, such as INSERT… SELECT, REPLACE… SELECT, LOAD DATA statements.
4. Mixed-mode inserts means that part of the value in the insert is self-increasing. For example, when the INSERT statement inserts multiple rows of data at the same time, a part of the self-increment column value is given, and the part is NULL value or INSERT… ON DUPLICATE KEY UPDATE.
The optional value of the parameter innodb_auoinc_lock_mode:
1. It 0is the realization of self-increment before version 5.1.22, that is, the AUTO-INC Locking method of table locks.
2. It 1is the parameter default value. For Simple inserts, this value uses a mutex to accumulate the counter in the memory. For Bulk inserts, the traditional AUTO-INC Locking method is still used. At this time, regardless of the rollback operation, the growth of self-value added is continuous. In this case, if the AUTO-INC Locking method is used to generate the self-increasing value, the operation of Simple inserts will still wait for the release of AUTO-INC Locking before the INSERT statement is over.
3. 2All INSERT-like self-increasing values are generated through mutex, which is the highest performance method, but it will bring some problems. Due to the existence of concurrent inserts, the self-increasing value of each insert It may not be continuous. If the master-slave replication is using Statement-Base Replication (the master-slave library uses the same SQL statement), the auto-increment column values of the master-slave library may be different due to concurrency, and Row- should be used in this case. Base Replication (the master and slave libraries make the same changes, and the row changes are recorded in the binlog instead of SQL statements).
MyISAM uses table locks, self-growth does not need to consider concurrent insertion issues.
In the InnoDB engine, the value of the self-increasing column must be an index, and it is the first column of the index. If it is the second column, an error will be reported. MyISAM does not have this problem:
CREATE TABLE tab (
a INT auto_increment,
b INT,
KEY(b, a)
) ENGINE = InnoDB;
Run it:
For foreign key columns, if you do not explicitly add an index to this column, the InnoDB engine will automatically add an index to it, so that table locks can be avoided.
To insert or update a foreign key column, you need to first query the records in the parent table, that is, the SELECT parent table. For the SELECT of the parent table, the consistent non-locking read is not used. This will cause data inconsistency. It will use SELECT… LOCK IN SHARE MODE, add an S lock to the parent table, if the X lock is already added to the parent table at this time, the operation on the child table will be blocked:
in the case of the above figure, the two transactions do not have COMMIT or ROLLBACK, and the transaction is at this time B will be blocked because transaction A adds an X lock on the row of the parent table with id=3, and transaction B needs to add an S lock on the row of the parent table with id=3, and transaction B will be blocked. If transaction B uses a consistent non-locking read when accessing the parent table, in the case of InnoDB's default transaction isolation level Repeatable Read, a row with id 3 will be read in the parent table, and the insert operation can be performed, and transaction A commits After that, there is no record with id 3 in the parent table, and there will be inconsistencies between the parent and child tables. If you query the innodb_locks table at this time:
InnoDB engine has three row lock algorithm designs:
1. Record Lock: lock on a single row record.
2. Gap Lock: gap lock, lock a range, excluding the record itself.
3.Next-Key Lock: Use with 2 to lock a range and lock the record itself. It is a locking algorithm that combines Gap Lock and Record Lock. Under this algorithm, the InnoDB engine uses this locking algorithm for row queries. For different SQL query statements, a shared Next-Key Lock and an exclusive Next- may be set. Key Lock.
Record Lock will always lock the index record. If the index is not set when the InnoDB engine table is created, the InnoDB engine implicit primary key will be used for locking.
To demonstrate Next-Key Lock, first create a table:
CREATE TABLE nkl (
a INT,
PRIMARY KEY(a)
) ENGINE = InnoDB;
Insert the following data into the table:
BEGIN;
INSERT INTO nkl
SELECT 1;
INSERT INTO nkl
SELECT 2;
INSERT INTO nkl
SELECT 3;
INSERT INTO nkl
SELECT 4;
INSERT INTO nkl
SELECT 7;
INSERT INTO nkl
SELECT 8;
COMMIT;
Then execute the following two transactions:
In the above case, transaction B will be locked regardless of whether 5 or 6 is inserted, because under the Next-Key Lock algorithm, (-∞, 6]all the values in the interval are locked, and it is possible to insert 9 at this time. For single-value index queries, Gap Lock is not needed, just add Record Lock. The InnoDB engine will choose the smallest algorithm model by itself.
The demonstration process in the above example is performed under the default transaction isolation level of InnoDB, that is, in Repeatable Read mode, the Next-Key Lock algorithm is the default row and record locking algorithm.
The lock may bring the following three problems:
1. Lost update: When multiple users modify a record at the same time, the update may be lost, that is, the update operation of the user is overwritten by the update operation of another user, such as the following situations:
(1) Transaction 1 Query a row of data, put it into local memory, and display it to user 1.
(2) Transaction 2 queries the same row of data, puts it into local memory, and displays it to user 2.
(3) User 1 modifies this row of records, updates the database and submits it.
(4) User 2 modifies this row of records, updates the database and submits it. At this time, the modification of user 1 is lost.
To avoid losing updates, you need to make the transaction a serial operation. You can add an exclusive lock to the record in step (1) (the for update clause in the figure below). At this time, transaction 2 needs to wait for step (1) (3) Step (2) can be performed to query data after completion:
2. Dirty read: Dirty data and dirty pages are different. Dirty pages refer to pages that have been modified in the buffer pool but have not been flushed to disk, that is, pages and pages in memory The data in the pages of the disk is inconsistent (of course, the redo log has been written before the data is flushed to the disk), and the dirty data refers to transactions that have been modified in the buffer pool but have not yet been committed.
Reading dirty pages does not affect data consistency, because valid data is in dirty pages, so asynchronously synchronized data can also bring performance improvements. Reading dirty data means that one transaction reads uncommitted data in another transaction, which violates the isolation of the database.
The transaction isolation level Read Uncommitted will happen dirty read.
3. Non-repeatable read: When the same data is read multiple times in the same transaction, in the interval between the two read data, the data read twice is different due to the modification of another transaction.
The transaction isolation level Read Committed will lead to non-repeatable reads. Generally, the problem of non-repeatable reading is acceptable. The default isolation level of SQL server and Oracle database is Read Committed.
The InnoDB engine avoids the problem of non-repeatable reading by using the Next-Key Lock algorithm. The official MySQL document defines non-repeatable reading as the Phantom Problem. Under this algorithm, the scan of the index locks not only the scanned index. , It also locks the range covered by these indexes, so inserts in this range are not allowed, which avoids the non-repeatable read problem caused by other transactions inserting data in this range. InnoDB uses the default transaction isolation level The Next-Key Lock algorithm avoids the problem of non-repeatable reading.
Due to the compatibility relationship between different locks, sometimes a lock in one transaction needs to wait for the lock in another transaction to release the resources it occupies, which is blocking.
In the InnoDB engine, the parameter innodb_lock_wait_timeout is used to control the maximum time of blocking waiting (default 50 seconds, dynamic parameters, which can be adjusted at runtime), and the parameter innodb_rollback_on_timeout is used to set whether to roll back the transaction in progress when the waiting timeout (default OFF, No rollback, static parameters).
By default, InnoDB does not roll back the blocking and timeout transaction, but commits it, which may cause an error. The following is an example:
add a Next-Key Lock to the above table in transaction A:
in transaction B, insert operation:
visible Transaction B has not been executed, but the executed part is committed.
Classic deadlock situation:
As shown in the figure above, InnoDB has detected a deadlock. Most deadlocks can be detected without human intervention. In the above example, transaction B is rolled back after deadlock occurs, and the lock is released, transaction A gets the resources. The InnoDB engine does not roll back most of the error exceptions, except for deadlocks, so when a deadlock exception occurs, there is no need to roll back the transaction again.
The common cause of deadlock in the Oracle database is that no index is added to the foreign key, and the InnoDB engine will automatically add an index to the foreign key column in the child table, and an exception will be thrown when the index is deleted.
Lock upgrade refers to expanding the scope of locks, such as upgrading row locks to page locks, and upgrading page locks to table locks. Some database designs consider locks to be a scarce resource. If you want to avoid the overhead of locks, you can upgrade locks. SQL server does this. It will automatically upgrade row, key or page-level locks to more coarse-grained locks when appropriate. , It protects system resources and prevents the system from using too much memory to maintain locks, which improves efficiency to a certain extent, but it will reduce concurrent performance.
After SQL server 2005, row locks are newly supported, but its design is completely different from the InnoDB engine. Lock escalation may still occur in the following situations:
1. The number of locks held by a single SQL statement on an object exceeds the threshold (default 5000 ), if it is a different object, no lock escalation will occur.
2. When the memory occupied by the lock resource exceeds 40% of the active memory.
The InnoDB engine does not have lock upgrades. For it, 1 lock is the same as 1,000,000 locks, and there is no overhead. This is similar to the Oracle database.