Hello everyone, I am Xiaolin.
Recently, the content of the article " What locks does MySQL have " has been added :
- Add record lock, gap lock, next-key lock
- Add insert intent lock
- Added a description of why the master-slave environment has insecurity when the auto-increment lock is innodb_autoinc_lock_mode = 2 mode
Therefore, the content is relatively comprehensive now, and basically all the locks used by MySQL have been described, and you can review them.
text
This time, let’s talk about MySQL locks , mainly in the form of Q&A, which seems relatively easy.
No more BBs, let's start!
In MySQL, according to the scope of locking, it can be divided into three types : global locks, table-level locks and row locks .
global lock
How is the global lock used?
To use a global lock, execute this command:
flush tables with read lock
After execution, the entire database is in a read-only state . At this time, other threads will be blocked when performing the following operations:
- Addition, deletion and modification operations on data, such as insert, delete, update and other statements;
- Change operations on the table structure, such as alter table, drop table and other statements.
If you want to release the global lock, you need to execute this command:
unlock tables
Of course, when the session is disconnected, the global lock will be released automatically.
What is the application scenario of the global lock?
The global lock is mainly used for logical backup of the whole database , so that during the backup of the database, the data in the backup file will not be different from the expected one due to the update of the data or table structure.
Give an example and everyone will know.
During the logical backup of the whole database, assume that no global lock is added to see what unexpected situations will arise.
If a user purchases a product during the logical backup of the full database, the general business logic of purchasing a product involves updating multiple database tables, such as updating the user’s balance in the user table, and then updating the purchased product in the product table inventory of goods.
Then, it is possible to have this sequence:
- Back up the data of the user table first;
- Then a user initiates the operation of purchasing a product;
- Then back up the data in the product table.
That is, between the backup user table and the product table, a user purchased a product.
In this case, the result of the backup is that the balance of the user in the user table is not deducted, but the inventory of the product in the product table is reduced. If the backup file is used to restore the database data later, the user has no shortage of money, but the inventory If there is less, it means that the user has bought a product for nothing.
Therefore, during the logical backup of the whole database, the above situation will not occur if the global lock is added.
What are the disadvantages of adding a global lock?
Adding a global lock means that the entire database is read-only.
Then if there is a lot of data in the database, backup will take a lot of time. The key is that during the backup period, the business can only read the data, but not update the data, which will cause the business to stagnate.
Since the use of global locks will affect the business when backing up database data, is there any other way to avoid it?
Yes, if the transaction supported by the database engine supports the isolation level of repeatable reading , then start the transaction before backing up the database, create a Read View first, and then use this Read View during the entire transaction execution, and because of the support of MVCC , the business can still update the data during the backup period.
Because under the isolation level of repeatable reading, even if other transactions update the data in the table, it will not affect the Read View when backing up the database. The data when the transaction was started.
The tool for backing up the database is mysqldump. When using the mysqldump –single-transactionparameter , it will start the transaction before backing up the database. This method is only applicable to storage engines that support "repeatable read isolation level transactions".
The default transaction isolation level of the InnoDB storage engine is repeatable read, so this method can be used to back up the database.
However, for an engine like MyISAM that does not support transactions, the global lock method must be used when backing up the database.
table lock
What are MySQL table-level locks? Specifically how to use it.
There are several types of table-level locks in MySQL:
- table lock;
- metadata lock (MDL);
- intent lock;
- AUTO-INC lock;
table lock
Let's talk about table locks first .
If we want to add a table lock to the student table (t_student), we can use the following command:
//表级别的共享锁，也就是读锁；
lock tables t_student read;
//表级别的独占锁，也就是写锁；
lock tables t_stuent write;
It should be noted that in addition to restricting the read and write operations of other threads, the table lock will also restrict the subsequent read and write operations of this thread.
That is to say, if this thread adds a "shared table lock" to the student table, then if this thread wants to perform a write operation on the student table, it will be blocked. Of course, other threads will also be blocked when writing to the student table. Will be blocked until the lock is released.
To release table locks, you can use the following command, which will release all table locks for the current session:
unlock tables
In addition, all table locks are released when the session exits.
However, try to avoid using table locks on tables that use the InnoDB engine, because the granularity of table locks is too large, which will affect concurrency performance. InnoDB is awesome because it implements finer-grained row-level locks .
metadata lock
Let's talk about metadata locks (MDL).
We don't need to use MDL explicitly, because when we operate on the database table, it will automatically add MDL to this table:
- When performing CRUD operations on a table, MDL read locks are added ;
- When doing structural changes to a table, MDL write locks are added ;
MDL is to ensure that when the user performs CRUD operations on the table, it prevents other threads from making changes to the table structure.
When a thread is executing the select statement (with MDL read lock), if other threads want to change the structure of the table (apply for MDL write lock), they will be blocked until the select statement is executed (release the MDL read lock) .
Conversely, when a thread changes the table structure (add MDL write lock), if other threads perform CRUD operations (apply for MDL read lock), they will be blocked until the table structure change is completed (release MDL write lock ).
MDL does not need to be explicitly called, so when is it released?
The MDL is released after the transaction is committed, which means that the MDL is always held during the execution of the transaction .
Then if the database has a long transaction (the so-called long transaction means that the transaction has been opened but has not been submitted), then when the table structure is changed, unexpected things may happen, such as the following sequence scene :
- First of all, thread A first enables the transaction (but has not submitted it), and then executes a select statement, at this time, the MDL read lock is first added to the table;
- Then, thread B also executes the same select statement, and it will not block at this time, because "reading" does not conflict;
- Then, thread C modifies the table field. At this time, because the transaction of thread A has not been submitted, that is, the MDL read lock is still occupied. At this time, thread C cannot apply for the MDL write lock and will be blocked.
Then after thread C is blocked, subsequent select statements for the table will be blocked. If there are a large number of requests for the select statement of the table at this time, a large number of threads will be blocked. At this time, the threads of the database It will fill up quickly.
Why thread C cannot apply for the MDL write lock, so the subsequent query operation for applying for the read lock will also be blocked?
This is because the operation of applying for an MDL lock will form a queue, and the priority of obtaining a write lock in the queue is higher than that of a read lock. Once an MDL write lock wait occurs, all subsequent CRUD operations on the table will be blocked.
Therefore, in order to safely change the table structure, before changing the table structure, first check the long transactions in the database to see if any transactions have added an MDL read lock to the table. If you can consider killing this long transaction, Then make changes to the table structure.
intent lock
Next, talk about intent locks .
- Before using the InnoDB engine table to add a "shared lock" to some records, you need to add an "intent shared lock" at the table level;
- Before using the InnoDB engine table to add an "exclusive lock" to some records, you need to add an "intent exclusive lock" at the table level;
That is, when performing insert, update, and delete operations, you need to add an "intent exclusive lock" to the table first, and then add an exclusive lock to the record.
The ordinary select does not add row-level locks. The ordinary select statement uses MVCC to achieve consistent reading and is lock-free.
However, select can also add shared locks and exclusive locks to records, as follows:
//先在表上加上意向共享锁，然后对读取的记录加共享锁
select ... lock in share mode;
//先表上加上意向独占锁，然后对读取的记录加独占锁
select ... for update;
Intention shared locks and intention exclusive locks are table-level locks, which will not conflict with row-level shared locks and exclusive locks, and there will be no conflicts between intention locks, only with shared table locks (*lock tables ... read* ) conflicts with exclusive table locks (*lock tables ... write*).
Table locks and row locks satisfy read-read sharing, read-write mutual exclusion, and write-write mutual exclusion.
If there is no "intent lock", then when adding "exclusive table lock", it is necessary to traverse all the records in the table to check whether there is an exclusive lock on the record, which will be very slow.
Then there is an "intent lock", because before adding an exclusive lock to the record, a table-level intentional exclusive lock will be added first, so when adding an "exclusive table lock", directly check whether the table has an intentional exclusive lock. It means that there are already records in the table that have been locked exclusively, so that there is no need to traverse the records in the table.
Therefore, the purpose of the intent lock is to quickly determine whether any records in the table are locked .
AUTO-INC lock
The primary key in the table is usually set to auto-increment, which is realized by declaring AUTO_INCREMENTthe attribute .
After that, when inserting data, you can not specify the value of the primary key, and the database will automatically assign an increasing value to the primary key, which is mainly realized through the AUTO-INC lock .
The AUTO-INC lock is a special table lock mechanism. The lock is not released after another transaction is committed, but is released immediately after the insert statement is executed .
When inserting data, a table-level AUTO-INC lock will be added , and then the AUTO_INCREMENTmodified field will be assigned an incremental value, and the AUTO-INC lock will be released only after the execution of the insert statement is completed.
Then, when a transaction is holding the AUTO-INC lock, other transactions will be blocked if they want to insert statements into the table, so as to ensure that the value of the AUTO_INCREMENTmodified
However, when inserting a large amount of data, the AUTO-INC lock will affect the insert performance, because the insert in another transaction will be blocked.
Therefore, starting from MySQL 5.1.22, the InnoDB storage engine provides a lightweight lock to achieve auto-increment.
Also when inserting data, a lightweight lock will be added to the AUTO_INCREMENTmodified field, and then an auto-increment value will be assigned to the field to release the lightweight lock without waiting for the entire insert statement to execute The lock is released only after completion .
The InnoDB storage engine provides an innodb_autoinc_lock_mode system variable, which is used to control whether to use AUTO-INC locks or lightweight locks.
-
When innodb_autoinc_lock_mode = 0, the AUTO-INC lock is used, and the lock is released after the statement is executed;
-
When innodb_autoinc_lock_mode = 2, a lightweight lock is used, and the lock is released after applying for the autoincrement primary key, and it does not need to wait for the statement to be executed before releasing it.
-
当 innodb_autoinc_lock_mode = 1：
-
- For ordinary insert statements, the self-increasing lock is released immediately after the application;
- For statements like insert ... select that insert data in batches, the self-increasing lock will not be released until the statement ends;
When innodb_autoinc_lock_mode = 2 is the highest performance method, but when the log format of binlog is statement and used together, data inconsistency will occur in the "master-slave replication scenario" .
As an example, consider the following scenario:
Session A inserts 4 rows of data into table t, then creates a table t2 with the same structure, and then two sessions simultaneously execute to insert data into table t2 .
If innodb_autoinc_lock_mode = 2, it means "release the lock after applying for the self-incrementing primary key, without waiting for the execution of the insert statement to complete". Then something like this might happen:
- session B first inserts two records, (1,1,1), (2,2,2);
- Then, session A applies for self-increment id to get id=3, and inserts (3,5,5);
- After that, session B continues to execute and inserts two records (4,3,3), (5,4,4).
It can be seen that the insert statement of session B generates discontinuous ids .
When this happens to the "main library", binlog will only record the insert statements of these two sessions when updating the t2 table. If binlog_format=statement, the recorded statement is the original statement. The order of recording is either to record the insert statement of session A first, or to record the insert statement of session B first.
But no matter which one it is, the binlog is taken to the "slave library" for execution. At this time, the slave library executes the statements in "order". Only after one SQL statement is executed, the next SQL statement will be executed. Therefore, the scenario where two sessions insert data into table t2 "simultaneously" like the master database will not happen on the slave database. Therefore, the insert statement of session B is executed on the standby database, and the ids in the generated results are all continuous. At this time, data inconsistency occurs in the master-slave library .
To solve this problem, the binlog log format should be set to row, so that the self-increment value allocated by the main library is recorded in the binlog. When the standby database is executed, what is the self-increment value of the main library and what is the self-increment value of the slave library.
Therefore, when innodb_autoinc_lock_mode = 2 and binlog_format = row, concurrency can be improved without data consistency problems .
row level lock
The InnoDB engine supports row-level locks, while the MyISAM engine does not support row-level locks.
As mentioned earlier, the ordinary select statement will not lock the record, because it belongs to the snapshot read. If you want to add a row lock to the record during the query, you can use the following two methods. The statement that the query will lock is called locked read .
//对读取的记录加共享锁
select ... lock in share mode;
//对读取的记录加独占锁
select ... for update;
The above two statements must be in a transaction, because when the transaction is committed, the lock will be released , so when using these two statements, add begin, start transaction or set autocommit = 0.
Shared locks (S locks) satisfy read-read sharing, and read-write mutual exclusion. The exclusive lock (X lock) satisfies the mutual exclusion of writing and writing, and the mutual exclusion of reading and writing.
There are three main types of row-level locks:
- Record Lock, record lock, that is, only lock a record;
- Gap Lock, gap lock, locks a range, but does not contain the record itself;
- Next-Key Lock: A combination of Record Lock + Gap Lock, which locks a range and locks the record itself.
Record Lock
Record Lock is called a record lock, which locks a record. And record locks are divided into S locks and X locks:
- When a transaction adds an S-type record lock to a record, other transactions can continue to add an S-type record lock to the record (S-type and S-locks are compatible), but cannot add an X-type record lock to the record (S-type not compatible with X locks);
- When a transaction adds an X-type record lock to a record, other transactions can neither add an S-type record lock to the record (S-type and X locks are incompatible), nor can it add an X-type record lock to the record (X type is not compatible with X-lock).
For example, when a transaction executes the following statement:
mysql > begin;
mysql > select * from t_test where id = 1 for update;
It is to add an X-type record lock to the record whose primary key id is 1 in the t_test table, so that other transactions cannot modify this record.
When the transaction commits, all locks generated during the transaction will be released.
Gap Lock
Gap Lock is called a gap lock, which only exists in the repeatable read isolation level. The purpose is to solve the phenomenon of phantom reading under the repeatable read isolation level.
Assume that there is a gap lock with a range id of (3, 5) in the table, then other transactions cannot insert the record with id = 4, which effectively prevents the occurrence of phantom reads.
Although gap locks have X-type gap locks and S-type gap locks, there is no difference. Gap locks are compatible, that is, two transactions can hold gap locks that contain a common gap range at the same time, and there is no mutual exclusion relationship. , proposed because the purpose of gap locks is to prevent phantom records from being inserted .
Next-Key Lock
Next-Key Lock is called Pro-Key Lock, which is a combination of Record Lock + Gap Lock, which locks a range and locks the record itself.
Assume that there is a next-key lock with the range id (3, 5] in the table, then other transactions can neither insert the record with id = 4 nor modify the record with id = 5.
Therefore, the next-key lock can not only protect the record, but also prevent other transactions from inserting new records into the gap in front of the protected record.
The next-key lock includes gap lock + record lock. If a transaction acquires an X-type next-key lock, another transaction will be blocked when it acquires an X-type next-key lock of the same range .
For example, if a transaction holds a next-key lock of type X with a range of (1, 10], another transaction will be blocked when it acquires a next-key lock of type X with the same range.
Although gap locks of the same range are mutually compatible among multiple transactions, for record locks, we need to consider the relationship between X-type and S-type, and X-type record locks conflict with X-type record locks.
insert intent lock
When a transaction inserts a record, it needs to determine whether the insertion position has been locked by other transactions (next-key lock also includes gap lock).
If there is, the insert operation will be blocked until the transaction that owns the gap lock commits (the moment when the gap lock is released), during which an insert intention lock will be generated , indicating that a transaction wants to insert a new record in a certain interval , but is now in a pending state.
For example, assume that transaction A has added a gap lock with a range id of (3, 5) to the table.
When transaction A has not yet been committed, transaction B inserts a new record with id = 4 into the table. At this time, it will be judged that the inserted position has been added with a gap lock by transaction A, so transaction B will generate an insertion intention lock. Then set the state of the lock to the waiting state ( PS: When MySQL adds a lock, the lock structure is generated first, and then the state of the lock is set. If the lock state is in the waiting state, it does not mean that the transaction has successfully acquired the lock. Only when the lock state In the normal state, it means that the transaction has successfully acquired the lock ), at this time, transaction B will be blocked until transaction A commits the transaction.
Although the name of the insertion intention lock has an intention lock, it is not an intention lock. It is a special gap lock and belongs to the row-level lock .
If the gap lock locks an interval, then the "insert intent lock" locks a point. Therefore, from this perspective, the insertion intention lock is indeed a special gap lock.
Another very important difference between insert intent locks and gap locks is that although "insert intent locks" also belong to gap locks, two transactions cannot be at the same time, one holds a gap lock, and the other owns the gap within the interval. Insert the intent lock (of course, it is possible to insert the intent lock if it is not in the gap lock interval).
References:
- "MySQL Technology Insider: innodb"
- "45 Lectures on MySQL Practical Combat"
- "Understanding MySQL from the root"