This article makes a summary of locks, transactions, and concurrency control. After reading many articles on the Internet, the descriptions are very inaccurate. If there is any inconsistency with your point of view, you are welcome to pat bricks with reason!
mysql server logical architecture
Each connection will generate a thread on the mysql server (the thread is managed internally through the thread pool). For example, when a select statement enters, mysql will first check whether the result set of the select is cached in the query cache, and if not, continue to perform parsing, The process of optimization and execution; otherwise, the result set will be retrieved from the cache.
MySQL concurrency control - shared locks, exclusive locks
shared lock
Shared locks are also called read locks. Read locks allow multiple connections to concurrently read the same resource at the same time without interfering with each other;
exclusive lock
An exclusive lock is also called a write lock. A write lock will block other write locks or read locks, ensuring that only one connection can write data at the same time, while preventing other users from reading and writing this data.
lock strategy
The overhead of locking is relatively expensive, and the locking strategy is actually a balance strategy between ensuring thread safety and obtaining the maximum performance.
-
MySQL lock strategy: talbe lock (table lock)
Table lock is the most basic lock strategy of MySQL, and it is also the lock with the least overhead. It locks the entire table;
The specific situation is: if a user is performing a write operation, it will acquire an exclusive "write lock", which may lock the entire table and block other users' read and write operations;
If a user is performing a read operation, it will first acquire the shared lock "read lock", which runs other read locks to read the table concurrently without interfering with each other. Read locks can be concurrent reads of a uniform resource as long as no write locks are entered.
Usually occurs in DDL statements\DML statements without indexing, such as this DML update table set columnA="A" where columnB="B".
If the columnB field does not have an index (or is not a composite index prefix), all records will be locked, that is, the lock table. If the execution of the statement can execute an index on the columnB field, the row that satisfies the where will be locked (row lock).
-
MySQL lock strategy: row lock (row lock)
Row locks can support concurrent processing to the greatest extent, but of course also bring the greatest overhead. As the name suggests, the granularity of row locks is in each row of data.
affairs
A transaction is a set of atomic SQL, or an independent unit of work.
A transaction means that either the mysql engine will execute all of the set of sql statements, or none of them will be executed (for example, if one of the statements fails).
For example, tim wants to transfer 100 yuan to bill:
1. Check whether Tim's account balance is greater than 100 yuan;
2. Tim's account is reduced by 100 yuan;
3. Bill's account is increased by 100 yuan;
These three operations are a transaction, which must be packaged and executed, either all of them succeed or all of them are not executed, and the failure of any one of them will cause all three operations to be "not executed" - rollback.
CREATE DATABASE IF NOT EXISTS employees;
USE employees;
CREATE TABLE `employees`.`account` (
`id` BIGINT (11) NOT NULL AUTO_INCREMENT,
`p_name` VARCHAR (4),
`p_money` DECIMAL (10, 2) NOT NULL DEFAULT 0,
PRIMARY KEY (`id`)
) ;
INSERT INTO `employees`.`account` (`id`, `p_name`, `p_money`) VALUES ('1', 'tim', '200');
INSERT INTO `employees`.`account` (`id`, `p_name`, `p_money`) VALUES ('2', 'bill', '200');
START TRANSACTION;
SELECT p_money FROM account WHERE p_name="tim";-- step1
UPDATE account SET p_money=p_money-100 WHERE p_name="tim";-- step2
UPDATE account SET p_money=p_money+100 WHERE p_name="bill";-- step3
COMMIT;
A good transaction system must meet the ACID characteristics:
ACID of the transaction
-
A: atomiciy atomicity
A transaction must ensure that all operations in it are either executed or all rolled back, and it is impossible that only a part of the operations are executed.
-
C: consistency
Data must be guaranteed to transition from one consistent state to another.
For example, when the second step was executed in the previous transaction, the system crashed, and the data would not show that Bill's account was 100 less, but Tim's account remained unchanged. Either maintain the original (all rolled back), or the bill is 100 less and the tim is 100 more, only these two consistency states
-
I: isolation isolation
When a transaction is not completed, it is usually guaranteed that other sessions cannot see the execution result of the transaction
-
D: durability
Once the transaction is committed, the data will be saved, and even if the system crashes after the commit, the data will not be lost.
-
Here, I recommend an architecture learning exchange group: 725633148, which will share some video recordings recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, principles of microservice architecture, JVM performance Optimization, distributed architecture, etc. have become necessary knowledge systems for architects. You can also receive free learning resources, which are currently benefiting a lot!
isolation level
查看系统隔离级别：
select @@global.tx_isolation;
查看当前会话隔离级别
select @@tx_isolation;
设置当前会话隔离级别
SET session TRANSACTION ISOLATION LEVEL serializable;
设置全局系统隔离级别
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
READ UNCOMMITTED (uncommitted read, dirty read)
Modifications within a transaction, even if not committed, are visible to other sessions.
Uncommitted data can be read - dirty reads. Dirty reads can cause many problems, and this isolation level is generally not applicable.
Example:
-- ------------------------- read-uncommitted实例 ------------------------------
-- 设置全局系统隔离级别
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
-- Session A
START TRANSACTION;
SELECT * FROM USER;
UPDATE USER SET NAME="READ UNCOMMITTED";
-- commit;
-- Session B
SELECT * FROM USER;
//SessionB Console 可以看到Session A未提交的事物处理，在另一个Session 中也看到了，这就是所谓的脏读
id name
2 READ UNCOMMITTED
34 READ UNCOMMITTED
READ COMMITTED (commit read or non-repeatable read, phantom read)
General databases use this isolation level by default (mysql is not). This isolation level ensures that if a transaction is not completely successful (commit is executed), the operations in the transaction are invisible to other sessions.
-- ------------------------- read-cmmitted实例 ------------------------------
-- 设置全局系统隔离级别
SET GLOBAL TRANSACTION ISOLATION LEVEL READ COMMITTED;
-- Session A
START TRANSACTION;
SELECT * FROM USER;
UPDATE USER SET NAME="READ COMMITTED";
-- COMMIT;
-- Session B
SELECT * FROM USER;
//Console OUTPUT:
id name
2 READ UNCOMMITTED
34 READ UNCOMMITTED
---------------------------------------------------
-- 当 Session A执行了commit，Session B得到如下结果：
id name
2 READ COMMITTED
34 READ COMMITTED
It also verifies that the data modified by the read committed level before the commit operation of the thing is not visible to other Sessions, and will only be visible to other Sessions after the commit is executed.
We can see that Session B queries twice and gets different data.
The read committed isolation level solves the problem of dirty reads, but it will produce two inconsistent read results for other Sessions (because another Session executes transactions, consistent changes).
REPEATABLE READ
The unified read SQL is executed multiple times in a transaction, and the returned results are the same.
This isolation level solves the problem of dirty reads and phantom reads. This refers to the rr level of innodb. Next-key locks are used in innodb to lock the "current read", lock the row and the insertion position that may generate phantom reads, and prevent new data insertion from generating phantom rows.
Detailed analysis below.
For details, please refer to the mysql manual
https://dev.mysql.com/doc/refman/5.7/en/innodb-storage-engine.html
SERIALIZABLE (serializable)
The strongest isolation level, by adding locks and write locks to each read row in the transaction, ensures that there will be no phantom read problems, but it will lead to a large number of timeouts and lock contention problems.
Multiversion Concurrency Control - MVCC
MVCC (multiple-version-concurrency-control) is a variant of row-level locking, which avoids locking operations in ordinary read conditions, so the overhead is lower.
Although the implementation is different, usually a non-blocking read is implemented
,for
Write operations lock only necessary rows
。
-
Consistent read (that is, read snapshots)
select * from table ….;
-
Current read (that is, to read the actual persistent data)
Special read operations, insert/update/delete operations, belong to the current read, which processes the current data and needs to be locked.
select * from table where ? lock in share mode;
select * from table where ? for update;
insert;
update ;
delete;
Note: select ... from where ... (no additional lock suffix) uses MVCC to ensure read snapshots (mysql is called consistent read). The so-called consistent read or read snapshot is to read the data snapshot before the current transaction starts. Updates after the start of this transaction will not be read. Details are described in select below.
For locked read SELECT with FOR UPDATE (exclusive lock) or LOCK IN SHARE MODE (shared lock), update, delete statements, it is necessary to consider whether it is an equivalent query of a unique index.
写锁-recordLock,gapLock,next key lock
For value queries that use unique indexes: for example, where columnA=”…”, if the index on columnA is used,
Then a row lock will be added to the record that satisfies where (for update is an exclusive lock, lock in shared is a shared lock, and other write operations add an exclusive lock). Here is the row-level lock, record lock.
For range queries (with non-unique indexes):
For example (do range query): where columnA between 10 and 30, it will cause data after 10 in other sessions to be unable to be inserted (next key lock), thus solving the problem of phantom reading.
Here the next key lock will include all the lines involved.
next key lock=recordLock+gapLock, not only locks the relevant data, but also locks the boundary, thus completely avoiding phantom reading
。
for no index
lock table
Usually occurs in DDL statements\DML statements without indexing, such as this DML update table set columnA="A" where columnB="B".
If the columnB field does not have an index (or is not a composite index prefix), all records will be locked, that is, the lock table. If the execution of the statement can execute an index on the columnB field, the row that satisfies the where will be locked (row lock).
The MVCC of INNODB is usually implemented by saving two hidden columns behind each row of data (in fact, three columns, the third column is used for transaction rollback, which is omitted here),
One holds the created version number of the row, the other holds the updated version number of the row (the version number of the last updated data)
This version number is the version number of each transaction, incrementing.
This ensures that innodb can read data correctly without locking for read operations.
MVCC select lock-free operation and maintenance version number
Below, under the default Repeatable Read isolation level of mysql, let's take a look at the MVCC operation:
-
Select (snapshot read, the so-called read snapshot is to read the data before the current transaction.):
a.
InnoDB only selects to find rows with a version number earlier than the current version number
, which ensures that the data read is either committed before the transaction starts (earlier than the current version number), or data that was created in the transaction itself (equal to the current version number).
b. The updated version number of the search row is either undefined or greater than the current version number (in order to ensure that the transaction can read the old data), which ensures that the transaction reads the data that has not been updated after the current transaction starts.
Note: The select here cannot have for update and lock in share statements.
In short, only the row data that satisfies the following conditions should be returned to achieve the effect of snapshot reading:
(行创建版本号< =当前版本号 && (行更新版本号==null or 行更新版本号>当前版本号 ) ) -
Insert
InnoDB creates version numbers for newly inserted rows in this transaction, saving the row with the current transaction version number as the row of the row.
-
Delete
InnoDB saves the current transaction version number for each deleted row as the row delete marker.
-
Update
There will be two pieces of data, keep the current version number as the new version number of the updated data, and save the current version number as the updated version number of the old data row.
当前版本号—写—>新数据行创建版本号 && 当前版本号—写—>老数据更新版本号(); Dirty reads vs phantom reads vs non-repeatable reads
Dirty read: The update data of an uncommitted intermediate state of a transaction is read by other sessions. When a transaction is accessing data and modifying the data, and the modification has not been submitted to the database (commit has not been executed), at this time, another session also accesses the data, because the data has not been submitted, then another The data read by a session is dirty data, and operations based on dirty data may also be incorrect.
Non-repeatable read: Simply put, the data read in a transaction may change, ReadCommitted is also called non-repeatable read.
Reading the same data multiple times within the same transaction returns different results. In other words, subsequent reads can read updated data that has been committed by another session transaction. On the contrary, "repeatable read" can ensure that the data read is the same when reading data multiple times in the same transaction, that is, subsequent reads cannot read the updated data that has been committed by another session transaction.
Phantom read: A query is executed in the transaction of session T1, and then a new row of records is inserted into session T2, and this row of records just meets the conditions of the query used by T1. T1 then uses the same query to retrieve the table again, but now sees the new row that transaction T2 just inserted. This new row is called a "phantom" because it appears to T1 to appear suddenly.
The RR level of innoDB cannot completely avoid phantom reads, which will be analyzed in detail below.
----------------------------------前置准备----------------------------------------
prerequisite：
-- 创建表
mysql>
CREATE TABLE `t_bitfly` (
`id` bigint(20) NOT NULL DEFAULT '0',
`value` varchar(32) DEFAULT NULL,
PRIMARY KEY (`id`)
)
-- 确保当前隔离级别为默认的RR级别
mysql> select @@global.tx_isolation, @@tx_isolation;
+-----------------------+-----------------+
| @@global.tx_isolation | @@tx_isolation |
+-----------------------+-----------------+
| REPEATABLE-READ | REPEATABLE-READ |
+-----------------------+-----------------+
1 row in set (0.00 sec)
---------------------------------------开始---------------------------------------------
session A | session B
|
|
mysql> START TRANSACTION; | mysql> START TRANSACTION;
Query OK, 0 rows affected (0.00 sec) | Query OK, 0 rows affected (0.00 sec)
|
|
mysql> SELECT * FROM test.t_bitfly; | mysql> SELECT * FROM test.t_bitfly;
Empty set (0.00 sec) | Empty set (0.00 sec)
|
| mysql> INSERT INTO t_bitfly VALUES (1, 'test');
| Query OK, 1 row affected (0.00 sec)
|
|
mysql> SELECT * FROM test.t_bitfly; |
Empty set (0.00 sec) |
|
| mysql> commit;
| Query OK, 0 rows affected (0.01 sec)
mysql> SELECT * FROM test.t_bitfly; |
Empty set (0.00 sec) |
-- 可以看到虽然两次执行结果返回的数据一致， |
-- 但是不能说明没有幻读。接着看： |
|
mysql> INSERT INTO t_bitfly VALUES (1, 'test'); |
ERROR 1062 (23000): |
Duplicate entry '1' for key 'PRIMARY' |
|
-- 明明为空的表，为什么说主键重复？——幻读出现 ！！！ |
How to ensure that the rr level will never generate phantom reading?
Add the for update (exclusive lock) or lock in share mode (shared lock) statement to the select...where statement used to achieve this. In fact, it locks data that may cause phantom reads and prevents data writing operations.
In fact, it is because the data write operation (insert, update) needs to acquire the write lock first. Since the part that may generate phantom read has already acquired a certain kind of lock, the premise of acquiring the write lock in another session is in the current session. Release all locks created by locking statements.
mysql deadlock problem
Deadlock is a circular waiting chain. I wait for your resources, but you wait for mine. We all wait for each other, and no one releases the resources we own, resulting in wireless waiting.
for example:
//Session A
START TRANSACTION;
UPDATE account SET p_money=p_money-100 WHERE p_name="tim";
UPDATE account SET p_money=p_money+100 WHERE p_name="bill";
COMMIT;
//Thread B
START TRANSACTION;
UPDATE account SET p_money=p_money+100 WHERE p_name="bill";
UPDATE account SET p_money=p_money-100 WHERE p_name="tim";
COMMIT;
When thread A executes the first statement UPDATE account SET p_money=p_money-100 WHERE p_name=”tim”; locks the row data of p_name=”tim”; and tries to get the data of p_name=”bill”;
, at this time, thread B also executes the first statement: UPDATE account SET p_money=p_money+100 WHERE p_name=”bill”;
The data of p_name=”bill” is locked, while trying to obtain the data of p_name=”tim”;
At this point, the two threads enter a deadlock, and no one can obtain the resources they want to obtain, and enter the wireless wait until the timeout!
innodb_lock_wait_timeout wait for the lock timeout to roll back the transaction:
The intuitive method is that when two transactions wait for each other, when one of the waiting time exceeds a certain threshold set, one of the transactions is rolled back, and the other transaction can continue to execute. This method is simple and effective. In innodb, the parameter innodb_lock_wait_timeout is used to set the timeout period.
Wait-for graph algorithm to actively perform deadlock detection:
Innodb also provides a wait-for graph algorithm to actively detect deadlocks. Whenever a lock request cannot meet the needs immediately and enters waiting, the wait-for graph algorithm will be triggered.
How to avoid deadlocks as much as possible
1) Access tables and rows in a fixed order. For example, for two transactions that update data, the order of updating data of transaction A is 1, 2; the order of updating data of transaction B is 2, 1. This is more likely to cause deadlocks.
2) Big affairs are divided into small ones. Large transactions are more prone to deadlocks. If business allows, split large transactions into small ones.
3) In the same transaction, try to lock all the resources needed at one time to reduce the probability of deadlock.
4) Lower the isolation level. If the business allows, it is also a good choice to lower the isolation level. For example, adjusting the isolation level from RR to RC can avoid many deadlocks caused by gap locks.
5) Add a reasonable index to the table. It can be seen that if the index is not used, a lock will be added to each row of the table, and the probability of deadlock will be greatly increased.
Explicit and Implicit Locks
Implicit locks: The locks we mentioned above are implicit locks that do not require additional statement locking.
show lock
：
SELECT ... LOCK IN SHARE MODE(加共享锁);
SELECT ... FOR UPDATE(加排他锁); Details have been mentioned above.
You can view the status of waiting for locks through the following sql
select * from information_schema.innodb_trx where trx_state="lock wait";
or
show engine innodb status;
transactions in mysql
show variables like "autocommit";
set autocommit=0; //0表示AutoCommit关闭
set autocommit=1; //1表示AutoCommit开启
-
Automatic commit (AutoCommit, mysql default)
By default, mysql adopts AutoCommit mode, that is, each sql is a transaction and does not need to be explicitly executed.
If autoCommit is off, then each sql opens a transaction by default, and this transaction will only be committed after explicitly executing "commit".
finally:
Here, I recommend an architecture learning exchange group: 725633148, which will share some video recordings recorded by senior architects: Spring, MyBatis, Netty source code analysis, high concurrency, high performance, distributed, principles of microservice architecture, JVM performance Optimization, distributed architecture, etc. have become necessary knowledge systems for architects. You can also receive free learning resources, which are currently benefiting a lot!