Optimistic concurrency control and pessimistic concurrency control are the main methods adopted by concurrency control. Optimistic locking and pessimistic locking are not only applied in relational databases, but also have related concepts in Hibernate, Memcache, etc.

Pessimistic lock

In the current high-concurrency architecture of the Internet, pessimistic locking has become very rare due to the impact of fail-fast thinking.

Pessimistic Locking (Pessimistic Locking), pessimistic locking refers to the data processing process, so that the data is in a locked state, generally using the database lock mechanism to achieve.

Pessimistic lock-the realization in the data table

To use pessimistic locking in MySQL, you must turn off MySQL autocommit, set autocommit = 0, MySQL uses autocommit autocommit mode by default, that is, if you perform an update operation, MySQL will automatically submit the results.

Take a chestnut:
Suppose there is a field quantity in the commodity table to indicate the current inventory of the commodity. Suppose there is a Dulex sleeve with an id of 100 and quantity = 8; if no lock is used, then the operation method

as follows:

//step1: 查出商品剩余量
 select quantity from items where id=100;
//step2: 如果剩余量大于0，则根据商品信息生成订单
 insert into orders(id,item_id) values(null,100);
 //step3: 修改商品的库存
 update Items set quantity=quantity-1 where id=100;

This way of writing is really normal in a small workshop, No Problems, but problems may occur in a high concurrency environment.

as follows:

In fact, in the ① or ② link, someone has already placed an order and reduced the inventory. At this time, step3 is still executed, which causes overselling .

But using pessimistic locks can solve this problem. In the above scenario, the product information is from query to modification, there is a process for generating orders in the middle, the principle of using pessimistic locks is that when we query items information, we put The current data is locked until we modify it and then unlock it. Then in this process, because the data is locked, there will be no third party to modify it. The premise of this is that the SQL statement to be executed needs to be placed in the same thing, otherwise the purpose of locking the data row cannot be achieved.

as follows:

//step1: 查出商品状态
select quantity from items where id=100 for update;
//step2: 根据商品信息生成订单
insert into orders(id,item_id) values(null,100);
//step3: 修改商品的库存
update Items set quantity=quantity-2 where id=100;

select ... for update is a method provided by MySQL to achieve pessimistic locking. At this time, in the items table, the data with the id of 100 is locked by us. Other transactions that need to execute select quantity from items where id = 100 for update must wait for this transaction to be executed. This way we can guarantee that the current data will not be modified by other transactions.

Pessimistic Lock-Extended Thinking

It should be noted that when I execute select quantity from items where id = 100 for update. If I execute select quantity from items where id = 100 (without for update) in the second transaction, the data can still be queried normally and will not be affected by the first transaction. In addition, MySQL also has a problem that all the scanned rows will be locked during the execution of the select ... for update statement, so using pessimistic locking in MySQL must ensure that the index is gone, not the full table scan, otherwise it will Lock the entire data table .

Pessimistic locking is not suitable for any scene, it also has some shortcomings, because pessimistic locking mostly depends on the database lock mechanism to ensure the maximum degree of exclusivity. If the locking time is too long, other users cannot access it for a long time, which affects the concurrent accessibility of the program. At the same time, this has a great impact on the database performance overhead, especially for long transactions. Optimistic locking is required.

Optimistic locking

Compared with pessimistic locks, optimistic locks believe that data will not cause conflicts under normal circumstances, so when the data is submitted for update, it will formally check whether the data conflicts. If a conflict is found, it will return an error message. To let users decide how to do it. Next we look at the implementation of optimistic locking in data tables and caches.

Optimistic locking-implementation in the data table

Using the data version number (version) mechanism is one of the most commonly used optimistic locking implementations. Generally, by adding a numeric type "version" field to the database table , when reading data, the value of the version field is read out together. Each time the data is updated, the version value is +1 . When we submit an update, the current version information of the corresponding record in the database table is compared with the version value retrieved for the first time. If the current version number of the database table is equal to the version value retrieved for the first time, it is updated Otherwise, it is considered to be outdated data and the update fails.

Give a chestnut

//step1: 查询出商品信息
select (quantity,version) from items where id=100;
//step2: 根据商品信息生成订单
insert into orders(id,item_id) values(null,100);
//step3: 修改商品的库存
update items set quantity=quantity-1,version=version+1 where id=100 and version=#{version};

Since you can use version , you can also use the timestamp field. This method also adds a timestamp field to the table. Similar to the version above, it also checks the timestamp of the data in the current database when the update is submitted and before the update. Compare the obtained time stamps, if they are consistent, then OK, otherwise it is a version conflict.

It should be noted that if your data table is a separate table for reading and writing, when the data written in the master table is not synchronized to the slave table in time, the update will always fail. At this time, you need to force read the data in the master table (put the select statement in things).

That is: put the select statement in the transaction, and query the master library!

Optimistic lock-lock granularity of optimistic lock

Optimistic locking is widely used for status synchronization in our bird system . We often encounter scenarios where the status of a logistics order is modified concurrently, so optimistic locking plays a huge role at this time.

Share a case of carefully selected optimistic locks to narrow the lock range

When commodity inventory is deducted, especially in scenarios with high concurrency such as spikes and cost-effectiveness, if the version number is used as an optimistic lock, only one transaction can be updated successfully at a time, and business perception is a large number of failed operations.

// 仍挑选以库存数作为乐观锁
//step1: 查询出商品信息
select (inventory) from items where id=100;
//step2: 根据商品信息生成订单
insert into orders(id,item_id) values(null,100);
//step3: 修改商品的库存
update items set inventory=inventory-1 where id=100 and inventory-1>0;

That's right! You have participated in Tmall, Taobao spike, and get a good deal. This is the SQL . By selecting optimistic locks, you can reduce the locking force and improve throughput ~

Optimistic locking needs to be used flexibly

In the current high-concurrency architecture of the Internet, pessimistic locking has become very rare due to the impact of fail-fast thinking.

Optimistic locking-extended training

In many systems in Ali, you can see commonly used features, params and other fields. If these fields are not versioned, it is very easy to have information coverage problems in concurrent scenarios.

such as:

Thread	Original features	Target features
T-A	a=1;	a=1;b=1;
T-B	a=1;	a=1;c=1;

We expect the result of the final update to be:

a=1;b=1;c=1;

If SQL is written at this time

update    
    lg_order
set    
    features=#features#
where    
    order_id=#order_id#

Then, as the order of TA and TB is different, the result we get may be a = 1; b = 1; or a = 1; c = 1;

If you use optimistic locking at this time and use the global field version for processing, you will find that there is a very high conflict rate with other fields of lg_order because the version field is global

How to deal with it? ? ?

Smart, you will find that when designing library tables in general, if there are fields with similar features, there will be a features_cc paired with it. Many younger programmers in the factory rarely pay attention to this field. , It should be much better now.

The function of features_cc is to control the optimistic locking version of features, thus avoiding the embarrassment of using version conflict with the entire field.

update    
    lg_order
set    
    features=#features#,    
    features_cc= features_cc +1
where    
    order_id=#order_id#    
    and features_cc =#ori_ features_cc#

It should be noted here that the owner needs to carefully review the SQL of its related table, requiring that all changes related to the features field of this table must be added with features_cc = features_cc +1 for calculation, otherwise it will cause concurrency conflicts and usually be protected Measures, otherwise won the bid .

In the actual environment, there are many such high-concurrency scenarios. Let's think about whether we have consciously added optimistic lock protection to the features field.

However, it needs to be mentioned that the intensive cultivation and control of this field is at the cost of increasing maintenance costs.

It took a long time for the two fields of features and attribute before BU students reached consensus and review code, requiring _cc for version control.

If the changes are too frequent, you can propose to maintain them separately to separate the hot and cold data.

Optimistic lock & pessimistic lock-small summary

	Pessimistic lock	Optimistic locking
concept	Locking records directly during query makes it impossible to query and update other transactions	Check the version or timestamp when submitting the update
grammar	select ... for update	Use version or timestamp for comparison
Implementor	The database itself	Developers
Applicable scene	Large amount of concurrency	Low concurrency
Analogy Java	Synchronized keywords	CAS algorithm

Shared lock & exclusive lock & intention lock & clearance lock

The characteristics of the transaction can be seen: https://blog.csdn.net/xushiyu1996818/article/details/103460349

Exclusive lock & shared lock

Shared locks (shared locks, S locks)

Xiaoming said: When you do n’t have a girlfriend, when you want to roll sheets with a woman, you can only go to the red light district, others can also find the red light district, this is sharing!

Shared lock is also called read lock. If transaction T1 adds S lock to row R, then

Other transactions T2 / T3 / Tn can only add an S lock to row R, not other locks

The transaction that obtains the S lock can only read data, not write data (you are stupid, of course, you cannot delete it).

select … lock in share mode;

Exclusive locks (X locks)

Xiaoming said: If you have money to find a female ticket, you have to monopolize the female ticket. Other people cannot use it. This is exclusive! Let's focus on exclusive locks.

An exclusive lock is also called a write lock. If transaction T1 adds an X lock to row R, then

No other transaction T2 / T3 / Tn can add any type of lock to row R until the X lock on row R of the T1 transaction is released.

The transaction that obtains the X lock can both read data and write data (you can also delete data).

select ... for update

Note: This grammar is also a pessimistic lock

Example table USER:

id	name	desc
1	Ma Yun	Richest man
2	Komei	First negative

// start T1
SELECT * FROM USER WHERE id = 1 lock in share mode; (S锁)
......

// start T2
UPDATE USER SET name = '小明' WHERE id = 1;
......

If T1 does not commit, the S lock will not be released,
then T2 will stare at the X lock and wait for T1 (transaction 1) to release the S lock.

at this time

// 接上文代码块
// start T3
// 此时，如果 T3 做同样查询，可以直接获取S锁进行查询
SELECT * FROM USER WHERE id = 1 lock in share mode; (S锁)

这个时候如果T1（事务1）要进行 DELETE 操作

// start T1
SELECT * FROM USER WHERE id = 1 lock in share mode; (S锁)
......
DELETE FROM USER WHERE id = 1；
......

At this time:
T1 finds that the X lock is occupied by T2, so T1 cannot wait for the X lock and waits for T2 to release the X lock, and T2 holds the X lock and waits for T1 to release the S lock.
After a deadlock occurs, InnoDB will generate an error message and release the lock ( later will talk about deadlocks and solutions encountered in business ).

The above X lock code can be displayed with the following diagram:

Use the time flow to display the interaction of the three threads as follows:

Intention lock

Intention Locks

Xiao Ming said: When he is looking for Cang teacher or female ticket, he must first check whether the other party is shared or monopolized by others, instead of going to someone to lock her in front, which can save costs!

Intent lock is a table lock, which is mostly used in innoDB. It is the behavior of the database itself. It does not require manual intervention. It will be released after the transaction ends.

Intentional locks are divided into intentional shared locks (IS locks) and intentional exclusive locks (IX locks)

Lock: indicates that some rows will be locked in the transaction
IX lock: indicates that X rows will be added to some rows in the transaction

The main role is to enhance the intent lock storage engine performance, innoDB in S and X locks a row lock, whenever a transaction comes, storage engine needs to traverse the locks held by all of the lines, lower performance , and therefore the introduction of intent lock , Before checking the row lock, check whether the intent lock exists, if it exists, block the thread.

Use of intention lock

Following the above ideas, let's look at the logic used

Give a chestnut

T1:
SELECT * FROM A WHERE id = 1 lock in share mode；（加S锁）

T2:
SELECT * FROM A WHERE id > 0 for update; (加X锁)

Looking at the two SQL transactions above, when T1 is executed, an S lock is added to the id = 1 line. Before T2 is executed, it is necessary to obtain the update lock of the entire table to determine, that is:
step1: determine whether table A has a table-level lock
Step2: Determine whether there is a row-level lock for each row of Table A.
When the amount of data is large (usually a table of 5-50 million data), this judgment of step2 is extremely inefficient .

no! No! Please! no! No! Please! no! No! Please! So, we need to lock the agreement.

Intention lock protocol

To obtain the S lock of some rows of table A, the transaction must acquire the IS lock of table A

To obtain the X lock of some rows of table A, the transaction must acquire the IX lock of table A

Now! do you get me sense? At
this time, step2 has changed to the judgment of intent lock
step2: found that table A has an IS lock, indicating that the table must have a row-level S lock, therefore, T2 applies for X lock blocking wait, no need to judge all Table, judging the efficiency has been greatly improved (does it save a lot of money)

Clearance lock

A gap lock is a lock on a gap between index records, or a lock on the gap before the first or after the last index record. For example, SELECT c1 FROM t WHERE c1 BETWEEN 10 and 20 FOR UPDATE; prevents other transactions from inserting a value of 15 into column t.c1, whether or not there was already any such value in the column, because the gaps between all existing values in the range are locked.

When we use range conditional conditions to retrieve data (non-clustered index, non-unique index) and request shared or exclusive locks, InnoDB will lock the index items of the data records that meet the conditions; Records that do not exist are called gaps, and InnoDB will also lock these gaps, that is, gap locks.
Next-Key locks are eligible row locks plus gap locks

Conditions for gap lock

Under InnoDB, the gap lock needs to meet three conditions:

The isolation level is RR
Current reading
The query condition can go to the index

The role of gap lock

MySQL official documentation: The purpose of the gap lock is to prevent other transactions from adding data to the gap.

In InnoDB in RR mode, gap lock can play two roles:

1. Guarantee data recovery and replication

2. Prevent phantom reading

Prevent execution of insert statements in gaps
Prevent existing data from being updated into the gap

Database data recovery and replication is achieved through binlog, which records the successfully executed DML statements (very widely used in Alibaba) . In data recovery, it is necessary to ensure the transaction sequence between the data, and gap locks can be avoided. Insert other transactions into a batch of data.

xushiyu1996818

Published 524 original articles · Like 80 · Visits 150,000+

His message board concerns

mysql lock mechanism optimistic lock & pessimistic lock, shared lock & exclusive lock & intention lock & gap lock

Optimistic lock & pessimistic lock