Distributed locks (resources are shared between processes instead of threads)

The architect takes you to play distributed locks

Most Internet systems are deployed in a distributed manner. Distributed deployment can indeed bring performance and efficiency improvements, but for this, we need to solve the problem of data consistency in a distributed environment .

When a resource is shared among multiple systems, in order to ensure that everyone accesses the resource data is consistent, then it must be required to be processed by only one client at the same time and cannot be executed concurrently . Otherwise, When someone writes and someone reads at the same moment, the data that everyone accesses will be inconsistent.

1. Why do we need distributed locks?

In stand-alone era , though not require a distributed lock, but also face similar problems, but in a single case, if there are multiple threads to simultaneously access a shared resource, we can use inter-locking threads of The mechanism is that when a thread acquires the resource, it immediately locks the resource, and when the resource is used up, unlock it again, and other threads can continue to use it. For example, in JAVA, some APIs (synchronize/Lock, etc.) for handling lock mechanisms are even provided.

But in the era of distributed systems, this lock mechanism between threads is useless. The system may have multiple copies and be deployed on different machines. These resources are no longer shared between threads, but belong to Resources shared between processes.

Therefore, in order to solve this problem, we must introduce "distributed locks."

分布式锁，是指在分布式的部署环境下，通过锁机制来让多客户端互斥的对共享资源进行访问。

What requirements should distributed locks meet?

• Exclusiveness: only one client can acquire the lock at the same time, and other clients cannot acquire the lock at the same time
• Avoid deadlock: this lock will be released after a limited period of time (normal release or abnormal release)
• High availability: The mechanism for acquiring or releasing locks must be highly available and perform well

2. What are the implementation methods of distributed locks?

There are three common mechanisms in distributed locks. From the perspective of the complexity of implementation, the difficulty increases from top to bottom:

• Based on database
• Based on Redis
• Implementation based on ZooKeeper

Either way, it is not perfect, we still have to choose according to the actual scenario of our business.

1. Distributed lock based on database

• Optimistic locking based on database
• Pessimistic lock based on database

The optimistic locking mechanism is actually implemented by introducing a version number (version) field in the database table.

When we want to read data from the database, the version field is also read out at the same time. If we want to update the read data and write it back to the database, we need to increase the version by 1, and at the same time, the new data and the new The version is updated to the data table, and it must be checked at the time of updating whether the version value in the current database is the previous version, and if it is, it will be updated normally. If it is not, the update fails, indicating that other processes have updated the data during this process.

As shown in the figure, assuming the same account, user A and user B have to withdraw money. The original balance of the account is 2000, user A has to withdraw 1500, and user B has to withdraw 1000. If there is no lock mechanism, in the concurrent Under circumstances, the balance may be deducted by 1500 and 1000 at the same time, resulting in an incorrect or even negative final balance. But if the optimistic locking mechanism is used here, when two users go to the database to read the balance, in addition to reading the 2000 balance, they also read the current version number version=1, waiting for user A or user B to modify When the database balances, no matter who operates first, the version number will be increased by 1, that is, version=2. Then another user will find that the version number is incorrect when updating, and it has become 2 instead of 1 when it was read out. Then this update fails, you have to read the latest database balance again.

使用「乐观锁」机制，必须得满足：
（1）锁服务要有递增的版本号version
（2）每次更新数据的时候都必须先判断版本号对不对，然后再写入新的版本号

Pessimistic lock is also called exclusive lock. In Mysql, it is based on for update to achieve locking. It
means that the data is modified by the outside world (including other transactions in this system and transaction processing from external systems) . Therefore, During the entire data processing process, the data is locked.
The realization of pessimistic locking often relies on the locking mechanism provided by the database (and only the locking mechanism provided by the database layer can truly guarantee the exclusivity of data access, otherwise, even if the locking mechanism is implemented in this system, it cannot be guaranteed that the external system will not be modified. data). mysql (for update) pessimistic lock summary and practice

//锁定的方法-伪代码
public boolean lock(){
    connection.setAutoCommit(false)//关闭自动提交属性
    for(){
        result =select * from user where  id = 100 for update;
        if(result){
         //结果不为空，则说明获取到了锁
         //进行相应操作
            return true;
        }
        //没有获取到锁，继续获取
        sleep(1000);
    }
    return false;
}

//释放锁-伪代码
connection.commit();//提交事务

In the above example, in the user table, id is the primary key. Through the for update operation, the database will add an exclusive lock to this record when querying.

(It should be noted that MySQL InnoDB default Row-Level Lock, only the field [clearly specify the primary key] or add an index, will be a row lock Row lock, otherwise it is a table level lock Table Lock, so this id field needs to be indexed) ；

1. Clearly specify the primary key, and there is this data, row lock
2. Clearly specify the primary key, if there is no such data, no lock
3. No primary key, table lock
4. Primary key is ambiguous, table lock

When this record is added with an exclusive lock, other threads cannot operate this record.

Then, in this case, we can think that the thread that acquired the exclusive lock has a distributed lock, and then we can execute the business logic we want to do. When the logic is completed, we can call the above statement to release the lock. .

Note : In a transaction, only SELECT… FOR UPDATE or LOCK IN SHARE MODE will wait for the end of other transactions when the same data is executed. Generally, SELECT… is not affected by this. For example, when I execute select status from t_goods where id=1 for update;. If I execute select status from t_goods where id=1 for update; again in another transaction, the second transaction will always wait for the submission of the first transaction. At this time, the second query is in a blocking state, but if I am In the second transaction, execute select status from t_goods where id=1; then the data can be queried normally and will not be affected by the first transaction.

2. Based on Redis

The lock mechanism implemented based on Redis mainly relies on the atomic operations of redis itself , such as:

SET user_key user_value NX PX 100

Since redis version 2.6.12, the SET command supports these parameters:
NX: Only when the key does not exist, the key is set. The effect of SET key value NX is equivalent to SETNX key value
PX millisecond: Set the expiration time of the key Millisecond milliseconds, when this time is exceeded, the set key will automatically become invalid

The above code example means that
when the user_key key does not exist in redis, a user_key key is set, and the value of this key is set to user_value, and the survival time of this key is 100ms

Why can this command help us implement the lock mechanism?
Because this command is executed successfully only when a certain key does not exist. Then when multiple processes simultaneously set the same key at the same time, only one process will always succeed.

After a certain process is set successfully, you can execute the business logic, and after the business logic is executed, go to unlock it.
Unlocking is very simple, you only need to delete this key, but you need to judge before deleting, the value corresponding to this key is the one set by yourself.

In addition, for distributed locks in redis cluster mode, redis's Redlock mechanism can be used.

3. Implementation based on ZooKeeper

Distributed locks implemented using ZooKeeper's temporary ordered nodes.

The principle is: when a client wants to perform a logical lock, it will generate a unique temporary ordered node in the directory of a specified node on zookeeper, and then judge whether it is the smallest sequence number among these ordered nodes one , if so, be acquired lock. If it is not, it means that the lock has not been acquired, then you need to find the node smaller than yourself in the sequence, and call the exist() method on it to register event listeners. When it is monitored that the node is deleted, then Once again, judge whether the node you created at the beginning has become the smallest in the sequence. If it is, then acquire the lock, if not, repeat the above steps.

When the lock is released, only the temporary node needs to be deleted.

As shown in the figure, the locker is a persistent node, node_1/node_2/.../node_n is the temporary node mentioned above, created by the client.
client_1/client_2/.../clien_n are all clients that want to acquire locks. Take client_1 as an example. If it wants to obtain a distributed lock, it needs to go to the locker to create a temporary node (if it is node_1). After creation, check whether its node number is the smallest under the locker. If it is, get it Up the lock. If it is not, find the node smaller than yourself (if it is node_2). After finding it, monitor node_2 until node_2 is deleted, then start to judge whether your node_1 is the smallest in the sequence again, and if it is, get it Lock, if it is not, continue to find a node.