Author: Kobayashi coding
Computer stereotyped essay website: https://xiaolincoding.com
Hello, I'm Xiaolin.
Today I want to talk to you about two questions:
- How to implement distributed locks with Redis?
- How does Redis solve the reliability problem of distributed locks in the cluster?
How to implement distributed locks with Redis?
Distributed lock is a mechanism for concurrency control in a distributed environment. It is used to control that a resource can only be used by one application at a time. As shown below:
Redis itself can be shared and accessed by multiple clients. It happens to be a shared storage system that can be used to store distributed locks. Moreover, Redis has high read and write performance and can handle highly concurrent lock operation scenarios.
The SET command of Redis has an NX parameter that can realize "insert when the key does not exist", so it can be used to implement distributed locks:
- If the key does not exist, it will show that the insertion is successful, which can be used to indicate that the locking is successful;
- If the key exists, the insertion failure will be displayed, which can be used to indicate the failure of locking.
When implementing distributed locks based on Redis nodes, we need to meet three conditions for locking operations.
- Locking includes three operations of reading the lock variable, checking the value of the lock variable and setting the value of the lock variable, but it needs to be done in an atomic operation, so we use the SET command with the NX option to achieve locking;
- The lock variable needs to set an expiration time, so as to prevent the client from getting an exception after obtaining the lock, which will cause the lock to be unable to be released. Therefore, we add the EX/PX option when the SET command is executed to set its expiration time;
- The value of the lock variable needs to be able to distinguish locking operations from different clients, so as to avoid accidental release operations when releasing the lock. Therefore, when we use the SET command to set the value of the lock variable, the value set by each client is a unique value. Used to identify the client;
A distributed command that satisfies these three conditions is as follows:
SET lock_key unique_value NX PX 10000
- lock_key is the key key;
- unique_value is the unique identifier generated by the client to distinguish lock operations from different clients;
- NX stands for setting the lock_key only when the lock_key does not exist;
- PX 10000 indicates that the expiration time of lock_key is set to 10s, which is to prevent the client from being abnormal and unable to release the lock.
The unlocking process is to delete the lock_key key (del lock_key), but it cannot be deleted randomly. It is necessary to ensure that the client performing the operation is the locked client. Therefore, when unlocking, we must first determine whether the unique_value of the lock is the locking client, and if so, delete the lock_key key.
It can be seen that there are two operations for unlocking. At this time, Lua scripts are needed to ensure the atomicity of unlocking, because Redis can execute Lua scripts atomically, which ensures the atomicity of lock release operations.
// 释放锁时，先比较 unique_value 是否相等，避免锁的误释放
if redis.call("get",KEYS[1]) == ARGV[1] then
return redis.call("del",KEYS[1])
else
return 0
end
In this way, the locking and unlocking of the distributed lock is completed on the Redis single node by using the SET command and the Lua script.
What are the advantages and disadvantages of implementing distributed locks based on Redis?
The advantages of implementing distributed locks based on Redis :
- Efficient performance (this is the core starting point for choosing a cache to implement distributed locks).
- Easy to implement. Many R&D engineers choose to use Redis to implement distributed locks, largely because Redis provides the setnx method, which is very convenient to implement distributed locks.
- Avoid single point of failure (because Redis is deployed across clusters, it naturally avoids single point of failure).
Disadvantages of implementing distributed locks based on Redis :
-
The timeout period is not good to set . If the timeout period of the lock is set too long, performance will be affected; if the timeout period is set too short, shared resources will not be protected. For example, in some scenarios, after a thread A acquires the lock, the execution time of the business code may be relatively long, resulting in the timeout of the lock being exceeded, and the lock will automatically become invalid. Note that the thread A has not finished executing, and the subsequent thread B accidentally holds the lock. Lock means that shared resources can be operated, so the shared resources between two threads cannot be protected.
-
- So how to set the timeout reasonably? We can set the timeout based on contract renewal: first set a timeout for the lock, then start a daemon thread, and let the daemon thread reset the timeout of the lock after a period of time. The implementation method is: write a daemon thread, and then judge the status of the lock. When the lock is about to expire, renew the contract and add the lock again. After the main thread is executed, destroy the renewal lock. However, this method is relatively complex.
-
Data in Redis master-slave replication mode is replicated asynchronously, which leads to the unreliability of distributed locks . If the Redis master node goes down without synchronizing to other nodes after the Redis master node acquires the lock, the new Redis master node can still acquire the lock at this time, so multiple application services can acquire the lock at the same time.
How does Redis solve the reliability of distributed locks in the cluster?
In order to ensure the reliability of distributed locks in a cluster environment, Redis has officially designed a distributed lock algorithm Redlock (red lock).
It is a distributed lock based on multiple Redis nodes. Even if a node fails, the lock variable still exists, and the client can still complete the lock operation.
The basic idea of the Redlock algorithm is to let the client and multiple independent Redis nodes request to apply for locks in turn. If the client can successfully complete the locking operation with more than half of the nodes, then we believe that the client has successfully obtained the distributed type lock, otherwise the lock fails .
In this way, even if a certain Redis node fails, because the lock data is also saved on other nodes, the client can still perform lock operations normally, and the lock data will not be lost.
The Redlock algorithm locks three processes:
-
The first step is that the client gets the current time.
-
The second step is that the client performs locking operations on N Redis nodes in sequence:
-
- The locking operation uses the SET command with NX, EX/PX options, and the unique identifier of the client.
- If a Redis node fails, in order to ensure that the Redlock algorithm can continue to run in this case, we need to set a timeout period for the "lock operation" (not a timeout period for "lock", but for "add lock operation" to set the timeout period).
-
The third step is that once the client completes the locking operation with all Redis nodes, the client needs to calculate the total time (t1) of the entire locking process.
Successful locking must meet two conditions at the same time ( brief description: if more than half of the Redis nodes successfully acquire the lock, and the total time spent does not exceed the effective time of the lock, then the locking is successful) :
- Condition 1: The client has successfully acquired locks from more than half (greater than or equal to N/2+1) of Redis nodes;
- Condition 2: The total time (t1) for the client to acquire the lock does not exceed the effective time of the lock.
After the lock is successfully acquired, the client needs to recalculate the valid time of the lock. The result of the calculation is "the initial valid time of the lock" minus the "total time spent by the client to acquire the lock (t1)".
After the lock fails, the client initiates the operation of releasing the lock to all Redis nodes. The operation of releasing the lock is the same as the operation of releasing the lock on a single node, as long as the Lua script for releasing the lock is executed.
Series "Illustrated Redis" articles:
Interview:
Data type articles:
- 20,000 words + 30 pictures｜A detailed description of the nine data types and application scenarios of Redis
- 20,000 words + 40 pictures｜Illustrating the realization of nine data structures of Redis
Persistence articles:
Function articles:
High availability articles:
Cache articles: