At present, almost all large websites and applications are deployed in a distributed manner. Data consistency in a distributed environment is a very important topic. The distributed CAP theory tells us that "no distributed system can satisfy Consistency, Availability and Partition tolerance at the same time, at most two of them can be satisfied at the same time." Therefore, many systems in At the beginning of the design, it is necessary to make a choice between these three. In the vast majority of scenarios in the Internet field, strong consistency needs to be sacrificed in exchange for high system availability. The system often only needs to ensure "eventual consistency", as long as the final time is within the range acceptable to users.
In many scenarios, in order to ensure the final consistency of data, we need a lot of technical solutions, such as distributed transactions, distributed locks, etc. Sometimes, we need to ensure that a method can only be executed by the same thread at the same time. In a stand-alone environment, Java actually provides many APIs related to concurrent processing, but these APIs are powerless in distributed scenarios, which means that pure Java APIs cannot provide distributed lock capabilities. So how to implement distributed lock?
Here we describe the implementation of distributed locks based on Redis, and explain and optimize the problems in the implementation.
In the beginning, I designed the distributed lock like this, you can see how it is implemented.
public class SyncLockManager {
private static VARedis redis = VARedis.getInstance ();
private static long lockTimeout = 5000;
private static int expireTimeout = 5000;
/** Attempt to acquire the lock represented by a keyword
* @param key
* @return
* @throws InterruptedException
*/
public static boolean tryLock(String key) throws InterruptedException{
boolean getLock = false;
long waitTotalTime = lockTimeout;
while(waitTotalTime>0){
long now = System.currentTimeMillis();
long lockReleaseTime = now + lockTimeout + 1;
long flag = redis.setNx(key, String.valueOf(lockReleaseTime));
if(flag==1){//Successfully set successfully, get the lock
getLock = true;
break;
}
String currentLockReleaseTime = redis.get(key);
if(currentLockReleaseTime!=null&&Long.parseLong(currentLockReleaseTime)<System.currentTimeMillis()){
String oldValue = redis.getSet(key, String.valueOf(lockReleaseTime));
if(oldValue!=null&&oldValue.equals(currentLockReleaseTime)){
getLock = true;
break;
}
}
Thread.sleep(100);
waitTotalTime = waitTotalTime - 100;
}
return getLock;
}
/**
* release lock
* @param key
*/
public static void releaseLock(String key){
long now = System.currentTimeMillis();
String currentLockReleaseTime = redis.get(key);
if(currentLockReleaseTime!=null&&now<Long.parseLong(currentLockReleaseTime)){
redis.delete(key);
}
}
}
The above implementation ideas can be summarized as follows: (1) The process of trying to acquire a lock is designed to repeatedly try to acquire the lock within the timeout period, and return when the lock is acquired, and continue to wait until the lock is successfully acquired or the timeout period is reached. Return failure; (2) In order to prevent the thread from being able to release the lock after locking, resulting in the subsequent thread never being able to acquire the lock, it is agreed that the thread will save the expiration time of the lock as the value of redis. If other threads find that the current time has exceeded the lock's value Expiration time, you have the right to reset the lock or release the lock;
Let's first think about whether there is a problem with this implementation of distributed locks?
To be honest, this is how I implemented it at the very beginning. I also considered various factors at that time and conducted corresponding distributed stress tests, but no problems were found. The reason basically does not appear on the existing network.
What is the problem? Don't sell off, that's the system time. Intuitively, if the system time is not synchronized in the case of distributed deployment, there will be a big problem with this distributed lock. When the thread acquires the lock, it is due to the locks added by other distributed application threads before the comparison. Both the expiration time and the system time in the current distributed application will fail, resulting in the lock being never succeeded and the lock being unable to be acquired. Of course, in the current network environment, the system time is inconsistent or the gap is relatively large, or it basically does not appear. If it can be guaranteed, then this implementation cannot be an implementation solution.
However, we can't be satisfied with this. How should we modify the distributed lock that is not affected by various factors?
In order to ensure that distributed locks are available, we must at least ensure that the implementation of the lock meets the following four conditions:
- mutual exclusivity. At any time, only one client can hold the lock.
- No deadlock will occur. Even if a client crashes while holding the lock and does not actively unlock it, other clients can be guaranteed to lock in the future.
- Fault tolerant. Clients can lock and unlock as long as most of the Redis nodes are up and running.
- The trouble should end it. Locking and unlocking must be done by the same client, and the client cannot unlock the locks added by others.
Let's first look directly at the modified Redis-based distributed lock implementation.
public class SyncLockManager {
private static final String LOCK_SUCCESS = "OK";
private static final Long RELEASE_SUCCESS = 1L;
private static final String SET_IF_NOT_EXIST = "NX";
private static final String SET_WITH_EXPIRE_TIME = "PX";
private static VARedis redis = VARedis.getInstance ();
private static long lockTimeout = 5000;
private static int expireTimeout = 5000;
/**
* Attempt to acquire a distributed lock and retry on failure
* Remarks: It is mainly used to solve the problem of high interface failure rate caused by the inability of the interface to obtain the lock in the case of high concurrency. You can delay waiting and retry to ensure the success rate of the interface.
* @param jedis Redis client
* @param lockKey lock
* @param requestId request ID
* @param expireTime Expiration time
* @return whether the acquisition was successful
*/
public static boolean tryGetDistributedLockWithRetry(String lockKey, String requestId) {
long waitTotalTime = lockTimeout;
boolean getLockSuccess = false;
while(waitTotalTime>0){
if(tryGetDistributedLock(lockKey, requestId, expireTimeout)){
return true;
}
try {
Thread.sleep(100);//Wait for retry
} catch (InterruptedException e) {
// do not process
}
waitTotalTime = waitTotalTime - 100;
}
return getLockSuccess;
}
/**
* Attempt to acquire a distributed lock
* @param jedis Redis client
* @param lockKey lock
* @param requestId request ID
* @param expireTime Expiration time
* @return whether the acquisition was successful
*/
public static boolean tryGetDistributedLock(String lockKey, String requestId, int expireTime) {
String result = redis.set(lockKey, requestId, SET_IF_NOT_EXIST, SET_WITH_EXPIRE_TIME, expireTime);
if (LOCK_SUCCESS.equals(result)) {
return true;
}
return false;
}
/**
* Release distributed lock
* @param jedis Redis client
* @param lockKey lock
* @param requestId request ID
* @return whether the release is successful
*/
public static boolean releaseDistributedLock(String lockKey, String requestId) {
String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
Object result = redis.eval(script, Collections.singletonList(lockKey), Collections.singletonList(requestId));
if (RELEASE_SUCCESS.equals(result)) {
return true;
}
return false;
}
}
As you can see, we lock with one line of code: jedis.set(String key, String value, String nxxx, String expx, int time), this set() method has a total of five formal parameters:
The first one is the key, we use the key as the lock because the key is unique.
The second is value, we pass requestId, many children's shoes may not understand, is it not enough to have key as a lock, why use value? The reason is that when we talked about reliability above, the distributed lock must meet the fourth condition to unlock the ringer. By assigning the value as requestId, we know which request the lock was added. can have a basis. requestId can be
UUID.randomUUID().toString()generated using the method.The third is nxxx, we fill in NX for this parameter, which means SET IF NOT EXIST, that is, when the key does not exist, we perform the set operation; if the key already exists, do nothing;
The fourth one is expx, which we pass as PX, which means that we need to add an expired setting to this key, and the specific time is determined by the fifth parameter.
The fifth is time, which corresponds to the fourth parameter and represents the expiration time of the key.
In general, executing the above set() method will only lead to two results: 1. There is no lock at present (the key does not exist), then the locking operation is performed, and a validity period is set for the lock, and the value indicates the lock client. 2. There is a lock, do nothing.
Careful children's shoes will find that our lock code meets the three conditions described in our reliability. First of all, set() adds the NX parameter, which can ensure that if there is an existing key, the function will not be called successfully, that is, only one client can hold the lock, which satisfies mutual exclusion. Second, since we set an expiration time for the lock, even if the lock holder crashes and fails to unlock it, the lock will be automatically unlocked (that is, the key will be deleted) due to the expiration time, and no deadlock will occur. Finally, because we assign the value to requestId, which represents the locked client request identifier, we can check whether it is the same client when the client is unlocked. Since we only consider the scenario of single-machine deployment of Redis, we do not consider fault tolerance for the time being.
For the unlock code, you can see that we only need two lines of code to unlock it! The first line of code, we wrote a simple Lua script code. The last time I saw this programming language was in "Hackers and Painters", but I didn't expect it to be used this time. In the second line of code, we pass the Lua code to jedis.eval()the method, and assign the parameter KEYS[1] to lockKey and ARGV[1] to requestId. The eval() method is to hand the Lua code to the Redis server for execution. So what is the function of this Lua code? In fact, it is very simple. First, obtain the value corresponding to the lock, check whether it is equal to the requestId, and delete the lock (unlock) if it is equal. So why use Lua language to implement it? Because the above operation is guaranteed to be atomic.
If Redis is deployed on multiple machines in your project, you can try to Redissonimplement distributed locks, which are Java components officially provided by Redis.
Thank you for reading. If you are interested in Java programming, middleware, databases, and various open source frameworks, please pay attention to my blog and Toutiao (Source Code Empire). The blog and Toutiao will regularly provide some related technical articles for later. Let's discuss and learn together, thank you.
If you think the article is helpful to you, please give me a reward, thank you.
