The use and principle of Redission as a distributed lock

Key points in implementing distributed locks

1. atomicity

For example, the following is not atomic

boolean lockStatus = stringRedisTemplate.opsForValue().setIfAbsent(orderId, userId);
stringRedisTemplate.expire(lockId, 30L, TimeUnit.SECONDS);

can be changed to

boolean lockStatus = stringRedisTemplate.opsForValue().setIfAbsent(orderId, userId, 30L, TimeUnit.SECONDS);

2. Expiration

The lock must have an expiration time, otherwise there is a risk of deadlock.
For example: if the lock is successfully locked but has not been released yet, and the service is down, the lock will remain permanently. After the service is up, when you try to seize the lock, you will find that there is already a lock and cannot be preempted, but the lock will remain forever. It's deadlocked before release.
So remember to use setnx to set the expiration time, or put set+expire into Lua to set it.

3. Lock renewal period

Why do I need to renew? Assume that the lock is set for 3 seconds, but the business code has not been executed for 4 seconds. Then the lock has expired, and other threads have added locks when requesting the interface (the setnx value is set in redis). At this time, it will not be executed concurrently. ? It is equivalent to thread unsafe! So lock renewal is required.

4. release lock

To release the lock correctly, what does it mean to be correct? Let’s look at an example:
Problem: Releasing the lock may release someone else’s lock. For example, the lock is set for 3 seconds, but the business code has not been executed for 4 seconds. Then the lock has expired, and other threads have added locks when requesting the interface ( The value of setnx is set again in redis), and then the first thread that has been executed for 4 seconds finishes running and releases the lock added by the second thread. At this time, other threads can grab the lock again, which is not safe!
Solution:
Lock renewal can effectively prevent other threads from being able to lock if the execution is not completed.
The key must be the same, because it is the same data. If the key is different, there is no need for distributed locks. The operations are different things, so you need to start with the value. Before unlocking, first determine whether the value of the key is Added by yourself, the value cannot be the thread ID, because the thread ID will be repeated in the distributed environment, so it can be changed to a business primary key such as userid, or it can be changed to a random number, because locking and unlocking are all in one method. When unlocking, This random number can be obtained.

Distributed lock simply implemented by redisTemplate

The distributed lock implemented using redisTemplate is like this

  boolean lock = false;
        try {
    
    
            lock = redisTemplate.opsForValue().setIfAbsent(lockUserBehaviorKey, "", 60, TimeUnit.SECONDS);
            if (!lock) {
    
    
                log.warn("获取redis锁失败:{}", new Date());
                return;
            }

        } catch (Exception e) {
    
    
            log.error("发生异常:{}", e);
        } finally {
    
    
            if (lock) {
    
    
                redisTemplate.delete(lockUserBehaviorKey);
            }
        }

So what are the problems with such a simple implementation of redis?
It is easy to see that it does not meet the lock renewal requirements, nor does it meet the reentrant requirements. It may also be unlocked by other threads, which is a bug. Of course,
we can also implement reentrant locks, blocking locks, etc. by ourselves, such as redis distributed Locks, however redission has already provided us with locks with these functions.

redission distributed lock

The locking mechanism of Redisson lock is shown in the figure above. The thread acquires the lock. If the acquisition is successful, the Lua script is executed and the data is saved to the redis database . If the acquisition fails: Keep trying to acquire the lock through the while loop (the waiting time can be customized, and failure will be returned after timeout). After the acquisition is successful, execute the Lua script and save the data to the redis database . The distributed lock provided by Redisson supports automatic lock renewal. That is to say, if the thread has not finished executing, then Redisson will automatically extend the timeout period for the target key in redis. This is called the Watch Dog mechanism in Redisson . At the same time, redisson also implements fair locks and read-write locks.

public void test() throws Exception{
    
    
        RLock lock = redissonClient.getLock("guodong");    // 拿锁失败时会不停的重试
        // 具有Watch Dog 自动延期机制 默认续30s 每隔30/3=10 秒续到30s
        lock.lock();
        // 尝试拿锁10s后停止重试,返回false 具有Watch Dog 自动延期机制 默认续30s
        boolean res1 = lock.tryLock(10, TimeUnit.SECONDS); 
        // 没有Watch Dog ，10s后自动释放
        lock.lock(10, TimeUnit.SECONDS);
        // 尝试拿锁100s后停止重试,返回false 没有Watch Dog ，10s后自动释放
        boolean res2 = lock.tryLock(100, 10, TimeUnit.SECONDS);
        Thread.sleep(40000L);
        lock.unlock();
    }

2. Wath Dog’s automatic extension mechanism.
If the node that obtains the distributed lock goes down and the lock happens to be in a locked state, a locked state will occur. In order to avoid this situation, the lock will set an expiration date. time. There is also a problem in this. Join a thread, get the lock, and set a 30s timeout. After 30s, the thread has not completed execution, and the lock timeout is released, which will cause problems. Redisson gave its own answer, which is automatic extension of watch dog . Mechanism
Redisson provides a watchdog that monitors locks. Its function is to continuously extend the validity period of the lock before the Redisson instance is closed. In other words, if a thread that obtains the lock has not completed the logic, then the watchdog It will help the thread to continuously extend the lock timeout, and the lock will not be released due to timeout.
By default, the watchdog renewal time is 30s, which can also be specified separately by modifying Config.lockWatchdogTimeout . In addition, Redisson also provides a locking method that can specify the leaseTime parameter to specify the locking time. After this time is exceeded, the lock will be automatically unlocked and the validity period of the lock will not be extended.
3. Reentrant locking mechanism
The reason why Redisson can implement a reentrant locking mechanism is related to two points:

1、Redis存储锁的数据类型是 Hash类型
2、Hash数据类型的key值包含了当前线程信息。

The surface data type here is the Hash type. The Hash type is equivalent to our java's <key, <key1, value>> type. The key here refers to 'redisson' and its validity period is
9 seconds. Let's take a look at our key1 value: 078e44a3-5f95-4e24-b6aa-80684655a15a:45 Its composition is:
guid + ID of the current thread. The following value is related to reentrant locking.

The above picture means the reentrant lock mechanism. Its biggest advantage is that the same thread does not need to wait for the lock, but can directly perform corresponding operations.

4 Disadvantages of Redis distributed locks
Redis distributed locks have a flaw, which is that in Redis sentinel mode:
client 1 writes a redisson lock to a master node, and it will be asynchronously copied to the corresponding slave node. However, once the master node goes down during this process, the master node switches and the slave node changes to the master node . Since redis master-slave is asynchronous synchronization, data may be lost.
When client 2 tries to lock, it can also lock on the new master node. This will cause multiple clients to lock the same distributed system. The lock is locked.
At this time, the system will definitely have problems in business semantics, resulting in the generation of various dirty data.
Redission also has problems. For example, redis has the problem of dirty data caused by master-slave synchronization. Then he told me that the newer generation architecture is Redlock. Redlock: the full name is Redis Distributed Lock; that is, a distributed lock implemented using
redis. ;
Also known as RedLock, very famous! It is the relatively safest and most reliable means for Redis to implement distributed locks.
His core idea is: create several independent Masters, such as five. Then lock them one by one. As long as more than half (here is 5/2+1 = 3), it means the lock is successful, and then when the lock is released, it is also released one by one. The advantage of this is that if one Master fails, there will be others, so there is no delay. It seems to have perfectly solved the above problem. But it is not 100% safe, as I will say later .
The specific details are

1. Get the current time (milliseconds)
2. Use the same key and random number to acquire locks on N Master nodes. The attempt time to acquire the lock here should be much shorter than the lock timeout. This is to prevent us from continuing to acquire locks after a certain Master hangs up, resulting in being blocked. Blocked for too long. In other words, assuming that the lock expires in 30 seconds and it takes 31 seconds for the three nodes to lock, it naturally means that the lock failed.
3. Only when the lock is acquired on most nodes (usually [(2/n)+1]) and the total acquisition time is less than the lock timeout, the lock acquisition is considered successful.
4. If the lock is acquired successfully, the lock timeout is the initial lock timeout minus the total time taken to acquire the lock.
5. If the lock acquisition fails, either because the number of successfully acquired nodes is less than half, or because the time taken to acquire the lock exceeds the lock release time, the key on the master that has been set will be deleted.

Two points need to be noted:

1. The machine times of multiple Redis Masters must be synchronized.
2. If the Redis red lock machine hangs up, the startup must be delayed for 1 minute (just longer than the lock timeout), because: if there are three Masters, writing to 2 of them is successful, and the lock is successful, but one hangs up, and one Master is still available, release it The one that is naturally hung up when locked will not execute del. When it is started again instantly, it will be found that the lock is still there (because the expiration time has not yet reached), which may cause unknown problems. So let Redis start delayed.

The main problems:
1. The implementation principle is extremely complicated, I believe everyone has seen it.
It is still an unsafe locking method. For example, if all five Masters are locked, the expiration time is 3 seconds. However, due to network jitter or other circumstances, the locks are only added to 3 machines and the expiration time is 3 seconds, which is invalid. The last two have not been locked yet, and the first three have expired. But at this time, other threads locked again and found that the first three were locked normally without lock. Because of the majority principle, the three were considered to be locked successfully. This results in two threads successfully locking at the same time. The first three are the locks of the subsequent threads, and the last two are the locks of the first thread. Isn't this confusing? Threads are no longer safe! Maybe you will say to open watchDog renewal, that seems to be no problem, but let me change the question, it is not expired, but one is hung up, and it has not been synchronized to the slave. The slave has been upgraded to master, and other threads found out There is no lock on this Slave, but 3 units can still be successfully locked, more than half of them. It's still concurrent, which is not safe. What to do? Don’t you want to be a slave? RedLock is too troublesome!

Source code analysis

Download this method

@Override
public void lockInterruptibly(long leaseTime, TimeUnit unit) throws InterruptedException {
    
    
    long threadId = Thread.currentThread().getId();
    // 获取锁
    Long ttl = tryAcquire(leaseTime, unit, threadId);
    if (ttl == null) {
    
     // 获取成功
        return;
    }

    // 异步订阅redis chennel
    RFuture<RedissonLockEntry> future = subscribe(threadId);
    commandExecutor.syncSubscription(future); // 阻塞获取订阅结果

    try {
    
    
        while (true) {
    
    // 循环判断知道获取锁
            ttl = tryAcquire(leaseTime, unit, threadId);
            // lock acquired
            if (ttl == null) {
    
    
                break;
            }

            // waiting for message
            if (ttl >= 0) {
    
    
                getEntry(threadId).getLatch().tryAcquire(ttl, TimeUnit.MILLISECONDS);
            } else {
    
    
                getEntry(threadId).getLatch().acquire();
            }
        }
    } finally {
    
    
        unsubscribe(future, threadId);// 取消订阅
    }
}

Summary lockInterruptibly: acquire the lock, subscribe to the message to release the lock if unsuccessful, and block before obtaining the message. After getting the release notification, loop to acquire the lock.
Let’s focus on how to acquire the lock: Long ttl = tryAcquire(leaseTime, unit, threadId)

private Long tryAcquire(long leaseTime, TimeUnit unit, long threadId) {
    
    
    return get(tryAcquireAsync(leaseTime, unit, threadId));// 通过异步获取锁，但get(future)实现同步
}

private <T> RFuture<Long> tryAcquireAsync(long leaseTime, TimeUnit unit, final long threadId) {
    
    
                                          if (leaseTime != -1) {
    
     //1 如果设置了超时时间，直接调用 tryLockInnerAsync
                                              return tryLockInnerAsync(leaseTime, unit, threadId, RedisCommands.EVAL_LONG);
                                          }
//2 如果leaseTime==-1，则默认超时时间为30s
RFuture<Long> ttlRemainingFuture = tryLockInnerAsync(LOCK_EXPIRATION_INTERVAL_SECONDS, TimeUnit.SECONDS, threadId, RedisCommands.EVAL_LONG);
//3 监听Future，获取Future返回值ttlRemaining(剩余超时时间)，获取锁成功，但是ttlRemaining，则刷新过期时间
ttlRemainingFuture.addListener(new FutureListener<Long>() {
    
    
    @Override
    public void operationComplete(Future<Long> future) throws Exception {
    
    
        if (!future.isSuccess()) {
    
    
            return;
        }

        Long ttlRemaining = future.getNow();
        // lock acquired
        if (ttlRemaining == null) {
    
    
            scheduleExpirationRenewal(threadId);
        }
    }
});
return ttlRemainingFuture;
}

The following is the most important redis method of acquiring locks, tryLockInnerAsync:

<T> RFuture<T> tryLockInnerAsync(long leaseTime, TimeUnit unit, long threadId, RedisStrictCommand<T> command) {
    
    
    internalLockLeaseTime = unit.toMillis(leaseTime);
    return commandExecutor.evalWriteAsync(
        getName(),
        LongCodec.INSTANCE,
        command,
        "if (redis.call('exists', KEYS[1]) == 0) then " +
        "redis.call('hset', KEYS[1], ARGV[2], 1); " +
        "redis.call('pexpire', KEYS[1], ARGV[1]); " +
        "return nil; " +
        "end; " +
        "if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then " +
        "redis.call('hincrby', KEYS[1], ARGV[2], 1); " +
        "redis.call('pexpire', KEYS[1], ARGV[1]); " +
        "return nil; " +
        "end; " +
        "return redis.call('pttl', KEYS[1]);",
        Collections.<Object>singletonList(getName()), internalLockLeaseTime, getLockName(threadId));
}

This method mainly calls redis to execute eval lua. Why eval is used because redis has atomicity for executing lua scripts. Translate this method:

-- 1. 没被锁{
    
    key不存在}
eval "return redis.call('exists', KEYS[1])" 1 myLock
-- (1) 设置Lock为key，uuid:threadId为filed, filed值为1
eval "return redis.call('hset', KEYS[1], ARGV[2], 1)" 1 myLock 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- (2) 设置key过期时间{
    
    防止获取锁后线程挂掉导致死锁}
eval "return redis.call('pexpire', KEYS[1], ARGV[1])" 1 myLock 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110

-- 2. 已经被同线程获得锁{
    
    key存在并且field存在}
eval "return redis.call('hexists', KEYS[1], ARGV[2])" 1 myLock 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- (1) 可重入，但filed字段+1
eval "return redis.call('hincrby', KEYS[1], ARGV[2],1)" 1 myLock 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- (2) 刷新过去时间
eval "return redis.call('pexpire', KEYS[1], ARGV[1])" 1 myLock 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110

-- 3. 已经被其他线程锁住{
    
    key存在，但是field不存在}：以毫秒为单位返回 key 的剩余超时时间
eval "return redis.call('pttl', KEYS[1])" 1 myLock

This is how the core acquires the lock. The following directly releases the lock method unlockInnerAsync:

-- 1. key不存在
eval "return redis.call('exists', KEYS[1])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- (1) 发送释放锁的消息，返回1，释放成功
eval "return redis.call('publish', KEYS[2], ARGV[1])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
 
-- 2. key存在，但field不存在，说明自己不是锁持有者，无权释放，直接return nil
eval "return redis.call('hexists', KEYS[1], ARGV[3])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
eval "return nil"
 
-- 3. filed存在，说明是本线程在锁，但有可能其他地方重入锁，不能直接释放，应该-1
eval "return redis.call('hincrby', KEYS[1], ARGV[3]，-1)" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
 
-- 4. 如果减1后大于0，说明还有其他重入锁，刷新过期时间,返回0。
eval "return redis.call('pexpire', KEYS[1], ARGV[2])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
 
-- 5. 如果不大于0，说明最后一把锁，需要释放
-- 删除key
eval "return redis.call('del', KEYS[1])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- 发释放消息
eval "return redis.call('publish', KEYS[2], ARGV[1])" 2 myLock redisson_lock__channel_lock 0 3000 3d7b5418-a86d-48c5-ae15-7fe13ef0034c:110
-- 返回1，释放成功

From the lock release code, we can see that a message will be sent after deleting the key, so as mentioned above, after the lock acquisition fails, the subscription to this message will be blocked.

Several uses of locks

When locking, you can only specify the locking duration, but not the waiting time. This is a difference between redissonLock and redLock. It should be noted that if the leaseTime
(locking duration) is not specified, 30S will be added by default, and a lock will be performed every 10S. Renewal means that watchDog is enabled by default
. If leaseTime is specified, it will not be automatically renewed. After the leaseTime is reached, if the execution is still not completed, the lock will be released. At this time, other threads can continue to lock the key.

1、tryLock

When using the tryLock() method without parameters, redisson will automatically add a scheduled task to regularly refresh the lock's expiration time. If the unlock fails, the lock will never be released because the regularly refreshed task always exists.
When using the tryLock (long waitTime, TimeUnit unit) method with two parameters, one more function than the one without parameters is to retry to acquire the lock within waitTime until timeout and return failure tryLock (long waitTime, long leaseTime, TimeUnit unit
) When passing the release time leaseTime, within waitTime, retry to acquire the lock until timeout and return failure. However, the task of regularly refreshing the lock expiration time will not be added.

2. lock method

Failure to lock will block.
Generally, there are several important parameters, leaseTime, awaitTime.

The parameterless method will enable the watchDog mechanism

     RLock sss = redissonClient.getLock("sss");
     sss.lock();

Specify leaseTime

 sss.lock(15,TimeUnit.SECONDS);

If leaseTime is specified, it will not be automatically renewed. After the leaseTime is reached, if the execution is still not completed, the lock will be released. At this time, other threads can continue to lock the key.

Blocking timeout

The lock method does not specify a timeout, so if a lock failure is detected, it will continue to try to acquire the lock while. wait

      sss.tryLock(5,5,TimeUnit.SECONDS);

tryLock supports specifying waiting time. When the waiting time is up, it returns false and will not block.