Why use distributed locks & Redis to implement distributed locks

Why use distributed locks

1. Why use distributed locks

In order to ensure that a method can only be executed by the same thread at the same time under high concurrency conditions, in the case of single-machine deployment of traditional monolithic applications, you can use Java concurrency processing-related APIs (such as ReentrantLock or synchronized) for mutual exclusion control. However, with the needs of business development, after the original single-machine deployment system is evolved into a distributed system, since the distributed system is multi-threaded, multi-process and distributed on different machines, this will make the concurrency control of the original stand-alone deployment The lock strategy fails. In order to solve this problem, a cross-JVM mutual exclusion mechanism is needed to control the access to shared resources. This is the problem to be solved by distributed locks.

2. What conditions should a distributed lock have?

Before analyzing the three implementations of distributed locks, let’s first understand what conditions distributed locks should have:
1. In a distributed system environment, a method can only be executed by one thread of one machine at a time;

2. Highly available lock acquisition and release;

3. High-performance acquisition and release of locks;

4. It has reentrant characteristics;

5. Equipped with lock failure mechanism to prevent deadlock;

6. It has the feature of non-blocking lock, that is, if the lock is not acquired, it will directly return the failure to acquire the lock.

Principles of database implementation of distributed locks:

Redis realizes the principle of distributed lock:

Problem Note:

• If the business failure lock is still there, a deadlock will occur. You can add an expiration time to automatically release the lock, but the automatic release may release other jvm locks, so you need to add a unique identifier to the lock, and check if it is right before deleting it. If the lock held by this machine is yes, then delete it, and ensure that the query and deletion are an atomic operation, you can use lua script
• A deadlock phenomenon occurs, causing the virtual machine instance to be unable to obtain resources again, and the expiration time can be set. Defect: Because the length of business execution time is uncertain, the setting of the expiration time is uncertain. Optimization: Use the try catch finally statement block, call the del method in the finally statement to delete the key to complete the purpose of releasing the lock, so that the next time the virtual machine instance requests resources, the lock can be acquired through the setNx() method and the response business logic will be executed!

watchdog:

Possible problems for Redis clusters:

• Problem : When the master-slave is switched, the master-slave synchronization is delayed, and the lock information may not be synchronized to the new master
• solve:
  • Use multiple redis instances to store and share, and lock each redis when locking
    • Step 1: Obtain the current system time (mainly to calculate the total time spent by the client locking multiple instances)
    • Step 2: Lock multiple instances in turn. After the lock is completed, calculate the total time spent by the client on locking multiple instances.
    • If the total time spent on locking is shorter than the effective time set by the lock, it means that the locking is successful

Correct implementation of Redis distributed lock

foreword

There are generally three ways to implement distributed locks: 1. Database optimistic locks; 2. Redis-based distributed locks; 3. ZooKeeper-based distributed locks. This blog will introduce the second method, which is to implement distributed locks based on Redis. Although there are already various blogs on the Internet that introduce the implementation of Redis distributed locks, their implementations have various problems. In order to avoid misleading people, this blog will introduce in detail how to correctly implement Redis distributed locks.

---

reliability

First of all, in order to ensure that distributed locks are available, we must at least ensure that the implementation of locks meets the following four conditions at the same time:

1. mutual exclusion. At any one time, only one client can hold the lock.
2. No deadlock will occur. Even if a client crashes while holding the lock and does not actively unlock it, it can ensure that other clients can lock it later.
3. It is fault-tolerant. As long as most of the Redis nodes are running normally, the client can lock and unlock.
4. The trouble should end it. Locking and unlocking must be the same client, and the client itself cannot unlock the lock added by others.

---

Code

Component dependencies

First of all, we need to introduce Jedisopen source components through Maven, and pom.xmladd the following code to the file:

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>2.9.0</version>
</dependency>

lock code

correct posture

Talk is cheap, show me the code. First show the code, and then slowly explain why it is implemented in this way:

public class RedisTool {

    private static final String LOCK_SUCCESS = "OK";
    private static final String SET_IF_NOT_EXIST = "NX";
    private static final String SET_WITH_EXPIRE_TIME = "PX";

    /**
     * 尝试获取分布式锁
     * @param jedis Redis客户端
     * @param lockKey 锁
     * @param requestId 请求标识
     * @param expireTime 超期时间
     * @return 是否获取成功
     */
    public static boolean tryGetDistributedLock(Jedis jedis, String lockKey, String requestId, int expireTime) {

        String result = jedis.set(lockKey, requestId, SET_IF_NOT_EXIST, SET_WITH_EXPIRE_TIME, expireTime);

        if (LOCK_SUCCESS.equals(result)) {
            return true;
        }
        return false;

    }

}

As you can see, we lock only one line of code: jedis.set(String key, String value, String nxxx, String expx, int time), the set() method has five formal parameters:

• The first one is the key, we use the key as a lock, because the key is unique.
• The second one is value. What we pass is requestId. Many children’s shoes may not understand. Isn’t it enough to have a key as a lock? Why use value? The reason is that when we talked about reliability above, the distributed lock needs to meet the fourth condition to unblock the bell. By assigning the value to requestId, we know which request added the lock and unlock it. When there is a basis. requestId can be UUID.randomUUID().toString()generated using method.
• The third is nxxx. We fill in NX for this parameter, which means SET IF NOT EXIST, that is, when the key does not exist, we perform a set operation; if the key already exists, no operation is performed;
• The fourth is expx. We pass PX as this parameter, which means we need to add an expiration setting to this key. The specific time is determined by the fifth parameter.
• The fifth is time, which corresponds to the fourth parameter and represents the expiration time of the key.

In general, executing the above set() method will only lead to two results: 1. There is currently no lock (the key does not exist), then the lock operation is performed, and a validity period is set for the lock, and the value indicates the lock client. 2. There is already a lock, do not do any operation.

Careful children's shoes will find that our lock code meets the three conditions described in our reliability. First of all, set() adds the NX parameter, which can guarantee that if the existing key exists, the function will not be called successfully, that is, only one client can hold the lock, satisfying mutual exclusion. Secondly, because we have set an expiration time for the lock, even if the lock holder subsequently crashes and does not unlock it, the lock will be automatically unlocked (that is, the key is deleted) due to the expiration time, and deadlock will not occur. Finally, because we assign the value as requestId, which represents the request ID of the locked client, then we can verify whether it is the same client when the client is unlocked. Since we only consider the scenario of Redis stand-alone deployment, we will not consider fault tolerance for now.

Error example 1

A more common error example is to use jedis.setnx()and jedis.expire()combination to achieve locking, the code is as follows:

public static void wrongGetLock1(Jedis jedis, String lockKey, String requestId, int expireTime) {

    Long result = jedis.setnx(lockKey, requestId);
    if (result == 1) {
        // 若在这里程序突然崩溃，则无法设置过期时间，将发生死锁
        jedis.expire(lockKey, expireTime);
    }

}

The function of the setnx() method is SET IF NOT EXIST, and the expire() method is to add an expiration time to the lock. At first glance, it seems to be the same as the result of the previous set() method. However, since these are two Redis commands, they are not atomic. If the program suddenly crashes after executing setnx(), the lock does not have an expiration time set. Then a deadlock will occur. The reason why some people implement this on the Internet is because the lower version of jedis does not support the multi-parameter set() method.

Error example 2

public static boolean wrongGetLock2(Jedis jedis, String lockKey, int expireTime) {

    long expires = System.currentTimeMillis() + expireTime;
    String expiresStr = String.valueOf(expires);

    // 如果当前锁不存在，返回加锁成功
    if (jedis.setnx(lockKey, expiresStr) == 1) {
        return true;
    }

    // 如果锁存在，获取锁的过期时间
    String currentValueStr = jedis.get(lockKey);
    if (currentValueStr != null && Long.parseLong(currentValueStr) < System.currentTimeMillis()) {
        // 锁已过期，获取上一个锁的过期时间，并设置现在锁的过期时间
        String oldValueStr = jedis.getSet(lockKey, expiresStr);
        if (oldValueStr != null && oldValueStr.equals(currentValueStr)) {
            // 考虑多线程并发的情况，只有一个线程的设置值和当前值相同，它才有权利加锁
            return true;
        }
    }
        
    // 其他情况，一律返回加锁失败
    return false;

}

This kind of error example is more difficult to find the problem, and the implementation is more complicated. Implementation idea: Use jedis.setnx()commands to implement locking, where the key is the lock, and the value is the expiration time of the lock. Execution process: 1. Try to lock through the setnx() method. If the current lock does not exist, it will return the lock successfully. 2. If the lock already exists, get the expiration time of the lock and compare it with the current time. If the lock has expired, set a new expiration time and return the lock successfully. code show as below:

So where is the problem with this code? 1. Since the client generates the expiration time by itself, it is necessary to force the time of each client in the distributed environment to be synchronized. 2. When the lock expires, if multiple clients execute jedis.getSet()the method at the same time, although only one client can finally lock, the expiration time of the client's lock may be overwritten by other clients. 3. The lock does not have an owner ID, that is, any client can unlock it.

unlock code

correct posture

It is better to show the code first, and then take you to explain why it is implemented in this way:

public class RedisTool {

    private static final Long RELEASE_SUCCESS = 1L;

    /**
     * 释放分布式锁
     * @param jedis Redis客户端
     * @param lockKey 锁
     * @param requestId 请求标识
     * @return 是否释放成功
     */
    public static boolean releaseDistributedLock(Jedis jedis, String lockKey, String requestId) {

        String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
        Object result = jedis.eval(script, Collections.singletonList(lockKey), Collections.singletonList(requestId));

        if (RELEASE_SUCCESS.equals(result)) {
            return true;
        }
        return false;

    }

}

As you can see, we only need two lines of code to unlock it! In the first line of code, we wrote a simple Lua script code. The last time I saw this programming language was in "Hackers and Painters", I didn't expect to use it this time. In the second line of code, we pass the Lua code jedis.eval()into the method, and assign the parameter KEYS[1] as lockKey, and ARGV[1] as requestId. The eval() method is to hand over the Lua code to the Redis server for execution.

So what is the function of this Lua code? In fact, it is very simple. First, get the value corresponding to the lock, check whether it is equal to the requestId, and delete the lock (unlock) if they are equal. So why use the Lua language to achieve it? Because it is necessary to ensure that the above operations are atomic. Regarding the problems caused by non-atomicity, you can read [Unlock Code - Error Example 2] . So why executing the eval() method can ensure atomicity is derived from the characteristics of Redis. The following is a partial explanation of the eval command on the official website:

[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-xwJufUUR-1685854990816)(http://o7x0ygc3f.bkt.clouddn.com/Redis%E5%88%86 %E5%B8%83%E5%BC%8F%E9%94%81%E7%9A%84%E6%AD%A3%E7%A1%AE%E5%AE%9E%E7%8E%B0%E6 %96%B9%E5%BC%8F/Redis%E5%88%86%E5%B8%83%E5%BC%8F%E9%94%81%E7%9A%84%E6%AD%A3%E7 %A1%AE%E5%AE%9E%E7%8E%B0%E6%96%B9%E5%BC%8F_01.png)]

To put it simply, when the eval command executes the Lua code, the Lua code will be executed as a command, and Redis will not execute other commands until the eval command is executed.

Error example 1

The most common unlocking code is to directly use jedis.del()the method to delete the lock. This method of unlocking without first judging the owner of the lock will cause any client to unlock it at any time, even if the lock is not its own.

public static void wrongReleaseLock1(Jedis jedis, String lockKey) {
    jedis.del(lockKey);
}

Error example 2

This kind of unlocking code is okay at first glance. Even I almost realized it like this before. It is almost the same as the correct posture. The only difference is that it is divided into two commands to execute. The code is as follows:

public static void wrongReleaseLock2(Jedis jedis, String lockKey, String requestId) {
        
    // 判断加锁与解锁是不是同一个客户端
    if (requestId.equals(jedis.get(lockKey))) {
        // 若在此时，这把锁突然不是这个客户端的，则会误解锁
        jedis.del(lockKey);
    }

}

As in the code comment, the problem is that if jedis.del()the lock is no longer owned by the current client when the method is called, the lock added by others will be released. So is there really such a scenario? The answer is yes. For example, client A locks, and after a period of time, client A unlocks. Before execution, the jedis.del()lock suddenly expires. At this time, client B tries to lock successfully, and then client A executes the del() method again. Then the lock of client B is released.

---

Summarize

This article mainly introduces how to use Java code to correctly implement Redis distributed locks, and also gives two classic error examples for locking and unlocking. In fact, it is not difficult to implement distributed locks through Redis, as long as the four conditions in reliability can be met. Although the Internet has brought us convenience, as long as we have a question, we can google it, but the answer on the Internet must be right? In fact, it is not so, so we should always maintain a questioning spirit, think more and verify more.

If Redis is deployed on multiple machines in your project, you can try to implement Redissondistributed locks. This is an official Java component provided by Redis. The link is given in the reference reading section.

Other blog post reference: In-depth analysis of Redis distributed lock principle - Zhihu (zhihu.com)

Watchdog, to renew the lock

[Advanced Java] Five minutes to sort out the implementation principle of the watchdog, and write the Redis lock renewal function by hand to build core competitiveness_哔哩哔哩_bilibili

• Solution: It can be realized based on HashedWheelTimer and adding spin
• HashedWheelTimer : time wheel, a tool class for asynchronously delaying task execution