In-depth analysis of the Redis distributed lock of the Jingdong spike system module, I didn't explain to you, you beat me!

1 | 0 background

In the current development process, in accordance with company specifications, it is necessary to rely on the cache component in the framework. It has to be said that the encapsulation of CRUD operations, connection pool, cache routing, and cache security management and control of the component masters are impeccable. But there is a small problem. The component does not implement distributed locks. Then you have to find a way to rely on the cache component to implement a distributed lock.

What, why do you want to implement it yourself? There are ready-made open source components that can be used directly. For example, Spring-Integration-Redis provides RedisLockRegistry and Redisson, which are not much faster than implementing them yourself. Then I have to declare that I don't like to reinvent wheels. The specific reason is that the cache components in the project cannot be replaced, and the connection pool may not be reused. The second is that if you are not familiar with the implementation principles of open source components, then there is a problem, and maintenance requires more costs.

Let me talk about the two scenarios that currently require distributed locks. One is WeChat access_token refresh (distributed locks can ensure that access_token is refreshed only once, and after the refresh is completed, it is placed in the cache, and other requests are directly read from the cache); the other is distributed Deployed timing tasks (distributed locks can ensure that only one node's timing tasks are executed at the same time).

2 | 0What is a distributed lock

 

In the case of stand-alone deployment, if you want to ensure that specific services are executed in sequence, use the synchronized keyword, Semaphore, and ReentrantLock provided by the JDK, or we can customize the lock based on AQS. In the case of single-machine deployment, the lock is shared between multiple threads, but in the case of distributed deployment, the lock is shared between multiple processes. Then the distributed lock must ensure the uniqueness of the lock resource, which can be shared among multiple processes.

3 | 0Distributed lock features

 

• Ensure that the same method can only be executed by one thread in one process in one machine at a certain time;
• To ensure that the lock is reentrant (to avoid deadlock);
• To ensure high availability of acquiring and releasing locks;

4 | 0Distributed lock implementation scheme comparison

 

• Mysql: Generally, the project will use the cache, it is impossible to use the database, and it is not realistic to rely on the database. Although it is simple to implement optimistic and pessimistic locking, the performance is poor.
• Redis: First, clustering can improve availability. Secondly, it is very simple to implement distributed locks with Redis. In addition, there are many frameworks that have helped us implement them, and they can be used directly, which is very convenient. At the same time, the periodic failure mechanism can solve the problem of failure to delete the network jitter lock, so I prefer Redis to implement it.
• Zookeeper: Like Mysql, it is impossible to add and maintain a set of Zookeeper clusters in order to use distributed locks. Secondly, the implementation is still more complicated. If the implementation is not good, it will cause a "herd effect". If it is not the original system, Zookeeper will be relied on, and if the pressure is not great, Zookeeper is generally not used to implement distributed locks.

5 | 0 Points to consider for distributed locks

 

• Lock release (finally);
• Lock timeout setting;
• Lock refresh (timed task, executed every 2/3 of the lock life cycle);
• If the lock times out, prevent the lock of other threads from being deleted (other threads will get the lock), consider the value value to be identified by the thread id, and judge whether it is the thread id of the current thread when the current thread releases the lock;
• Reentrant

6 | 0Redis distributed lock

 

6|1RedisLockRegistry

 

RedisLockRegistry is a redis distributed lock implementation class provided in spring-integration-redis. It is mainly a better lock realized by the double lock of redis lock + local lock.

 

OBTAIN_LOCK_SCRIPT is a locked lua script. KEYS[1] represents the key value of the current lock, ARGV[1] represents the current client ID, and ARGV[2] represents the expiration time.

The basic logic is: get the corresponding client ID from redis according to KEYS[1]. If the existing client ID is equal to ARGV[1], then reset the expiration time to ARGV[2]; if the value does not exist, Set the corresponding value of KEYS[1] to ARGV[1], and the expiration time to ARGV[2].

 

 

The process of acquiring the lock is also very simple. First, acquire the lock through the local lock (localLock, which corresponds to the ReentrantLock instance), and then execute the OBTAIN_LOCK_SCRIPT script through RedisTemplate to acquire the redis lock.

Why use local locks? First of all, for reentrant locks, and secondly to reduce the pressure on redis services.

 

The process of releasing the lock is also relatively simple. The first step is to judge whether the current thread holds the lock through the local lock, and the second step is to judge the count of the lock held by the current thread through the local lock.

If the count of the lock held by the current thread is> 1, it means that the local lock has been acquired by the current thread multiple times. At this time, only the local lock is released (the count of the lock held by the current thread after release is -1).

If the current thread holds the lock count = 1, release the local lock and redis lock.

 

RedisLockRegistry is used as shown above.

First define the Bean corresponding to RedisLockRegistry, which needs to rely on the Redis ConnectionFactory.

Then inject the RedisLockRegistry instance into the service layer.

The business logic is wrapped by the lock method and the unlock method. It should be noted that the unlock method should be written in the finally code block.

6|2Redisson

 

Redisson is a Java In-Memory Data Grid based on Redis.

It makes full use of a series of advantages provided by the Redis key-value database, and provides users with a series of common tools with distributed characteristics based on the common interfaces in the Java utility toolkit.

As a toolkit for coordinating single-machine multi-threaded concurrent programs, it has obtained the ability to coordinate distributed multi-machine multi-threaded concurrent systems, greatly reducing the difficulty of designing and developing large-scale distributed systems.

At the same time, the combination of various characteristic distributed services further simplifies the collaboration between programs in a distributed environment.

　　

First feel the use of redis distributed locks through Redisson Api.

Define RedissonBuilder and build RedissonClient through the address of redis cluster.

 

Define a Bean of type RedissonClient.

 

In the business code, the distributed lock is obtained through RedissonClient.

 

Since the understanding of Redisson distributed lock implementation principle is not very thorough, here is an article recommended: Redisson distributed lock implementation analysis.

6 | 3Comparison between Redisson and RedisLockRegistry

 

• RedisLockRegistry is realized by local lock (ReentrantLock) and redis lock, double lock, Redission is realized by Netty Future mechanism, Semaphore (jdk semaphore), redis lock.
• RedisLockRegistry and Redssion are both implemented reentrant locks.
• RedisLockRegistry does not handle the lock refresh. Redisson uses Netty's TimerTask and Timeout tools to complete the regular lock refresh task.
• RedisLockRegistry only implements distributed locks, while Redisson handles distributed locks and also provides rich APIs such as queues, collections, and lists.

7 | 0 Hands-on implementation of distributed locks

 

7 | 1 Implementation principle

 

Local lock (ReentrantLock) + redis lock

7 | 2 Get the lock lua script

 

 

7 | 3 lock refresh lua script

 

 

7 | 4 lock release lua script

 

 

7 | 5 Local lock definition

 

Each lock key corresponds to a unique local lock

 

7 | 6 Thread ID definition

 

In a distributed environment, each thread corresponds to a unique identifier

 

7 | 7 Lock refresh timing task definition

 

The timing task execution is completed through JDK ConcurrentTaskScheduler, and the ScheduledFuture complete timing task destruction. The taskId corresponds to the thread ID.

 

7 | 8Define distributed lock annotation

 

 

7 | 9 Distributed lock section

 

 

Obtain the lock key value and lock expiration time information through the RedisLock annotation instance lockInfo.

7 | 10 Acquisition lock process

 

 

1. Obtain the lock key value through the lockInfo.key() method, and obtain the corresponding local lock (ReentrantLock) through the lock key value
2. Local lock acquisition lock object
3. Enter the loop of acquiring redis lock
4. Execute the lua script to acquire the lock through the cache service component
5. If the redis lock is acquired, determine whether the current thread acquires the lock for the first time and turns on lock refresh, and the corresponding registered lock refresh timing task
6. If the redis lock is not obtained, sleep lockInfo.sleep() for milliseconds and try again

7 | 11 Release lock process

 

 

1. Obtain the local lock corresponding to the current lock key value
2. Determine whether the current thread is the holder of the local lock
3. If the number of reentries of the local lock is greater than 1, only the local lock is released
4. If the number of reentrants of the local lock is equal to 1, release the local lock and redis lock

7 | 12 Distributed lock test

 

Define the test class, annotate the test method with @RedisLock annotation, make the key value of the lock "redis-lock-test", and sleep randomly in the test method.

 

Start 20 threads and call the test method at the same time.

 

The results of the multi-threaded redis distributed lock test are as follows.

 

　　

Define the reentrant test class, get the current proxy object in the method, and call the test method recursively.

 

In the test method, call the test method marked with @RedisLock in the reentrant test class.

 

The results of the distributed lock reentrant test are as follows.

 

8 | 0 Practical application of distributed locks

 

8 | 1 Define access_token refresh service

 

 

The @RedisLock annotation is marked on the refreshAccessToken method, indicating that this method will be executed serially in a distributed environment.

First get the access_token from the cache.

If the access_token in the cache is empty or equal to the invalid access_token, a new access_token is generated through TokenAPI and placed in the cache.

If the access_token in the cache is not empty and is not equal to the invalid access_token, the access_token in the cache is directly returned.

8 | 2Define access_token to obtain service

 

 

If the access_token in the cache is empty, refresh the access_token directly and put it in the cache.

If the access_token in the cache is not empty and equal to the invalid access_token, refresh the access_token and put it in the cache, otherwise directly return the access_token in the cache.

8 | 3 Distributed lock application scenarios

 

In a distributed environment, problems involving concurrency between threads and processes can be solved by distributed locks. If it is the concurrency problem of sharing resources between single-node threads, it can be solved by the thread lock provided by the JDK. If it is the concurrency problem of sharing resources between multiple nodes and threads, you need to use distributed locks. For example, the most common spikes, grab red envelopes, back-end services involving inventory deduction, amount deduction, and other high-concurrency serialization operations can all use distributed locks to solve problems. The example described in this article is mainly applied to WeChat official account and WeChat applet access_token refresh, WeChat sharing jsapi_ticket refresh, distributed lock can ensure that access_token and jsapi_ticket have only one thread to perform refresh actions under high concurrency, avoiding access_token or access_token after multiple refreshes. jsapi_ticket is invalid.