foreword
Today I bring you a good article about distributed locks. The blogger has written several articles about the content of distributed systems, and has received a lot of praise from everyone. Distributed systems still account for a relatively large proportion of current development. If you haven't been exposed to distributed systems, you are welcome to study the blogger's microservice column. If you know a thing or two about microservices, then The blogger's recent series of digging ancestral graves is very suitable for you. Let's dig and dig and dig in the land of Java together!
distributed lock
lock concept
The lock in the code is actually very similar to our real life. The purpose of using the lock is to ensure the safety of personal and property, which is actually the same as the use in the code.
The use of locks is that we want the situation to be: who locks the lock and who opens it. This is the safest situation, whether it is locked outside or locked indoors.
In the code, when multiple threads make a variable at the same time, the process needs to be synchronized, and the operations are mutually exclusive, and its essence is the idea of locking.
Distributed lock concept
No distributed system can satisfy Consistency, Availability, and Partition tolerance at the same time, and can only satisfy two at the same time. We have already told you about this when we talked about distributed transactions. I don’t allow you to know about CAP. Portal: Java Development- I don’t know if it counts as a detailed explanation
What the distributed lock has to do is to ensure the security of the data when multiple threads modify the data at the same time, and only one thread can operate the data at the same time, which is what we call security. Lock it during operation, unlock it when it is used up, and leave the next opportunity to acquire the lock to others.
This made the blogger think of a more interesting thing in life: squatting in the pit! Who doesn't want other people to come in and be convenient with you when they are squatting in the pit? This case is highly consistent with the idea of distributed locks, is there any? Is there any? Hahahaha~~
Features of distributed locks
The characteristics of distributed locks are actually very simple and easy to understand. Let's list them:
- Mutual exclusion : only one service can access a resource at a time
- Atomicity : Consistency requires that the behavior of locking and unlocking is atomic. Atomicity refers to one or a series of operations that cannot be interrupted by the smallest particle
- Security : who locks, who releases
- Fault tolerance : When the service holding the lock crashes, the lock can still be released, so as to avoid deadlock
- High availability : High availability is generally used to describe distributed systems. Here, it means that acquiring and releasing locks is safe and stable, and also includes fault-tolerant capabilities.
- High performance : The performance of acquiring and releasing locks is better, because some operations are inefficient locks, which will be discussed below
- Persistence : locks are automatically renewed/extended according to business needs
Application Scenarios of Distributed Locks
Regarding the usage scenarios of distributed locks, here are a few common ones:
- Bank withdrawals and mobile phone transfers are carried out at the same time. You can see if you can get double the money. Obviously not, you need distributed locks
- Users of different clusters under the cluster go to perform flash sales of products, and there is only one product inventory, so obviously only one person can succeed in flash sales, and distributed locks are needed
I believe that everyone can fully imagine this process, so I will not draw the flow chart anymore.
Execution process of distributed lock
Although bloggers really don’t want to draw pictures, there must be pictures here. No matter how many words there are, it is not as good as a logical and clear flow chart. Below, we use a picture to illustrate the execution process of distributed locks:
Is it very simple after reading it, is it the same as you thought? Don't think it's so difficult, it's actually very simple! ! !
Database implements distributed lock
Regarding the implementation, in fact, it still needs to be done according to the specific business, so the blogger is not only here, but also includes the following. If it involves a specific business, no specific code will be written. If it is general, it will be displayed in the form of memory code.
Table lock implementation
According to the table lock implementation, it can be roughly divided into the following steps:
- Create a table without too many parameters, such as:
CREATE TABLE tb ( check_no varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL, PRIMARY KEY (check_no) USING BTREE ) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic; SET FOREIGN_KEY_CHECKS = 1;SET FOREIGN_KEY_CHECKS=0; //Turn off foreign key constraint checking
-
SET FOREIGN_KEY_CHECKS=1; //Enable foreign key constraint checking
- When accessing data, store a number in this table
- As long as the insert is successful, it means that the lock acquisition is successful, otherwise the lock acquisition fails
- Due to the primary key conflict, only one same number can be stored in a table, and the lock is generated in this way
- After the logic execution of the program that successfully acquires the lock, deleting the data of the number means that the lock is released. This is where other threads can compete for the lock. Whoever inserts successfully acquires the lock and can perform their own tasks.
This table is created by bloggers who have seen others. Personally, I feel that the parameter setting is a little complicated. From all aspects, a simple table is enough to realize it. I hope to discuss the reasons with you, and please feel free to enlighten me.
conditional realization
Query based on conditions is based on row locks. This condition is similar to the version number below. Let’s talk about conditional queries first. Let's take a look at the following SQL:
update tb_stock set stock=stock-#{saleNum} where id = #{id} and stock-#{saleNum}>=0"
You should know that in innoDB, the update operation is the symbol of the row lock, so when the row lock is used, no matter how many threads there are to operate the data, but only one thread is operating at the same time, the total inventory minus The remaining inventory after the purchase quantity must be >=0, which ensures that the product will not be oversold. This is the idea of optimistic locking, but optimistic locking is not locking, it is just an idea, nothing more, you only need this method to work. Optimistic locking has two ideas, that is, version number and CAS. Obviously, this is the practice of CAS.
About optimistic locking and pessimistic locking, bloggers won’t go into details, if you don’t understand, click this link to check it out: Java development-basic data structure in database
version number implementation
The implementation of the version number is very similar to the conditions. When there is no basis for judgment, a version field can be added. Let's look at the following SQL:
update tb_user set name=#{name},version=version+1 where id=#{id} and version=#{version}
After reading it, I actually feel that the version number and the principle of the condition are exactly the same. They are all based on the principle of row locking, but they have certain problems, which we will discuss below.
Disadvantages of database implementation of distributed lock
From the above, we can easily see that its shortcomings are obvious. First of all, it is the frequent operation of the database. It is also very busy, and there are still some problems in performance.
The above problems are not unsolvable, but to solve these problems, the implementation method will become more and more complicated, and performance issues have to be considered. If it is a stand-alone system, it can also be considered. Since it is distributed, why use this method?
Zookeeper implements distributed locks
Realization principle
Zookeeper's ability to implement distributed locks benefits from its designer's ingenious design, mainly using zookeeper's znode node characteristics and watch mechanism to complete.
znode node
It is also very simple to say, znode nodes are divided into two categories, namely: temporary nodes and persistent nodes, and they are divided into two categories: ordered nodes and disordered nodes, see the detailed description below:
- Persistent node: Once created, it exists permanently in zookeeper unless manually deleted
- Persistent ordered nodes: Once created, they exist permanently in zookeeper unless manually deleted. The difference is that each node has a node serial number by default, and it is sequentially incremented
- Temporary node: After the node is created, once the server restarts or goes down, it will be automatically deleted
- Temporary ordered node: After the node is created, once the server restarts or goes down, it will be automatically deleted. The difference is that each node has a node serial number by default, and it increases in order
watch monitoring mechanism
The watch monitoring mechanism is mainly used to monitor the state changes of nodes and to trigger subsequent events. When node B monitors node A, once node A is modified, deleted, or the list of child nodes changes, node B will receive a Notification of node changes, and then complete other tasks. Here, B listens to A releasing the lock, and then tries to acquire the lock.
Way of working
Although the ordered nodes are ordered, before creating the first node, you need to create a parent node first, and then create a temporary ordered node under the parent node. The reason why the temporary node is used is to ensure that after the server goes down or restarts , the original lock can be released. As the number of nodes increases, the first lock to be acquired must be the node with the smallest serial number. After the task is completed, delete the temporary node, and then go to find the next smallest node, and the monitoring mechanism is to monitor the previous one according to the latter one. For example, B listens to A, C listens to B, D listens to C, E listens to D... and so on. After the previous one is completed, the next node will be notified to acquire the lock and execute the task. As shown below:
Implementation of lock idea (inefficient lock)
When using zookeeper, it is also very common to have an inefficient implementation method. Its principle is to use only one lock node. When creating a lock node, if the lock node does not exist, the creation is successful, which means that the current thread has acquired the lock. If If the creation of the lock node fails, it means that other threads have acquired the lock, and the thread will monitor the release of the lock node, which means deleting the node. When the lock node is released, other threads continue to try to create and lock the lock node.
Let's do a simulation
Create a new project, the pom file is as follows:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.6.0</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.codingfire</groupId>
<artifactId>zookeeper_lock</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>zookeeper_lock</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!--zookeeper-->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.5.5</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.101tec</groupId>
<artifactId>zkclient</artifactId>
<version>0.10</version>
<exclusions>
<exclusion>
<artifactId>slf4j-log4j12</artifactId>
<groupId>org.slf4j</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>2.6.0</version>
</plugin>
</plugins>
</build>
</project>
Create an abstract class to provide external methods for adding and unlocking:
package com.codingfire.zookeeper_lock;
import org.I0Itec.zkclient.ZkClient;
public abstract class AbstractLock {
//zookeeper服务器地址
public static final String ZK_SERVER_ADDR="localhost:2181";
//zookeeper超时时间
public static final int CONNECTION_TIME_OUT=30000;
public static final int SESSION_TIME_OUT=30000;
//创建zk客户端
protected ZkClient zkClient = new ZkClient(ZK_SERVER_ADDR,SESSION_TIME_OUT,CONNECTION_TIME_OUT);
/**
* 获取锁
* @return
*/
public abstract boolean tryLock();
/**
* 等待加锁
*/
public abstract void waitLock();
/**
* 释放锁
*/
public abstract void releaseLock();
//加锁实现
public void getLock() {
String threadName = Thread.currentThread().getName();
if (tryLock()){
System.out.println(threadName+": 获取锁成功");
}else {
System.out.println(threadName+": 等待获取锁");
waitLock();
getLock();
}
}
}
Create a subclass that implements the abstract method:
package com.codingfire.zookeeper_lock;
import org.I0Itec.zkclient.IZkDataListener;
import java.util.concurrent.CountDownLatch;
public class LowLock extends AbstractLock {
private static final String LOCK_NODE="/lock_node";
private CountDownLatch countDownLatch;
@Override
public boolean tryLock() {
if (zkClient == null){
return false;
}
try {
zkClient.createEphemeral(LOCK_NODE);
return true;
} catch (Exception e) {
return false;
}
}
@Override
public void waitLock() {
//注册监听
IZkDataListener listener = new IZkDataListener() {
//节点数据改变触发
@Override
public void handleDataChange(String dataPath, Object data) throws Exception {
}
//节点删除触发
@Override
public void handleDataDeleted(String dataPath) throws Exception {
if (countDownLatch != null){
countDownLatch.countDown();
}
}
};
zkClient.subscribeDataChanges(LOCK_NODE,listener);
//如果节点存在，则线程阻塞等待
if (zkClient.exists(LOCK_NODE)){
countDownLatch = new CountDownLatch(1);
try {
countDownLatch.await();
System.out.println(Thread.currentThread().getName()+": 等待获取锁");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
//节点不存在，删除监听
zkClient.unsubscribeDataChanges(LOCK_NODE,listener);
}
@Override
public void releaseLock() {
System.out.println(Thread.currentThread().getName()+": 释放锁");
zkClient.delete(LOCK_NODE);
zkClient.close();
}
}
Then we test the above code in the main method of the startup class:
package com.codingfire.zookeeper_lock;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import java.util.concurrent.TimeUnit;
@SpringBootApplication
public class ZookeeperLockApplication {
public static void main(String[] args) {
//模拟多个10个客户端
for (int i = 0; i < 20; i++) {
Thread thread = new Thread(new LockRunnable());
thread.start();
}
}
private static class LockRunnable implements Runnable {
@Override
public void run() {
AbstractLock abstractLock = new LowLock();
abstractLock.getLock();
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
abstractLock.releaseLock();
}
}
}
Then you can run the test. Not surprisingly, the tasks are executed one by one. Since the debug log of zookeeper is still printed when outputting, the page is very messy, so I won’t take a screenshot. Just run it yourself and see the result. The log can’t be removed yet, obviously the tool for printing the log has been removed:
<!--zookeeper-->
<dependency>
<groupId>org.apache.zookeeper</groupId>
<artifactId>zookeeper</artifactId>
<version>3.5.5</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.101tec</groupId>
<artifactId>zkclient</artifactId>
<version>0.10</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
Children's shoes who know the reason can leave a message for the blogger. Just now our code is based on a lock implemented by a single node, which will cause a problem: once the lock is released, other waiting threads will receive a notification and compete for the only lock. This is due to the change of the watched znode node, And causing a large number of notification operations is called the herd effect, which will seriously reduce the performance of zookeeper. So this inefficient locking approach is generally not used in distributed systems.
zookeeper efficient lock implementation
In order to solve the herd problem of inefficient locks, we will queue these tasks, which will use ordered nodes. Among the ordered nodes, the smallest node acquires the lock and executes the task. After the execution is completed, the lock is released (delete the node), and the smallest node of the next watch will receive the notification and acquire the lock to avoid competition. This is an efficient lock. The implementation process is as follows:
Tool classes for efficient locks also need to inherit from abstract classes:
package com.codingfire.zookeeper_lock;
import org.I0Itec.zkclient.IZkDataListener;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;
public class HighLock extends AbstractLock {
private static String PARENT_NODE_PATH="";
public HighLock(String parentNodePath){
PARENT_NODE_PATH = parentNodePath;
}
//当前节点路径
private String currentNodePath;
//前一个节点的路径
private String preNodePath;
private CountDownLatch countDownLatch;
@Override
public boolean tryLock() {
//判断父节点是否存在
if (!zkClient.exists(PARENT_NODE_PATH)){
//父节点不存在，创建持久节点
try {
zkClient.createPersistent(PARENT_NODE_PATH);
} catch (Exception e) {
}
}
//创建第一个临时有序节点
if (currentNodePath==null || "".equals(currentNodePath)){
//在父节点下创建临时有序节点
currentNodePath = zkClient.createEphemeralSequential(PARENT_NODE_PATH+"/","lock");
}
//不是第一个临时有序节点
//获取父节点下的所有子节点列表
List<String> childrenNodeList = zkClient.getChildren(PARENT_NODE_PATH);
//因为有序号，所以进行升序排序
Collections.sort(childrenNodeList);
//判断是否加锁成功，当前节点是否为父节点下序号最小的节点
if (currentNodePath.equals(PARENT_NODE_PATH+"/"+childrenNodeList.get(0))){
//当前节点是序号最小的节点
return true;
}else {
//当前节点不是序号最小的节点，获取其前置节点，并赋值
int length = PARENT_NODE_PATH.length();
int currentNodeNumber = Collections.binarySearch(childrenNodeList, currentNodePath.substring(length + 1));
preNodePath = PARENT_NODE_PATH+"/"+childrenNodeList.get(currentNodeNumber-1);
}
return false;
}
@Override
public void waitLock() {
//注册监听
IZkDataListener listener = new IZkDataListener() {
@Override
public void handleDataChange(String dataPath, Object data) throws Exception {
}
@Override
public void handleDataDeleted(String dataPath) throws Exception {
if (countDownLatch != null){
countDownLatch.countDown();
}
}
};
zkClient.subscribeDataChanges(preNodePath,listener);
//判断前置节点是否存在，存在则阻塞
if (zkClient.exists(preNodePath)){
countDownLatch = new CountDownLatch(1);
try {
countDownLatch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
//删除监听
zkClient.unsubscribeDataChanges(preNodePath,listener);
}
@Override
public void releaseLock() {
zkClient.delete(currentNodePath);
zkClient.close();
}
}
The calling method is as follows:
//获取当前方法名
String methodName = Thread.currentThread().getStackTrace()[1].getMethodName();
AbstractLock lock = new HighLock("/"+methodName);
try {
//加锁
lock.getLock();
//执行操作
} catch (Exception e) {
e.printStackTrace();
} finally {
//释放锁
lock.releaseLock();
}It's that simple, and the above are basically the encapsulation classes commonly used in the industry, the difference will not be great, and they can be used directly. Before testing, please make sure your zookeeper is enabled.
Redis implements distributed locks
Single node Redis lock
If it is a single-node redis, then the implementation of distributed locks is quite simple, because redis itself is single-threaded, based on this feature, in the case of concurrency, each request needs to be queued to enter, and only one thread can enter at the same time and acquire the lock.
Redis implements distributed locks based on three core APIs:
- setNx(): save the key-value in redis, only when the key does not exist, it will be set successfully, otherwise it will return 0, reflecting its mutual exclusion
- expire(): Set the expiration time of the key to avoid deadlocks
- delete(): delete the key to release the lock
lock
Locking is performed through jedis.set. If the return value is OK, it means that the locking is successful. Otherwise, the locking fails. If it fails, it will continue to spin and try to acquire the lock. At the same time, if the lock is still not acquired within a certain period of time, it will exit the spin. No more attempts to acquire the lock. When locking, requestId needs to be used to identify the lock token held by each thread.
package com.codingfire.zookeeper_lock;
import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.params.SetParams;
public class SingleRedisLockUtil {
JedisPool jedisPool = new JedisPool("localhost",6379);
//锁过期时间
protected long internalLockLeaseTime = 30000;
//获取锁的超时时间
private long timeout = 999999;
/**
* 加锁
* @param lockKey 锁键
* @param requestId 请求唯一标识
* @return
*/
SetParams setParams = SetParams.setParams().nx().px(internalLockLeaseTime);
public boolean tryLock(String lockKey, String requestId){
String threadName = Thread.currentThread().getName();
Jedis jedis = this.jedisPool.getResource();
Long start = System.currentTimeMillis();
try{
for (;;){
String lockResult = jedis.set(lockKey, requestId, setParams);
if ("OK".equals(lockResult)){
System.out.println(threadName+": 获取锁成功");
return true;
}
//否则循环等待，在timeout时间内仍未获取到锁，则获取失败
System.out.println(threadName+": 获取锁失败，等待中");
long l = System.currentTimeMillis() - start;
if (l>=timeout) {
return false;
}
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}finally {
jedis.close();
}
}
}
Before creating the class, you need to add a few dependencies, otherwise an error will be reported:
<dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>3.0.1</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-pool2</artifactId> </dependency> <!--Redis分布式锁--> <dependency> <groupId>org.redisson</groupId> <artifactId>redisson-spring-boot-starter</artifactId> <version>3.13.1</version> </dependency>
unlock
Unlocking is also somewhat knowledgeable. It is necessary to prevent the current thread from releasing other people's locks. When thread A successfully locks, thread A will unlock after a period of time. Thread A's lock has expired. At this time, thread B also locks, because The lock of thread A has expired, so thread B can lock successfully. At this point, thread A may release the lock of thread B. This is why requestId is introduced when locking. Therefore, when unlocking, it is necessary to judge whether the value of the current lock key is the same as the value passed in. If they are the same, it means that they are the same person and can be unlocked. Otherwise it cannot be unlocked.
When unlocking, if you first query for comparison, and then delete the same after the comparison, it looks right, but there is a pitfall, that is, atomicity is ignored. Atomicity requires that our series of operations must be completed in one step. Otherwise, accidents are prone to occur in the middle, so queries and deletions are usually done with lua scripts. Let's see how to write the code:
Add the unlock method in the tool class:
/**
* 解锁
* @param lockKey 锁键
* @param requestId 请求唯一标识
* @return
*/
public boolean releaseLock(String lockKey,String requestId){
String threadName = Thread.currentThread().getName();
System.out.println(threadName+"：释放锁");
Jedis jedis = this.jedisPool.getResource();
String lua =
"if redis.call('get',KEYS[1]) == ARGV[1] then" +
" return redis.call('del',KEYS[1]) " +
"else" +
" return 0 " +
"end";
try {
Object result = jedis.eval(lua, Collections.singletonList(lockKey),
Collections.singletonList(requestId));
if("1".equals(result.toString())){
return true;
}
return false;
}finally {
jedis.close();
}
}test
Before testing, please make sure your Redis service is enabled, and then run the following code:
package com.codingfire.zookeeper_lock;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
public class RedisTest {
public static void main(String[] args) {
//模拟多个5个客户端
for (int i=0;i<5;i++) {
Thread thread = new Thread(new LockRunnable());
thread.start();
}
}
private static class LockRunnable implements Runnable {
@Override
public void run() {
SingleRedisLockUtil singleRedisLock = new SingleRedisLockUtil();
String requestId = UUID.randomUUID().toString();
boolean lockResult = singleRedisLock.tryLock("lock", requestId);
if (lockResult){
try {
TimeUnit.SECONDS.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
singleRedisLock.releaseLock("lock",requestId);
}
}
}
A corner of the console output:
There is no way to take a particularly complete screenshot, and everyone will understand after looking at it.
single node problem
The above single-node solution is just a basic solution, and there are still some problems. If the lock expires, it will be released automatically. What should I do if the task has not been executed yet? At this time, the same task may be executed repeatedly, so the setting of the supermarket time of the lock is more particular, but there is no way to predict the length of the task. It is best to dynamically extend the time when the lock is about to expire, because the lock is in If it has not been released before the expiration date, it must be because the task has not been completed. There is no doubt about this.
The best way is to create a daemon thread, and at the same time define a timed task to increase the expiration time for unreleased locks every once in a while. When the business is executed, the lock is released, and then the daemon thread is closed. This solves the problem of lock renewal.
Although a single node is good, it is also possible for Redis to have a cluster, right? After all, Redis is also a kind of database. For data security and cache to improve query efficiency, it is not too much for me to be a master-slave, right? Then we will continue to look down.
Redisson distributed lock
Redisson is a third-party class library recommended by the redis official website to implement distributed locks. It has powerful functions, implements various locks, and is very simple to use. It is relatively easier for Redisson to implement single-node distributed locks. It can be operated directly based on the lock()&unlock() method.
Let's take a look at Redisson's single-node distributed lock first.
Redisson single-node distributed lock
Dependence is indispensable:
<!--Redis分布式锁-->
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson-spring-boot-starter</artifactId>
<version>3.13.1</version>
</dependency>
The configuration file is arranged on:
server:
redis:
host: localhost
port: 6379
database: 0
jedis:
pool:
max-active: 500
max-idle: 1000
min-idle: 4
Add the following code to the startup class, and the final startup class is as follows:
package com.codingfire.zookeeper_lock;
import org.redisson.Redisson;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import java.util.concurrent.TimeUnit;
@SpringBootApplication
public class ZookeeperLockApplication {
@Value("${spring.redis.host}")
private String host;
@Value("${spring.redis.port}")
private String port;
@Bean
public RedissonClient redissonClient(){
RedissonClient redissonClient;
Config config = new Config();
String url = "redis://" + host + ":" + port;
config.useSingleServer().setAddress(url);
try {
redissonClient = Redisson.create(config);
return redissonClient;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
public static void main(String[] args) {
SpringApplication.run(ZookeeperLockApplication.class,args);
}
}
Write the unlocking tool class:
package com.codingfire.zookeeper_lock;
import org.redisson.api.RLock;
@Component
public class RedissonLock {
@Autowired
private RedissonClient redissonClient;
/**
* 加锁
* @param lockKey
* @return
*/
public boolean addLock(String lockKey){
try {
if (redissonClient == null){
System.out.println("redisson client is null");
return false;
}
RLock lock = redissonClient.getLock(lockKey);
//设置锁超时时间为5秒，到期自动释放
lock.lock(10, TimeUnit.SECONDS);
System.out.println(Thread.currentThread().getName()+": 获取到锁");
//加锁成功
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
public boolean releaseLock(String lockKey){
try{
if (redissonClient == null){
System.out.println("redisson client is null");
return false;
}
RLock lock = redissonClient.getLock(lockKey);
lock.unlock();
System.out.println(Thread.currentThread().getName()+": 释放锁");
return true;
}catch (Exception e){
e.printStackTrace();
return false;
}
}
}
Test session:
I originally wanted to write the test in the main method, but found that the static method cannot call the external non-static method, so forget it, and created a test class RedissonLockTest:
package com.codingfire.zookeeper_lock;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
@SpringBootTest
@RunWith(SpringRunner.class)
public class RedissonLockTest {
@Autowired
private RedissonLock redissonLock;
@Test
public void easyLock(){
//模拟多个5个客户端
for (int i = 0; i < 5; i++) {
Thread thread = new Thread(new LockRunnable());
thread.start();
}
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}
}
private class LockRunnable implements Runnable {
@Override
public void run() {
redissonLock.addLock("lock");
try {
TimeUnit.SECONDS.sleep(3);
} catch (InterruptedException e) {
e.printStackTrace();
}
redissonLock.releaseLock("lock");
}
}
}
Look at the console output:
The five threads are executed sequentially, and the test is successful.
watchdog mechanism
Although it is more convenient for Redisson to implement distributed locks on a single node, there are still certain problems. We should see the setting expiration time when using locks. Do you have any impression? Expiration! ! ! The same problem as in zookeeper above, zookeeper uses a daemon thread to renew the lock, here we use the watchdog mechanism.
The watchdog mechanism is easier to use than the daemon thread, and it is done in two steps:
- Do not set the expiration time, the default is 30s
- Change the locking method from lock() to tryLock()
In addition, the expiration time can be modified:
config.setLockWatchdogTimeout(3000L);
Distributed system and red lock algorithm
Once the single-node Redis goes down, the lock cannot be operated, so configurations like master-slave are inevitable. However, there are certain problems in the master-slave mode: the master copies data to the slave asynchronously. When a thread holds a lock and has not copied the data to the slave, the master crashes, and the slave will be promoted to the master at this time. , the new master does not have the lock information of the previous thread, so other threads can re-lock, what should I do? Fortunately, there is a red lock algorithm to solve it.
Redlock is an algorithm based on multi-node redis to implement distributed locks , which can effectively solve the problem of redis single point of failure. The official suggestion is to build five redis servers to implement the red lock algorithm, which is more expensive~~~
The red lock algorithm is very strict about the calculation of the expiration time. Fortunately, we don’t need to calculate these by ourselves. The blogger is being lazy here, so I won’t talk about it, but we must ensure that AOF is enabled when using Redis. If Redis goes down, Restart after the ttl time, the disadvantage is that Redis cannot provide external services within the ttl time, but it is also acceptable in the case of multiple nodes.
Implementation of Red Lock Algorithm
Create a configuration class:
package com.codingfire.zookeeper_lock;
import org.redisson.Redisson;
import org.redisson.RedissonRedLock;
import org.redisson.api.RLock;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;
import org.springframework.context.annotation.Configuration;
@Configuration
public class RedissonRedLockConfig {
public RedissonRedLock initRedissonClient(String lockKey){
Config config1 = new Config();
config1.useSingleServer().setAddress("redis://ip:8000").setDatabase(0);
RedissonClient redissonClient1 = Redisson.create(config1);
Config config2 = new Config();
config2.useSingleServer().setAddress("redis://ip:8001").setDatabase(0);
RedissonClient redissonClient2 = Redisson.create(config2);
Config config3 = new Config();
config3.useSingleServer().setAddress("redis://ip:8002").setDatabase(0);
RedissonClient redissonClient3 = Redisson.create(config3);
Config config4 = new Config();
config4.useSingleServer().setAddress("redis://ip:8003").setDatabase(0);
RedissonClient redissonClient4 = Redisson.create(config4);
Config config5 = new Config();
config5.useSingleServer().setAddress("redis://ip:8004").setDatabase(0);
RedissonClient redissonClient5 = Redisson.create(config5);
RLock rLock1 = redissonClient1.getLock(lockKey);
RLock rLock2 = redissonClient2.getLock(lockKey);
RLock rLock3 = redissonClient3.getLock(lockKey);
RLock rLock4 = redissonClient4.getLock(lockKey);
RLock rLock5 = redissonClient5.getLock(lockKey);
RedissonRedLock redissonRedLock = new RedissonRedLock(rLock1,rLock2,rLock3,rLock4,rLock5);
return redissonRedLock;
}
}
Since multiple Redis servers are required, you can simulate them locally through Docker.
Create a new test class for testing:
package com.codingfire.zookeeper_lock;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.redisson.RedissonRedLock;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;
import java.io.IOException;
import java.util.concurrent.TimeUnit;
@SpringBootTest
@RunWith(SpringRunner.class)
public class RedLockTest {
@Autowired
private RedissonRedLockConfig redissonRedLockConfig;
@Test
public void testRedLock(){
//模拟多个5个客户端
for (int i = 0; i < 5; i++) {
Thread thread = new Thread(new RedLockTest.RedLockRunnable());
thread.start();
}
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}
}
private class RedLockRunnable implements Runnable {
@Override
public void run() {
RedissonRedLock redissonRedLock = redissonRedLockConfig.initRedissonClient("demo");
try {
boolean lockResult = redissonRedLock.tryLock(100, 10, TimeUnit.SECONDS);
if (lockResult){
System.out.println("获取锁成功");
TimeUnit.SECONDS.sleep(3);
}
} catch (InterruptedException e) {
e.printStackTrace();
}finally {
redissonRedLock.unlock();
System.out.println("释放锁");
}
}
}
}
Red Lock Principle
The red lock algorithm allows the number of failed nodes to be locked to be limited to (N-(N/2+1)). If five nodes are currently added, the number of failed nodes is allowed to be 2, which cannot exceed half, and all nodes are traversed when locking Execute lua script to lock to ensure atomicity. If the lock is successfully acquired, it will be added to the acquired lock set. If an exception is thrown, in order to prevent other nodes from successfully locking, all nodes need to be unlocked, and the lock fails. If the locking fails, there is no need to continue to apply for locks from the following nodes. The red lock algorithm requires at least N/2+1 nodes to be successfully locked before the final lock application is considered successful. At this point, the total time spent acquiring locks from each node will be calculated. If it is greater than or equal to the maximum waiting time, the lock application will fail. Otherwise, the lock application will succeed and the distributed lock will be successfully added.
epilogue
Look, distributed locks are actually not that difficult. If you can see this, then I believe you will gain something. Read it a few times, which is good for understanding distributed locks. After learning distributed locks, you can be happy again I went out and pretended to be 13, so happy! ! ! Learning is a step-by-step process. After you finish learning, it doesn’t mean you will. After a while, you will probably forget it. Just like a blogger, you will forget many things after a while after writing. It also requires constant review to deepen your impression. Learning is investing in yourself. Only when the diamond is hard can you take on the porcelain work. Come on, boys!