Concurrency issues in a cluster environment
Redis implements distributed locks
More extreme cases of accidental deletion
Lua script solves atomicity problem
foreword
We have implemented one person, one order in the stand-alone mode before, but if the cluster mode is turned on, the stand-alone mode solution is obviously not applicable. The first is the lock solution. In the cluster mode, if synchronized is used as the lock, each stand-alone If you have your own lock, this will cause the lock to fail. At this time, you must use distributed locks. This article uses the setnx operation of redis to manually implement distributed locks, and finally uses redisson to do distributed locks. gave a solution
Concurrency issues in a cluster environment
In the previous article, the code for implementing one person and one order is to add synchronized to ensure thread safety, but it will not work in cluster mode. The syn lock will fail, and the analysis of the cause of the lock failure
Since we have deployed multiple tomcats now, and each tomcat has its own jvm , then suppose there are two threads inside the tomcat of server A. Since these two threads use the same code, their locks The objects are the same, and mutual exclusion can be achieved, but if there are two threads inside the tomcat of server B, their lock objects are written the same as those of server A, but the lock objects are not the same , so Thread 3 and thread 4 can achieve mutual exclusion, but they cannot achieve mutual exclusion with thread 1 and thread 2. This is why the syn lock fails in a cluster environment. In this case, we need to use distributed locks to solve this problem.
distributed lock
definition
Distributed lock: A lock that is visible and mutually exclusive to multiple processes in a distributed system or cluster mode.
The core idea of distributed lock is to let everyone use the same lock . As long as everyone uses the same lock, then we can lock the thread, prevent the thread from proceeding, and let the program execute serially. This is the core of distributed lock train of thought
Conditions to be met
Visibility : Multiple threads can see the same result, that is, multiple threads must get the same information . Note: The visibility mentioned here is not the memory visibility referred to in concurrent programming, but only between multiple processes sense the meaning of change
Mutual exclusion : Mutual exclusion is the most basic condition of distributed locks, making programs execute serially
High availability : the program is not easy to crash, and high availability is guaranteed at all times
High performance : Since locking itself reduces performance, all distributed locks require higher locking performance and lock release performance
Security : Security is also an integral part of the program
Common Distributed Locks
Mysql: mysql itself has a lock mechanism, but because of the general performance of mysql , it is rare to use mysql as a distributed lock when using distributed locks
Redis: Redis as a distributed lock is a very common way to use it. Now enterprise-level development basically uses redis or zookeeper as a distributed lock. Using the setnx method, if the key is inserted successfully, it means that the lock has been obtained . If someone If the insertion is successful, if other people fail to insert, it means that the lock cannot be obtained. Use this logic to implement distributed locks
Zookeeper: Zookeeper is also a better solution for implementing distributed locks in enterprise-level development.
Redis implements distributed locks
There are two basic methods that need to be implemented when implementing distributed locks:
-
Acquire the lock:
-
Mutual exclusion: ensures that only one thread can acquire the lock
-
Non-blocking: try once, return true on success, false on failure
-
-
Release the lock:
-
manual release
-
Timeout release: add a timeout period when acquiring a lock to prevent deadlock
-
core idea
We use the setNx method of redis. When multiple threads enter, we use this method. When the first thread enters, there is this key in redis, and it returns 1. If the result is 1, it means that he has grabbed the key. lock, then he goes to execute the business, then deletes the lock, exits the lock logic, and waits for a certain period of time to try again after the thread that has not grabbed the lock
Code
ILock interface, formulate the specification of lock operation
/**
* @author
* @version 1.0
* @description: 锁的基本接口
* @date 2023/9/4 9:16
*/
public interface ILock {
//尝试获取锁
boolean tryLock(Long timeoutSec);
//释放锁
void unlock();
}
Primary version of redis distributed lock
/**
* @author
* @version 1.0
* @description: redis分布式锁初级版本
* @date 2023/9/4 9:20
*/
public class SimpleRedisLock implements ILock {
private String name;
private StringRedisTemplate stringRedisTemplate;
private static final String KEY_PREFIX = "lock:";
public SimpleRedisLock(String name, StringRedisTemplate stringRedisTemplate) {
this.name = name;
this.stringRedisTemplate = stringRedisTemplate;
}
@Override
public boolean tryLock(Long timeoutSec) {
//获取当前线程id
long id = Thread.currentThread().getId();
Boolean success = stringRedisTemplate.opsForValue().setIfAbsent(KEY_PREFIX + name, id + "", timeoutSec, TimeUnit.SECONDS);
//防止自动拆箱出现空指针
return Boolean.TRUE.equals(success);
}
@Override
public void unlock() {
//释放锁
stringRedisTemplate.delete(KEY_PREFIX + name);
}
}When executing the setnx operation, the order is spliced with the user id to realize one order per person.
@Override
public Result seckillVoucher(Long voucherId) {
SeckillVoucher seckillVoucher = seckillVoucherService.getById(voucherId);
//判断是否开始，开始时间如果在当前时间之后就是尚未开始
if (seckillVoucher.getBeginTime().isAfter(LocalDateTime.now())) {
return Result.fail("秒杀尚未开始");
}
//判断是否结束，结束时间如果在当前时间之前就是已经结束
if (seckillVoucher.getEndTime().isBefore(LocalDateTime.now())) {
return Result.fail("秒杀已经结束");
}
//判断库存是否充足
if (seckillVoucher.getStock() < 1) {
return Result.fail("库存不足");
}
Long userId = UserHolder.getUser().getId();
//创建锁对象
SimpleRedisLock lock = new SimpleRedisLock("order:" + userId, stringRedisTemplate);
//尝试获取锁
boolean isLock = lock.tryLock(1200L);
//获取锁失败
if(!isLock){
return Result.fail("不允许重复下单");
}
try {
return voucherOrderService.createVoucherOrder(voucherId);
}finally {
//释放锁
lock.unlock();
}
}Accidental deletion
logical explanation
The thread holding the lock is blocked inside the lock, causing its lock to be released automatically when it expires . At this time, other threads, thread 2, try to acquire the lock, and then get the lock , and then thread 2 executes while holding the lock. During the process, thread 1 reacts and continues to execute, but during the execution of thread 1, it reaches the logic of deleting the lock . At this time, the lock that should belong to thread 2 will be deleted . This is the description of the situation where someone else’s lock is accidentally deleted.
solution
The solution is to judge whether the current lock belongs to itself when each thread releases the lock . If it does not belong to itself, the lock will not be deleted. Assuming the above situation is still the case, thread 1 is stuck and the lock is automatically released. Thread 2 enters the internal execution logic of the lock. At this time, thread 1 reacts and deletes the lock. However, thread 1 sees that the current lock does not belong to itself, so it does not perform the logic of deleting the lock. When thread 2 reaches the logic of deleting the lock , if the time point of automatic release of the lock has not been passed, it is judged that the current lock belongs to itself, so the lock is deleted. That is, to judge whether the lock is your own before releasing the lock, you need to add a unique thread ID when acquiring the lock. This project uses uuid+thread id, because in cluster mode, thread IDs may conflict, and you need to put together uuid
Code
core idea
Modify the previous distributed lock implementation to meet the requirements: store the thread ID (can be represented by UUID) when acquiring the lock, and first obtain the thread ID in the lock when releasing the lock, and judge whether it is consistent with the current thread ID
-
Release lock if consistent
-
If inconsistent, the lock is not released
Core logic: When storing the lock, put the ID of your own thread. When deleting the lock, judge whether the current lock ID is stored by yourself. If it is, delete it. If not, don’t delete it.
@Override
public boolean tryLock(Long timeoutSec) {
//获取当前线程id
String id = ID_PREFIX+Thread.currentThread().getId();
Boolean success = stringRedisTemplate.opsForValue().setIfAbsent(KEY_PREFIX + name, id, timeoutSec, TimeUnit.SECONDS);
//防止自动拆箱出现空指针
return Boolean.TRUE.equals(success);
}
@Override
public void unlock() {
//从redis中获取锁信息
String id = stringRedisTemplate.opsForValue().get(KEY_PREFIX + name);
//获取当前线程标识
String ThreadId = ID_PREFIX+Thread.currentThread().getId();
//自己的锁才释放
if(id.equals(ThreadId)) {
//释放锁
stringRedisTemplate.delete(KEY_PREFIX + name);
}
}More extreme cases of accidental deletion
After thread 1 now holds the lock, in the process of executing business logic, he is preparing to delete the lock, and has reached the process of conditional judgment. For example, he has obtained the current lock that really belongs to him and is preparing to delete it Lock, but at this time his lock expires, then thread 2 comes in at this time, but thread 1 will continue to execute later, when he is stuck, he will directly execute the line of code that deletes the lock, which is equivalent to the condition The judgment did not work. This is the atomicity problem when deleting locks. The reason for this problem is that thread 1's lock acquisition, lock comparison, and lock deletion are actually not atomic. We need to prevent the When the situation occurs, it is necessary to ensure that the two operations of verifying the lock and deleting the lock are atomic operations that cannot be interrupted
Lua script solves atomicity problem
Redis provides the Lua scripting function to write multiple Redis commands in one script to ensure the atomicity of multiple commands execution. Lua is a programming language. For its basic grammar, you can refer to the website: Lua Tutorial | Novice Tutorial . Here we will focus on the calling functions provided by Redis. We can use lua to operate redis and ensure its atomicity, so that we can It is an atomic action to achieve lock-taking than lock-deleting . As a Java programmer, there is no simple requirement. You don’t need to be too proficient, you just need to know what it does.
The business process for releasing the lock is as follows
1. Obtain the thread mark in the lock
2. Determine whether it is consistent with the specified mark (current thread mark)
3. If consistent, release the lock (delete)
4. Do nothing if inconsistent
In the end, the lua script that we operate redis to lock and delete locks will become like this
-- 这里的 KEYS[1] 就是锁的key，这里的ARGV[1] 就是当前线程标示
-- 获取锁中的标示，判断是否与当前线程标示一致
if (redis.call('GET', KEYS[1]) == ARGV[1]) then
-- 一致，则删除锁
return redis.call('DEL', KEYS[1])
end
-- 不一致，则直接返回
return 0The code changes are as follows
private static final DefaultRedisScript<Long> UNLOCK_SCRIPT;
static {
UNLOCK_SCRIPT = new DefaultRedisScript<>();
UNLOCK_SCRIPT.setLocation(new ClassPathResource("unlock.lua"));
UNLOCK_SCRIPT.setResultType(Long.class);
}
public void unlock() {
// 调用lua脚本
stringRedisTemplate.execute(
UNLOCK_SCRIPT,
Collections.singletonList(KEY_PREFIX + name),
ID_PREFIX + Thread.currentThread().getId());
}Distributed lock-redission
The distributed lock based on setnx has the following problems:
Reentrancy problem : The reentrancy problem means that the thread that acquires the lock can enter the code block of the same lock again. The significance of reentrant lock is to prevent deadlock. For example, in code such as HashTable, his method is to use synchronized Modified, if he calls another method in one method, then if it is not reentrant at this time, wouldn't it be a deadlock? So the main significance of reentrant locks is to prevent deadlocks. Both our synchronized and Lock locks are reentrant.
Non-retryable : It means that the current distribution can only try once. We think that the reasonable situation is: when the thread fails to acquire the lock, he should be able to try to acquire the lock again.
Timeout release: We increase the expiration time when adding locks, so that we can prevent deadlocks, but if the freeze time is too long, although we use lua expressions to prevent deletion of other people's locks by mistake, but After all, it is not locked, there is a security risk
Master-slave consistency: If Redis provides a master-slave cluster, when we write data to the cluster, the host needs to asynchronously synchronize the data to the slave, and if the host crashes before the synchronization passes, a deadlock will occur question
The concept of redisson
Redisson is a Java in-memory data grid (In-Memory Data Grid) implemented on the basis of Redis. It not only provides a series of distributed common Java objects, but also provides many distributed services, including the implementation of various distributed locks.
Redission provides a variety of functions of distributed locks, and does not require us to manually implement locks. At the same time, redisson's implementation of distributed locks can also perfectly solve the above-mentioned setnx problems
quick start
Import dependencies, it is not recommended to directly import start, which will overwrite the original configuration of springboot for redisson
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson</artifactId>
<version>3.13.6</version>
</dependency>Configure the Redisson client:
@Configuration
public class RedissonConfig {
@Bean
public RedissonClient redissonClient(){
// 配置
Config config = new Config();
config.useSingleServer().setAddress("redis://localhost:6379")
.
// 创建RedissonClient对象
return Redisson.create(config);
}
}Just need to replace the lock we wrote by ourselves with the lock of redisson
@Resource
private RedissonClient redissonClient;
@Override
public Result seckillVoucher(Long voucherId) {
// 1.查询优惠券
SeckillVoucher voucher = seckillVoucherService.getById(voucherId);
// 2.判断秒杀是否开始
if (voucher.getBeginTime().isAfter(LocalDateTime.now())) {
// 尚未开始
return Result.fail("秒杀尚未开始！");
}
// 3.判断秒杀是否已经结束
if (voucher.getEndTime().isBefore(LocalDateTime.now())) {
// 尚未开始
return Result.fail("秒杀已经结束！");
}
// 4.判断库存是否充足
if (voucher.getStock() < 1) {
// 库存不足
return Result.fail("库存不足！");
}
Long userId = UserHolder.getUser().getId();
//创建锁对象 这个代码不用了，因为我们现在要使用分布式锁
//SimpleRedisLock lock = new SimpleRedisLock("order:" + userId, stringRedisTemplate);
RLock lock = redissonClient.getLock("lock:order:" + userId);
//获取锁对象
boolean isLock = lock.tryLock();
//加锁失败
if (!isLock) {
return Result.fail("不允许重复下单");
}
try {
//获取代理对象(事务)
IVoucherOrderService proxy = (IVoucherOrderService) AopContext.currentProxy();
return proxy.createVoucherOrder(voucherId);
} finally {
//释放锁
lock.unlock();
}
}Summarize
We have been going all the way, using the addition of expiration time to prevent the occurrence of deadlock problems, but after the expiration time, there may be the problem of accidentally deleting other people's locks. This problem we started by using locks before deletion, comparing locks, and deleting locks This logic is used to solve it, that is, to judge whether the current lock belongs to you before deleting, but there is still an atomicity problem, that is, we cannot guarantee that taking a lock is an atomic action compared to locking and deleting a lock. Finally Solve this problem through lua expressions, but there are still many problems in using setnx to implement distributed locks, such as non-reentrant, non-retryable, etc., and it is commonly used in enterprises to use redisson as distributed locks, without repeated creation Wheels, but we implement a distributed lock by ourselves through setnx, which will help us understand the principle of distributed locks more deeply.