1. Distributed lock concept
In a multi-threaded environment, in order to ensure the thread safety of the data, the lock ensures that only one can access and update the shared data at the same time. In a stand-alone system, we can use synchronized locks and Lock locks to ensure thread safety.
The synchronized lock is a built-in lock provided by Java. It provides a locking mechanism between threads in a single JVM process and controls multi-thread concurrency. Only applicable to concurrency control in a stand-alone environment.
If you want to provide locking in multiple nodes, concurrently control shared resources in a distributed system, ensure that only one access can be called at the same time, avoid multiple callers competing for calls and data inconsistencies, and ensure data consistency, you need distributed Lock .
Distributed lock: A lock mechanism that controls access to shared resources by different processes in a distributed system. Calls between different processes need to maintain mutual exclusivity. At any time, only one client can hold the lock.
Shared resources include:
- database
- file hard drive
- Shared memory
Distributed lock features:
- Mutual exclusivity: the lock can only be deleted by the client holding it, not by other clients
- Lock timeout release: The lock held after timeout can be released to prevent unnecessary waste of resources and deadlock.
- Reentrancy: After a thread acquires a lock, it can request a lock again.
- High performance and high availability: Locking and unlocking need to be as low-cost as possible, while also ensuring high availability to avoid distributed lock failures.
- Security: The lock can only be removed by the client holding it, not by other clients.
Simulate order placement in concurrent environment
①: Add Redis dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
②: Add configuration:
spring:
redis:
host: localhost
port: 6379
password:
timeout: 2000s
# 配置文件中添加 lettuce.pool 相关配置，则会使用到lettuce连接池
lettuce:
pool:
max-active: 8 # 连接池最大连接数(使用负值表示没有限制) 默认为8
max-wait: -1ms # 接池最大阻塞等待时间(使用负值表示没有限制) 默认为-1ms
max-idle: 8 # 连接池中的最大空闲连接 默认为8
min-idle: 0 # 连接池中的最小空闲连接 默认为 0
③: Redis configuration class:
@Configuration
public class RedisConfig {
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(redisConnectionFactory);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
// json 序列化配置
Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class);
jackson2JsonRedisSerializer.setObjectMapper(om);
// String 序列化
StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
// 所有的 key 采用 string 的序列化
template.setKeySerializer(stringRedisSerializer);
// 所有的 value 采用 jackson 的序列化
template.setValueSerializer(jackson2JsonRedisSerializer);
// hash 的 key 采用 string 的序列化
template.setHashKeySerializer(stringRedisSerializer);
// hash 的 value 采用 jackson 的序列化
template.setHashValueSerializer(jackson2JsonRedisSerializer);
template.afterPropertiesSet();
return template;
}
}
Redis tool class:
@Component
public class RedisUtil {
@Autowired
private RedisTemplate<String, Object> redisTemplate;
// 普通缓存获取
public Object get(String key) {
return key == null ? null : redisTemplate.opsForValue().get(key);
}
// 普通缓存放入
public boolean set(String key, Object value) {
try {
redisTemplate.opsForValue().set(key, value);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
public boolean setIfAbsent(String key, Object value, long timeout, TimeUnit unit) {
return Boolean.TRUE.equals(redisTemplate.opsForValue().setIfAbsent(key, value, timeout, unit));
}
public void del(String... key) {
if (key != null && key.length > 0) {
if (key.length == 1) {
redisTemplate.delete(key[0]);
} else {
//springboot2.4后用法
redisTemplate.delete(Arrays.asList(key));
}
}
}
}
④: Add order interface
@Slf4j
@RestController
@RequestMapping("/order")
public class OrderController {
@Autowired
private RedisUtil redisUtil;
@GetMapping("/initProductStock")
public void initProductStock() {
redisUtil.set("stock", 100);
}
@GetMapping("/create_order")
public void createOrder() {
// 获取当前库存
int stock = (Integer) redisUtil.get("stock");
if (stock > 0) {
// 减库存
int realStock = stock - 1;
redisUtil.set("stock", realStock);
// TODO 添加订单记录
log.info("扣减成功，剩余库存：" + realStock);
return;
}
log.error("扣减失败，库存不足");
}
}
Interface Description:
/order/initProductStock: First initialize an inventory in Redis/order/create_order: Order interface: Get the inventory from the cache first. If the inventory is greater than 0, then the inventory is reduced by 1.
⑤: Concurrency testing
Use JMeter to conduct concurrent environment testing, 10 threads, and loop 5 times.
⑥: Test results, print log as follows:
After calling the interface 50 times using JMeter, under normal circumstances, the inventory should be: 50 = 100 - 50.
But the log shows that the final inventory is: 95.
This is because in a concurrent environment, when multiple threads place orders, the previous thread has not updated the inventory, and the subsequent thread has requested in and obtained the unupdated inventory. Subsequent deductions from the inventory are not deductions from the most recent inventory. The more threads, the less inventory is deducted . This is the oversold problem that occurs in high concurrency scenarios.
Obviously, the above problem is a thread safety issue. The first thing we can think of is to add a synchronized lock to it.
Yes, no problem, but we know that synchronized locks belong to the JVM level, which is what we call "single-machine locks". If it is deployed in a multi-machine environment, can data consistency be guaranteed?
The answer is definitely no. At this time, we need to use our Redis distributed lock
Several solutions for implementing distributed locks using Redis all use the SETNX command (setting the key equal to a certain value). Only the number of parameters passed in the high-level scheme is different, and abnormal situations are taken into account.
SETNX is the abbreviation of SET IF NOT EXISTS. Command format: SETNX key value. If the key does not exist, SETNX returns 1 successfully. If the key already exists, it returns 0.
setIfAbsent()It is the merge command of setnx + expire
2. Redis distributed lock solution one: SETNX + EXPIRE
Question: Why add expiration time?
If Redis crashes before releasing the lock, a deadlock will occur.
setnxCommands and expirecommands must be atomic operations.
The pseudocode is as follows:
if（jedis.setnx(key_resource_id,lock_value) == 1）{
//加锁
expire（key_resource_id，100）; //设置过期时间
try {
do something //业务请求
}catch(){
}
finally {
jedis.del(key_resource_id); //释放锁
}
}
setnxand expiretwo commands are separated, "not an atomic operation". If after executing setnxthe lock and expirewhen the expiration time is set, the process crashes or needs to be restarted for maintenance, then the lock will be "immortal" and "other threads will never be able to obtain the lock."
@GetMapping("/create_order")
public void createOrder() {
String key = "lock_key";
// 1.加锁
boolean lock = tryLock(key, 1, 60L, TimeUnit.SECONDS);
if (lock) {
try {
// 获取当前库存
int stock = (Integer) redisUtil.get("stock");
if (stock > 0) {
// 减库存
int realStock = stock - 1;
redisUtil.set("stock", realStock);
// TODO 添加订单记录
log.info("扣减成功，剩余库存：" + realStock);
return;
}
log.error("扣减失败，库存不足");
} catch (Exception e) {
log.error("扣减库存失败");
} finally {
// 3.解锁
unlock(key);
}
} else {
log.info("未获取到锁...");
}
}
public boolean tryLock(String key, Object value, long timeout, TimeUnit unit) {
return redisUtil.setIfAbsent(key, value, timeout, unit);
}
public void unlock(String key) {
redisUtil.del(key);
}
After running with JMeter, the results are as follows:
The thread that acquired the lock has successfully deducted the inventory, and the thread that did not acquire the lock only prints the log.
3. Redis distributed lock solution 2: SETNX + EXPIRE + verify unique random value
Option 1 still has certain flaws:Assume that thread A successfully acquires the lock and has been executing business logic, but 60s have passed and the execution has not been completed. However, at this time, the lock has expired. Thread B requested again. Obviously, thread B can also acquire the lock successfully and start executing the business logic code. Then the problem comes: during the execution of thread B, thread A has finished executing, and the lock of thread B will be released!
Since the lock may be accidentally deleted by other threads, we set a random number for the value that is unique to the current thread. When deleting, verify it and it will be OK.
@GetMapping("/create_order")
public void createOrder() {
String key = "lock_key";
String value = "ID_PREFIX" + Thread.currentThread().getId();
// 1.加锁
boolean lock = tryLock(key, value, 60L, TimeUnit.SECONDS);
if (lock) {
// ...
}
public void unlock(String key, String value) {
String currentValue = (String)redisUtil.get(key);
if (StringUtils.hasText(currentValue) && currentValue.equals(value)) {
redisUtil.del(key);
}
}
What needs to be noted here is: when releasing the lock, get first and then delete. This is not an atomic operation and cannot guarantee process safety. In order to be more precise, lua script is used instead.
lua script
Lua script is a lightweight and compact language that has been built into redis. Its execution is run through the eval/evalsha command of redis. The operation is encapsulated into a Lua script. In any case, it is an atomic operation executed at once.
lockDel.lua is as follows:resources/lua/lockDel.lua
if redis.call('get', KEYS[1]) == ARGV[1]
then
-- 执行删除操作
return redis.call('del', KEYS[1])
else
-- 不成功，返回0
return 0
end
public void unlock(String key, String value) {
// 解锁脚本
DefaultRedisScript<Long> unlockScript = new DefaultRedisScript<>();
unlockScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("lua/lockDel.lua")));
unlockScript.setResultType(Long.class);
// 执行lua脚本解锁
Long execute = redisTemplate.execute(unlockScript, Collections.singletonList(key), value);
}
or:
public void unlock(String key, String value) {
// 解锁脚本
String script = "if redis.call('get',KEYS[1]) == ARGV[1] then return redis.call('del',KEYS[1]) else return 0 end";
redisTemplate.execute(new DefaultRedisScript<Long>(script, Long.class), Collections.singletonList(key), value);
}
4. Redis distributed lock solution three: Redisson
There are still problems in the second option: "The lock is expired and released, and the business is not completed." If the set timeout is relatively short and the business execution time is relatively long. For example, the timeout is set to 5s, and the business execution takes 10s. At this time, the business has not been completed, and other requests will acquire the lock. Two requests request business data at the same time, which does not satisfy the mutual exclusivity of the distributed lock and cannot guarantee the thread's security. Safety
4.1 Redisson concept
In fact, let's imagine whether we can open a scheduled daemon thread for the thread that obtained the lock, and check whether the lock still exists every once in a while. If it exists, the expiration time of the lock will be extended to prevent the lock from being released early after expiration. The current open source framework Redisson solves this problem. The underlying schematic diagram of Redisson is as follows:
As soon as the thread is successfully locked, a watch dogwatchdog will be started. It is a background thread and will check every 10 seconds. If thread 1 still holds the lock, the life time of the lock key will be continuously extended. Therefore, Redisson uses watch dog to solve the problem of "lock expired release, business not completed"
Although Redis has the best performance as a distributed lock. But it is also the most complex. The above summary of Redis mainly has the following problems:
- If the expiration time is not set, there will be a deadlock.
- Set expiration time
- Lock accidentally deleted
- The business continues to execute, causing multiple threads to execute concurrently.
RedissionDistributed locks are implemented online . Redisson is a Java in-memory data grid (In-Memory Data Grid) implemented on the basis of Redis. It not only provides a series of distributed common Java objects, but also provides many distributed services. Redisson is implemented based on the netty communication framework, so it supports non-blocking communication and has better performance than Jedis.
Redisson distributed lock four-layer protection:
- Anti-deadlock
- Prevent accidental deletion
- Reentrant (a thread can acquire the same lock again after acquiring the lock without waiting for the lock to be released)
- Automatic renewal
Redisson implements Redis distributed locks and supports stand-alone and cluster modes
4.2 Redisson implementation
Redisson documentation directory: Redisson documentation directory
Using Redission distributed lock is divided into three steps:
- Get lock
redissonClient.getLock("lock") - Lock
rLock.lock() - Unlock
rLock.unlock()
Introduce dependencies:
<dependency>
<groupId>org.redisson</groupId>
<artifactId>redisson</artifactId>
<version>3.20.0</version>
</dependency>
Redisson configuration class:
@Configuration
public class RedissionConfig {
@Value("${spring.redis.host}")
private String redisHost;
@Value("${spring.redis.password}")
private String password;
private int port = 6379;
@Bean
public RedissonClient getRedisson() {
Config config = new Config();
// 单机版
config.useSingleServer()
.setAddress("redis://" + redisHost + ":" + port);
//.setPassword(password);
config.setCodec(new JsonJacksonCodec());
return Redisson.create(config);
}
}
Cluster version:
@Bean
public RedissonClient getRedisson() {
Config config = new Config();
config.useClusterServers()
.setScanInterval(2000) // 集群状态扫描间隔时间，单位是毫秒
//可以用"rediss://"来启用SSL连接
.addNodeAddress("redis://127.0.0.1:7000", "redis://127.0.0.1:7001")
.addNodeAddress("redis://127.0.0.1:7002");
return Redisson.create(config);
}
Order interface:
@Slf4j
@RestController
@RequestMapping("/order")
public class OrderController {
@Autowired
private RedissonClient redissonClient;
@GetMapping("/create_order")
public void createOrder() {
String key = "lock_key";
RLock rLock = redissonClient.getLock(key);
// 1.加锁
rLock.lock();
try {
// 获取当前库存
int stock = (Integer) redisUtil.get("stock");
if (stock > 0) {
// 减库存
int realStock = stock - 1;
redisUtil.set("stock", realStock);
// TODO 添加订单记录
log.info("扣减成功，剩余库存：" + realStock);
return;
}
log.error("扣减失败，库存不足");
} catch (Exception e) {
log.error("扣减库存失败");
} finally {
// 3.解锁
rLock.unlock();
}
}
}
After running concurrently using JMeter:
The distributed lock implemented by Redission can be called directly without the need for lock exceptions, timeout concurrency, lock deletion and other issues. It has encapsulated the code to handle the above problems and can be called directly.