Practical Chapter-25.Redis Message Queue-Understanding Message Queue_bilibili_bilibili
1. Message queue and blocking queue are different
1) The message queue is not in the jvm, so the memory is not limited by the jvm, avoiding the risk of memory overflow.
2) The message queue will persist itself to ensure data security. Avoid orders in the blocking queue from disappearing due to service downtime or restart.
3) The message queue will require confirmation (signature) when sending messages to consumers. Otherwise, it will be considered not received and will continue to be sent next time to ensure that each message is consumed at least once.
2List structure
Redis's list is a doubly linked list.
BLPUSH and BRPOP will automatically block when accessing
3.PubSub
A message can be obtained by a single consumer or multiple consumers.
4.Stream type
4.1 Single consumption model
4.1.1Write
4.1.2 Reading
Stream will not delete the message after reading it, so you and others can read it again.
4.2 Consumer group model
5. Use Stream’s message queue
It turns out that a Lua script is used to determine the qualifications, and then messages are sent to the blocking queue in Java.
Now because stream is used as the message queue, it is also in redis, so you can directly send messages to the stream in the lua script. This simplifies the function of java and reduces the interaction with redis.
Since you want to send to the message queue, you need to know the queue number of the message queue, and you need to pass one more parameter.
---资格判断，库存判断，以及对消息队列Stream发消息等一系列redis操作
--1.参数列表
--1.1优惠券id
local voucherId = ARGV[1]
--1.2用户id
local userId = Argv[2]
--1.3订单id
local orderId = ARGV[3]
--2.数据key
--2.1库存key
local stockKey = "seckill:stock" .. voucherId
--2.2订单key
local orderKey = "seckill:order" .. voucherId
--3.脚本业务
--3.1判断库存是否充足 get stock
if (tonumber(redis.call('get', stockKey)) <= 0) then
--库存不足返回1
return 1
end
--3.2判断用户是否下单
if (redis.call('sismember', orderKey, userId) == 1) then
--存在，返回2
return 2
end
--3.3扣库存incrby stockKey -1
redis.call('incrby', stockKey, -1)
--3.4下单，保存用户信息 sadd orderKey userId
redis.call('sadd', orderKey, userId)
--3.5 发送消息到redis stream队列，xadd stream.orders * k1 v1 k2 v2......
redis.call('xadd', 'stream.orders', '*', 'userId', userId, 'voucherId', voucherId, 'id', orderId)
return 0
package com.hmdp.service.impl;
import cn.hutool.core.bean.BeanUtil;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import com.hmdp.dto.Result;
import com.hmdp.entity.VoucherOrder;
import com.hmdp.mapper.VoucherOrderMapper;
import com.hmdp.service.ISeckillVoucherService;
import com.hmdp.service.IVoucherOrderService;
import com.hmdp.utils.RedisIdWorker;
import com.hmdp.utils.UserHolder;
import lombok.extern.slf4j.Slf4j;
import org.redisson.api.RLock;
import org.redisson.api.RedissonClient;
import org.springframework.aop.framework.AopContext;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.redis.connection.stream.*;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import javax.annotation.PostConstruct;
import javax.annotation.Resource;
import java.time.Duration;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
/**
* <p>
* 服务实现类
* </p>
*/
/*
* 主要是
* 读取的lua脚本
* 取消息队列的线程
*
*/
@Slf4j
@Service
public class VoucherOrderServiceImpl extends ServiceImpl<VoucherOrderMapper, VoucherOrder> implements IVoucherOrderService {
@Resource
private ISeckillVoucherService seckillVoucherService;
//制造订单号
@Resource
private RedisIdWorker redisIdWorker;
@Resource
private StringRedisTemplate stringRedisTemplate;
//分布式锁
@Resource
private RedissonClient redissonClient;
//线程池
private static final ExecutorService SECKILL_ORDER_EXECUTOR = Executors.newSingleThreadExecutor();
private IVoucherOrderService proxy;
private static final DefaultRedisScript<Long> SECKILL_SCRIPT;
//lua脚本资源
static {
SECKILL_SCRIPT = new DefaultRedisScript<>();
SECKILL_SCRIPT.setLocation(new ClassPathResource("seckill.lua"));
SECKILL_SCRIPT.setResultType(Long.class);
}
//刚开始类加载就调用
@PostConstruct
private void init() {
SECKILL_ORDER_EXECUTOR.submit(new VoucherOrderHandler());
}
//线程功能（去消息队列取消息下订单）
private class VoucherOrderHandler implements Runnable {
//消息队列的名字“orders”
String queueName = "stream.orders";
//取消息下订单
@Override
public void run() {
while (true) {
try {
//1.获取redis消息队列中的订单信息 XREADGROUP group g1 c1 count 1 block 2000 stream.orders >
List<MapRecord<String, Object, Object>> list = stringRedisTemplate.opsForStream().read(
Consumer.from("g1", "c1"),
StreamReadOptions.empty().count(1).block(Duration.ofSeconds(2)),
StreamOffset.create(queueName, ReadOffset.lastConsumed())
);
//2.判断是否获取成功
if (list == null || list.isEmpty()) {
//获取失败，再来一次
continue;
}
//3.解析消息的订单信息
MapRecord<String, Object, Object> record = list.get(0);
Map<Object, Object> values = record.getValue();
VoucherOrder voucherOrder = BeanUtil.fillBeanWithMap(values, new VoucherOrder(), true);
//3.创建订单存到数据库
handleVoucherOrder(voucherOrder);
//4.ACK确认 SACK stream.orders g1 id
stringRedisTemplate.opsForStream().acknowledge(queueName, "g1", record.getId());
} catch (Exception e) {
log.error("处理订单异常");
handlePendingList();
}
}
}
//出现异常处理pendinglist
private void handlePendingList() {
while (true) {
try {
//1.获取pendingList中的订单信息 XREADGROUP group g1 c1 count 1 stream.orders 0
List<MapRecord<String, Object, Object>> list = stringRedisTemplate.opsForStream().read(
Consumer.from("g1", "c1"),
StreamReadOptions.empty().count(1),
StreamOffset.create(queueName, ReadOffset.from("0"))
);
//2.判断是否获取成功
if (list == null || list.isEmpty()) {
//获取失败，pendList没有信息结束循环
break;
}
//3.解析消息的订单信息
MapRecord<String, Object, Object> record = list.get(0);
Map<Object, Object> values = record.getValue();
VoucherOrder voucherOrder = BeanUtil.fillBeanWithMap(values, new VoucherOrder(), true);
//3.创建订单存到数据库
handleVoucherOrder(voucherOrder);
//4.ACK确认 SACK stream.orders g1 id
stringRedisTemplate.opsForStream().acknowledge(queueName, "g1", record.getId());
} catch (Exception e) {
log.error("处理pendList异常");
try {
Thread.sleep(20);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
}
}
/*
*
private BlockingQueue<VoucherOrder> orderTasks = new ArrayBlockingQueue<>(10224 * 1024);
private class VoucherOrderHandler implements Runnable {
@Override
public void run() {
while (true) {
try {
//1.获取队列中的信息
VoucherOrder voucherOrder = orderTasks.take();
//2.创建订单存到数据库
handleVoucherOrder(voucherOrder);
} catch (InterruptedException e) {
log.error("处理订单异常");
}
}
}
}
*/
//加Redisson分布式锁
//加锁成功调用 下面的创建订单函数
private void handleVoucherOrder(VoucherOrder voucherOrder) {
//1.1获取用户id
Long userId = voucherOrder.getUserId();
//自己定义的SimpleRedisLock锁类
//SimpleRedisLock lock = new SimpleRedisLock("order:" + userId, stringRedisTemplate);
//使用的redisson获取的锁类
//1.2创建锁对象
RLock lock = redissonClient.getLock("lock:order:" + userId);
//1.3获取锁
boolean isLock = lock.tryLock();
//1.4判断获取锁是否成功
if (!isLock) {
//获取锁失败，返回错误
log.error("不允许重复下单");
return;
}
try {
proxy.createVoucherOrder(voucherOrder);
} finally {
//3.释放锁
lock.unlock();
}
}
/*
* 除了下面seckillVoucher其他都是线程里的操作
* 以下是redis判断的操作seckillVoucher
* */
//调用lua脚本判断资格，发送消息
@Override
public Result seckillVoucher(Long voucherId) {
//1.1获取用户id
Long userId = UserHolder.getUser().getId();
//获取订单id
long orderId = redisIdWorker.nextId("order");
//1.2执行lua脚本
Long result = stringRedisTemplate.execute(
SECKILL_SCRIPT,
Collections.emptyList(),
voucherId.toString(), userId.toString(), String.valueOf(orderId)
);
//2.判断执行结果是否为0
int r = result.intValue();
if (r != 0) {
//2.1不为0，没有购买资格
return Result.fail(r == 1 ? "库存不足" : "不能重复下单");
}
//4.2获取代理对象（确保事务Transactional生效，保证createVoucherOrder 提交完事务之后再释放锁）
proxy = (IVoucherOrderService) AopContext.currentProxy();
//5返回订单id
return Result.ok(orderId);
}
/*
*
@Override
public Result seckillVoucher(Long voucherId) {
//1.1获取用户id
Long userId = UserHolder.getUser().getId();
//1.2执行lua脚本
Long result = stringRedisTemplate.execute(
SECKILL_SCRIPT,
Collections.emptyList(),
voucherId.toString(), userId.toString()
);
//2.判断执行结果是否为0
int r = result.intValue();
if (r != 0) {
//2.1不为0，没有购买资格
return Result.fail(r == 1 ? "库存不足" : "不能重复下单");
}
//3--4为0，有购买资格，把下单信息保存到阻塞队列中
//3.创建订单
VoucherOrder voucherOrder = new VoucherOrder();
//3.1订单id
long orderId = redisIdWorker.nextId("order");
voucherOrder.setId(orderId);
//3.2保存用户id
voucherOrder.setUserId(userId);
//3.3优惠券id
voucherOrder.setVoucherId(voucherId);
//4.保存订单信息
//4.1放入阻塞队列
orderTasks.add(voucherOrder);
//4.2获取代理对象（确保事务Transactional生效，保证createVoucherOrder 提交完事务之后再释放锁）
proxy = (IVoucherOrderService) AopContext.currentProxy();
//5返回订单id
return Result.ok(orderId);
}
*/
/*
*
@Override
public Result seckillVoucher(Long voucherId) {
//1.查询优惠卷
SeckillVoucher voucher = seckillVoucherService.getById(voucherId);
//2.判断秒杀是否开始
if (voucher.getBeginTime().isAfter(LocalDateTime.now())) {
//尚未开始
return Result.fail("秒杀尚未开始");
}
//3.判断秒杀是否已经结束
if (voucher.getEndTime().isBefore(LocalDateTime.now())) {
//已经结束
return Result.fail("秒杀已经结束");
}
//4.判断库存是否充足
if (voucher.getStock() < 1) {
return Result.fail("库存不足");
}
//5.查询用户id
Long userId = UserHolder.getUser().getId();
//6.对相同用户id的操作加锁，防止并行执行，一人多单
//6.1创建锁对象
//自己定义的SimpleRedisLock锁类
//SimpleRedisLock lock = new SimpleRedisLock("order:" + userId, stringRedisTemplate);
//使用的redisson获取的锁类
RLock lock = redissonClient.getLock("lock:order:" + userId);
//6.2获取锁
boolean isLock = lock.tryLock();
//6.3判断获取锁是否成功
if (!isLock) {
//获取锁失败，返回错误
return Result.fail("一个用户只能下一单");
}
try {
//7获取代理对象（确保事务Transactional生效，保证createVoucherOrder 提交完事务之后再释放锁）
IVoucherOrderService proxy = (IVoucherOrderService) AopContext.currentProxy();
return proxy.createVoucherOrder(voucherId);
} finally {
//8.释放锁
lock.unlock();
}
}
*/
//创建mysql订单方法，被加Redisson锁方法调用
@Transactional
public void createVoucherOrder(VoucherOrder voucherOrder) {
//1.一人一单判断
Long userId = voucherOrder.getUserId();
//1.1查询订单 判断存在用户id和优惠券id
Integer count = query().eq("user_id", userId)
.eq("voucher_id", voucherOrder.getVoucherId())
.count();
//1.2判断是否存在
if (count > 0) {
log.error("用户已经购买过一次");
return;
}
//2.扣减库存 乐观锁
boolean success = seckillVoucherService.update()
.setSql("stock = stock - 1") //set stock = stock-1
.eq("voucher_id", voucherOrder.getVoucherId()) //where voucher_id==voucherId
.gt("stock", 0) //where stock >0
.update();
if (!success) {
//扣减失败
log.error("库存不足！");
return;
}
//插入数据
save(voucherOrder);
}
}6. Overall business process
6.1 redis judgment process
1. First obtain the order id and user id, and call the Lua script to perform redis operations. Lua includes judging purchase qualifications/inventory adequacy, deducting inventory and placing orders, and sending order messages to Stream.
2. Stream forms a message queue, and any exceptions are automatically placed in the pending-list.
6.2 Thread process
1. The thread reads the message queue ( read message queue ). If it can read the message, continue reading; if it reads the order message, it parses the message content, calls order function 3 to write to the database, and then replies with a confirmation ack; if If an exception occurs, go to step 2 of penging-list.
2. Process the penging-list, parse the message, place an order, and process; if there is an exception, repeat the cycle and continue processing; until the processing is completed, the process will exit.
3. Add Redisson distributed lock and call the create order database operation.
