RocketMQ achieves reliable message final consistency

The schematic diagram of RocketMQ to achieve the ultimate consistency of reliable messages:

Without further ado, go directly to the code. In this case, nacos in RocketMQ and Spring cloud Alibaba components are used to register and discover services, mybatis-plus, etc. Two databases are used in the case;

1. Project structure:

The rocketmq-transaction project is divided into three sub-modules. The base-framework-mysql-support module (as a basic module, referenced by other service modules) stores database-related jar packages and configuration classes. The order-service module is an order microservice, storage-service Is an inventory microservice module;

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.lucifer</groupId>
    <artifactId>rocketmq-transaction</artifactId>
    <packaging>pom</packaging>
    <version>1.0-SNAPSHOT</version>
    <modules>
        <module>order-service</module>
        <module>storage-service</module>
        <module>base-framework-mysql-support</module>
    </modules>


    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Finchley.RELEASE</spring-cloud.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>

            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-dependencies</artifactId>
                <version>2.1.3.RELEASE</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>

        </dependencies>
    </dependencyManagement>


</project>

2. Module Introduction

（1）base-framework-mysql-support：

This module has only one configuration about mybatis-plus: the code is as follows:

package com.lucifer.config;

import com.baomidou.mybatisplus.extension.plugins.PerformanceInterceptor;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * @author lucifer
 * @date 2020/4/14 21:54
 * @description mybatis-plus 配置
 */
@Configuration
@MapperScan(value = {"com.lucifer.mapper"})
public class MybatisPlusConfig {
    
    /**
     * SQL执行效率插件
     */
    @Bean
   // @Profile({"dev", "test"})// 设置 dev test 环境开启
    public PerformanceInterceptor performanceInterceptor() {
        return new PerformanceInterceptor();
    }
}

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>rocketmq-transaction</artifactId>
        <groupId>com.lucifer</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>base-framework-mysql-support</artifactId>

    <dependencies>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid-spring-boot-starter</artifactId>
            <version>1.1.10</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.39</version>
        </dependency>

        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.12</version>
        </dependency>
    </dependencies>

</project>

(1) order-service: order module

pojo: package for storing entity classes

/**
 * 订单表
 */
@Data
@NoArgsConstructor
@TableName("order_tbl")
public class Order {

    @TableId(type = IdType.AUTO)
    private Integer id;
    private String userId;
    private String commodityCode;
    private Integer count;
    private BigDecimal money;
    @TableField(exist = false)
    private String txNum;
}

/**
 * @author lucifer
 * @date 2020/4/15 13:04
 * @description 事务日志表
 */
@Data
//@Builder
@NoArgsConstructor
//@Accessors(chain = true)
@TableName("tx_log")
public class TxLog {
    @TableId
    private String txNum;
    private Date createTime;
}

mapper：

package com.lucifer.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.lucifer.pojo.Order;

public interface OrderMapper extends BaseMapper<Order> {
}

package com.lucifer.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.lucifer.pojo.TxLog;

public interface TxLogMapper extends BaseMapper<TxLog> {
}

service: interface

package com.lucifer.service;


import com.lucifer.pojo.Order;

public interface OrderService {


    /**
     * 发送订单消息
     *
     * @param order
     */
    void sendOrder(Order order);


    /**
     * 新增订单
     *
     * @param order
     */
    void insertOrder(Order order);

}

Implementation class:

/**
 * @author lucifer
 * @date 2020/4/14 19:31
 * @description
 */
@Service
public class OrderServiceImpl implements OrderService {

    @Resource
    private RocketMQTemplate rocketMQTemplate;
    @Resource
    private OrderMapper orderMapper;
    @Resource
    private TxLogMapper txLogMapper;


    @Override
    public void sendOrder(Order order) {
        String str = JSON.toJSONString(order);
        Message<String> message = MessageBuilder.withPayload(str).build();
        /**
         * 发送一条事务消息
         * String txProducerGroup: 生产组
         * String destination：topic
         * Message<?> message: 消息内容
         * Object arg: 参数
         */
        rocketMQTemplate.sendMessageInTransaction("producer_group_tx1", "topic_tx", message, null);
    }

    @Transactional(rollbackFor = Exception.class)
    @Override
    public void insertOrder(Order order) {
        //用事务id幂等处理
        if (txLogMapper.selectById(order.getTxNum()) != null) {
            return;
        }
        orderMapper.insert(order);
        //插入事务日志
        TxLog txLog = new TxLog();
        txLog.setTxNum(order.getTxNum());
        System.out.println("order.getTxNum():" + order.getTxNum());
        txLog.setCreateTime(new Date());
        txLogMapper.insert(txLog);

        //模拟异常，检查事务是否回滚
        QueryWrapper<Order> queryWrapper = new QueryWrapper<>();
        queryWrapper.eq("commodity_code", "product-1");
        if (orderMapper.selectList(queryWrapper).size()== 6) {
            throw new RuntimeException("人为模拟异常");
        }
    }

}

Rocketmq's transaction listener: (important)

/**
 * @author lucifer
 * @date 2020/4/15 0:59
 * @description TODO
 */
@Slf4j
@Component
@RocketMQTransactionListener(txProducerGroup = "producer_group_tx1")
public class ProducerTransactionListener implements RocketMQLocalTransactionListener {

    @Resource
    private OrderService orderService;
    @Resource
    private TxLogMapper txLogMapper;

    /**
     * 事务消息发送mq成功后的回调方法
     *
     * @param msg
     * @param arg
     * @return 返回事务状态
     * RocketMQLocalTransactionState.COMMIT：提交事务，提交后broker才允许消费者使用
     * RocketMQLocalTransactionState.ROLLBACK：回滚事务，回滚后消息将被删除，并且不允许别消费
     * RocketMQLocalTransactionState.Unknown：中间状态，表示MQ需要核对，以确定状态
     */
    @Transactional(rollbackFor = Exception.class)
    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message msg, Object arg) {

        try {
            String str = new String((byte[]) msg.getPayload());
            Order order = JSON.parseObject(str, Order.class);

            orderService.insertOrder(order);
            //当返回RocketMQLocalTransactionState.COMMIT，自动向mq发送commit，mq将消息的状态改为可消费
            return RocketMQLocalTransactionState.COMMIT;
        } catch (Exception e) {
            e.printStackTrace();
            return RocketMQLocalTransactionState.ROLLBACK;
        }
    }

    /**
     * 事务状态回查，查询是否下单成功
     *
     * @param msg
     * @return 返回事务状态
     */
    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message msg) {
        String str = new String((byte[]) msg.getPayload());
        Order order = JSON.parseObject(str, Order.class);
        //事务id
        String txNo = order.getTxNum();
        TxLog txLog = txLogMapper.selectById(txNo);
        if (txLog != null) {
            return RocketMQLocalTransactionState.COMMIT;
        } else {
            return RocketMQLocalTransactionState.UNKNOWN;
        }
    }
}

application.yml: configuration file

server:
  port: 8081

spring:
  application:
    name: order-service
  datasource:
    druid:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://192.168.160.131:3306/order?autoReconnect=true&useUnicode=true&createDatabaseIfNotExist=true&characterEncoding=utf8&serverTimezone=UTC
      username: root
      password: 123456
  cloud:
    nacos:
      discovery:
        server-addr: 192.168.160.131:8848
#  main:
#    allow-bean-definition-overriding: true
logging:
  level:
    com.lucifer.mapper: debug

rocketmq:
  producer:
    group: producter_tx
  name-server: 192.168.160.131:9876

Springboot startup class:

/**
 * @author lucifer
 * @date 2020/4/14 19:28
 * @description TODO
 */
@EnableDiscoveryClient
@SpringBootApplication
public class OrderApplication {
    public static void main(String[] args) {
        SpringApplication.run(OrderApplication.class, args);
    }

}

Controller layer:

/**
 * @author lucifer
 * @date 2020/4/14 19:32
 * @description TODO
 */
@RestController
@RequestMapping(value = "order")
public class OrderController {

    @Resource
    OrderService orderService;

    /**
     * 下单：插入订单表、扣减库存，模拟回滚
     *
     * @return
     */
    @GetMapping("/placeOrder/commit")
    public Boolean placeOrderCommit() {
        //将uuid作为事务id,发送到mq
        String uuid = UUID.randomUUID().toString();
        Order order = new Order();
        order.setCommodityCode("product-1");
        order.setUserId("1");
        order.setCount(1);
        order.setTxNum(uuid);
        order.setMoney(new BigDecimal(12.5));
        System.out.println("准备下单了=======》" + order);
        orderService.sendOrder(order);
        return true;
    }
}

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <artifactId>rocketmq-transaction</artifactId>
        <groupId>com.lucifer</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <artifactId>order-service</artifactId>

    <dependencies>
        <dependency>
            <groupId>com.lucifer</groupId>
            <artifactId>base-framework-mysql-support</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-spring-boot-starter</artifactId>
            <version>2.0.2</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.62</version>
        </dependency>
        <!-- nacos -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
            <version>0.2.2.RELEASE</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

(3) storage-service: inventory microservice module

mapper:

package com.lucifer.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.lucifer.pojo.Storage;

public interface StorageMapper extends BaseMapper<Storage> {
}

package com.lucifer.mapper;

import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.lucifer.pojo.TxLog;

public interface TxLogMapper extends BaseMapper<TxLog> {
}

sing:

/**
 * 库存表
 */
@Data
@Accessors(chain = true)
@TableName("storage_tbl")
public class Storage {

    private Long id;
    private String commodityCode;
    private Long count;
}

ps: order and txlog entity classes can be copied from order-service;

service:

public interface StorageService {
    /**
     * 扣减库存
     *
     * @param commodityCode
     * @param count
     * @param txNum 事务id
     */
    void deduct(String commodityCode, int count,String txNum);
}

Implementation class:

/**
 * @author lucifer
 * @date 2020/4/14 20:07
 * @description TODO
 */
@Service
@Slf4j
public class StorageServiceImpl implements StorageService {

    @Resource
    private StorageMapper storageMapper;
    @Resource
    private TxLogMapper txLogMapper;

    @Transactional(rollbackFor = Exception.class)
    @Override
    public void deduct(String commodityCode, int count, String txNum) {
        log.info("扣减库存，商品编码：{},数量：{}", commodityCode, count);
        TxLog txLog = txLogMapper.selectById(txNum);
        if (txLog != null) {
            return;
        }
        //扣减库存
        QueryWrapper<Storage> wrapper = new QueryWrapper<>();
        wrapper.setEntity(new Storage().setCommodityCode(commodityCode));
        Storage storage = storageMapper.selectOne(wrapper);
        if (storage == null) {
            throw new RuntimeException("商品" + commodityCode + ",不存在");
        }
        storage.setCount(storage.getCount() - count);

        storageMapper.updateById(storage);

        //添加事务记录，用于幂等
        TxLog tLog = new TxLog();
        tLog.setTxNum(txNum);
        tLog.setCreateTime(new Date());
        txLogMapper.insert(tLog);

        //模拟异常，检查事务是否回滚
        if(storageMapper.selectById(1).getCount()==4996){
            throw new RuntimeException("人为模拟异常");
        }
    }
}

Rocketmq monitoring class:

/**
 * @author lucifer
 * @date 2020/4/15 0:59
 * @description TODO
 */
@Slf4j
@Component
@RocketMQMessageListener(consumerGroup = "consumer_group_tx2", topic = "topic_tx")
class ConsumerTransactionListener implements RocketMQListener<String> {

    @Resource
    private StorageService storageService;

    @Override
    public void onMessage(String message) {
        log.info("开始消费消息:{}", message);
        //解析消息
        Order order = JSON.parseObject(message, Order.class);

        //扣减库存
        storageService.deduct(order.getCommodityCode(), order.getCount(), order.getTxNum());
    }
}

Springboot startup class:

/**
 * @author lucifer
 * @date 2020/4/14 20:23
 * @description 库存服务
 */
@EnableDiscoveryClient
@SpringBootApplication
public class StorageApplication {
    public static void main(String[] args) {
        SpringApplication.run(StorageApplication.class, args);
    }
}

application.yml:

server:
  port: 8082

spring:
  application:
    name: storage-service
  datasource:
    druid:
      driver-class-name: com.mysql.cj.jdbc.Driver
      url: jdbc:mysql://192.168.160.131:3306/storage?autoReconnect=true&useUnicode=true&createDatabaseIfNotExist=true&characterEncoding=utf8&serverTimezone=UTC
      username: root
      password: 123456
  cloud:
    nacos:
      discovery:
        server-addr: 192.168.160.131:8848
#  main:
#    allow-bean-definition-overriding: true
logging:
  level:
    com.lucifer.mapper: debug

rocketmq:
  producer:
    group: consumer_tx
  name-server: 192.168.160.131:9876

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <artifactId>rocketmq-transaction</artifactId>
        <groupId>com.lucifer</groupId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <artifactId>storage-service</artifactId>

    <dependencies>
        <dependency>
            <groupId>com.lucifer</groupId>
            <artifactId>base-framework-mysql-support</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-spring-boot-starter</artifactId>
            <version>2.0.2</version>
        </dependency>
        <!-- nacos -->
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
            <version>0.2.2.RELEASE</version>
        </dependency>
    </dependencies>


    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Start the order-service service and start the storage-service service;

testing scenarios:

(1) Order-service local transaction fails, order-service will not send order message

(2) Storage-service receives the message of placing an order, the deduction of inventory fails, and will continue to retry the deduction of inventory (of course, this number of attempts is limited), the console will continue to print the retry information: If you continue to repeat this consumption, If it continues to fail for a certain number of times (the default is 16 times), it will be delivered to the DLQ dead letter queue. At this time, manual intervention is required.

Scene (2) Screenshot of the database:

Lucifer_Lucifer

Published 187 original articles · Like 146 · Visit 490,000+

His message board concerns