Use RocketMQ (rocketmq-spring-boot-starter) in Springboot to solve distributed transaction problems

Preface
The previous article introduced the use process of rocketmq-spring-boot-starter. This article introduces how to use RocketMQ to solve transaction problems in a distributed environment.

If you are not familiar with rocketmq-spring-boot-starter, it is recommended to read my last article first: Springboot Integration RocketMQ Tutorial

1: Scene simulation

Scenario : Suppose we have such a business now: users will get points for recharging network fees, and 1 yuan = 1 point, recharge 100 yuan in the user service, and add 100 points to the user in the point service

Analysis : For such cross-service and cross-library operations, we need to ensure that these two operations either succeed together or fail together. The RocketMQ solution is: RocketMQ transaction message + local transaction + monitoring consumption to achieve final consistency

Before implementing, let’s introduce RocketMQ’s transaction

Two: RocketMQ transaction introduction

1. Basic concepts
(1) Half Message: also called Prepare Message, translated as "half message" or "preparation message", which refers to a message that cannot be delivered temporarily, that is, the message is successfully sent to the MQ server and cannot be delivered to the consumer for the time being For consumption, only when the server receives the second confirmation from the producer, can it be consumed by the consumer
(2) Message Status Check: Check back the message status. The second acknowledgment of the transactional message may be lost due to network disconnection or restart of the producer application. When the MQ server finds that a message remains in the half-message state for a long time, it will send a request to the message producer to Check the final status of the message ("commit" or "rollback")

2. Execution flowchart (tool ProcessOn)

1. The producer sends a half-message to the MQ Server, which cannot be delivered temporarily and will not be consumed
2. After the half-message is sent successfully, the producer executes the local transaction
3. The producer sends a commit or rollback message to the MQ Server for secondary confirmation based on the execution result of the local transaction
4. If the MQ Server receives the commit, it will mark the half message as deliverable, and the consumer can consume at this time; if it receives the rollback, it will discard the half message directly and will not consume
5. If the MQ Server does not receive the second confirmation message, the MQ Server will periodically (default 1 minute) send a checkback message to the producer to check the local transaction status, and then the producer will send a commit or send to the MQ Server again according to the result of the local transaction checkback rollback message

Three: Business code implementation

1. Create table
(1) user table

CREATE TABLE `t_user` (
   `id` int(11) NOT NULL AUTO_INCREMENT COMMENT '用户表',
   `name` varchar(16) NOT NULL COMMENT '姓名',
   `id_card` varchar(32) NOT NULL COMMENT '身份证号',
   `balance` int(11) NOT NULL DEFAULT '0' COMMENT '余额',
   `state` tinyint(1) DEFAULT NULL COMMENT '状态（1在线，0离线）',
   `vip_flag` tinyint(1) NOT NULL DEFAULT '0' COMMENT 'VIP用户标识（1是，0否）',
   `create_time` datetime NOT NULL COMMENT '创建时间',
   `last_login_time` datetime DEFAULT NULL COMMENT '最后一次登录时间',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4

(2) Point table

CREATE TABLE `t_credit` (
   `id` int(11) NOT NULL AUTO_INCREMENT COMMENT '积分表',
   `user_id` int(11) NOT NULL COMMENT '用户id',
   `username` varchar(16) NOT NULL COMMENT '用户姓名',
   `integration` int(11) NOT NULL DEFAULT '0' COMMENT '积分',
   PRIMARY KEY (`id`)
 ) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4

(3) Transaction log table

CREATE TABLE `t_mq_transaction_log` (
   `transaction_id` varchar(64) NOT NULL COMMENT '事务id',
   `log` varchar(64) NOT NULL COMMENT '日志',
   PRIMARY KEY (`transaction_id`)
 ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4

1. I am a simulation, and I put it in a library here. As for why a transaction log table is built , you will know later
2. After the table is created, for your convenience, first manually write a piece of data in the user table and point table
3. The project structure is skipped here, including entity classes, mapper interfaces, etc.

2. Create a new MQ transaction producer: MQTXProducerService

@Slf4j
@Component
public class MQTXProducerService {
    
    

    private static final String Topic = "RLT_TEST_TOPIC";
    private static final String Tag = "charge";
    private static final String Tx_Charge_Group = "Tx_Charge_Group";

    @Autowired
    private RocketMQTemplate rocketMQTemplate;

    /**
     * 先向MQ Server发送半消息
     * @param userCharge 用户充值信息
     */
    public TransactionSendResult sendHalfMsg(UserCharge userCharge) {
    
    
        // 生成生产事务id
        String transactionId = UUID.randomUUID().toString().replace("-", "");
        log.info("【发送半消息】transactionId={}", transactionId);

        // 发送事务消息（参1：生产者所在事务组，参2：topic+tag，参3：消息体(可以传参)，参4：发送参数）
        TransactionSendResult sendResult = rocketMQTemplate.sendMessageInTransaction(
                Tx_Charge_Group, Topic + ":" + Tag,
                MessageBuilder.withPayload(userCharge).setHeader(RocketMQHeaders.TRANSACTION_ID, transactionId).build(),
                userCharge);
        log.info("【发送半消息】sendResult={}", JSON.toJSONString(sendResult));
        return sendResult;
    }
}

1. Here I use UUID to generate the transaction id, which is the id of the transaction log table above (the actual situation may use the snowflake algorithm or define the ID according to the business)
2. Method parameter userCharge, additionally added, can be understood as dto, just two fields: userId, chargeAmount, representing user id and recharge amount
3. Note here: There are two parameters in the method of sending a half message, reference 3 and reference 4. Those who have read the previous integration tutorial should know that this parameter 3 is for consumers, and this parameter 4 is for local affairs. Here I am It is the same as the simulation, the actual business may be different

3. Create a new local transaction listener: MQTXLocalService

@Slf4j
@RocketMQTransactionListener(txProducerGroup = "Tx_Charge_Group") // 这里的txProducerGroup的值要与发送半消息时保持一致
public class MQTXLocalService implements RocketMQLocalTransactionListener {
    
    

    @Autowired
    private UserService userService;
    @Autowired
    private MQTransactionLogMapper mqTransactionLogMapper;

    /**
     * 用于执行本地事务的方法
     */
    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message message, Object obj) {
    
    
        // 获取消息体里参数
        MessageHeaders messageHeaders = message.getHeaders();
        String transactionId = (String) messageHeaders.get(RocketMQHeaders.TRANSACTION_ID);
        log.info("【执行本地事务】消息体参数：transactionId={}", transactionId);

        // 执行带有事务注解的本地方法：增加用户余额+保存mq日志
        try {
    
    
            UserCharge userCharge = (UserCharge) obj;
            userService.addBalance(userCharge, transactionId);
            return RocketMQLocalTransactionState.COMMIT; // 正常：向MQ Server发送commit消息
        } catch (Exception e) {
    
    
            log.error("【执行本地事务】发生异常，消息将被回滚", e);
            return RocketMQLocalTransactionState.ROLLBACK; // 异常：向MQ Server发送rollback消息
        }
    }

    /**
     * 用于回查本地事务执行结果的方法
     */
    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message message) {
    
    
        MessageHeaders headers = message.getHeaders();
        String transactionId = headers.get(RocketMQHeaders.TRANSACTION_ID, String.class);
        log.info("【回查本地事务】transactionId={}", transactionId);

        // 根据事务id查询事务日志表
        MQTransactionLog mqTransactionLog = mqTransactionLogMapper.selectByPrimaryKey(transactionId);
        if (null == mqTransactionLog) {
    
     // 没查到表明本地事务执行失败,通知回滚
            return RocketMQLocalTransactionState.ROLLBACK;
        }
        return RocketMQLocalTransactionState.COMMIT; // 查到表明本地事务执行成功，提交
    }
}

@Service
public class UserService {
    
    

    @Autowired
    private UserMapper userMapper;
    @Autowired
    private MQTransactionLogMapper mqTransactionLogMapper;

    /**
     * 用户增加余额+事务日志
     */
    @Transactional(rollbackFor = Exception.class)
    public void addBalance(UserCharge userCharge, String transactionId) {
    
    
        // 1. 增加余额
        userMapper.addBalance(userCharge.getUserId(), userCharge.getChargeAmount());
        // 2. 写入mq事务日志
        saveMQTransactionLog(transactionId, userCharge);
    }

    @Transactional(rollbackFor = Exception.class)
    public void saveMQTransactionLog(String transactionId, UserCharge userCharge) {
    
    
        MQTransactionLog transactionLog = new MQTransactionLog();
        transactionLog.setTransactionId(transactionId);
        transactionLog.setLog(JSON.toJSONString(userCharge));
        mqTransactionLogMapper.insertSelective(transactionLog);
    }

}

1. The code here is the main key point. The local transaction is to add the balance to the user and then insert the mq transaction log. Only when these two operations succeed, will it return to COMMIT, and if it fails abnormally, it will return to ROLLBACK
2. The backcheck method may not be executed, but it must be. Backcheck is to query the transaction log table based on the transaction id (transactionId) we generated and passed before. The advantage of this is that it doesn’t matter how many tables are involved in the business. My log table It is also bound to your local transaction . I only need to query this transaction table. If I can find it, it means that the local transaction has been executed successfully.
3. Here is a point: the above addBalance method is flawed. If the saveMQTransactionLog method is abnormal, although the @Transactional annotation is added to the addBalance method, the transaction will not take effect. This involves the principle of spring’s transaction mechanism (essentially implemented through AOP+ dynamic proxy ), but I am also here for simulation, so I filter out this detail

4. Create a new transaction message consumer: MQTXConsumerService

@Slf4j
@Component
@RocketMQMessageListener(topic = "RLT_TEST_TOPIC", selectorExpression = "charge", consumerGroup = "Con_Group_Four") // topic、tag保持一致
public class MQTXConsumerService implements RocketMQListener<UserCharge> {
    
    

    @Autowired
    private CreditMapper creditMapper;

    @Override
    public void onMessage(UserCharge userCharge) {
    
    
        // 一般真实环境这里消费前，得做幂等性判断，防止重复消费
        // 方法一：如果你的业务中有某个字段是唯一的，有标识性，如订单号，那就可以用此字段来判断
        // 方法二：新建一张消费记录表t_mq_consumer_log，字段consumer_key是唯一性，能插入则表明该消息还未消费，往下走，否则停止消费
        // 我个人建议用方法二，根据你的项目业务来定义key，这里我就不做幂等判断了，因为此案例只是模拟，重在分布式事务

        // 给用户增加积分
        int i = creditMapper.addNumber(userCharge.getUserId(), userCharge.getChargeAmount());
        if (1 == i) {
    
    
            log.info("【MQ消费】用户增加积分成功，userCharge={}", JSONObject.toJSONString(userCharge));
        } else {
    
    
            log.error("【MQ消费】用户充值增加积分消费失败，userCharge={}", JSONObject.toJSONString(userCharge));
        }
    }
}

1. Consumers are actually relatively simple, similar to ordinary consumers, just pay attention to attribute configuration
2. Here you may question that there is nothing wrong with the previous sending and local transactions, either commit or rollback, but what if the consumption fails here? In fact, there is almost no chance of problems here. First of all, RocketMQ is highly available. If your system is really huge, you can cluster it. Furthermore, whether the consumption here is successful or not, the source code has been processed internally, as long as there is no abnormality , it will consume, and it also has a retry mechanism; finally, you can expand the consumption logic here. When the consumption is unsuccessful, you can save the record, remind it regularly or manually process it

Four: Test

Add to RocketMQController:

@PostMapping("/charge")
public Result<TransactionSendResult> charge(UserCharge userCharge) {
    
    
    TransactionSendResult sendResult = mqtxProducerService.sendHalfMsg(userCharge);
    return Result.success(sendResult);
}

Use postman to call: http://localhost:8080/rocketmq/charge


and see that it is normal, then go to the database and find that 100 has been added from the balance and points, and the transaction log table is also recorded, success!

Summary: In fact, after understanding the implementation process of transactions, you will find that it is quite simple to use RocketMQ to solve distributed transactions. After all, MQ is very friendly, and MQ has many uses, and every project can have it. Of course, there are other popular and professional distributed transaction solutions now, so Seata has to be mentioned, but if your project does not specifically need Seata, if MQ can solve it, then you can introduce one less Seata component Well, why not do it?

Here is a little personal opinion: the choice of a distributed transaction solution must consider and suit your actual business, then my suggestion is: if the business is a user doing a certain operation, it will definitely go down, even if something happens The problem is also backward, so it is no problem to use RocketMQ 100%. Let me give you the simplest example: the user paid the money, but there was a problem with the follow-up program. It is impossible for you to contact the user to say that there is a problem with the system. How about I refund the money to you first, and you can place an order to buy it again later. If so, I think no one will use your system later. In short, in actual production, it must be flexibly used according to the business.