[RocketMQ] Advanced usage: Four questions explain transaction messages in detail

 

The biggest difference between RocketMQ and other message middleware is that it supports transaction messages, which is also a message-based eventual consistency scheme in distributed transactions.

1. What is the transaction message?

Transaction message: A message with transaction characteristics, that is, after the Producer is sent to the broker, the message can be rolled back or submitted (the Consumer will only be visible after the commit).

2. What is the use of transaction messages?

RocketMQ official example: User A initiates an order, pays 100 yuan, and gets 100 points after the operation is completed. Account service and membership service are two independent microservice modules with their own databases. According to the above mentioned problem possibilities , There will be these situations:

• If you deduct the money first, and then send the message, the money may have just been deducted, the machine went down, the message was not sent, and the points did not increase as a result.

• If you send a message first and then deduct the money, the points may increase, but the money is not deducted, and 100 points are given to others for nothing.

// 先扣款，再加积分伪代码：
@Transational
pay() {
    mysql.payMoney() // 数据库中添加用户信息
    reduceRepo() // 数据库中减少库存  
}
-------------------------------------// 若先发消息再加积分，那在这行宕机怎么办？
producer.send(msg) // 发送100积分

==> So the above method is not feasible, we can send a message (plus points) to the Broker first, but set the message to the consumer invisible state

• If the local transaction (deduction) is processed successfully, then let the Consumer see it
• If the local transaction (deduction) fails, the current message will be rolled back

There may be a problem here. After the producer's local transaction succeeds, it fails to send a transaction confirmation message to the broker. What should I do?
This time means that consumers cannot consume the news normally. Therefore, RocketMQ provides a message review mechanism . If the transaction message is always in an intermediate state, the broker will initiate a retry to query the processing status of the transaction on the broker. Once the transaction is found to be successful, the current message is set to be visible

The overall model diagram is as follows:

From the above example, we can see that transaction messages are generally used before local transactions. This can also be understood as a nested transaction, sending a message is an outer transaction, and a local transaction is a memory transaction.

3. Does Java use transaction messages?

For the above example, let's take a look at how to implement it through specific code.

TransactionProducer

public class TransactionProducer {
    public static void main(String[] args) throws Exception {  
        // 这里用的是事务Producer（TransactionMQProducer）
        TransactionMQProducer transactionProducer=new TransactionMQProducer("tx_producer_group");
        transactionProducer.setNamesrvAddr("43.105.136.120:9876");    
        // 自定义线程池，用于异步执行事务操作     
        transactionProducer.setExecutorService(Executors.newFixedThreadPool(10); );  
        // 核心！！添加事务消息监听     
        transactionProducer.setTransactionListener(new TransactionListenerLocal());
        transactionProducer.start();   
        
        for(int i=0;i<20;i++) {       
            String orderId= UUID.randomUUID().toString();    
            String body="{'operation':'doOrder','orderId':'"+orderId+"'}";     
            // 构建消息
            Message message = new Message("pay_tx_topic", "TagA",orderId, body.getBytes(RemotingHelper.DEFAULT_CHARSET));  
            // 发送消息， 注：是发送事务消息
            transactionProducer.sendMessageInTransaction(message, orderId+"&"+i);   
            Thread.sleep(1000); // 1秒一次
        }
    } 
}

TransactionListenerLocal (transaction message core)

// 本地事务监听，实现TransactionListener接口
public class TransactionListenerLocal implements TransactionListener {
    private static final Map<String,Boolean> results=new ConcurrentHashMap<>();
    
    // 执行本地事务   
    @Override   
    public LocalTransactionState executeLocalTransaction(Message msg, Object arg) { 
        System.out.println(":执行本地事务："+arg.toString());    
        String orderId=arg.toString();  
        // 模拟数据入库操作（成功/失败）
        boolean rs=saveOrder(orderId);
        return rs? LocalTransactionState.COMMIT_MESSAGE:LocalTransactionState.UNKNOW;  
        // 这个返回状态表示告诉broker这个事务消息是否被确认，允许给到consumer进行消费    
        // LocalTransactionState.ROLLBACK_MESSAGE 回滚    
        // LocalTransactionState.UNKNOW  未知   
    }  
    
    // 提供事务执行状态的回查方法，提供给broker回调   
    @Override   
    public LocalTransactionState checkLocalTransaction(MessageExt msg) {  
        String orderId=msg.getKeys();     
        System.out.println("执行事务执行状态的回查，orderId:"+orderId);  
        boolean rs=Boolean.TRUE.equals(results.get(orderId));   
        System.out.println("回调："+rs);     
        return rs?LocalTransactionState.COMMIT_MESSAGE:     
      							  LocalTransactionState.ROLLBACK_MESSAGE;  
    }   
    
    private boolean saveOrder(String orderId){    
        //如果订单取模等于0，表示成功,否则表示失败  
        boolean success=Math.abs(Objects.hash(orderId))%2==0;   
        results.put(orderId,success);    
        return success;  
    } 
}

TransactionConsumer

public class TransactionConsumer {
    public static void main(String[] args) throws MQClientException, IOException {   
        DefaultMQPushConsumer defaultMQPushConsumer=new      
            DefaultMQPushConsumer("tx_consumer_group");   
        defaultMQPushConsumer.setNamesrvAddr("43.105.136.120:9876");       
        defaultMQPushConsumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_O FFSET);   
        defaultMQPushConsumer.subscribe("pay_tx_topic","*");    
         
        defaultMQPushConsumer.registerMessageListener((MessageListenerConcurrently)
                  (msgs, context) -> {           
                      msgs.stream().forEach(messageExt -> {    
                          try {                 
                              String orderId=messageExt.getKeys();
                              // 拿到消息
                              String body=new String(messageExt.getBody(), 
                                                     RemotingHelper.DEFAULT_CHARSET); 
                              // 扣减库存
                              System.out.println("收到消息:"+body+"，开始扣减库存："+orderId);  
                          } catch (UnsupportedEncodingException e) {    
                              e.printStackTrace();                
                          }             
                      });   
                      return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;    
                  });     
        defaultMQPushConsumer.start();    
        System.in.read();  
    }
}

4. Three states of transaction message?

In the TransactionListenerLocal class above, we see that both of the rewritten methods need to return LocalTransactionState, which means telling the broker how to deal with this existing transaction message:

1. ROLLBACK_MESSAGE: Roll back the transaction

   When the executeLocalTransaction method returns ROLLBACK_MESSAGE, it means that the transaction is rolled back directly

2. COMMIT_MESSAGE: Commit the transaction

3. UNKNOW: The broker will periodically check the status of the Producer message until it succeeds or fails.

   When UNKNOW is returned, the Broker will check back checkLocalTransaction after a period of time, and execute the transaction operation (rollback or commit) according to the return status of checkLocalTransaction

As in the example, when ROLLBACK_MESSAGE is returned, the consumer will not receive the message and will not call the check-back function. When COMMIT_MESSAGE is returned, the transaction is committed and the consumer receives the message. When it returns UNKNOW, the check-back function is called after a period of time. , And return the submission or rollback status according to the status. The message that returns the submission status will be consumed by the consumer, so the consumer can consume part of the message at this time.