Messaging middleware transaction processing

Option 1 – The business side implements it by itself

Suppose the messaging middleware does not provide "transactional messaging" functionality, for example you are using Kafka. So how to solve this problem?

The solution is as follows: 
(1) Prepare a message table on the Producer side, and put the two operations of update DB and insert message in a DB transaction.

(2) Prepare a background program to continuously transmit the messages in the message table to the message middleware. Failed, keep retrying and retransmitting. Duplicate messages are allowed, but messages are not lost or out of order.

(3) The Consumer side prepares a weight judgment table. The processed messages are recorded in the judgment list. Implement business idempotency. But here is another question of atomicity: how to ensure the atomicity of the two operations of message consumption + insert message to judgment table?

The consumption is successful, but the insert judgment fails, what should I do? Regarding this, there was a discussion in Kafka's source code analysis series, Part 1, when the problem of exactly once.

Through the above three steps, we have basically solved the atomicity problem of the two operations of update db and sending network messages.

But one disadvantage of this scheme is that it needs to design the DB message table, and also needs a background task to continuously scan local messages. This leads to additional burdens on the business side due to the coupling of message processing and business logic.

Scenario 2 – RocketMQ transaction message

In order to solve this problem without coupling with the business, RocketMQ proposes the concept of "transaction message".

Specifically, the sending of the message is divided into two phases: the Prepare phase and the confirmation phase.

Specifically, the above 2 steps are decomposed into 3 steps: 
(1) Send Prepared message 
(2) update DB 
(3) Confirm or cancel the Prepared message according to the success or failure of the update DB result.

Some people may ask, what should I do if the first 2 steps are executed successfully and the last step fails? Here comes the key point of RocketMQ: RocketMQ will scan all Prepared messages regularly (default is 1 minute) and ask the sender, do you want to confirm that this message is sent? Or cancel this message?

The specific code is implemented as follows:

That is, a checkListener is defined, and RocketMQ will call back this Listener to implement the above-mentioned scheme.

// 也就是上文所说的，当RocketMQ发现`Prepared消息`时，会根据这个Listener实现的策略来决断事务
TransactionCheckListener transactionCheckListener = new TransactionCheckListenerImpl();
// 构造事务消息的生产者
TransactionMQProducer producer = new TransactionMQProducer("groupName");
// 设置事务决断处理类
producer.setTransactionCheckListener(transactionCheckListener);
// 本地事务的处理逻辑，相当于示例中检查Bob账户并扣钱的逻辑
TransactionExecuterImpl tranExecuter = new TransactionExecuterImpl();
producer.start()
// 构造MSG，省略构造参数
Message msg = new Message(......);
// 发送消息
SendResult sendResult = producer.sendMessageInTransaction(msg, tranExecuter, null);
producer.shutdown();
public TransactionSendResult sendMessageInTransaction(.....)  {

    // 逻辑代码，非实际代码
    // 1.发送消息
    sendResult = this.send(msg);
    // sendResult.getSendStatus() == SEND_OK
    // 2.如果消息发送成功，处理与消息关联的本地事务单元
    LocalTransactionState localTransactionState = tranExecuter.executeLocalTransactionBranch(msg, arg);
    // 3.结束事务
    this.endTransaction(sendResult, localTransactionState, localException);
}

Summary: Comparing scheme 2 and scheme 1, the biggest change of RocketMQ is that the "scanning message table" is not allowed to be done by the business side, but the message middleware helps.

As for the message table, in fact, it has not been omitted. Because the message middleware needs to ask the sender whether the transaction is executed successfully, it still needs a "local message table in disguise" to record the execution status of the transaction.

manual intervention

Some people may want to say again, no matter scheme 1 or scheme 2, the sender successfully puts the message into the queue, but what if the consumer fails to consume it?

Consumption fails, try again, and it keeps failing, what should I do? Do you want to roll back the entire process automatically?

The answer is human intervention. From the perspective of engineering practice, the cost of this automatic rollback of the entire process is very huge, which is not only complicated to implement, but also introduces new problems. For example, if the automatic rollback fails, what should I do?

For such extremely low-probability cases, manual processing is more reliable and simpler than implementing a highly complex automated rollback system.