two-phase commit protocol
The two-phase commit protocol is a distributed algorithm that coordinates all distributed atomic transaction participants and decides to commit or cancel (rollback). (1) Protocol participants
In the two-phase commit protocol, the system generally includes two types of machines (or nodes): one is the coordinator, usually only one in a system; the other is the transaction participants (participants, cohorts or workers), generally Contains multiple, which can be understood as the number of data copies in the data storage system. The protocol assumes that each node will record a write-ahead log and store it persistently, even if the node fails, the log will not be lost. The protocol also assumes that nodes will not fail permanently and that any two nodes can communicate with each other.
(2) Two-stage execution
1. Request phase (commit-request phase, or voting phase, voting phase)
In the request phase, the coordinator will notify the transaction participants to prepare to commit or cancel the transaction, and then enter the voting process.
During the voting process, participants will inform the coordinator of their decisions: agree (transaction participant's local job execution is successful) or cancel (local job execution failure).
2. Commit phase
In this phase, the coordinator will make a decision based on the voting results of the first phase: commit or cancel.
If and only if all participants agree to submit the transaction, the coordinator will notify all participants to submit the transaction, otherwise the coordinator will notify all participants to cancel the transaction.
Participants will perform corresponding operations after receiving messages from the coordinator.
(3) Disadvantages of two-phase commit
1. Synchronous blocking problem. During execution, all participating nodes are transaction blocking.
When participants occupy public resources, other third-party nodes have to be blocked when accessing public resources.
2. Single point of failure. Due to the importance of the coordinator, once the coordinator fails.
Participants will be blocked forever. Especially in the second stage, if the coordinator fails, all participants are still in the state of locking transaction resources, and cannot continue to complete transaction operations. (If the coordinator hangs up, a coordinator can be re-elected, but it cannot solve the problem that the participants are blocked due to the downtime of the coordinator)
3. The data is inconsistent. In the second phase of the two-phase commit, after the coordinator sends the commit request to the participants, a local network exception occurs or the coordinator fails during the process of sending the commit request, which results in only some participants receiving the commit request.
After these participants receive the commit request, they will execute the commit operation. However, other machines that have not received a commit request cannot perform transaction commits. As a result, there is a phenomenon of data consistency in the entire distributed system.
(4) Problems that cannot be solved by two-phase commit
When the coordinator makes mistakes and the participants also make mistakes, the two-phase cannot guarantee the integrity of transaction execution.
Consider that the coordinator went down after sending the commit message, and the only participant who received the message also went down at the same time.
Then even if the coordinator generates a new coordinator through the election agreement, the status of this transaction is uncertain, and no one knows whether the transaction has been committed.
Transaction message TransactionProducer
Distributed Transaction Final Consistency Solution Based on Reliable Message Service
-
The producer executes the local transaction, modifies the order payment status, and submits the transaction
-
The producer sends a transaction message to the broker, and the message is not visible to the consumer until the message is sent to the broker. -
The producer confirms the transaction message, making the transaction message sent to the broker visible to the consumer
-
The consumer obtains the message to consume, and executes ack to confirm after consumption
-
There may be a problem here. After the producer's local transaction succeeds, what should I do if it fails to send a transaction confirmation message to the broker
? At this time, it means that consumers cannot normally consume this message. Therefore, RocketMQ provides a message check mechanism. If
the transaction message is always in an intermediate state, the broker will initiate a retry to query the processing status of the transaction on the broker. Once
the transaction is found to be successful, set the current message to be visible
practice:
producer
public class TransactionProducer {
public static void main(String[] args) throws Exception {
TransactionMQProducer transactionProducer=new
TransactionMQProducer("tx_producer_group");
transactionProducer.setNamesrvAddr("10.211.55.4:9876");
ExecutorService executorService = newFixedThreadPool(10);
//自定义线程池，用于异步执行事务操作
transactionProducer.setExecutorService(executorService);
transactionProducer.setTransactionListener(new TransactionListenerLocal());
transactionProducer.start();
for(int i=0;i<20;i++) {
String orderId= UUID.randomUUID().toString();
String body="{'operation':'doOrder','orderId':'"+orderId+"'}";
Message message = new Message("pay_tx_topic", "TagA",orderId,body.getBytes(RemotingHelper.DEFAULT_CHARSET));
transactionProducer.sendMessageInTransaction(message,orderId+"&"+i);
Thread.sleep(1000);
}
}
}TransactionListener
public class TransactionListenerLocal implements TransactionListener {
private static final Map<String,Boolean> results=new ConcurrentHashMap<>();
//执行本地事务
@Override
public LocalTransactionState executeLocalTransaction(Message msg, Object
arg) {
System.out.println(":执行本地事务："+arg.toString());
String orderId=arg.toString();
boolean rs=saveOrder(orderId);//模拟数据入库操作
return rs?LocalTransactionState.COMMIT_MESSAGE:LocalTransactionState.UNKNOW;
// 这个返回状态表示告诉broker这个事务消息是否被确认，允许给到consumer进行消费
// LocalTransactionState.ROLLBACK_MESSAGE 回滚
//LocalTransactionState.UNKNOW 未知
}
//提供事务执行状态的回查方法，提供给broker回调
@Override
public LocalTransactionState checkLocalTransaction(MessageExt msg) {
String orderId=msg.getKeys();
System.out.println("执行事务执行状态的回查，orderId:"+orderId);
boolean rs=Boolean.TRUE.equals(results.get(orderId));
System.out.println("回调："+rs);
return rs?LocalTransactionState.COMMIT_MESSAGE:LocalTransactionState.ROLLBACK_MESSAGE;
}
private boolean saveOrder(String orderId){
//如果订单取模等于0，表示成功,否则表示失败
boolean success=Math.abs(Objects.hash(orderId))%2==0;
results.put(orderId,success);
return success;
}
}comsumer
public class TransactionConsumer {
public static void main(String[] args) throws MQClientException, IOException
{
DefaultMQPushConsumer defaultMQPushConsumer=new
DefaultMQPushConsumer("tx_consumer_group");
defaultMQPushConsumer.setNamesrvAddr("10.211.55.4:9876");
defaultMQPushConsumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
defaultMQPushConsumer.subscribe("pay_tx_topic","*");
defaultMQPushConsumer.registerMessageListener((MessageListenerConcurrently)
(msgs, context) -> {
msgs.stream().forEach(messageExt -> {
try {
String orderId=messageExt.getKeys();
String body=new String(messageExt.getBody(),
RemotingHelper.DEFAULT_CHARSET);
System.out.println("收到消息:"+body+"，开始扣减库存："+orderId);
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
});
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
});
defaultMQPushConsumer.start();
System.in.read();
}
}Three states of RocketMQ transaction message
-
ROLLBACK_MESSAGE: rollback transaction
-
COMMIT_MESSAGE: Commit the transaction
-
UNKNOW: The broker will regularly check back the status of the Producer message until it succeeds or fails completely.
- When the executeLocalTransaction method returns ROLLBACK_MESSAGE, it means that the transaction is rolled back directly;
- When COMMIT_MESSAGE is returned, the transaction is committed;
- When UNKNOW is returned, Broker will check back checkLocalTransaction after a period of time, and perform transaction operations (rollback or commit) according to the return status of checkLocalTransaction. For example, in the example, when ROLLBACK_MESSAGE is returned, the consumer will not receive the message and will not call Checkback function, when COMMIT_MESSAGE is returned, the transaction is committed, and the consumer receives the message. When UNKNOW is returned, the checkback function will be called after a period of time, and the status of submission or rollback will be returned according to the status judgment. The message returning the submission status will be Consumer consumption, so at this time the consumer can consume part of the message;