[Yuan Chuang Conference Preview] 1024 Programmers’ Day (two days before), meet at the Open Source China office, let’s talk about AI! >>>

1 background

Before describing the concept of distributed transactions, let us first review some concepts related to transactions.

1.1 Basic concepts of transactions

It is a program execution unit, in which all operations are either successfully executed or all executions fail. Only half of the operations succeed and the other half fail. For example, if a transaction code performs two database update operations, then these two database operations will either be executed successfully or be rolled back.

1.2 Basic characteristics of transactions

We know that transactions have four very important characteristics, which we often call (ACID).

Atomicity: All operations in a transaction are either completely completed or not completed, and will not end at any intermediate stage. If an error occurs during the execution of the transaction, it will be restored (Rollback) to the state before the transaction started, as if the transaction had never been executed.
Consistency: The integrity of the database is not compromised before the transaction begins and after the transaction ends. This means that the data written must fully comply with all preset rules, including the accuracy and concatenation of the data, and that the subsequent database can spontaneously complete the predetermined work.
Isolation: The database's ability to allow multiple concurrent transactions to read, write and modify its data at the same time. Isolation can prevent data inconsistency due to cross execution when multiple transactions are executed concurrently. Transaction isolation is divided into different levels, including read uncommitted, read committed, repeatable read and serializable.
Durability: After the transaction is completed, the modifications to the data are permanent and will not be lost even if the system fails.

2 Distributed transactions

In fact, the concept of distributed transactions is essentially the same as that of database transactions. Since it is a transaction, it needs to meet the basic characteristics of transactions (ACID). However, the expression of distributed transactions is very different from that of local transactions. .

In the era of local transactions, if we need to operate multiple records of the database at the same time, and these operations can be put into a transaction, then we can achieve it through the transaction mechanism provided by the database.

With the advancement of microservice architecture, a local logical execution unit has been split into multiple independent microservices, and these microservices operate different databases and tables respectively.

For example, when assigning an individual transportation task, you need to generate requirements, plans, and tasks at the same time, and also call inquiry services and insurance services. Then, once any service exception occurs, transactional problems will occur. Although our current logic can be invalidated by operations, what about after automation in the future?

Distributed transactions are designed to solve the data consistency problem between different nodes in microservice architecture (all forms are distributed systems). This consistency problem essentially solves the problem that traditional transactions need to solve, that is, if a request is in multiple microservice call chains, the data processing of all services will either succeed or be rolled back. Of course, the form of distributed transaction problems may be quite different from traditional transactions, but the essence of the problems is the same, and they all require solving the problem of data consistency.

There are many ways to implement distributed transactions, the most representative of which is the XA distributed transaction protocol proposed by Oracle Tuxedo system. The XA protocol includes two implementations: two-phase commit (2PC) and three-phase commit (3PC). Next, we will introduce the principles of these two implementation methods respectively.

3 Two-phase submission (2PC)

Two-phase commit is also called 2PC (two-phase commit protocol). 2PC is a very classic strongly consistent and centralized atomic commit protocol. The centralization mentioned here means that there are two roles in the protocol: one is the distributed transaction coordinator (coordinator) and N participants (participants).

3.1 2PC operating principle

Two-stage submission, as the name suggests, involves two stages of submission: the first stage, the preparation stage (voting stage); the second stage, the submission stage (execution stage).

3.1.1 Prepare phase

The initiator of the distributed transaction sends a request to the distributed transaction coordinator (Coordinator, also called transaction management TransactionManager).
The Coordinator sends transaction preprocessing requests to Participant A and Participant B respectively, called Prepare, and some data are also called "Vote Request".
At this time, these participant nodes will generally open local database transactions and then start executing database local transactions. Each database participant executes transactions locally and writes local Undo/Redo logs (Undo logs record data before modification. Used for database rollback, the Redo log records modified data and is used to write the data file after committing the transaction), but after the execution is completed, the local transaction of the database will not be submitted immediately, but the "Vote Commit" will be performed to the Coordinator first. Feedback and inform the processing results.
If all participants give "Vote Commit" feedback to the coordinator, then the process enters the second stage.

3.1.2 Commit phase

1) If all participants report success, the coordinator will send a "global commit confirmation notification (global_commit)" to all participants, and the participant Participant will complete the submission of its own local database transaction and reply with the submission result. ack" message to the coordinator Coordinator, and then the coordinator Coordinator will return the result of the completion of distributed transaction processing to the caller. If any participant returns failure, the transaction is rolled back.

2) If the participant feeds back the "Vote_Abort" message to the coordinator, a failure message is returned. At this time, the distributed transaction coordinator Coordinator will initiate a transaction rollback message ("global_rollback") to all participants. At this time, each participant will roll back the local transaction, release resources, and send "ack" to the coordinator. "Confirm the message, and the coordinator will return the result of the distributed transaction processing failure to the caller.

The above is the basic process of two-phase submission. So according to this two-phase submission protocol, can the data consistency problem of distributed systems be solved?

3.2 Problems with 2PC

In fact, 2PC only solves the data consistency problem of a transaction in a distributed system across multiple services by adding the role of the transaction coordinator (Coordinator) through a two-stage processing process.

The following points are some of the problems encountered in the XA-two-phase commit protocol:

Performance issues: All participant nodes in 2PC are transaction blocking. When a communication timeout occurs on a participant node, the other participants will be passively blocked and occupy resources that cannot be released.
Coordinator single point of failure problem: Due to heavy reliance on the coordinator, once the coordinator fails, the participants are still in a locked resource state and cannot complete the transaction commit operation. Although a coordinator will be re-elected after the coordinator fails, it cannot solve the problem of participants being blocked due to the failure of the previous coordinator.
Network interruption leads to split brain: In the second phase, after the coordinator sends the commit command to the participants, once the network jitter occurs at this time, some participants receive the commit request and execute it, but other participants who have not received the commit request cannot Execute transaction commit. This leads to data inconsistency in the entire distributed system.

4 Three-stage submission (3PC)

Three-stage submission, also known as 3PC, adds the CanCommit stage on the basis of 2PC and introduces a timeout mechanism. Once the transaction participant fails to receive the commit request from the coordinator, it will automatically perform a local commit, which relatively effectively solves the problem of the coordinator's single point of failure.

4.1 3PC operating principle

4.1.1 CanCommit phase

The coordinator issues CanCommit to the participants and performs transaction inquiry operations. Only after all participants respond yes can the next stage be entered. (The table is not locked at this stage, unlike 2pc which starts locking the table in the first stage. The first stage of 3pc is to avoid locking the table on the premise that individual participants do not have the ability to commit transactions.) In simple terms Just check the health of your own state.
If any participant responds No, the entire distributed transaction will be interrupted, and the coordinator will send an "abort" request to all participants.

4.1.2 PreCommit phase

In phase one, if all participants return Yes, then the PreCommit phase will be entered for transaction pre-commit. At this time, the distributed transaction coordinator will send a PreCommit request to all participants. After receiving it, the participants will start to perform transaction operations and record Undo and Redo information into the transaction log. After the participant completes the transaction operation (it is in the state of uncommitted transaction at this time), it will feedback "Ack" to the coordinator to indicate that it is ready to submit, and is waiting for the coordinator's next instruction.
The result of any participant's feedback is No, or the coordinator times out while waiting for feedback from the participant node (only the coordinator can time out in 2PC, and there is no timeout mechanism for participants). The entire distributed transaction will be aborted, and the coordinator will send an "abort" request to all participants.

4.1.3 DoCommit phase

In Phase 2, if all participants can perform PreCommit submission, the coordinator will change from "PreCommit Status" -> "Commit Status". Then send a "doCommit" request to all participants. After receiving the commit request, the participant performs the transaction submission operation and feeds back the "Ack" message to the coordinator. The coordinator completes the transaction after receiving the Ack message from all participants.
Similarly, if a participant node fails to complete the PreCommit feedback or the feedback times out, the coordinator will send abort requests to all participant nodes, thus interrupting the transaction.

Compared with 2PC, 3PC sets a timeout for both the coordinator and the participant, which solves the problem that the participant cannot communicate with the coordinator node for a long time (the coordinator hangs up). The problem of releasing resources is because the participants themselves have a timeout mechanism and will automatically perform local commits to release resources after timeout. This mechanism also reduces the blocking time and scope of the entire transaction.
In addition, through the design of the three stages of CanCommit, PreCommit, and DoCommit, compared with 2PC, an additional buffer stage is set up to ensure that the status of each participating node is consistent before the final submission stage.

Disadvantages of 3PC:

While 3PC removes blocking, it also introduces a new problem, that is, after the participant receives the precommit message, if a network partition occurs, the node where the coordinator is located and the participant cannot communicate normally. In this case , the participant will still commit the transaction, which will inevitably lead to data inconsistency.

5 Compensation Matters (TCC)

TCC, like 2PC and 3PC, is just an implementation solution for distributed transactions.

5.1 TCC principle:

TCC (Try-Confirm-Cancel) is also called compensation transaction. The core idea is: "For each operation, a corresponding confirmation and compensation (revocation operation) must be registered." It is divided into three operations:

Try stage: mainly testing the business system and reserving resources, such as freezing inventory.
Confirm stage: confirm execution of business operations.
Cancel stage: Cancel the execution of business operations.

The processing flow of TCC transactions is similar to that of 2PC two-stage submission, but 2PC is usually at the cross-database DB level, and TCC is essentially an application-level 2PC, which needs to be implemented through business logic. The advantage of this distributed transaction implementation is that it allows applications to define the granularity of database operations, making it possible to reduce lock conflicts and improve throughput.

The disadvantage is that it is very intrusive to the application. Each branch of the business logic needs to implement the three operations of try, confirm, and cancel. In addition, its implementation is relatively difficult, and different rollback strategies need to be implemented according to different failure reasons such as network status and system failures. In order to meet the consistency requirements, the confirm and cancel interfaces must also be idempotent.

The specific schematic diagram of TCC is as follows:

5.2 Notes:

1. Business operations are completed in two stages:

Before accessing TCC, business operations can be completed in only one step. However, after accessing TCC, it is necessary to consider how to divide it into two phases. The check and reservation of resources are carried out in the one-stage Try operation, and the actual operation is The execution of business operations is performed in the second-stage Confirm operation;
the TCC service must ensure that after the first-stage Try operation is successful, the second-stage Confirm operation must be successful;

2. Allow empty rollback;

When the transaction coordinator calls the one-stage Try operation of the TCC service, a network timeout may occur due to packet loss. At this time, the transaction coordinator will trigger a two-stage rollback and call the Cancel operation of the TCC service
; A Cancel request is received when a Try request is made. This scenario is called an empty rollback; the TCC service should allow the execution of an empty rollback when implemented;

3. Anti-suspension control;

When the transaction coordinator calls the one-stage Try operation of the TCC service, a timeout may occur due to network congestion. At this time, the transaction coordinator will trigger a two-stage rollback and call the Cancel operation of the TCC service; after that, congestion will occur in The first-stage Try packet on the network is received by the TCC service, and the second-stage Cancel request is executed before the first-stage Try request.
When implementing the TCC service, users should allow empty rollback, but refuse to execute empty rollback. The next phase of the Try request comes;

4. Idempotent control:

Whether it is the retransmission of network data packets or the compensation execution of abnormal transactions, the Try, Confirm or Cancel operations of the TCC service will be repeatedly executed. When implementing the TCC service, users need to consider idempotent control, that is, the execution of Try, Confirm, and Cancel The business results are the same if executed once and multiple times;

5. Business data visibility control;

The first-stage Try operation of the TCC service will reserve resources. Before the second-stage operation is executed, if other transactions need to read the reserved resource data, how to display the business data in the intermediate state to the user requires the business to Think clearly when implementing; the usual design principle is "it is better not to display at all or less than to display too much or wrongly";

6. Concurrent access control of business data;

After the resources are reserved in the first-stage Try operation of the TCC service, the reserved resources will not be released until the second-stage operation is executed; if other distributed transactions modify these business resources at this time, distributed transaction concurrency problems will occur;
When implementing TCC services, users need to consider the concurrency control of business data and try to minimize the logical lock granularity to maximize the concurrency of distributed transactions;

6 Hmily

Hmily (How much I love you)
high-performance distributed transaction tcc open source framework. Developed based on Java language (JDK1.8), it supports dubbo, springcloud, motan and other RPC frameworks for distributed transactions.

Frame properties

Support nested transactions (Nested transaction support).
Using the disruptor framework for asynchronous reading and writing of transaction logs has no difference in performance from the RPC framework.
Supports SpringBoot-starter project startup and is easy to use.
RPC framework supports: dubbo, motan, springcloud.
Local transaction storage support: redis, mongodb, zookeeper, file, mysql.
Transaction log serialization support: java, hessian, kryo, protostuff.
It adopts Aspect AOP aspect idea to integrate seamlessly with Spring and naturally supports clustering.
It has a built-in classic distributed transaction scenario demo project and a swagger-ui visual interface for quick experience.

6.1 Hmily principle and flow chart

Schematic:

flow chart:

7 References

https://dromara.org/website/zh-cn/docs/hmily/index.html
https://houbb.github.io/2018/10/30/hmily
https://developer.aliyun.com/article/609854
https://blog.csdn.net/bjweimengshu/article/details/86698036
https://blog.csdn.net/u014296316/article/details/90185589

Author: JD Logistics Song Le

Source: JD Cloud Developer Community Ziyuanqishuo Tech Please indicate the source when reprinting

A brief discussion on distributed transactions and solutions | JD Logistics Technology Team