Distributed transaction interview questions

1. Business introduction

A transaction is a program execution unit (unit) that operates on a data item in the database.

Transactions should have four attributes: atomicity, consistency, isolation, and durability. These four properties are often called ACID properties.

1.1. Glossary of terms

• Transaction: A transaction is a reliable and independent unit of work composed of a set of operations. The transaction has ACID characteristics, namely atomicity, consistency, isolation and durability.
• Local transactions: When a transaction is managed locally by the resource manager it is called a local transaction. The advantage of local transactions is that it supports strict ACID characteristics, is efficient and reliable, the state can be maintained only in the resource manager, and the application programming model is simple. However, local transactions do not have the processing capability of distributed transactions, and the smallest unit of isolation is limited by resource managers.
• Global transaction: When a transaction is globally managed by the global transaction manager, it becomes a global transaction. The transaction manager is responsible for managing the global transaction status and participating resources, and coordinates the consistent commit and rollback of resources.
• TX protocol: the interface between the application or the application server and the transaction manager.
• XA protocol: the interface between the global transaction manager and the resource manager. XA is a distributed transaction specification proposed by the X/Open organization. This specification mainly defines the interface between the global transaction manager and the local resource manager. Mainstream database products all implement the XA interface. The XA interface is a bidirectional system interface that serves as a communication bridge between the transaction manager and multiple resource managers. The reason why XA is needed is that theoretically speaking, two machines cannot achieve a consistent state in a distributed system, so a single point for coordination is introduced. Transactions managed and coordinated by the global transaction manager can span multiple resources and processes. The global transaction manager generally uses the XA two-phase protocol to interact with the database.
• AP: Application program, which can be understood as a program using DTP (Data Tools Platform).
• RM: Resource manager, which can be a DBMS or message server management system. Applications control resources through the resource manager. The resources must implement the interface defined by XA. The resource manager is responsible for controlling and managing actual resources.
• TM: Transaction Manager, responsible for coordinating and managing transactions, providing AP programming interfaces and managing resource managers. The transaction manager controls global transactions, manages transaction life cycles, and coordinates resources.
• Two-Phase Commit Protocol: XA’s mechanism for coordinating multiple resources in a global transaction. A two-stage submission solution is adopted between TM and RM to solve the consistency problem. Two-node submission requires a coordinator (TM) to control the operation results of all participating (RM) nodes and guide these nodes whether final submission is required. The limitations of two-phase submission are the protocol cost, the persistence cost in the preparation phase, the persistence cost of the global transaction state, and the vulnerability caused by multiple potential failure points. After preparation, failures before submission cause a series of isolation and recovery problems.
• BASE theory: BA refers to basic business availability and supports partition failure. S represents a flexible state, which allows for desynchronization in a short period of time. E represents eventual consistency. The data is ultimately consistent, but is inconsistent in real time. Atomicity and durability must be fundamentally guaranteed. In order to meet the needs of availability, performance and service degradation, only consistency and isolation requirements must be reduced.
• CAP theorem: For a shared data system, you can only have at most two of the CAPs at the same time. Any two have their own applicable scenarios. Real business systems are usually a mixture of ACID and CAP. The most important thing in a distributed system is to meet business needs, rather than pursuing highly abstract and absolute system characteristics. C means consistency, that is, the data seen by all users is the same. A stands for availability, which means that an available copy of the data can always be found. P represents partition fault tolerance, which can tolerate failures such as network outages.

1.2. The difference between distributed transactions and distributed locks:

Distributed locks solve the problem of distributed resource preemption; distributed transactions and local transactions solve the problem of process submission.

1.3. Four characteristics of transactions:

Atomic

• A transaction must be an atomic operation sequence unit. During one execution, all operations included in the transaction will either be executed successfully or not executed at all. If any one of them fails, the entire transaction will be rolled back. Only if all operations are executed successfully, the entire transaction will be rolled back. The business is successful.

Consistency

• The execution of a transaction cannot destroy the integrity and consistency of the database data. The database must be in a consistent state before and after the transaction is executed.

IsolationIsolation

• In a concurrent environment, concurrent transactions are isolated from each other, and the execution of one transaction cannot be interfered with by other transactions. That is, when different transactions concurrently manipulate the same data, each transaction has its own complete data space. That is, the operations and data used within a transaction are isolated from other concurrent transactions, and transactions executed concurrently cannot interfere with each other. .

• 4 transaction isolation levels in SQL:

  (1) Read uncommitted

允许脏读。如果一个事务正在处理某一数据，并对其进行了更新，但同时尚未完成事务，因此事务没有提交，与此同时，允许另一个事务也能够访问该数据。例如A将变量n从0累加到10才提交事务，此时B可能读到n变量从0到10之间的所有中间值。

(2) Read submitted

允许不可重复读。只允许读到已经提交的数据。即事务A在将n从0累加到10的过程中，B无法看到n的中间值，之中只能看到10。同时有事务C进行从10到20的累加，此时B在同一个事务内再次读时，读到的是20。

(3) Repeatable reading

允许幻读。保证在事务处理过程中，多次读取同一个数据时，其值都和事务开始时刻时是一致的。禁止脏读、不可重复读。幻读即同样的事务操作，在前后两个时间段内执行对同一个数据项的读取，可能出现不一致的结果。保证B在同一个事务内，多次读取n的值，读到的都是初始值0。幻读，就是不同事务，读到的n的数据可能是0，可能10，可能是20

(4) Serialization

 最严格的事务，要求所有事务被串行执行，不能并发执行。

If there is no concurrency control on transactions, let’s take a look at the abnormal situations that may occur in database concurrent operations.

• One type of lost update: Two things read the same data, one modifies field 1 and the other modifies field 2, and the one submitted later restores the field that was submitted first.
• Type II lost update: Two things read the same data, both modify the same field, and the one submitted later overwrites the modification submitted first.
• Dirty read: An uncommitted value is read. If the transaction is rolled back, a dirty read will occur.
• Non-repeatable read: Between two queries, the content of the data was modified by another transaction, resulting in content inconsistency.
• Phantom read: Between two queries, records are inserted or deleted by another transaction, resulting in inconsistency in the result set.

DurabilityPersistence

• Durability: Durability, also known as permanence, means that once a transaction is submitted, its state changes to the corresponding data in the database should be permanent.

• Even if a system crash or machine downtime occurs, as long as the database can be restarted, it can be restored to the state when the transaction ended successfully.

1.4. MySQL local transaction implementation plan

In most scenarios, our application only needs to operate a single database, and the transaction in this case is called a local transaction (Local Transaction). The ACID properties of local transactions are directly supported by the database.

Students who have learned about MySQL transactions will know that in order to achieve local transactions, MySQL has done a lot of work, such as rollback logs, redo logs, MVCC, read-write locks, etc.

Transaction implementation principle of MySQL database

Taking MySQL's InnoDB (InnoDB is a storage engine of MySQL) as an example, introduce the transaction implementation principle of a single database.

InnoDB is the ACID feature of transactions guaranteed by logs and locks, as follows:

(1) Through the database lock mechanism, the isolation of transactions is guaranteed;

(2) Through the Redo Log (redo log), the durability of the transaction is guaranteed;

(3) Guarantee the atomicity of transactions through Undo Log (undo log);

(4) Ensure transaction consistency through Undo Log;

How does Undo Log ensure the atomicity of transactions?

The specific method is: before operating any data, first back up the data to a place (the place where the data backup is stored is called Undo Log), and then modify the data. If an error occurs or the user executes a Rollback statement, the system can use the backup in the Undo Log to restore the data to the state before the transaction started.

How does Redo Log ensure the durability of transactions?

The specific method is: Redo Log records the backup of new data (opposite to Undo Log). Before the transaction is committed, you only need to persist the Redo Log, and there is no need to persist the data. When the system crashes, although the data is not persisted, the Redo Log is persisted. The system can restore all data to the state before the crash based on the contents of the Redo Log.

1.5. Dirty read, phantom read, non-repeatable read

When multiple transactions operate concurrently, the following three problems will occur in the database: dirty reads, phantom reads, and non-repeatable reads.

1.5.1. Dirty Read

Transaction A reads the uncommitted data of transaction B:

Before transaction A reads the data and transaction B modifies the data before submitting the data, transaction A reads the data again and will read the data that transaction B has modified. If transaction B rolls back or modifies the data again at this time, Then commit, the data read by transaction A is dirty data, this situation is called dirty read (Dirty Read).

1.5.2. Phantom Read

When transaction A performs a range query, transaction B adds new records that meet the range conditions. When transaction A performs a range query based on the conditions again, it will find new records that meet the conditions submitted in transaction B (Phantom) Row).

1.5.3. Unrepeatable Read

After transaction A reads some data, it reads the data again and finds that the data read has been changed or deleted in transaction B.

The difference between phantom reading and non-repeatability:

Phantom read: In the same transaction, under the same conditions, the number of records queried twice is different;
Non-repeatable read: In the same transaction, under the same conditions, the data queried twice is different;

1.6. Transaction isolation level

In order to solve the problems caused by transaction concurrency in the database, in the standard SQL specification, four transaction isolation levels are defined. Each level specifies the modifications made in a transaction and which ones are visible within and between transactions. , which are invisible.

Lower isolation levels generally support higher concurrency and have lower system overhead.

Control the isolation level of transactions by modifying the MySQL system parameters. In MySQL8, this parameter is transaction_isolation, and in MySQL5, this parameter is tx_isolation:

MySQL8：
-- 查看系统隔离级别：
SELECT @@global.transaction_isolation;

-- 查看当前会话隔离级别
SELECT @@transaction_isolation;

-- 设置当前会话事务隔离级别
SET SESSION TRANSACTION ISOLATION LEVEL SERIALIZABLE;

-- 设置全局事务隔离级别
SET GLOBAL TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;

Four isolation levels of transactions:

Uncommitted read (READ UNCOMMITTED): All transactions can see uncommitted modifications of other transactions. Generally rarely used;

READ COMMITTED: Oracle's default isolation level, transactions can only see each other's submitted changes;

Repeatable read (REPEATABLE READ): MySQL default isolation level, multiple queries in the same transaction will see the same data rows; non-repeatable read can be solved, but phantom reads may occur;

SERIALIZABLE: The highest isolation level, serial execution of transactions. After the previous transaction is executed, subsequent transactions will be executed. Every piece of data read will be locked, which will cause a lot of timeouts and lock contention problems;

2. Basic concepts of distributed transactions

2.1. Transaction complexity in distributed environment

Diversity on the storage side.

The first is the diversity of storage. In the case of local transactions, all data will fall into the same DB. However, in the case of distributed transactions, the data may fall into multiple DBs, or may fall into Redis, MQ, etc.

Storage end diversity, as shown in the figure below:

Scalability of transaction links

In the case of local transactions, usually all transaction-related business operations will be encapsulated into a Service method. In a distributed situation, the request link is extended and elongated, and an operation is split into multiple services. They appear linear or mesh-like, and rely on network communication to form a whole. In this case, matters undoubtedly become more complicated.

Transaction link scalability, as shown in the figure below:

Based on the above two complexities, it is unrealistic to expect a unified distributed transaction solution that can satisfy various storage media and various complex links in an almost non-invasive way like local transactions.

At least, at present, there is no very mature solution. Therefore, under normal circumstances, under distributed conditions, transactions will be split and resolved, and different solutions will be adopted according to different situations.

2.2. What is a distributed transaction?

For distributed systems, it is necessary to ensure data consistency in the distributed system, ensure that the data is always consistent in the subsystem, and avoid business problems. In a distributed system, pairs of numbers either succeed together or fail together, and must be a holistic transaction.

Distributed transactions mean that transaction participants, transaction-supporting servers, resource servers, and transaction managers are located on different nodes in different distributed systems.

Simply put, a large operation in a distributed system consists of different small operations. These small operations are distributed on different service nodes and belong to different applications. Distributed transactions need to ensure that all of these small operations succeed. Or all else fails.

For example: On an e-commerce website, when a user places an order for a product, he or she needs to create an order data in the order table, and at the same time modify the remaining inventory quantity of the current product in the inventory table. There are two steps: one to add and one to modify. We must ensure that these two steps must succeed or fail at the same time, otherwise business problems will occur.

When implementing any transaction mechanism, the ACID characteristics of transactions should be considered, including: local transactions and distributed transactions. For distributed transactions, even if they cannot all be well satisfied, we must consider the extent to which they are supported.

Typical distributed transaction scenario:

Cross-database transaction
Cross-database transaction means that a certain function of an application needs to operate multiple libraries, and different business data are stored in different libraries. The author has seen a relatively complex business in which 9 libraries were operated at the same time.

The following figure demonstrates a service operating two libraries at the same time:

Splitting databases and tables
Usually when a database has a large data volume or the expected future data volume is relatively large, it will be split horizontally, that is, into databases and tables.

As shown in the figure below, database B is split into 2 libraries:

For the case of sub-database and sub-table, developers generally use some database middleware to reduce the complexity of SQL operations.

For example, for sql: insert into user(id,name) values ​​(1, "tianshouzhi"), (2, "wangxiaoxiao"). This SQL is the syntax for operating a single database. In the case of a single database, transaction consistency can be guaranteed.

However, due to the sub-database and table division, the developer hopes to insert record No. 1 into sub-database 1 and record No. 2 into sub-database 2. Therefore, the database middleware needs to rewrite it into two SQL statements and insert them into two different sub-databases respectively. At this time, it is necessary to ensure that both libraries succeed or fail. Therefore, basically all database middleware faces the problem Distributed transaction issues.

Micro-service
micro-service architecture is currently a relatively popular concept. For example, in the case mentioned by the author above, an application operates 9 libraries at the same time. The business logic of such an application must be very complex, which is a great challenge for developers. It should be split into different independent services to simplify the business logic. . After splitting, independent services make remote calls through the RPC framework to communicate with each other. The following figure demonstrates an architecture in which three services call each other:

Service A needs to directly operate the database to complete a certain function, and needs to call Service B and Service C at the same time, and Service B operates two databases at the same time, and Service C also operates a library. It is necessary to ensure that these cross-service operations on multiple databases either succeed or fail. In fact, this may be the most typical distributed transaction scenario.

Distributed transaction implementation solutions must consider performance issues. If performance is severely degraded in order to strictly ensure ACID characteristics, then it is unacceptable for some businesses that require fast response.

3. CAP theorem

3.1. Theoretical basis of distributed transactions

The four major characteristics of database transaction ACID cannot meet the actual needs of distributed transactions. At this time, some new experts have proposed some new theories.

3.2. CAP theorem

The CAP theorem was proposed by Professor Eric Brewer of the University of California, Berkeley. He pointed out that WEB services cannot satisfy the following three attributes at the same time:

• Consistency: The client knows that a series of operations will happen at the same time (take effect)
• Availability: Every operation must end with a predictable response
• Partition tolerance: Even if a single component becomes unavailable, the operation can still be completed

Specifically, in a distributed system, a Web application can only support the above two attributes at most at the same time. Therefore, designers must choose between consistency and usability.

What Professor Eric Brewer proposed in July 2000 was just a conjecture. Two years later, Seth Gilbert and Nancy Lynch of MIT theoretically proved the CAP theory, and a distributed system can only satisfy the requirements of CAP at most. 2 items. After that, CAP theory officially became a recognized theorem in the field of distributed computing.

Therefore, the CAP theorem is applicable to distributed systems so far!

The consistency, availability, and partition tolerance of CAP are as follows:

consistency

Data consistency refers to "all nodes see the same data at the same time", that is, after the update operation is successful and returned to the client, the data of all nodes at the same time is completely consistent, and no intermediate state can exist.

In a distributed environment, consistency refers to the ability of multiple copies to remain consistent. Under the requirement of consistency, when a system performs an update operation in a consistent state of data, it should ensure that the data of the system is still processed in a consistent state.

For example, for an e-commerce system user to place an order, operations such as inventory reduction, user fund account deduction, and point increase must be consistent after the user's order is completed. There cannot be a situation like the inventory has been reduced, but the user's capital account has not been deducted and the points have not been increased. If this occurs, it is considered inconsistent.

Data consistency is divided into strong consistency, weak consistency, and eventual consistency.

If it is indeed possible to ensure that the data seen by the client is consistent at all times as described above, then it is called strong consistency.
If intermediate states are allowed to exist and only require that the data is ultimately consistent after a period of time, it is called eventual consistency.
In addition, if partial data inconsistency is allowed, it is called weak consistency.

Availability

The services provided by the system must always be available, and results must be returned within a limited time for each user operation request.

Two measured dimensions:

(1) Within a limited time
, for a user's operation request, the system must be able to return the corresponding processing result within the specified time (response time). If this time range is exceeded, the system is considered unavailable. That is, the response time must be within a reasonable value so as not to disappoint users.

Just imagine, if an order operation takes 10 minutes to complete in order to ensure the consistency of distributed transactions, then users will obviously not be able to bear it.

(2) Return normal results
: The system is required to return a normal response result after completing the processing of the user's request. Normal response results can usually clearly reflect the result of processing the request, that is, success or failure, rather than a return result that confuses the user. For example, if a system error such as OutOfMemory is returned, the system is considered unavailable.

"Return result" is another very important indicator of usability. It requires the system to return a normal response result after processing the user request, regardless of whether the result is success or failure.

Partition tolerance

That is, when a distributed system encounters any network partition failure, it still needs to be able to ensure that it provides external services that meet the consistency and availability, unless the entire network environment fails.

Network partitioning refers to a distributed system in which different nodes are distributed in different subnetworks (computer rooms/remote networks). Due to some special reasons, there is a state of network disconnection between these subnetworks, but the status of each subnetwork The internal network is normal, causing the network environment of the entire system to be divided into several isolated areas. The joining and exiting of each node forming a distributed system can be regarded as a special network partition.

3.3. Application of CAP

Give up P

If you give up the partition fault tolerance, you give up the distribution and the scalability of the system

Give up A

If the availability is abandoned, when a network partition or other failure occurs, the affected service needs to wait for a certain period of time, and the policy service cannot be provided externally during this period, that is, it is unavailable

Give up C

If the consistency is abandoned (here refers to strong consistency), the system cannot guarantee the real-time consistency of the data. When the data reaches the final consistency, there is a time window, and the data is inconsistent within the time window.

For distributed systems, P cannot be given up, so architects usually make a trade-off between availability and consistency.

The CAP theory tells us that
many large websites and applications are currently deployed in a distributed manner, and data consistency in distributed scenarios has always been an important topic.

Based on the CAP theory, many systems must make trade-offs among these three at the beginning of design:

No distributed system can satisfy consistency, availability, and partition tolerance at the same time. It can only satisfy two at most. In most scenarios in the Internet field, strong consistency needs to be sacrificed in exchange for high system availability, and the system often only needs to ensure eventual consistency.

Q: Why can't both consistency and availability be guaranteed in a distributed system?

Answer: First of all, there is a premise. For distributed systems, partition fault tolerance is the most basic requirement, so basically we can only choose between consistency (C) and availability (A) when designing a distributed system. Trade-offs.

If consistency is guaranteed (C): For nodes N1 and N2, when writing data to N1, the operation on N2 must be suspended. Only when N1 synchronizes data to N2 can N2 read and write requests, and N2 is blocked. Requests submitted by the client during the pause operation will receive a failure or timeout. Obviously, this is contrary to usability.

If availability (A) is guaranteed: then the read and write operations of N2 cannot be suspended, but if N1 is writing data at the same time, this violates the consistency requirement.

3.4. CAP trade-off

Through the CAP theory, we know that the three characteristics of consistency, availability and partition tolerance cannot be met at the same time. So which one should we abandon?

For most large-scale Internet application scenarios, there are many hosts and scattered deployments, and the current cluster scale is getting larger and larger, so node failures and network failures are normal, and service availability must be guaranteed to reach N nines, that is, P and A are guaranteed, discard C (the next best thing is to ensure eventual consistency). Although it will affect the customer experience in some places, it is not serious enough to cause user flow.

For scenes involving money that cannot be compromised, C must guarantee. If the network fails, it is better to stop the service. This is the guarantee of CA, and P is discarded. It seems that there have been no less than 10 accidents in the domestic banking industry in the past few years, but the impact is not large, there are not many reports, and the general public knows little. Another option is to guarantee CP and discard A. For example, the network failure is only reading but not writing.

3.5. A and C in CAP and ACID are completely different.

The difference between A:

• The A in ACID refers to atomicity, which means that the transaction is regarded as an indivisible minimum unit of work. All operations in the transaction are either submitted successfully or rolled back after failure;
• The A in CAP refers to Availability, which refers to whether the entire cluster can still respond to client read and write requests after a part of the nodes in the cluster fails;

The difference between C:

• ACID consistency is about database rules, and the database always transitions from one consistent state to another consistent state;
• The consistency of CAP is to copy data between distributed multiple servers so that these servers have the same data. Due to network speed limitations, the time consumed by this replication on different servers is not fixed. The cluster organizes clients to view different data. Data that has not been synchronized on the node maintains a logical view, which is a concept of consistency in the distributed field;

In short:

Consistency in ACID refers to database integrity before and after transaction execution, while CAP consistency refers to the consistency of data on distributed nodes. Different backgrounds, no comparison

4. BASE theorem

CAP is a distributed system design theory, and BASE is an extension of the AP solution in the CAP theory. The methods and strategies we adopt for C are to ensure eventual consistency;

eBay's architect Dan Pritchett published an article on ACM based on his practical summary of large-scale distributed systems and proposed BASE theory. BASE theory is an extension of CAP theory. The core idea is that even if strong consistency (StrongConsistency, CAP) cannot be achieved, Consistency is strong consistency), but applications can achieve eventual consistency (Eventual Consitency) in a suitable way.

4.1. BASE theorem

BASE is an acronym for the three phrases Basically Available (basically available), Soft state (soft state) and Eventually consistent (final consistency). BASE evolved based on the CAP theorem. The core idea is that strong consistency cannot be achieved immediately, but each application can use appropriate methods according to its own business characteristics to make the system achieve final consistency.

1. Basically Available

基本可用是指分布式系统在出现不可预知的故障的时候，允许损失部分可用性，但不等于系统不可用。

(1) Loss in response time

当出现故障时，响应时间增加

(2) Functional loss

   当流量高峰期时，屏蔽一些功能的使用以保证系统稳定性（服务降级）

2. Soft state

It means that the data in the system is allowed to exist in an intermediate state, and it is believed that the existence of this intermediate state will not affect the overall availability of the system.

As opposed to the hard state, it means that the data in the system is allowed to have an intermediate state, and that the existence of the intermediate state will not affect the overall availability of the system, that is, the process of allowing the system to synchronize data between data copies of different nodes has a delay. hour.

3.Eventually consistent (eventual consistency)

It is emphasized that all data copies in the system can eventually reach a consistent state after a period of synchronization. Its essence is that the system needs to ensure that the final data can be consistent, but it does not need to ensure the strong consistency of the system data in real time.

Eventual consistency can be divided into the following types:

(1) Causal consistency (Causal consistency)
means that process A notifies process B after updating the data, then the range of the data by process B will be the latest value updated by process A.
(2) Read your writes:
After process A updates a piece of data, it can always access the latest value it has updated.
(3) Session consistency (Session consistency)
frames data consistency in the session, and achieves the consistency of reading and writing in a session. That is, after the update is performed, the client can always read the latest value of the data item in the same session
(4) Monotonic read consistency (Monotonic read consistency)
If a process reads a certain value of a data item from the system , then the system should not return an older value for any subsequent data access by the process.
(5) Monotoic write consistency:
A system needs to ensure that write operations from the same process are executed sequentially.
The BASE theory proposes to gain availability by sacrificing consistency, and allows data to be inconsistent for a period of time, but eventually reaches a consistent state.

4.2. Characteristics of BASE theory:

BASE theory is oriented to large-scale, highly available and scalable distributed systems, which is opposite to the ACID characteristics of traditional things.

It is completely different from the strong consistency model of ACID, but obtains availability by sacrificing strong consistency and allows data to be inconsistent for a period of time, but eventually reaches a consistent state.

But at the same time, in actual distributed scenarios, different business units and components have different requirements for data consistency. Therefore, in the specific distributed system architecture design process, ACID characteristics and BASE theory are often combined together.

4.3. The relationship between BASE theory and CAP

BASE theory is the result of the trade-off between consistency and availability in CAP. It is derived from the summary of distributed practices in large-scale Internet systems and is gradually evolved based on the CAP theorem. The core idea of ​​BASE theory is: even if strong consistency cannot be achieved, each application can adopt appropriate methods according to its own business characteristics to achieve final consistency of the system.

BASE theory is actually an extension and supplement to CAP theory, mainly to AP. The strong consistency of the data is sacrificed to ensure the availability of the data. Although there is intermediate loading, the data is ultimately consistent.

4.4. Differences and connections between ACID and BASE

ACID is a commonly used design concept in traditional databases, pursuing a strong consistency model. BASE supports large-scale distributed systems and proposes to obtain high availability by sacrificing strong consistency.

ACID and BASE represent two completely opposite design philosophies. In the scenario of distributed system design, system components have different consistency requirements, so ACID and BASE are used together.

6. Classification of distributed transactions

In a distributed scenario, multiple services serve one process at the same time. For example, in an e-commerce ordering scenario, payment service is required for payment, inventory service deducts inventory, order service generates orders, and logistics service updates logistics information. If a certain service fails to execute, or the request is lost due to network failure, data inconsistency may occur in the entire system.

The above scenario is a distributed consistency problem. Ultimately, the root cause of distributed consistency lies in the distributed operation of data, which causes local transactions that cannot guarantee the atomicity of the data.

There are two ways to solve the distributed consistency problem, one is distributed transactions, and the other is to try to avoid distributed transactions through business processes. Distributed transactions directly solve problems, while business avoidance actually solves the problem (solving the person who raised the problem). In fact, in real business scenarios, if business avoidance is not very troublesome, the most elegant solution is business avoidance.

Distributed transaction implementation solutions are divided into rigid transactions and flexible transactions in terms of type :

• The CP theory of rigid affairs satisfying CAP

• Flexible transactions satisfy BASE theory (basically available, eventually consistent)

6.1. Rigid affairs

Rigid transactions: Usually no business transformation, strong consistency, native support for rollback/isolation, low concurrency, suitable for short transactions.

Principle: Rigid transactions satisfy the CP theory of CAP

Rigid transactions refer to making distributed transactions have strong data consistency like local transactions. From the perspective of CAP, that is to say, reaching CP state.

Rigid transactions: XA protocol (2PC, JTA, JTS), 3PC, but due to synchronization blocking and low processing efficiency, they are not suitable for large website distributed scenarios.

6.2. Flexible transactions

Flexible transactions do not require strong consistency, but require final consistency, allowing intermediate states, which is the Base theory, in other words, the AP state.

Compared with rigid transactions, the characteristics of flexible transactions are: business transformation, ultimate consistency, implementation of compensation interface, implementation of resource locking interface, high concurrency, and suitable for long transactions.

Flexible transactions are divided into:

• Compensation type
• Asynchronous guaranteed
• Best effort notification type.

Flexible transactions: TCC/FMT, Saga (state machine mode, Aop mode), local transaction messages, message transactions (semi-message)

6.3. Rigid transactions: XA model, XA interface specification, XA implementation

6.3.1, XA model or X/Open DTP model

X/OPEN is an organization. The X/Open International Alliance Ltd. is a European foundation established to provide standards for the UNIX environment. Its main goal is to promote the development of open system protocols for UNIX languages, interfaces, networks, and applications. It also promotes application interoperability between different UNIX environments, as well as supports the Institute of Electrical and Electronics Engineers (IEEE) Portable Operating System Interface (POSIX) specification for UNIX.

X/Open DTP (Distributed Transaction Process) is a distributed transaction model. This model mainly uses two-phase commit (2PC - Two-Phase-Commit) to ensure the integrity of distributed transactions.

In the X/Open DTP (Distributed Transaction Process) model, there are three roles:

• ​ AP: Application, application. That is the business layer. Which operations belong to a transaction is defined by AP.

• TM: Transaction Manager, transaction manager. Receive AP transaction requests, manage global transactions, manage transaction branch status, coordinate RM processing, notify RM which operations belong to which global transactions and transaction branches, etc. This is also the core part of the entire transaction scheduling model.

• RM: Resource Manager, resource manager. Usually it is a database, but it can also be other resource managers, such as message queues (such as JMS data sources), file systems, etc.

XA divides the roles involved in the transaction into AP, RM, and TM.

AP, that is, application, is our business service.

RM refers to resource manager, i.e. DB, MQ, etc.

TM is the transaction manager.

AP operates TM by itself. When a transaction is needed, AP requests TM to initiate a transaction, and TM is responsible for the submission and rollback of the entire transaction.

The XA specification mainly defines the interface between the (global) transaction manager (Transaction Manager) and the (local) resource manager (Resource Manager). The XA interface is a bidirectional system interface that forms a communication bridge between the Transaction Manager and one or more Resource Managers.

The reason why XA needs to introduce a transaction manager is because, in a distributed system, theoretically (refer to Fischer et al.'s paper), two machines cannot theoretically reach a consistent state, and a single point needs to be introduced for coordination. The transaction manager controls global transactions, manages transaction lifecycle, and coordinates resources. The resource manager is responsible for controlling and managing the actual resources (such as databases or JMS queues)

6.3.2, XA specification

What is XA

In very official terms:

• The XA specification is a distributed transaction processing (DTP, Distributed Transaction Processing) standard defined by the X/Open organization.

• The XA specification describes the interface between the global transaction manager and the local resource manager. The purpose of the XA specification is to allow multiple resources (such as databases, application servers, message queues, etc.) to be accessed in the same transaction, so that ACID properties remain valid across applications.

• The XA specification uses a two-phase commit (2PC, Two-Phase Commit) protocol to ensure that all resources commit or roll back any particular transaction at the same time.

• The XA specification was proposed in the early 1990s. Currently, almost all mainstream databases provide support for the XA specification.

XA Specification is a distributed transaction processing specification proposed by X/OPEN. XA standardizes the communication interface between TM and RM, forming a two-way communication bridge between TM and multiple RMs, thus ensuring the four ACID characteristics under multiple database resources. Currently well-known databases, such as Oracle, DB2, mysql, etc., all implement the XA interface and can be used as RMs.

XA is a distributed transaction of the database with strong consistency. During the entire process, the data is locked. That is, during the entire process from prepare to commit and rollback, TM always holds the lock of the database. If anyone else wants to modify it, This piece of data in the database must wait for the lock to be released, and there is a risk of long transactions.

The following functions enable the transaction manager to operate on the resource manager:
1) xa_open, xa_close: establish and close the connection with the resource manager.
2) xa_start, xa_end: start and end a local transaction.
3) xa_prepare, xa_commit, xa_rollback: pre-submit, commit and roll back a local transaction.
4) xa_recover: Roll back a pre-committed transaction.
5) The functions starting with ax_ enable the resource manager to dynamically register in the transaction manager and operate on XID (TRANSACTION IDS).
6) ax_reg, ax_unreg; allow a resource manager to dynamically register or deregister in a TMS (TRANSACTION MANAGER SERVER).

The processing flow of each stage of XA

6.3.3, Implementation of XA protocol

2PC/3PC protocol

The Two-Phase Commit (2PC) protocol is a data consistency protocol defined by the XA specification.

The three-phase commit (3PC) protocol is an extension of the 2PC protocol.

Set

Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services under a microservice architecture. Seata will provide users with AT, TCC, SAGA and XA transaction modes

Before Seata was open sourced, the corresponding internal version of Seata had always played the role of distributed consistency middleware within the Alibaba economy, helping the economy to smoothly go through Double 11 in previous years and providing strong support to the business of each BU. The commercial product GTS has been sold on Alibaba Cloud and Financial Cloud successively

Jta specification

As a transaction specification on the Java platform, JTA (Java Transaction API) also defines support for XA transactions. In fact, JTA is modeled based on the XA architecture. In JTA, the transaction manager is abstracted as the javax.transaction.TransactionManager interface. And implemented through the underlying transaction service (ie JTS). Like many other Java specifications, JTA only defines interfaces, and specific implementations are provided by suppliers (such as J2EE vendors). Currently, JTA implementations mainly include the following: 1. JTA implementation provided by J2EE containers
( JBoss)
2. Independent JTA implementation: such as JOTM, Atomikos.

These implementations can be used in environments that do not use a J2EE application server to provide distributed transaction guarantees. Such as Tomcat, Jetty and ordinary java applications.

JTS specification

Transactions are an essential part of programming. Based on this, in order to standardize transaction development, Java has added specifications about transactions, namely JTA and JTS

JTA defines a set of interfaces, which stipulates several main roles: TransactionManager, UserTransaction, Transaction, XAResource, and defines the specifications that need to be followed between these roles, such as the delegation of Transaction to TransactionManager.

JTS is also a set of specifications. It is mentioned above that interaction between roles is required in JTA, so how should they interact? JTS is a specification that specifies the details of interaction.

Generally speaking, JTA agrees on the interfaces of program roles more from a framework perspective, while JTS agrees on the interfaces between program roles from a specific implementation perspective, and both perform their own duties.

Atomikos distributed transaction implementation

Atomikos has two well-known distributed business products:

TransactionEssentials: open source free product
ExtremeTransactions: commercial version, requires a fee.
The relationship between these two products is as shown in the figure below:

As you can see, the open source version supports JTA/XA, JDBC, and JMS transactions.

atomikos also supports integration with spring transactions.

The top-level abstraction of the spring transaction manager is the PlatformTransactionManager interface, which provides an important implementation class:

DataSourceTransactionManager: used to implement local transactions
JTATransactionManager: used to implement distributed transactions
Obviously, what we need to configure here is JTATransactionManager.

public class JTAService {
    
      
    @Autowired   
    private UserMapper userMapper;//操作db_user库   
    @Autowired  
    private AccountMapper accountMapper;//操作db_account库  

    @Transactional   
    public void insert() {
    
        
        User user = new User();     
        user.setName("wangxiaoxiao");     
        userMapper.insert(user);  
        //模拟异常，spring回滚后，db_user库中user表中也不会插入记录     
        Account account = new Account();     
        account.setUserId(user.getId());    
        account.setMoney(123456789);    
        accountMapper.insert(account); 
    }
}

Main limitations of XA

All data sources must be obtained, and the data sources must also support the XA protocol. Currently, only the InnoDB storage engine in MySQL supports the XA protocol.
The performance is relatively poor. All involved data must be locked to ensure strong consistency and long transactions will occur.

Seata AT mode

Seata AT mode is an enhanced 2pc mode.

AT mode: evolution of the two-phase commit protocol, without always locking the table

Phase 1: Business data and rollback log records are committed in the same local transaction, releasing local locks and connection resources
Phase 2: Committing is asynchronous and completed very quickly. Or rollback reverse compensation through one-stage rollback log

LCN（2pc）

TX-LCN , official document, github, more than 3,000 stars, since the framework is compatible with LCN (2pc), TCC, and TXC transaction modes after 5.0, in order to distinguish the LCN mode, the LCN distributed transaction is hereby renamed as TX-LCN distribution business framework.

TX-LCN is positioned as a transaction coordination framework. The framework itself does not produce transactions, but is the coordinator of local transactions, so as to achieve the effect of transaction consistency.

TX-LCN mainly has two modules, Tx-Client(TC) and Tx-Manager™.

TM (Tx-Manager): It is an independent service. It is the controller of distributed transactions. It coordinates the submission and rollback of distributed transactions.
TC (Tx-Client): Integrated by the business system, the transaction initiator and participants are all controlled by TxClient. terminal control

6.4, 2PC (standard XA model)

2PC is Two-Phase Commit, two-phase submission.

6.4.1. Detailed explanation: two stages

It is widely used in the database field in order to ensure that all nodes based on the distributed architecture can maintain atomicity and consistency when processing transactions. Most relational databases are based on 2PC to complete distributed transaction processing.

As the name suggests, 2PC is divided into two stages of processing, stage one: submitting transaction request, stage two: executing transaction submission;

If phase one times out or an exception occurs, 2PC phase two: interrupt the transaction

Phase 1: Submit transaction request

Business inquiries. The coordinator sends transaction content to all participants, asks whether the commit operation can be performed, and begins to wait for each participant to respond;
execute the transaction. Each participant node performs transaction operations and counts Undo and Redo operations into the local transaction log;
each participant feeds back the response to the transaction query to the coordinator. Returns Yes if executed successfully, otherwise returns No.

Phase 2: Execute transaction submission

The coordinator decides in phase two whether to finally execute the transaction commit operation. This stage includes two situations:

Execute transaction commit

All participants reply Yes, then the transaction commit is executed.

Send a commit request. The coordinator sends a Commit request to all participants;
the transaction is committed. After receiving the Commit request, the participant will formally perform the transaction submission operation, and after completing the submission operation, release the resources occupied during the entire transaction execution; feedback the
transaction submission results. After the participant completes the transaction submission, the writing coordinator sends an Ack message to confirm;
the transaction is completed. The coordinator completes the transaction after receiving the Ack from all participants.

Phase 2: Interrupting the transaction

Things will always be unexpected, when there is a participant who sends a No response to the coordinator, or waits for a timeout. The coordinator will abort the transaction whenever it cannot receive a Yes response from all participants.

Send a rollback request. The coordinator sends a Rollback request to all participants;
rollback. After receiving the request, the participant uses the local Undo information to perform the Rollback operation. And after the rollback is completed, the system resources occupied by the transaction are released;
the rollback results are fed back. After completing the rollback operation, the participant sends an Ack message to the coordinator;
interrupting the transaction. After the coordinator receives the rollback Ack message from all participants, the transaction is interrupted.

6.4.2, 2pc solves the problem of strong consistency of distributed data

As the name suggests, two-phase submission is divided into two stages when processing distributed transactions: voting (voting stage, sometimes called prepare stage) and commit stage.

There are two roles in 2pc, transaction coordinator (seata, atomikos, lcn) and transaction participants. Transaction participants usually refer to the application database.

6.4.3 Characteristics of 2PC second-stage submission

The 2PC solution is more suitable for single application

In the 2PC solution, there is a transaction manager role, which is responsible for coordinating transactions of multiple databases (resource managers). The transaction manager first asks each database if it is ready? If each database replies ok, then the transaction is officially submitted and operations are performed on each database; if any one of the databases replies not ok, then the transaction is rolled back.

The 2PC solution is more suitable for distributed transactions across multiple libraries in a single application, and because it relies heavily on the database level to handle complex transactions, the efficiency is very low, and it is definitely not suitable for high concurrency scenarios.

The 2PC solution is rarely used in practice. Generally speaking, if such an operation occurs across multiple libraries within a system, it is illegal. I can introduce to you, now microservices, a large system is divided into hundreds of services, dozens of services. Generally speaking, our regulations and specifications require that each service can only operate its own corresponding database.

If you want to operate the libraries corresponding to other services, direct connection to the libraries of other services is not allowed, which violates the specifications of the microservice architecture. If you cross-access and access hundreds of services at will, everything will be messed up. Such a set of services cannot If it cannot be managed or managed, it may happen that the data is corrected by others, or your own database is written down by others, etc.

If you want to operate the library of other people's services, you must do it by calling the interface of other services. Cross-access to other people's databases is absolutely not allowed.

2PC has obvious advantages and disadvantages:

The advantages are mainly reflected in the simple implementation principle;
there are many disadvantages:

• During the execution of the 2PC submission, all logic involved in the transaction operation is in a blocked state, that is to say, each participant is waiting for the response of other participants and cannot perform other operations;

• The coordinator is a single point. Once a problem occurs, other participants will not be able to release transaction resources or complete transaction operations;

• The data is inconsistent. When executing transaction submission, if a local network exception occurs after the coordinator sends a Commit request to all participants or the coordinator has not finished sending the Commit request, it will crash, resulting in only some participants receiving and executing the request. As a result, data inconsistency will occur in the entire system;

• keep. 2PC does not have a complete fault-tolerant mechanism. When a participant fails, the coordinator cannot quickly learn about the failure and can only strictly rely on the timeout setting to decide whether to further execute the commit or interrupt the transaction.

In fact, distributed transactions are a very complex matter. Two-phase commit only solves the problem that a transaction in a distributed system needs to span multiple service nodes by adding the role of the transaction coordinator (Coordinator) through a two-stage processing process. data consistency issues. However, considering abnormal situations, this process is not so impeccable.

Suppose that in the second phase, if the Coordinator hangs up after receiving the Partcipant's "Vote_Request" or there is an abnormality in the network, then the Partcipant node will always be in a state of local transaction suspension, thus occupying resources for a long time. Of course, this situation will only occur in extreme cases, but as a robust software system, the handling of exception cases is the real test of the correctness of the solution.

To sum up: Some problems encountered in the XA-two-phase commit protocol

Performance issues

From the process, we can see that its biggest disadvantage is that the nodes are blocked in the middle of its execution. Each node operating the database occupies database resources at this time. Only when all nodes are ready, the transaction coordinator will notify the global commit, and the resources will be released only after the participants commit local transactions. Such a process will take a long time and have a large impact on performance.

Coordinator single point of failure problem

The transaction coordinator is the core of the entire XA model. Once the transaction coordinator node hangs up, the participants will not receive notifications of submission or rollback, causing the participant nodes to always be in an intermediate state where the transaction cannot be completed.

Data inconsistency caused by missing messages

In the second stage, if a local network problem occurs and some transaction participants receive the commit message, while other transaction participants do not receive the commit message, it will lead to data inconsistency between nodes.

6.5、3PC

In view of the shortcomings of 2PC, researchers proposed 3PC, namely Three-Phase Commit.

As an improved version of 2PC, 3PC redivides the original two-stage process into three stages: CanCommit, PreCommit and do Commit.

6.5.1. Detailed explanation: three stages

Phase 1: CanCommit

Business inquiries. The coordinator sends a canCommit request containing the transaction content to all participants, asking whether transaction submission can be performed, and waits for a response;
each participant responds to the transaction inquiry. Under normal circumstances, if the participant thinks that the transaction can be executed smoothly, it returns Yes, otherwise it returns No.

Phase 2: PreCommit

At this stage, the coordinator will decide whether to execute the PreCommit operation of the transaction based on the feedback from the previous stage. There are two possibilities:

Execute transaction pre-commit

Send a pre-commit request. The coordinator issues a PreCommit request to all nodes and enters the prepared phase;

Transaction pre-commit. After receiving the PreCommit request, the participant will perform the transaction operation and write the Undo and Redo logs to the local transaction log;

Each participant successfully executes the transaction operation, and at the same time sends feedback to the coordinator in the form of an Ack response, and the colleagues wait for the final Commit or Abort instruction.

interrupt transaction

Add any participant to send a No response to the coordinator, or wait for a timeout. The coordinator can interrupt the transaction when it does not receive responses from all participants:

Send an interrupt request. The coordinator sends an Abort request to all participants;

Interrupt transaction. Whether it receives an Abort request from the coordinator or a timeout occurs while waiting for the coordinator's request, participants will interrupt the transaction;

Phase 3: doCommit

At this stage, the transaction will actually be committed, and there are two possibilities.

Execute commit

Send a commit request. If the coordinator receives Ack responses from all participants, it will transition from pre-commit to commit state and send doCommit requests to all participants;

Transaction commit. After receiving the doCommit request, the participant will formally perform the transaction submission operation and release the occupied resources after completing the submission operation;

Feedback transaction submission results. The participant will send an Ack message to the coordinator after completing the transaction submission;

Complete the transaction. After the coordinator receives the Ack messages from all participants, the transaction is completed.

interrupt transaction

At this stage, assuming that the coordinator in the normal state receives a No response from any participant, or does not receive a feedback message within the timeout period, the transaction will be interrupted:

Send an interrupt request. The coordinator sends abort requests to all participants;

Transaction rollback. After receiving the abort request, the participant will use the Undo message in phase 2 to perform transaction rollback and release the occupied resources after completing the rollback;

Feedback transaction rollback results. The participant sends an Ack message to the coordinator after completing the rollback;

Midrange affairs. After the coordinator receives the Ack message from all participants, the transaction is interrupted.

6.5.2. The difference between 2PC and 3PC:

3PC effectively reduces the participant blocking range caused by 2PC, and can continue to reach consensus after a single point of failure; however,
3PC brings new problems. After the participants receive the preCommit message, if the network is partitioned, the coordinator Subsequent communication cannot be carried out with the participants. In this case, the participants will still execute transaction submission after waiting for timeout, which will lead to data inconsistency.

The difference between 2PC and 3PC:

The three-phase commit protocol introduces a timeout mechanism in both the coordinator and participants, and splits the first phase of the two-phase commit protocol into two steps: query, then lock the resource, and finally commit. The three stages of three-phase submission are: can_commit, pre_commit, and do_commit.

In the doCommit phase, if the participant cannot receive the doCommit or abort request from the coordinator in time, it will continue to commit the transaction after waiting for timeout. (In fact, this should be decided based on probability. When entering the third stage, it means that the participant has received the PreCommit request in the second stage. Then the prerequisite for the coordinator to generate the PreCommit request is that he receives the PreCommit request before the second stage starts. The CanCommit response to all participants is Yes. (Once a participant receives PreCommit, it means that he knows that everyone actually agrees to the modification.) Therefore, in one sentence, when entering the third stage, due to network timeout and other reasons, although The participant does not receive a commit or abort response, but he has reason to believe that the probability of successful submission is high.)

​

The single point of failure problems that 3PC mainly solves:

Compared with 2PC, 3PC mainly solves the single point of failure problem and reduces blocking, because once a participant cannot receive information from the coordinator in time, he will execute commit by default. Instead of holding transactional resources all the time and in a blocked state.

However, this mechanism can also cause data consistency problems, because, due to network reasons, the abort response sent by the coordinator is not received by the participants in time, and then the participants perform the commit operation after waiting for timeout. This results in data inconsistency with other participants who received the abort command and performed rollback.

"What is the optimization of 3PC compared to 2PC?"
Compared with 2PC, 3PC has timeouts set for both the coordinator (Coordinator) and participants (Partcipant), while 2PC only has the timeout mechanism for the coordinator. What problem does this solve?

This optimization point mainly avoids the problem of participants being unable to release resources when they are unable to communicate with the coordinator node for a long time (the coordinator hangs up), because the participants themselves have a timeout mechanism that will automatically proceed after the timeout. Local commit to release resources. This mechanism also reduces the blocking time and scope of the entire transaction.

In addition, through the design of the three stages of CanCommit, PreCommit, and DoCommit, compared with 2PC, an additional buffer stage is set up to ensure that the status of each participating node is consistent before the final submission stage.

The above is an improvement of 3PC compared to 2PC (relatively alleviating the first two problems in 2PC), but 3PC still does not completely solve the problem of data inconsistency. If during the DoCommit process, participant A cannot receive the coordinator's communication, then participant A will automatically submit, but the submission fails and other participants succeed, and the data will be inconsistent at this time.

6.6. Problems with XA specification

However, the XA specification appeared in 1994 and has not become popular on a large scale. It must have certain flaws:

• Data locking: Before the transaction ends, the data is locked according to the data isolation level in order to ensure consistency.

• Protocol blocking: Local transactions are blocked and waiting until the global transaction commits or calls back.

• High performance loss: mainly reflected in the increased RT cost of transaction coordination, and concurrent transaction data uses locks to compete and block.

The XA protocol is relatively simple, and once a commercial database implements the XA protocol, the cost of using distributed transactions will be relatively low. However, XA also has a fatal shortcoming, that is, its performance is not ideal. Especially in the transaction order link, the amount of concurrency is often very high, and XA cannot meet high concurrency scenarios. XA is currently supported ideally in commercial databases, but is less than ideally supported in mysql databases. MySQL's XA implementation does not record prepare phase logs, and switching back between the active and standby databases results in data inconsistency between the active and standby databases. Many nosql also do not support XA, which makes the application scenarios of XA very narrow.

In fact, it is not unnecessary. For example, many cross-resources based on CICS on IBM mainframes are distributed transactions based on the XA protocol. In fact, XA is also a standard for distributed transaction processing, but it is rarely used in the Internet. The reason is There are the following:

• Performance (blocking protocols, increased response times, lock times, deadlocks);
• Database support completeness (there were defects before MySQL 5.7);
• The coordinator relies on independent J2EE middleware (early heavyweight Weblogic, Jboss, later lightweight Atomikos, Narayana and Bitronix);
• Operation and maintenance are complex, and DBAs lack experience in this area;
• Not all resources support the XA protocol;

To be precise, XA is a specification and protocol. It just defines a series of interfaces. However, most of the current implementations of XA are databases or MQ, so when XA is mentioned, it often refers to the underlying distributed transaction solution based on the resource layer. In fact, some data sharding frameworks or middleware now also support the XA protocol. After all, its compatibility and universality are better.

6.7. Classification of flexible transactions

In Internet scenarios such as e-commerce, rigid transactions have exposed bottlenecks in database performance and processing capabilities.

Flexible transactions have two characteristics: basically available and flexible state.

• Basic availability means that a part of the availability is allowed to be lost when a distributed system fails.

• Flexible state refers to allowing the system to exist in an intermediate state. This intermediate state will not affect the overall availability of the system, such as the master-slave synchronization delay of database read-write separation, etc. The consistency of flexible transactions refers to eventual consistency.

Flexible transactions are mainly divided into compensation type and notification type.

Compensation transactions are divided into: TCC, Saga;

Notification-type transactions are divided into: MQ transaction messages and best-effort notification types.

Compensation transactions are all synchronous, and notification transactions are asynchronous.

6.8. Notification transactions

The mainstream implementation of notification-type transactions is to notify other transaction participants of the execution status of their own transactions through MQ (Message Queue). MQ components are introduced to effectively decouple transaction participants. Each participant can execute asynchronously, so notification-type transactions Transactions are also called asynchronous transactions.

Notification transactions are mainly suitable for scenarios that need to update data asynchronously and have low real-time requirements for data. They mainly include:

There are two types: asynchronous guaranteed transactions and best-effort notification transactions.

• Asynchronous guaranteed transactions: Mainly suitable for ensuring the final consistency of data in internal systems, because the internals are relatively controllable, such as orders and shopping carts, receipt and clearing, payment and settlement, etc.;

• Best effort notification: Mainly used for external systems, because the external network environment is more complex and untrustworthy, so we can only do our best to notify to achieve final data consistency, such as recharge platforms and operators, payment docking, etc. across network system levels docking;

6.9. Asynchronous guaranteed transactions

It refers to modifying a series of synchronous transaction operations into asynchronous operations based on message queues to avoid the degradation of data operation performance caused by synchronous blocking in distributed transactions.

6.9.1, MQ transaction message solution

The MQ-based transaction messaging solution mainly relies on MQ's semi-message mechanism to ensure the consistency of delivered messages and participants' own local transactions. The implementation principle of the semi-message mechanism actually draws on the idea of ​​​​2PC, which is a broad expansion of two-phase submission.

Half message: The logic after the original queue message is executed. If the subsequent local logic makes an error, the message will not be sent. If it passes, MQ will be informed to send it;

process

• The transaction initiator first sends a half message to MQ;
• MQ notifies the sender that the message is sent successfully;
• Execute local transaction after sending half message successfully;
• Return commit or rollback based on the local transaction execution result;
• If the message is rollback, MQ will discard the message without delivery; if it is commit, MQ will send the message to the message subscriber;
• The subscriber performs local transactions based on the message;
• After the subscriber executes the local transaction successfully, mark the message as consumed from MQ;
• If the execution end hangs up or times out during the execution of a local transaction, the MQ server will continuously ask the producer to obtain the transaction status;
• The consumption success mechanism on the Consumer side is guaranteed by MQ;

6.9.2. Examples of using asynchronous guaranteed transactions

For example, suppose there is a business rule: after a certain order is successful, a certain amount of points will be added to the user.

In this rule, the service that manages the order data source is the transaction initiator, and the service that manages the points data source is the transaction follower.

From this process, we can see that transactions implemented based on message queues have the following operations:

• The order service creates an order and submits a local transaction
• The order service publishes a message
• Point service adds points after receiving the message

We can see that its overall process is relatively simple, and the business development workload is not large:

Write the logic of order creation in the order service.
Write the logic of adding points in the points service.
You can see that the transaction form process is simple and the performance consumption is small. The traffic peaks and valleys between the initiator and the follower can be filled with queues, and business development work is done at the same time. The volume is basically the same as that of a stand-alone transaction, and there is no need to write a reverse business logic process

Therefore, transactions implemented based on message queues are our top priority in addition to stand-alone transactions.

6.9.3. Implement MQ asynchronous guaranteed transactions based on Alibaba RocketMQ

There are some third-party MQs that support transaction messages. These message queues support semi-message mechanisms, such as RocketMQ and ActiveMQ. However, some commonly used MQs do not support transactional messages, such as RabbitMQ and Kafka.

Taking Ali's RocketMQ middleware as an example, the idea is roughly as follows:

1. The producer (system A in this example) sends a half message to the broker. This half message does not mean that the message content is incomplete. It contains the complete message content. The sending logic on the producer side is the same as that of ordinary messages.

2. The broker stores half messages. The storage logic of half messages is the same as that of ordinary messages, but the attributes are different. The topic is fixed at RMQ_SYS_TRANS_HALF_TOPIC, and the queueId is also fixed at 0. The messages in this tiopic are invisible to consumers, so the The message is never consumed. This ensures that the consumer will not be able to consume the half message before the half message is submitted successfully.

3. After the broker-side half message is successfully stored and returned, system A executes the local transaction and determines the submission status of the half message as commit or rollback based on the execution result of the local transaction.

4. System A sends a request to end the half message and brings the submission status (submit or rollback)

5. After receiving the request, the broker first checks out the message from the queue of RMQ_SYS_TRANS_HALF_TOPIC and sets it to the completion status. If the message status is submitted, copy the half message from the RMQ_SYS_TRANS_HALF_TOPIC queue to the queue of the message's original topic (then the message can be consumed normally); if the message status is rollback, do nothing.

6. The half-message end request sent by the producer is oneway, that is, it is ignored after being sent. This alone cannot guarantee that the half-message will be submitted. RocketMq provides a cover-up solution. This solution is called the message reverse check mechanism. Broker At startup, a TransactionalMessageCheckService task will be started, which will regularly read out all timed-out unfinished half messages from the half message queue. For each unfinished message, the Broker will send a message check request to the corresponding Producer. According to Check the results to determine whether the half-message needs to be submitted or rolled back, or continue to check later

7. The consumer (system B in this example) consumes messages and performs local data changes (as for whether B can successfully consume and whether to retry if consumption fails, this is an issue that needs to be considered for normal message consumption)

In rocketMq, whether the producer executes local transactions after receiving the broker's stored half message and returns successfully, or the broker checks the message status back to the producer, it is all done through the callback mechanism.

When sending a semi-message, a callback class TransactionListener will be passed in. Two of the methods must be implemented when using it. The executeLocalTransaction method will be executed after the broker returns the semi-message storage successfully, and we will execute local transactions in it; the checkLocalTransaction method will be executed after the broker returns the semi-message storage. It is executed when the producer initiates a reverse check, in which we query the status of the library table. The return value of both methods is the message status, which tells the broker that it should submit or roll back the half message.

6.9.4. Local message table solution

Sometimes our current MQ components do not support transactional messages, or we want to intrude on the business side as little as possible. At this time, we need another solution "based on DB local message table".

The local message table was originally proposed by eBay to solve the problem of distributed transactions. It is one of the more commonly used solutions in the industry. Its core idea is to split distributed transactions into local transactions for processing.

Local message table process

Sending party:

• A message table is required to record information related to message status.
• The business data and the message table are in the same database, so make sure they are in the same local transaction. Directly utilize local transactions to write business data and transaction messages directly to the database.
• After processing the business data and writing the message table in the local transaction, write the message to the MQ message queue. Use a dedicated delivery worker thread to deliver transaction messages to MQ, and delete transaction message table records according to the delivery ACK
• The message will be sent to the message consumer. If the sending fails, it will be retried.

Message consumer:

• Process messages in the message queue and complete your own business logic.
• If the local transaction is processed successfully, it indicates that it has been processed successfully.
• If local transaction processing fails, execution is retried.
• If it is a failure at the business level, send a business compensation message to the message producer to notify operations such as rollback.

The producer and consumer regularly scan the local message table, and resend unprocessed messages or failed messages. If there is a reliable automatic reconciliation and replenishment logic, this solution is still very practical.

Advantages and disadvantages of local message table:

Advantages :

• The construction cost of the local message table is relatively low, which realizes reliable message delivery and ensures the final consistency of distributed transactions.
• There is no need to provide a back-check method, further reducing business intrusion.
• In some scenarios, annotations and other forms can be further used for decoupling, and it is possible to achieve intrusive implementation without business code.

shortcoming:

• The local message table is coupled with the business, making it difficult to make it universal and cannot be independently scalable.
• The local message table is based on a database, and the database requires reading and writing disk IO, so there is a performance bottleneck under high concurrency.

6.9.5, MQ transaction message VS local message table

What they have in common:

1. Transaction messages all rely on MQ for transaction notification, so they are all asynchronous.
2. There is a possibility of duplicate delivery of transaction messages on the delivery side. A supporting mechanism is needed to reduce the duplicate delivery rate and achieve more friendly message delivery and deduplication.
3. The consumer of transaction messages needs to carry out consumption deduplication design or service idempotence design because duplication of delivery is unavoidable.

The difference between the two:

MQ transaction message:

• MQ needs to support the semi-message mechanism or similar features, and have better deduplication processing for repeated delivery;
• It is relatively intrusive to the business and requires modification by the business side to provide the corresponding review function for successful local operations;

DB local message table:

• A database is used to store transaction messages, which reduces the requirements for MQ, but increases storage costs;
• Transaction messages use asynchronous delivery, which increases the possibility of repeated message delivery;

6.10. Best effort notification

The goal of the best-effort notification scheme is that the initiating party uses a certain mechanism to use its best efforts to notify the recipient of the business processing results.

Best-effort informed eventual consistency:

The essence is to achieve eventual consistency by introducing a periodic verification mechanism, which is less intrusive to the business and is suitable for scenarios with low sensitivity to eventual consistency and short business links.

Best-effort notification transactions are mainly used for external systems. Because the external network environment is more complex and untrustworthy, we can only do our best to notify and achieve ultimate data consistency, such as recharge platforms and operators, payment docking, merchant notifications, etc. Business interaction scenarios between platforms and cross-enterprise systems;

Asynchronous guaranteed transactions are mainly suitable for ensuring the final consistency of data in internal systems, because the internals are relatively controllable, such as orders and shopping carts, receipt and clearing, payment and settlement, etc.

Ordinary messages cannot solve the consistency problem of local transaction execution and message sending. Because message sending is a process of network communication, the process of sending messages may fail or time out. After the timeout, the message may be sent successfully or failed. The sender of the message cannot be determined, so at this time, whether the message sender commits the transaction or rolls back the transaction, inconsistency may occur.

Therefore, the difficulty of notification-type transactions lies in: ensuring the consistency of message delivery and local transactions of participants.

Because the core points are the same, both are to ensure the consistent delivery of messages. Therefore, the delivery process of the best-effort notification transaction is the same as the asynchronous guarantee type, so there are also two branches:

• Transaction message solution based on MQ itself

• DB-based local transaction message table solution

6.10.1, MQ transaction message solution

To implement best-effort notification, MQ's ACK mechanism can be used.

The best-effort notification transaction is similar to the asynchronous guaranteed process before delivery, and the key lies in the post-delivery processing.

Because the asynchronous guarantee type lies in the internal transaction processing, MQ and the system are directly connected and do not require strict permissions, security and other aspects of thinking design. The best-effort notification transaction lies in the connection with the third-party system, so the best-effort notification transaction has several characteristics:

• After the business active party completes the business processing, it sends a notification message to the business passive party (third-party system), allowing message loss.

• The active side of the business provides incremental time intervals (5min, 10min, 30min, 1h, 24h) for retrying the interface of calling the passive side of the business; after N times of notification, it will no longer notify, alarm + logging + manual intervention.

• The business passive party provides an idempotent service interface to prevent repeated consumption of notifications.

• The business active party needs to have a regular verification mechanism to check the business data; prevent the business passive party from rolling back the business when it cannot fulfill its responsibilities, and ensure the final consistency of the data.

• The active party in the business activity, after completing the business processing, sends a message to the passive party in the business activity, allowing the message to be lost.
• The active party can set up time-stepped notification rules, and repeat the notification according to the rules after the notification fails, until it is notified N times and no longer is notified.
• The active party provides a proofreading query interface for the passive party to proofread and query on demand to recover lost business messages.
• If the passive party of the business activity receives the data normally, it will return the response normally and end the transaction.
• If the passive party does not receive normally, according to the timing strategy, query the active party of business activities to recover the lost business messages.

features

• Service modes used: queryable operations, idempotent operations;
• The processing results of the passive party do not affect the processing results of the active party;
• Suitable for systems with low time sensitivity to eventual business consistency;
• Suitable for operations between systems across enterprises, or operations between relatively independent systems within an enterprise, such as bank notifications, merchant notifications, etc.;

6.10.2. Local message table solution

To implement best-effort notification, a mechanism can be employed that periodically checks the local message table.

Sending party:

• A message table is required to record information related to message status.
• The business data and the message table are in the same database, so make sure they are in the same local transaction. Directly utilize local transactions to write business data and transaction messages directly to the database.
• After processing the business data and writing the message table in the local transaction, write the message to the MQ message queue. Use a dedicated delivery worker thread to deliver transaction messages to MQ, and delete transaction message table records according to the delivery ACK
• The message will be sent to the message consumer. If the sending fails, it will be retried.
• The producer regularly scans the local message table, and resends unprocessed or failed messages. If there is a reliable automatic reconciliation and replenishment logic, this solution is still very practical.

The best-effort notification transaction lies in the connection with the third-party system, so the best-effort notification transaction has several characteristics:

• After the business active party completes the business processing, it sends a notification message to the business passive party (third-party system), allowing message loss.

• The active side of the business provides incremental time intervals (5min, 10min, 30min, 1h, 24h) for retrying the interface of calling the passive side of the business; after N times of notification, it will no longer notify, alarm + logging + manual intervention.

• The business passive party provides an idempotent service interface to prevent repeated consumption of notifications.

• The business active party needs to have a regular verification mechanism to check the business data; prevent the business passive party from rolling back the business when it cannot fulfill its responsibilities, and ensure the final consistency of the data.

6.10.3. Best effort notification transaction VS asynchronous guaranteed transaction

In my understanding, the best-effort notification transaction is actually a business implementation developed based on asynchronous guaranteed transactions and suitable for external docking. Their main differences are business differences, as follows:

• From the perspective of participants: best-effort notification transactions are suitable for cross-platform and cross-enterprise business interactions between systems; asynchronous guaranteed transactions are more suitable for internal service delivery in the same network system.
• From the message level: best-effort notification transactions need to be actively pushed and provide a multi-level retry mechanism to ensure data notification; asynchronous guaranteed transactions only require message consumers to actively consume.
• From the data level: best-effort notification transactions require additional periodic verification mechanisms to verify the data to ensure the final consistency of the data; asynchronous guaranteed transactions only need to ensure reliable delivery of messages, and do not need to verify the data themselves. deal with.

6.11. Problems with notification transactions

Notification transactions cannot solve the consistency problem of local transaction execution and message sending.

Because message sending is a process of network communication, the process of sending messages may fail or time out. After the timeout, the message may be sent successfully or failed. The sender of the message cannot be determined, so at this time, whether the message sender commits the transaction or rolls back the transaction, inconsistency may occur.

6.11.1. Message sending consistency

The core role of message middleware in distributed systems is asynchronous communication, application decoupling and concurrent buffering (also called traffic peak shaving). In a distributed environment, communication needs to be carried out through the network, which introduces uncertainty in data transmission, which is the partition fault tolerance in the CAP theory.

Message sending consistency means that the business action that generates the message is consistent with the message sending action. That is to say, if the business operation is successful, the message generated by this business operation must be sent out, otherwise it will be lost.

The conventional MQ queue processing process cannot achieve message consistency. Therefore, it is necessary to use semi-message and local message tables to ensure consistency.

6.11.2. The problem of repeated message sending and the idempotent design of business interfaces

Unconfirmed messages will be re-delivered according to rules.

For the above process, repeated sending of messages will cause repeated calls to the business processing interface. The main reason why messages are sent repeatedly during message consumption is that the message middleware does not update the delivery status in time after the consumer successfully receives and processes the message. If messages are allowed to be sent repeatedly, the consumer should implement an idempotent design of the business interface.

6.12. Compensation type

However, transactions implemented based on messages cannot solve all business scenarios, such as the following scenario: when an order is completed, the user's cash is deducted at the same time.

The transaction initiator here is the service that manages the order library, but whether the entire transaction is submitted cannot be determined solely by the order service, because it must also ensure that the user has enough money to complete the transaction, and this information is in the service that manages cash. . Here we can introduce transactions based on compensation implementation, and the process is as follows:

• Create order data, but do not submit local transactions yet
• The order service sends a remote call to the cash service to deduct the corresponding amount.
• Submit the order database transaction after the above steps are successful.

The above is a normal and successful process. If the abnormal process needs to be rolled back, an additional remote call will be sent to the cash service to add the previously deducted amount.

The above process is more complicated than the transaction process based on message queue, and the development workload is also more:

• Write the logic for creating orders in the order service
• Write the logic for deducting money in the cash service
• Write the logic of compensation return in cash service

It can be seen that this transaction process is more complex than the distributed transaction implemented based on messages. It requires additional development of relevant business rollback methods, and it also loses the function of peak-shaving and valley-filling of traffic between services. But it is only a little more complicated than message-based transactions. If you cannot use message queue-based eventual consistency transactions, you can give priority to using compensation-based transaction forms.

What is the compensation model?

The compensation mode uses an additional coordination service to coordinate various business services that need to ensure consistency. The coordination service calls each business microservice in sequence. If a business service calls abnormally (including business exceptions and technical exceptions), all previous calls will be cancelled. Successful business services.

Compensation modes generally include two subdivided solutions: TCC and Saga.

6.13. TCC transaction model

6.13.1. What is the TCC transaction model?

The concept of TCC (Try-Confirm-Cancel) comes from a paper titled "Life beyond Distributed Transactions: an Apostate's Opinion" published by Pat Helland.

The TCC distributed transaction model consists of three parts:

1. Main business service: The main business service is the initiator of the entire business activity and the orchestrator of the service. It is responsible for initiating and completing the entire business activity.

2. Slave business service: Slave business service is a participant in the entire business activity. It is responsible for providing TCC business operations and implementing three interfaces: preliminary operation (Try), confirmation operation (Confirm), and cancellation operation (Cancel) for the main business service to call. .

3. Business activity manager: The business activity manager manages and controls the entire business activity, including recording and maintaining the transaction status of the TCC global transaction and the sub-transaction status of each slave business service, and calling the Confirm of all slave business services when the business activity is submitted. Operation, the Cancel operation of all slave business services is called when the business activity is canceled.

TCC proposes a new transaction model based on business-level transaction definitions. The lock granularity is completely controlled by the business itself. The purpose is to solve the problem of large-granularity resource locking across tables and databases in complex businesses.

TCC divides the transaction running process into two stages: Try and Confirm/Cancel. The logic of each stage is controlled by the business code, which avoids long transactions and can obtain higher performance.

6.13.2. TCC work flow

Compared with traditional models such as XA, the TCC (Try-Confirm-Cancel) distributed transaction model is characterized by that it does not rely on the resource manager (RM) to support distributed transactions, but achieves distribution through the decomposition of business logic. affairs.

The TCC model believes that for a specific business logic in a business system, when it provides external services, it must accept some uncertainty, that is, the call to the preliminary operation of the business logic is only a temporary operation, and the main business service that calls it retains the subsequent right of cancellation. If the main business service believes that the global transaction should be rolled back, it will require the cancellation of the previous temporary operation, which corresponds to the cancellation operation of the slave business service. When the main business service believes that the global transaction should be submitted, it will give up the right to cancel the previous temporary operation, which corresponds to the confirmation operation of the slave business service. Every initial operation will eventually be confirmed or canceled.

Therefore, for a specific business service, the TCC distributed transaction model requires the business system to provide three pieces of business logic:
Preliminary operation Try: complete all business checks and reserve necessary business resources.

Confirm operation Confirm: The business logic is actually executed, without any business check, and only uses the business resources reserved in the Try stage. Therefore, as long as the Try operation succeeds, Confirm must succeed. In addition, the Confirm operation needs to satisfy idempotence to ensure that a distributed transaction can and can only succeed once.

Cancel operation Cancel: Release the business resources reserved in the Try phase. Similarly, the Cancel operation also needs to satisfy idempotence.

The TCC distributed transaction model consists of three parts:

Try phase: Call the Try interface, try to execute the business, complete all business checks, and reserve business resources.

Confirm or Cancel phase: The two are mutually exclusive, you can only enter one of them, and both satisfy idempotence, allowing retry after failure.

Confirm operation: Make a confirmation submission to the business system, confirm the execution of business operations, do not perform other business checks, and only use the business resources reserved in the Try stage.

Cancel operation: Execute business cancellation and release reserved resources when business execution errors require rollback.

If the Try phase fails, you can cancel it. What if the Confirm and Cancel phases fail?

A transaction log will be added to TCC. If an error occurs in the Confirm or Cancel phase, a retry will be performed, so these two phases need to support idempotence; if the retry fails, manual intervention is required for recovery and processing.

6.13.3. TCC affairs cases

However, the transaction form based on compensation cannot meet all needs, such as the following scenario: when an order is completed, the user's cash is deducted, but when the transaction is not completed or canceled, the customer cannot be allowed to see the money decrease. .

At this time we can introduce TCC, the process is as follows:

• Order service creates orders
• The order service sends a remote call to the cash service to freeze the customer's cash
• Submit order service data
• The order service sends a remote call to the cash service to deduct the customer's frozen cash.

The above is a normal process. If it is an abnormal process, you need to send a remote call request to the cash service to cancel the frozen amount.

The above process is more complicated than the process of transactions based on compensation implementation, and the development workload is also more:

• The order service writes the logic for creating orders
• Cash service writes the logic to freeze cash
• Cash service writes the logic for deducting cash
• Cash service writes the logic to unfreeze cash

TCC is actually the most complex situation. It can handle all business scenarios. However, regardless of performance considerations or development complexity considerations, this type of transaction should be avoided as much as possible.

6.13.4. Requirements of TCC transaction model

• Queryable operations: Service operations have a globally unique identifier, and the operation has a unique and determined time.

• Idempotent operation: The business results produced by repeated calls multiple times are the same as the results produced by calling them once. The first is to achieve idempotence through business operations, the second is that the system caches the results of all requests and processing, and finally, after detecting repeated requests, it automatically returns the previous processing results.

• TCC operation: Try phase, try to execute the business, complete the inspection of all services, and achieve consistency; reserve necessary business resources to achieve quasi-isolation. Confirm phase: The business is actually executed without any checks. Only the business resources reserved in the Try phase are used. The Confirm operation must also satisfy idempotence. Cancel phase: Cancel the execution of the business and release the business resources reserved in the Try phase. The Cancel operation must satisfy idempotence. The difference between TCC and 2PC (two-phase commit) protocol: TCC is located in the business service layer rather than the resource layer. TCC does not have a separate preparation phase. The Try operation has the ability to operate and prepare resources. The Try operation in TCC can flexibly select business resources and lock them. granularity. The development cost of TCC is higher than that of 2PC. In fact, TCC is also a two-stage operation, but TCC is not equivalent to 2PC operation.

• Compensable operations: Do stage: Business processing is actually executed, and the business processing results are visible externally. Compensate stage: offset or partially cancel the business results of forward business operations, and the compensation operations satisfy idempotence. Constraints: The compensation operation is commercially feasible, and the risks and costs caused by unisolated business execution results or incomplete compensation are controllable. In fact, TCC's Confirm and Cancel operations can be regarded as compensation operations.

6.13.5. TCC transaction model VS DTP transaction model

Compare the TCC transaction model and the DTP transaction model, as follows:

These two pictures look very different, but in fact they are similar in many places!

• The main business service in the TCC model is equivalent to the AP in the DTP model, and the slave business service in the TCC model is equivalent to the RM in the DTP model.

  • In the DTP model, the application AP operates the resources on multiple resource managers RM; while in the TCC model, the main business service operates the resources on multiple slave business services. For example, in the flight booking case, the Meituan App is the main business service, while Sichuan Airlines and China Eastern Airlines are the slave business services. The main business service needs to use the ticket resources on the slave business service. The difference is that the resource provider in the DTP model is a relational database similar to Mysql, while the resource provider in the TCC model is other business services.

• In the TCC model, the try, confirm, and cancel interfaces provided by the business service are equivalent to the prepare, commit, and rollback interfaces provided by the RM in the DTP model.

  • The XA protocol stipulates that the RM in the DTP model needs to provide prepare, commit, and rollback interfaces for TM to call to achieve two-phase submission.

  • In the TCC model, the slave business service is equivalent to RM, which provides similar try, confirm, and cancel interfaces.

• transaction manager

  • There is a transaction manager in both the DTP model and the TCC model. the difference is:

  • In the DTP model, phase 1 (prepare) and phase 2 (commit, rollback) are both called by TM.

  • In the TCC model, the try interface in phase 1 is the main business service call (green arrow), and the (confirm, cancel interface) in phase 2 is the transaction manager TM call (red arrow). This is the two-phase asynchronous function of the TCC distributed transaction model. If the first phase of the slave business service is successfully executed, the main business service can be submitted and completed, and then the transaction manager framework will asynchronously execute the second phase of each slave business service. . A certain amount of isolation and consistency is sacrificed here, but the availability of long transactions is improved.

6.13.6. Comparison between TCC and 2PC

In fact, the essence of TCC is similar to that of 2PC:

• T stands for Try, and the two Cs stand for Confirm and Cancel.

• Try means trying. Each participant in the request link executes the Try logic in turn. If successful, the Confirm logic will be executed. If there is a failure, the Cancel logic will be executed.

TCC and XA two-phase commit have the same purpose, the following figure lists the comparison between the two

At stage 1:

• In XA, each RM prepares to submit its own transaction branch, which is actually preparing to submit resource update operations (insert, delete, update, etc.);

• In TCC, however, it is the main business activity that requests (try) each slave business service to reserve resources.

At stage 2:

• XA judges whether to submit or rollback according to whether each RM is prepared successfully in the first stage. If the prepare is successful, then commit each transaction branch, otherwise rollback each transaction branch.

• In TCC, if all business resources are successfully reserved in the first phase, then confirm each slave business service, otherwise cancel (cancel) the resource reservation requests of all slave business services.

The difference between TCC and 2PC is:

• XA is a distributed transaction at the resource level with strong consistency. During the entire two-phase submission process, the resource lock will always be held. Based on database lock implementation, the database needs to support the XA protocol. Since relevant data needs to be locked during the entire transaction execution process, generally high concurrency performance will be poor.

• TCC is a distributed transaction at the business level, with eventual consistency, and does not always hold resource locks, and has better performance. However, it is highly intrusive to microservices. Each transaction of microservices must implement three methods: try, confirm, and cancel. The development cost is high, and the cost of future maintenance and transformation is also high. In order to achieve the consistency requirements of transactions, try, confirm . The cancel interface must implement idempotent operations. Since the transaction manager needs to record transaction logs, a certain amount of performance will be lost and the entire TCC transaction time will be lengthened.

• TCC will weaken the locking of resources in each step to achieve a purpose that can withstand high concurrency (based on eventual consistency).

The specific instructions are as follows:

XA is a distributed transaction at the resource level with strong consistency. During the entire two-phase submission process, the resource lock will always be held.

The internal process of two-phase submission in XA transactions is shielded from developers, and developers cannot perceive this process from the code level. In the two-stage submission process of the transaction manager, from prepare to commit/rollback, resources are actually always locked. If someone else needs to update these two records, they must wait for the lock to be released.

TCC is a distributed transaction at the business level, with eventual consistency and will not always hold resource locks.

The two-phase submission in TCC is not completely shielded from developers, which means that from the code level, developers can feel the existence of two-phase submission. During the execution of try, confirm/cancel, their respective local transactions are generally opened to ensure the ACID characteristics of the internal business logic of the method. in:

1. The local transaction of the try process is to ensure the correctness of the business logic of resource reservation.

2. The local transaction logic executed by confirm/cancel confirms/cancels reserved resources to ensure eventual consistency, which is the so-called Compensation-Based Transactions. Since it is multiple independent local transactions, resources will not be locked all the time.

In addition, it is mentioned here that the local transaction executed by confirm/cancel is a compensatory transaction:

Compensation is an independent local transaction that supports ACID features, which is used to logically cancel the impact of the previous ACID transaction of the service provider. For a long-running transaction, instead of implementing a huge distributed ACID transaction, It is better to use a compensation-based solution, treat each service call as a short local ACID transaction, and submit it immediately after execution.

6.13.7, TCC usage scenarios

TCC can solve distributed transactions in some scenarios, but one of its problems is that each participant needs to implement the Try, Confirm and Cancel interfaces and logic respectively, which is very intrusive to the business.

The TCC solution relies heavily on rollback and compensation code. The final result is: the rollback code logic is complex and the business code is difficult to maintain. Therefore, the TCC solution has fewer usage scenarios, but there are also usage scenarios.

For example, when dealing with money, payment and transaction-related scenarios, everyone will use the TCC solution to strictly ensure that all distributed transactions are either successful or automatically rolled back, strictly ensure the correctness of funds, and ensure that there will be no problems with funds. .

6.14. SAGA long transaction model

SAGA can be seen as an asynchronous compensation transaction implemented using a queue.

6.14.1. Saga related concepts

In 1987, Hector Garcia-Molina and Kenneth Salem of Princeton University published a Paper Sagas about how to deal with long lived transactions. Saga is a long-lived transaction that can be decomposed into a collection of sub-transactions that can be interleaved. Each of these subtransactions is a real transaction that maintains database consistency.

The Saga model splits a distributed transaction into multiple local transactions. Each local transaction has a corresponding execution module and compensation module (corresponding to Confirm and Cancel in TCC). When any local transaction in the Saga transaction goes wrong, Previous transactions can be restored by calling relevant compensation methods to achieve eventual transaction consistency.

Such a SAGA transaction model sacrifices certain isolation and consistency, but improves the availability of long-running transactions.

The Saga model consists of three parts:

• LLT (Long Live Transaction): A transaction chain composed of local transactions.
• Local transaction: The transaction chain consists of sub-transactions (local transactions), LLT = T1+T2+T3+...+Ti.
• Compensation: Each local transaction Ti has a corresponding compensation Ci.

There are two execution orders for Saga:

• T1, T2, T3, …, Tn

• T1, T2, …, Tj, Cj, …, C2, C1, where 0 < j < n

Saga has two recovery strategies:

• Backward Recovery: Undo all previous successful sub-transactions. If any local subtransaction fails, the completed transaction is compensated. For example, the execution sequence of abnormal situations is T1, T2, T3,…Ti,Ci,…C3,C2,C1.

• Forward Recovery: retrying failed transactions, suitable for scenarios that must succeed, in which case Ci is not required. Execution sequence: T1, T2,…,Tj (failure), Tj (retry),…,Ti.

Obviously, there is no need to provide compensating transactions for forward recovery. If in your business, sub-transactions (eventually) will always succeed, or compensating transactions are difficult to define or impossible, forward recovery is more suitable for your needs.

Compensating transactions theoretically never fail, however, in a distributed world, servers can go down, networks can fail, and even data centers can lose power. What can we do in this situation? The last resort is to provide fallback measures, such as manual intervention.

6.14.2. Conditions of use of Saga

Saga looks promising for our needs. Can this be done for all long-lived affairs? There are some limitations here:

• Saga only allows two levels of nesting, top-level Saga and simple sub-transactions
• In the outer layer, full atomicity cannot be satisfied. That is, sagas may see partial results from other sagas
• Each sub-transaction should be an independent atomic behavior
• In our business scenario, each business environment (such as flight booking, car rental, hotel booking and payment) is a naturally independent behavior, and each transaction can use the database of the corresponding service to ensure atomic operations.

There are also things to consider with compensation:

• The compensating transaction undoes the behavior of transaction Ti from a semantic perspective, but it may not return the database to the state when Ti was executed. (For example, if a transaction triggers the launch of a missile, this may not be undone)

But that's not a problem for our business. In fact, actions that are difficult to undo may also be compensated. For example, a transaction that sends an email can be compensated by sending another email explaining the problem.

6.14.3. Guarantee for ACID:

Saga's guarantees for ACID are the same as TCC's:

• Atomicity: Guaranteed under normal circumstances.
• Consistency (Consistency): At a certain point in time, the data in database A and database B will violate the consistency requirements, but they will be consistent in the end.
• Isolation: At a certain point in time, transaction A can read the partially submitted results of transaction B.
• Durability (Durability), like local transactions, the data is persisted as long as it is committed.

Saga does not provide ACID guarantees because atomicity and isolation cannot be satisfied. The original paper is described as follows:

full atomicity is not provided. That is, sagas may view the partial results of other sagas

Through saga log, saga can ensure consistency and durability.

6.14.4, the solution of SAGA model

The core idea of ​​the SAGA model is to convert distributed transactions into local transactions through a certain solution, thereby reducing the complexity of the problem.

For example, taking the scenario of DB and MQ as an example, the business logic is as follows:

• Insert a piece of data into DB.

• Send a message to MQ.

Since the above logic corresponds to two storage terminals, namely DB and MQ, it cannot be solved simply through local transactions. Then, according to the SAGA model, there are two solutions.

Option 1: Semi-message mode.

In the new version of RocketMQ, this mode is supported.

First, we understand what a half message is. To put it simply, it adds a status to the message. When the sender puts a message into MQ for the first time, the message is in a pending confirmation state. In this state, the message cannot be consumed by consumers. The sender must interact with MQ a second time and change the message from the pending confirmation state to the confirmed state before the message can be consumed by the consumer. Messages that are pending confirmation are called semi-messages.

The complete transaction logic of the half-message is as follows:

• Send a half message to MQ.

• Insert data into DB.

• Send a confirmation message to MQ.

We found that through the form of semi-message, the DB operation is sandwiched between two MQ operations. Assuming that step 2 fails, then the message in MQ will always be in a semi-message state and will not be consumed by consumers.

So, have half messages always existed in MQ? Or what if step 3 fails?

In order to solve the above problem, MQ introduces a scanning mechanism. That is, MQ will scan all half messages at regular intervals, and ask the sender about the scanned half messages that have existed for too long. If the query receives a confirmation reply, the message will be changed to a confirmation status, such as If a failed reply is received, the message will be deleted.

As mentioned above, a problem with the semi-message mechanism is that it requires the business side to provide an interface for querying message status, which is still very intrusive to the business side.

Solution 2: Local message table

In the DB, add a message table to store messages. as follows:

• Insert data into the DB business table.

• Insert data into the DB message table.

• Send the message in the message table to MQ asynchronously. After receiving the ack, delete the message in the message table.

As above, through the above logic, a distributed transaction is split into two major steps. Numbers 1 and 2 constitute a local transaction, thus solving the problem of distributed transactions.

This solution does not require the business end to provide a message query interface. It only needs to slightly modify the business logic, and is minimally intrusive.

6.14.5. SAGA case

SAGA is suitable for scenarios where the final status of the business initiator does not need to be returned immediately. For example: your request has been submitted, please check later or pay attention to notifications.

Rewrite the above compensation transaction scenario using SAGA, and the process is as follows:

• The order service creates an order record with an unknown final status and commits the transaction
• The cash service deducts the required amount and submits the transaction
• The order service updates the order status to success and submits the transaction

The above is a successful process. If the cash service fails to deduct the amount, then the last step of the order service will update the order status to failure.

Its business coding workload is a little more than compensating transactions and includes the following:

• The logic for the order service to create the initial order
• The order service confirms the logic of order success
• Logic of order service failure to confirm order
• The logic of deducting cash from cash services
• Cash Service Compensation Return Cash Logic

However, it has performance advantages over the compensation transaction form. During the execution of all local sub-transactions, there is no need to wait for the execution of the sub-transactions it calls, which reduces the locking time. This is in businesses with many and long transaction processes. The performance advantages are even more obvious. At the same time, it uses queues for communication, which has the effect of peak-cutting and valley-filling.

Therefore, this form is suitable for business scenarios that do not need to synchronously return the final result of the initiator's execution, can be compensated, have high performance requirements, and do not mind additional coding.

But of course SAGA can also be slightly transformed into a form similar to TCC that can reserve resources.

6.14.6. Comparison between Saga and TCC

The disadvantage of Saga compared to TCC is the lack of reserved actions, which makes the implementation of compensation actions more troublesome: Ti is commit. For example, a business is to send emails. In TCC mode, save the draft first (Try) and then send (Confirm). If you cancel, Just delete the draft (Cancel). However, Saga just sends an email (Ti) directly. If you want to cancel, you have to send another email to explain the cancellation (Ci), which is a bit troublesome to implement.

If the above example of sending an email is changed to: Service A sends an Event to ESB (Enterprise Service Bus, which can be considered as a message middleware) immediately after completing Ti. The downstream service listens to this Event and does some of its own work before sending it. Event to ESB, if A service executes compensation action Ci, then the level of the entire compensation action is very deep.

However, the lack of reserved actions can also be considered an advantage:

• Some businesses are very simple. Applying TCC requires modifying the original business logic, but Saga only needs to add a compensation action.

• The minimum number of communications for TCC is 2n, while for Saga it is n (n=number of sub-transactions).

• Some third-party services do not have a Try interface, so TCC mode is more tricky to implement, while Saga is very simple.

• The absence of reserved actions means that there is no need to worry about resource release, and exception handling is simpler.

Saga vs. TCC

Saga and TCC are both compensatory transactions. Their differences are:

Disadvantages:

• Isolation is not guaranteed;

Advantage:

• Submit local transactions in one phase, lock-free, high performance;
• Event-driven mode, participants can execute asynchronously and achieve high throughput;
• Saga is less intrusive to the business, and only needs to provide a reverse operation Cancel; while TCC needs to carry out global process transformation on the business;

6.15. Comparison of overall plans

Attributes	2PC	TCC	Saga	Asynchronous guaranteed transactions	best effort notification
Transactional consistency	powerful	weak	weak	weak	weak
Complexity	middle	high	middle	Low	Low
business intrusiveness	Small	big	Small	middle	middle
Limitations of use	big	big	middle	Small	middle
performance	Low	middle	high	high	high
Maintenance cost	Low	high	middle	Low	middle

七、set

Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services. Seata will provide users with AT, TCC, SAGA and XA transaction modes to create a one-stop distributed solution for users.

7.1. Introduction to seata

Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services under a microservice architecture. Seata will provide users with AT, TCC, SAGA and XA transaction modes.
Before Seata was open source, the corresponding internal version of Seata had been playing the role of distributed consistency middleware within the Alibaba economy, helping the economy to go through the years smoothly. Double 11 provided strong support to each BU business. The commercial product GTS has been sold on Alibaba Cloud and Financial Cloud
. Related links:

What is seata: https://seata.io/zh-cn/docs/overview/what-is-seata.html

Download https://seata.io/zh-cn/blog/download.html

Official example https://seata.io/zh-cn/docs/user/quickstart.html

7.1.1. Seata’s three major modules

Seata AT uses an enhanced two-phase commit implementation.

Seata is divided into three modules:

• TC: Transaction Coordinator. Responsible for the generation of our transaction ID, transaction registration, submission, rollback, etc.

• TM: transaction initiator. Defines the boundaries of transactions and is responsible for informing TC of the start, submission, and rollback of distributed transactions.

• RM: Resource manager. To manage the resources of each branch transaction, each RM will be registered in TC as a branch transaction.

In Seata's AT mode, TM and RM are both part of the SDK and business services together, we can think of them as Clients. TC is an independent service that exposes itself to Clients through service registration and discovery.

Among the three major modules in Seata, TM and RM are integrated with the business system as the client of Seata, and TC is independently deployed as the server of Seata.

7.1.2. Seata’s distributed transaction execution process

In Seata, the execution process of distributed transactions:

• TM starts distributed transactions (TM registers global transaction records with TC);

• Arrange transaction resources such as databases and services according to business scenarios (RM reports resource readiness status to TC);

• TM ends the distributed transaction and the first phase of the transaction ends (TM notifies TC to commit/rollback the distributed transaction);

• TC summarizes transaction information and decides whether the distributed transaction should be submitted or rolled back;

• TC notifies all RMs to commit/rollback resources, and the second phase of the transaction ends;

Seata's three roles of TC, TM, and RM are very similar to the XA model. The following figure is the general transaction process of the XA model.

In the X/Open DTP (Distributed Transaction Process) model, there are three roles:

• ​ AP: Application, application. That is the business layer. Which operations belong to a transaction is defined by AP.

​ - TM: Transaction Manager, transaction manager. Receive AP transaction requests, manage global transactions, manage transaction branch status, coordinate RM processing, notify RM which operations belong to which global transactions and transaction branches, etc. This is also the core part of the entire transaction scheduling model.

​ - RM: Resource Manager, resource manager. Usually it is a database, but it can also be other resource managers, such as message queues (such as JMS data sources), file systems, etc.

7.1.3, 4 distributed transaction solutions

Seata will have 4 distributed transaction solutions, namely AT mode, TCC mode, Saga mode and XA mode.

7.2. Seata AT mode

Seata AT mode is the earliest supported mode. AT mode refers to Automatic (Branch) Transaction Mode automated branch transactions.

The Seata AT model is an enhanced 2pc model, or an enhanced XA model.

Generally speaking, the AT mode is an evolution of the 2pc two-phase commit protocol. The difference is that the Seata AT mode will not always lock the table.

7.2.1. Prerequisites for using eata AT mode

• Based on a relational database that supports native ACID transactions.

  • For example, in versions before MySQL 5.1, the default search engine is MyISAM. From versions after MySQL 5.5, the default search engine changes to InnoDB. The characteristics of the MyISAM storage engine are: table-level locks, no support for transactions and full-text indexes. Therefore, tables based on MyISAM do not support Seata AT mode.

• Java applications that access databases through JDBC.

7.2.2. Seata AT model diagram

Evolution of the two-phase commit protocol:

• Phase 1: Business data and rollback log records are committed in the same local transaction, releasing local locks and connection resources.

• Second stage:

  • Commits are asynchronous and complete very quickly.
  • Or rollback reverse compensation through a one-stage rollback log

The model diagram of the complete AT in the transaction mode formulated by Seata:

7.2.3. Example of Seata AT mode

Let’s use a relatively simple business scenario to describe the working process of Seata AT mode.

There is a recharge business. There are currently two services, one is responsible for managing the user's balance, and the other is responsible for managing the user's points.

When a user recharges, the balance on the user's account is first increased, and then the user's points are increased.

Seata AT is divided into two phases, the main logic is all in the first phase, and the second phase is mainly for rollback or log cleaning.

First stage process:

The first stage process is as follows:

1) TM in the balance service applies to TC to start a global transaction, and TC will return a global transaction ID.

2) Before the balance service executes the local business, the RM will first register the branch transaction with the TC.

3) The balance service sequentially generates undo log, executes local transactions, generates redo log, and finally submits local transactions directly.

4) The RM of the balance service reports to the TC that the transaction status is successful.

5) The balance service initiates a remote call and passes the transaction ID to the points service.

6) The points service will also register branch transactions with TC before executing local business.

7) The points service generates undo log, executes local transactions, generates redo log, and finally submits local transactions directly.

8) The RM of the points service reports to the TC that the transaction status is successful.

9) The points service returns the successful remote call to the balance service.

10) The TM of the balance service applies to the TC for the commit/rollback of the global transaction.

There is also TM in the points service, but since it is not used, it can be ignored.

If we use the annotated transaction of the Spring framework, the remote call will occur before the local transaction is committed. However, whether the remote call is initiated first or the local transaction is submitted first, this actually has no effect.

The second stage process is as follows:

The logic of the second stage is relatively simple.

There is a long connection between Client and TC. If it is a normal global submission, TC will notify multiple RMs to asynchronously clean up the local redo and undo logs. If it is a rollback, the TC can notify each RM to roll back the data.

This will lead to a question. Since local transactions are submitted directly by ourselves, how to roll back later? Since we record undo and redo log before and after local business operations, we can roll back through undo log.

Since undo and redo log and business operations are in the same transaction, they must succeed or fail at the same time.

But there is still a problem, because during the period from local submission of each transaction to notification of rollback, this data may have been modified by other transactions. If you directly use undo log to roll back, it will lead to data inconsistency.

At this time, RM will use redo log to verify whether the data is the same, so as to know whether the data has been modified by other transactions. Note: undo log is the data before modification and can be used for rollback; redo log is data after modification and is used for rollback verification.

If the data has not been modified by other transactions, it can be rolled back directly; if it is dirty data, it will be processed according to different strategies.

7.2.4. Application of Seata AT mode in e-commerce ordering scenarios

The following describes the working principle of Seata AT mode and the use of e-commerce ordering scenarios

As shown below:

In the above figure, the coordinator shopping-service first calls the participant repo-service to deduct inventory, and then calls the participant order-service to generate an order. The global transaction process after using Seata in XA mode for this business flow is as shown in the figure below:

The global transaction execution process described in the above figure is:

1) shopping-service registers a global transaction with Seata and generates a global transaction identifier XID

2) Execute local transactions of repo-service.repo_db and order-service.order_db to the pending submission stage. The transaction content includes query operations on repo-service.repo_db and order-service.order_db and writing undo_log records of each library.

3) repo-service.repo_db and order-service.order_db register branch transactions with Seata and include them in the global transaction scope corresponding to the XID

4) Submit local transactions of repo-service.repo_db and order-service.order_db

5) repo-service.repo_db, order-service.order_db report the submission status of branch transactions to Seata

6) Seata summarizes the submission status of all DB branch transactions and decides whether the global transaction should be submitted or rolled back

7) Seata notifies repo-service.repo_db and order-service.order_db to commit/rollback local transactions. If rollback is required, a compensatory method is adopted.

Among them, 1) 2) 3) 4) 5) belong to the first stage, and 6) 7) belong to the second stage.

7.2.5. Seata’s data isolation

The main implementation logic of seata's at mode is the data source agent, and the data source agent will be implemented based on relational transaction databases such as MySQL and Oracle. The isolation level based on the database is read committed. In other words, support for local transactions is a necessary condition for seata to implement at mode, which will also limit the usage scenarios of seata's at mode.

Write isolation
From the previous workflow, we can easily know that Seata's write isolation level is global and exclusive.

First, let’s understand the process of writing isolation

分支事务1-开始
| 
V 获取 本地锁
| 
V 获取 全局锁    分支事务2-开始
|               |
V 释放 本地锁     V 获取 本地锁
|               |
V 释放 全局锁     V 获取 全局锁
                |
                V 释放 本地锁
                |
                V 释放 全局锁

As shown above, the lock acquisition process of a distributed transaction is as follows:
1) Obtain the local lock first, so that you can modify the local data, but you cannot submit the local transaction yet.
2) Then, whether you can submit it depends on whether you can obtain it. Global lock
3) Obtaining the global lock means that it can be modified, then submit the local transaction and release the local lock
4) When the distributed transaction is submitted, release the global lock. This allows other transactions to acquire global locks and commit their modifications to local data.

As you can see, there are two key points here
1) Before the local lock is acquired, the global lock will not be contended for
2) Before the global lock is acquired, the local lock will not be submitted

This means that data modifications will be mutually exclusive. This will prevent dirty data from being written. Global locks can isolate write data in distributed modifications.

Principles of writing isolation:

• Before a phase one local transaction is committed, it is necessary to ensure that the global lock is obtained first.

• If the global lock cannot be obtained, the local transaction cannot be submitted.

• Attempts to obtain global locks are limited to a certain range. If they exceed the range, they will be given up, the local transaction will be rolled back, and the local lock will be released.

Let’s illustrate with an example:

• Two global transactions, tx1 and tx2, respectively update the m field of table a, and the initial value of m is 1,000.

tx1 starts first, starts the local transaction, obtains the local lock, and updates the operation m = 1000 - 100 = 900. Before the local transaction is committed, the global lock of the record is obtained, and the local commit releases the local lock.

Starting after tx2, the local transaction is started, the local lock is obtained, and the update operation is m = 900 - 100 = 800. Before the local transaction is committed, try to get the global lock of the record. Before tx1 is globally committed, the global lock of the record is held by tx1. tx2 needs to retry and wait for the global lock.

tx1 two-phase global commit, releasing the global lock. tx2 gets the global lock and commits the local transaction.

If the two-stage global rollback of tx1 occurs, tx1 needs to reacquire the local lock of the data and perform a reverse compensation update operation to implement branch rollback.

At this time, if tx2 is still waiting for the global lock of the data while holding the local lock, the branch rollback of tx1 will fail. The rollback of the branch will be retried until the global lock and other locks of tx2 time out, the global lock is given up and the local transaction is rolled back to release the local lock, and the branch rollback of tx1 is finally successful.

Because the global lock of the whole process is held by tx1 until the end of tx1, the problem of dirty writing will not occur.

read isolation level

Based on the database local transaction isolation level of Read Committed (Read Committed) or above, the default global isolation level of Seata (AT mode) is Read Uncommitted (Read Uncommitted).

If the application is in a specific scenario, the global read must be required to be committed. Currently, Seata's method is through the proxy of the SELECT FOR UPDATE statement.

The execution of the SELECT FOR UPDATE statement will apply for a global lock. If the global lock is held by other transactions, release the local lock (roll back the local execution of the SELECT FOR UPDATE statement) and try again. During this process, the query is blocked until the global lock is obtained, that is, the read related data has been submitted, and then it will not return.

For overall performance considerations, Seata's current solution does not proxy all SELECT statements, only FOR UPDATE SELECT statements.

7.2.6. Spring Cloud integrates Seata AT mode

AT mode refers to Automatic (Branch) Transaction Mode. The prerequisite for using AT mode is

• Based on a relational database that supports native ACID transactions.

• Java applications that access databases through JDBC.

How to use seata-at

1. Introduce the seata framework, configure the basic configuration of seata, and establish the undo_log table

2. Consumers introduce the global transaction annotation @GlobalTransactional

3. The producer introduces the global transaction annotation @GlobalTransactional

7.3. Seata TCC mode

7.3.1. Introduction

TCC is a two-stage transaction like Seata AT transaction. Its main differences from AT transaction are:

TCC seriously intrudes into business code

• Data operations at each stage must be coded by themselves, and the transaction framework cannot handle them automatically.

TCC higher performance

• There is no need to add global locks to the data, allowing multiple transactions to operate on the data at the same time.

eata TCC is a two-stage submission model as a whole. A distributed global transaction. The global transaction is composed of several branch transactions. The branch transactions must meet the requirements of the two-stage commit model, that is, each branch transaction needs to have its own:

• One-stage prepare behavior
• Two-phase commit or rollback behavior
  

According to the different behavior modes of the two phases, we divide branch transactions into Automatic (Branch) Transaction Mode and TCC (Branch) Transaction Mode.

AT mode ( refer to link TBD ) is based on a relational database that supports local ACID transactions:

• One-stage prepare behavior: In local transactions, business data updates and corresponding rollback log records are submitted together.

• Second-stage commit behavior: it ends successfully immediately, and the rollback log is automatically and asynchronously cleaned in batches.

• Second-stage rollback behavior: By rolling back logs, compensation operations are automatically generated to complete data rollback.

Correspondingly, the TCC mode does not depend on the transaction support of the underlying data resources:

• One-stage prepare behavior: call custom prepare logic.

• Two-phase commit behavior: call custom commit logic.

• Two-stage rollback behavior: call custom rollback logic.

The so-called TCC mode refers to supporting the integration of customized branch transactions into the management of global transactions.

First stage Try

Taking account service as an example, the user account amount should be deducted when placing an order:

If a user purchases a product worth 100 yuan, 100 yuan will be deducted.

The TCC transaction first reserves the deduction amount of 100 yuan, or freezes the 100 yuan first:

Second stage Confirm

If the first stage can be successfully completed, it means that the "deduction amount" business (branch business) will definitely be successful in the end. When the global transaction is committed, TC will control the current branch transaction to commit. If the commit fails, TC will try again and again until the commit is successful.

When the global transaction is committed, the frozen amount can be used to finally implement business data operations:

Phase 2Cancel

If the global transaction is rolled back, the frozen amount will be unfrozen and restored to the previous state. TC will control the rollback of the current branch transaction. If the rollback fails, TC will try again and again until the rollback is completed.

Multiple concurrent transactions

Multiple TCC global transactions are allowed to be concurrent. When they execute the deduction amount, they only need to freeze their respective amounts:

8. Interview questions

8.1. Do you understand distributed transactions? How do you solve the distributed transaction problem?

Seata AT models and Seata TCC are most commonly used in production.

• Strong consistency model, Seata AT strong consistency solution model is used for strong consistency mainly in core modules, such as transactions/orders, etc.

• Weak consistency model. The weak consistency scheme of Seata TCC is generally used in edge modules such as inventory. Through the coordination of TC, the final consistency can be guaranteed, and business decoupling can also be achieved.

Strong consistency scenario

For those particularly strict scenarios, Seata AT mode is used to ensure strong consistency;

Prepare examples: If you find a scenario that strictly requires data to be absolutely correct (such as inventory, orders, and coupons in e-commerce transactions), you can answer the question by using mature middleware such as the Seata AT model.

 阿里开源了分布式事务框架seata经历过阿里生产环境大量考验的框架。  seata支持Dubbo，Spring Cloud。 

It is Seata AT mode, ensuring strong consistency and supporting data modification across multiple libraries;

• Order library: add order

• Commodity library: deduct inventory

• Coupon Library: Withhold Coupons

Weak consistency scenario

For scenarios where the data consistency requirements are not particularly strict, or where sub-transactions are executed by different systems, you can answer the question of using Seata TCC to ensure weak consistency.

Prepare an example: a scenario where data consistency is not strictly required, or where sub-transactions are executed by different systems, such as an e-commerce order payment service that updates the order status and sends a successful payment message. Only weak consistency needs to be ensured.

Seata TCC mode ensures weak consistency and supports data modification across multiple services and systems. In the above scenario, Seata TCC mode transactions are used

• Order service: modify order status

• Notification service: send payment status

eventual consistency scenario

Based on the eventual consistency of reliable messages, each sub-transaction can be asynchronous for a long time, but the data must not be lost. Asynchronous guaranteed transactions can be used.

You can use MQ-based asynchronous guaranteed transactions, such as e-commerce platform notification of payment results:

• Points service: increase points

• Accounting Services: Generate Accounting Records