basic theory
CAP theory
Consistency: All data backups in a distributed system maintain a consistent state at the same time. If the state cannot be guaranteed to be consistent, an error is returned directly;
Availability (Availability): Even if some nodes in the cluster fail, it can also ensure that the client accesses the system and gets a correct response, allowing the data state to be inconsistent within a certain period of time;
Partition tolerance (Partition tolerance): When a distributed system encounters any network partition failure, it can still provide external services that meet consistency and availability, unless the entire network environment fails;
Four characteristics of local transactions (ACID)
Transactions should be atomic, consistent, isolated, and durable , or ACID for short.
Atomicity can be understood as all operations within a transaction are either executed or not executed.
Consistency can be understood as data that satisfies integrity constraints, that is, there will be no data in an intermediate state, and the integrity of data before and after a transaction must remain consistent. .
Isolation refers to the fact that multiple transactions will not interfere with each other when executed concurrently, that is, the data inside a transaction is isolated from other transactions.
Durability refers to the fact that after a transaction is completed, the data is stored forever, and other subsequent operations or failures will not affect the result of the transaction.
BASE theory
Basically Available: When the distributed system fails, the core is guaranteed to be available, and partial availability is allowed to be lost. (loss in response time, loss in functionality)
Soft State: The data in the system allows the existence of intermediate states, which do not affect the overall availability of the system. (paying, processing, etc.)
Eventually Consistent (Eventually Consistent): The data in the system cannot always be in a soft state, there must be a time limit, and data consistency should be guaranteed after the time limit expires. (Payment becomes successful payment)
Compared with the ADIC strong consistency model of local transactions, the BASE theory proposes to obtain availability by sacrificing certain strong consistency;
Different business units and business components have different requirements for data consistency, so BASE theory and ACID features will be used in combination in distributed systems.
Idempotent design
Idempotent is a concept in mathematics and computer science. f(n) = 1^n, no matter how much n is equal to, f(n) will always be equal to 1;
In the program, the same method is executed with the same parameters, and the result of each execution is the same, that is, it is idempotent;
Taking the idempotent design of order status processing as an example, no matter how many times the orderProcess() method is executed, the inventory will only be deducted once and true will be returned.
Distributed transaction classification
Two-phase submission 2PC (Two-Phase-Commit) | Three-phase submission 3PC (Three-Phase-Commit)
Three-phase commit introduces two mechanisms
1. Introduce a timeout mechanism. At the same time, a timeout mechanism is introduced in both the coordinator and the participants.
2. Insert a preparatory phase between the first and second phases. It is guaranteed that the state of each participating node is consistent before the final commit phase.
The main problems to be solved:
It avoids the problem that participants cannot release resources when they cannot communicate with the coordinator node for a long time (the coordinator hangs up), because the participants themselves have a timeout mechanism that will automatically perform local commit after timeout to release resources . And this mechanism also reduces the blocking time and scope of the entire transaction.
shortcoming:
The performance is poor, and there will be long-term lock tables.
Compensation transaction-TCC (Try-Confirm-Cancel)|Saga
TCC and Saga are actually compensation mechanisms adopted, and their core idea is: for each operation, a corresponding confirmation and compensation (undo) operation must be registered. Both confirmation and compensation adopt idempotent design.
Disadvantages: large amount of code, poor maintainability.
message transaction
The message consistency scheme is to ensure the consistency of upstream and downstream application data operations through message middleware. The basic idea is to put local operations and sending messages in one transaction, ensuring that either both local operations and message sending succeed or both fail. The downstream application subscribes to the message to the message system, and performs corresponding operations after receiving the message.
The message scheme is essentially to convert a distributed transaction into two local transactions, and then rely on the retry mechanism of the downstream business to achieve final consistency.
Representative product: RocketMQ
Distributed transaction product framework
Jingdong jdts
The service is connected to any node in the cluster through lb to ensure the correct execution of the business. When a certain node is abnormal, the cluster can provide services normally, and supports horizontal and vertical expansion of the cluster.
Set
An open source distributed transaction solution, dedicated to providing high-performance and easy-to-use distributed transaction services. Seata will provide users with AT, TCC, SAGA and XA transaction modes to create a one-stop distributed solution for users.
Global Transaction Service GTS
It is used to achieve high-performance transaction consistency in a distributed environment, especially in a microservice architecture. It can be used in conjunction with RDS, MySQL, PostgreSQL and other data sources, Spring Cloud, Dubbo, HSF and other RPC frameworks, MQ message queues and other middleware products to easily implement distributed database transactions, multi-database transactions, message transactions, and service link levels business and various combinations.
Author: JD Retail Gu Wei
Source: JD Cloud Developer Community