Introduction to business scenarios
Single system order
In the single e-commerce system, the user orders successfully, because the entire ordering operation is completed in the same method, and the table corresponding to the commodity and the table corresponding to the order are in the same database, according to the atomicity of the database , Consistency, Isolation or Durabilily. After the ordering service is successful, the inventory is deducted and the data for generating the order operation will be effective at the same time. The process is as follows:
Distributed system order
In a distributed system, the ordering process is micro-serviceized, and the ordering process is split into order services and commodity services. At this time, the database is also decomposed and divided into corresponding order databases and commodity databases. If it is directly replaced with the corresponding microservice according to the method in the same transaction in the above process, the flow chart is as follows:
When using RPC or HTTP requests to call the goods service and order service in the order service, the data of the goods may be inconsistent with the data of the order database due to abnormal network or server downtime . There may be an order service. Due to the abnormal data rollback, no data is generated in the order database when the goods are called, and the goods service processing is successful. The inventory deduction in the goods database is successful. An example of the reason for the network abnormality or server downtime is as follows:
Distributed transaction
Two-phase database submission
How to solve the problem of inconsistent data after splitting into microservices as above? At this time, it is necessary to introduce distributed transactions. If the inventory deduction and order generation are to take effect in the distributed system at the same time, it can be achieved through the two-stage submission method of the database . According to the CAP theory (for details, please refer to the in-depth understanding of CAP theory and applicable scenarios ) 2PC belongs to the CP mode .
Students who understand the distributed affairs of the database must know that the 2PC supported by the database is also called XA Transactions. Among them, XA is a two-phase submission agreement, the agreement is divided into the following two stages:
- The first stage: the transaction coordinator requires each database involved in the transaction to precommit (precommit) this operation and reflect whether it can be committed.
- The second stage: the transaction coordinator requires each database to submit data.
Among them, if any database vetoes this submission, then all database operation data will be rolled back. This method will have serious performance impact, and also has the following disadvantages:
- Synchronous blocking problem! During execution, all participating nodes are transaction blocking. When participants occupy public resources, access to public resources by other third-party nodes is also blocked.
- Single point of failure! Due to the importance of the coordinator, once the coordinator fails, the participants will continue to block. Especially in the second stage, when the coordinator fails, all participants are still in the state of locking transaction resources, and cannot continue to complete transaction operations.
- The data is inconsistent! In the two-phase submission process, when the coordinator sent a commit request to the participants, a network abnormality or server downtime occurred , which would result in partial data submission success and partial submission failure, resulting in data inconsistency.
Message middleware + HttpClient to implement distributed transactions
In distributed systems, the usability of the system is generally paid more attention. Based on this, people have proposed the BASE theory, which is used to further expand the CAP theorem. BASE theory refers to:
- Basically Available
- Soft state
- Eventually consistent
BASE theory is the result of a trade-off between consistency and availability in CAP. The core idea of the theory is: we cannot achieve strong consistency, but each application can use appropriate methods to make the system achieve according to its business characteristics Eventual consistency . In the project, Kafka + HttpClient was used to implement distributed transactions based on BASE theory. The idea of implementation is as follows:
- The service call initiator sends the http information of the called service through Kafka messages , and the success and fail service information of the callback service call initiator . At this time, the business object status of the service call originator uses the soft status , for example, the order status is in the order.
- Create a consumer that forwards http request message middleware to request the http request of the called service. After the called service is successfully executed, it determines the callback service to call the success service or fail service of the initiator according to the status returned by the HTTP request, so that the status of the service object of the service initiator can reach the final consistency , for example, the order is successful or the order fails .
- In order to ensure that the message is consumed, it is necessary to use a synchronous transmission strategy when sending the message. When consuming http request information, after the message needs to confirm consumption, manually submit the partition offset to ensure that the message must be consumed. At this time, the http request service of the callee, and the success and fail services of the service initiator need to satisfy the idempotent design .
Take the following single service as an example, the flow chart is as follows:
The rest of the implementation methods can refer to chatting about distributed transactions, and then talk about the solution
