The service-oriented architecture of the 2.0 platform will inevitably be divided into libraries, which will inevitably face a distributed transaction processing problem. Therefore, both design and coding are far more work than 1.0 monolithic application architecture. However, when doing anything, the focus is not on implementation, but on thinking. Therefore, to solve the distributed transaction problem, you must first think clearly about how to do it.
There are actually a lot of distributed transaction processing methods. There is no need to worry about finding a solution. The key is to find a method that meets your own business scenarios and suits your own business scenarios. The project I did before involves a distributed transaction processing method. The one that has a clearer memory is a mobile recharge service. It is probably the business that the user completes the online payment through the bank and then recharges his mobile phone number. This involves the two consistent transactions of bank recharge and mobile phone charge recharge. What is used is to deal with this kind of transaction problem through the reconciliation every morning. The realization idea is to recharge the mobile phone bill. This end will upload an order file to an FTP server every morning, and then notify the bank to get the statement file for reconciliation. , And then implement various settlements. Of course, specific issues have to be analyzed in detail. Wouldn't it work if our current business scenarios were to directly apply this method to deal with it? Certainly not!
Our business scenario process: the user scans the code and shakes the car through the APP, consumes 1 yuan, and immediately settle accounts for all parties involved in each order.
1. The user account is deducted 1 yuan.
2. The platform is divided into 2 cents.
3. The city partner account is divided into 4 cents.
4. The merchant account is divided into 4 cents.
Remarks: In the 2.0 architecture system, we have already said that the database is divided into user database, platform database, city partner database, and merchant database.
So like this kind of business scenario, what we are currently using is to implement distributed transactions based on the final consistency of logs (events). Of course, there are many knowledge points involved, including database single-machine transactions, MQ message notifications, and scheduled scheduling. , Asynchronous, etc.
Before describing in detail the eventual consistency based on logs (events), we must first talk about flexible transactions and rigid transactions. In fact, these concepts are not that complicated. The so-called rigid transactions refer to transactions that strictly follow the ACID principles, such as those in a stand-alone environment. Database transaction is the kind of spring transaction implementation of our 1.0 platform before. Flexible transactions are a little more complicated and refer to transactions that follow the BASE theory. There are many common implementation methods: two-phase commit (2PC), TCC compensation commit, message-based asynchronous guarantee, and best-effort notification.
When I say so much, I just want to say: big transaction = small transaction (atomic transaction) + asynchronous (message notification).
There are two event tables in each library, eventPublish (event table to be published) and eventProcess (event table to be processed). The table design is as follows, and you can also design it specifically for your own business scenarios.
Well, for a simple example, the APP scans the code to start the shaker (involving two microservices).
1. Users reduce funds (database A)
2. Merchants increase funds (database B)
The idea of achieving final consistency based on logs (events) is as follows:
1. The user service starts the transaction after receiving the user request, reduces the funds, and creates a record with a status of 1 (to be published) in the eventPublish table. The payload records the event content and submits the transaction.
2. Timer scan in the user service, start the transaction in the method, and then check whether eventPublish has a record with status 1. After checking the record, get the payload information and publish the message to the merchant through MQ (the merchant has MQ real-time monitoring) .
3. The merchant service monitors the user's fund reduction event from MQ (first judge whether the database already exists, if it exists, send a message to the user through MQ, and return a successful reception message to the user), create a status in the eventProcess table as 1 (to be processed) record, the payload records the event content, if it is saved successfully, it will return a successful reception message to the client (the client has MQ real-time monitoring). (The purpose is that the client will set the status of eventPublish to 2, and it will not be scanned next time)
4. The user side listens to the message after successfully receiving it, and sets the status of eventPublish to 2 (published), and it will not be scanned next time, or it will continue to send messages to the merchant to tell the merchant to increase funds.
5. Timer scan in the merchant service, start the transaction in the method, and then check whether eventProcess has a record with a status of 1 (to be processed). After the record is checked, the payload information is obtained and the business is processed to increase funds for the merchant. After the processing is successful Modify the status of eventProcess in the database to 2 (completed), and finally commit the transaction.
This is probably the idea of implementation. This processing method has a relatively high throughput of network requests, and is basically processed asynchronously in segments.
Of course, our business scenario is more complicated than this. A process involves more atomic libraries, but the general idea is similar to the processing. The whole design is as follows:
Our specific business situation and the entire process are also being optimized.
1. Abnormal disaster tolerance processing, such as segmented entry into a certain section in the middle, unable to proceed to the next step due to various reasons, and continuous execution through MQ and scheduling multiple times. At this time, we will check if it is invalid if trying to retry multiple times , Will enter the secondary message queue, and can manually participate in processing in the background.
2. The flexibility of the single link method is further strengthened. For example, the user's next segment is not to the merchant, or may be a city partner, or other. At this time, it can be judged by the MQ transfer parameters, and the arrival point of the next segment.
In short, based on the MQ distributed business processing scenario, the stability of the bottleneck lies in the MQ itself, so this piece must ensure its continuous high availability and stability. I will continue to share the details of the subsequent problems encountered in the project.