In the current application system, whether it is an enterprise-level application or an Internet application, the final data consistency is a problem that every application system has to face. The bullet solution is not something that can be solved by introducing specific middleware or a specific open source framework . It depends more on business scenarios and provides solutions according to the scenarios. According to the author's understanding in recent years, I have summarized several points. More application systems pay more attention to the consistency of data when coding, so that the system is robust.
1. Basic theory
At present, several theories about transactions include: ACID transaction characteristics, CAP distributed theory, and BASE , etc. ACID is reflected in database transactions, while CAP and BASE are theories of distributed transactions . Combined with business systems, such as order management, warehouse management, etc., these theories can be used for reference to solve problems.
1. ACID features
2. CAP characteristics
- C (consistency) Consistency refers to the atomicity of data, which is guaranteed by transactions in classic databases. When a transaction is completed, regardless of success or rollback, the data will be in a consistent state. In a distributed environment, consistency is Refers to whether the data of multiple nodes is consistent;
- A (availability) service remains available all the time. When the user sends a request, the service can return the result within a certain period of time;
- P (partition tolerance) In distributed applications, the system may not be able to operate due to some distributed reasons. Good partition tolerance makes the application a distributed system, but it seems to be a functioning whole
3. BASE features
- BA: Basic Availability Basic business availability;
- S: Soft state flexible state;
- E: Eventual consistency final consistency;
Second, the common practice of eventual consistency
1. Single database transaction
If the application system is a single database, then this is well guaranteed, and the transactional characteristics of the database are used to meet the consistency of the transaction, and the consistency at this time is strong consistency. For java application systems, it is rarely hard-coded directly through the start, commit and rollback of transactions, and most of them are guaranteed through spring's transaction templates or declarative transactions;
2. Multiple database transactions
For multi-database transactions, spring 3.0 + Atomikos + JTA can be used for support according to the two-phase commit protocol;
3. Eventual consistency based on transactional message queues
With the help of the message queue, the message is sent in the place where the business logic is processed. After the business logic is successfully processed, the message is submitted to ensure that the message is sent successfully, and then the message queue is delivered for processing. If it is successful, it will end. If it is not successful, try again. , until it succeeds, but it only applies to the business logic where the first stage is successful and the second stage must be successful. Corresponds to the C process in the above figure.
4. Final consistency based on message queue + timing compensation mechanism
The difference between the previous part and the above queue based on transactional messages is that the retry in the second stage is no longer the retry logic of the message middleware itself, but a separate compensation task mechanism. In fact, in most logics, the probability of failure in the second stage is relatively small, so it can be more clear to list the independent compensation tasks separately, and it can be more clear how many tasks have failed until now. Corresponds to the E process in the above figure.
5. Introduction of asynchronous callback mechanism
Application A calls B. In the return result of the synchronous call, B returns successfully to A. In general, it ends at this time. In fact, it is no problem in 99.99% of the cases, but sometimes in order to ensure 100%, remember Live at least 100% in the system design. At this time, system B will call back to A again and tell A that you have succeeded in calling my logic. In fact, this logic is very similar to the three-way handshake in the TCP protocol. Process B in the figure above.
6. Confirmation mechanism similar to double check mechanism
It is still the process of asynchronous callback in the above figure. A calls B synchronously, and B returns successfully. This call is over, but in order to ensure that after a period of time, this time can be a few seconds, or it can be processed regularly every day, and call B again to check whether the previous call was successful. For example, A calls B to update the order status, which is successful at this time. After a delay of a few seconds, A queries B to confirm whether the status is what he just expected. Process D in the figure above.
3. Disadvantages of Distributed Transactions
1. Disadvantages of two-phase commit protocol
Two-phase commit involves network communication of multiple nodes. If the communication time is too long, the relative time of the transaction will be too long, and the time to lock resources will also be long. In high concurrency services, there will be serious problems performance bottle
2. Message queue
In a high concurrency environment, we generally use message queues to avoid the execution of distributed transactions.
When using message queues, we need to save reliable credentials (messages of distributed transactions) in the following ways:
Take Alipay and Yu’ebao as examples to illustrate.
When Alipay completes the action of deducting money, it records the message data, and stores the message data and business data in the same database instance.
Begin Transaction update A set amount=amount-1000 where uid=100; insert into message(uid,amount,status) values (1,1000,1) End Transaction Commit;
Send the message that Alipay has completed the deduction of money to Yu'ebao in time. After Yu'ebao has completed the processing, it will return a success message. After Alipay receives the message, it will delete the corresponding message record in the message table to complete the deduction operation.
The traditional way is, when I'm done, I'll send you a message. The solution to the consistency means that I will send you a message first, and then confirm with you that I am done when I am done. This is an improved transactional messaging middleware.
See: http://coolshell.cn/articles/10910.html