Today, I would like to share a distributed thing with you. I will introduce common distributed transaction implementation schemes and their advantages and disadvantages as well as applicable scenarios, and will bring out some of their first variant implementations.
I will also take a look at the improved model of distributed database to 2PC and see how distributed database is done.
Then analyze the specific implementation of a wave of distributed transaction framework Seata to see how distributed transactions are implemented. After all, the agreement is useful if it is implemented.
First of all, let's talk about what transactions and distributed transactions are.
Affairs
The ACID of transactions must be familiar to everyone. This is actually a definition in the strict sense, which means that the realization of transactions must have atomicity, consistency, isolation, and durability.
However, transactions in the strict sense are difficult to achieve. As we know databases, there are various isolation levels. The higher the performance, the lower the isolation level. So often we will find our own balance from it, and will not follow the transactions in the strict sense.
And in our daily discussions, the so-called transaction often simply refers to a series of operations that are all executed successfully or all failed, and there will be no success or failure.
After we have clarified our daily definition of transaction, let's take a look at what is distributed transaction.
Distributed transaction
Due to the rapid development of the Internet, the previous monolithic architecture could not withstand so many demands, such a complex business, and such a large amount of traffic.
The advantage of the monolithic architecture is that it is quickly built and online in the early stage, and methods and modules are all internally called, and it is more efficient without network overhead.
From a certain aspect, deployment is also convenient, after all, it's just a package and throw it up.
However, with the development of the enterprise, the complexity of the business is getting higher and higher, and the internal coupling is extremely serious, which causes the whole body to be affected, development is not easy, and testing is not easy.
And it cannot be dynamically scaled according to hotspot services. For example, the amount of access to goods and services is extremely large. If it is a single architecture, we can only replicate the entire application for multiple cluster deployments, which wastes resources.
Therefore, the split is imperative, and the microservice architecture is here.
After the split, the boundaries between services are clear, and each service can run and deploy independently, so it can be elastically scaled at the service level.
Local calls between services become remote calls, the link is longer, and the time for one call is longer, but the overall throughput is greater.
However, other complications will be introduced after the split, such as service link monitoring, overall monitoring, fault-tolerant measures, elastic scaling, and other operational and maintenance monitoring issues, as well as issues such as distributed transactions, distributed locks and business related issues. Wait.
It often solves one pain point and introduces other pain points, so the evolution of the architecture is the result of trade-offs, and it depends on which pain points your system can bear.
Today we are talking about the pain point of distributed transactions.
Distributed transactions are composed of multiple local transactions. Distributed transactions span multiple devices and experience a complex network between them. It is conceivable that the road to strict transactions is difficult and long.
The stand-alone version of the transaction will not strictly follow the strict implementation of the transaction, let alone the distributed transaction, so in reality we can only implement the incomplete version of the transaction.
After clarifying transactions and distributed transactions, let's take a look at common distributed transaction schemes: 2PC, 3PC, TCC, local messages, transaction messages.
2PC
2PC, Two-phase commit protocol, that is, two-phase commit protocol. It introduces a transaction coordinator role to manage each participant (that is, each database resource).
The whole is divided into two phases, namely the preparation phase and the commit/rollback phase.
Let's take a look at the first stage, the preparation stage.
The transaction coordinator sends a preparation command to each participant. After each participant receives the command, the relevant transaction operation will be executed. You can think that everything except the commit of the transaction has been done.
Then each participant will return a response to inform the coordinator whether he is ready to succeed.
After the coordinator receives the response of each participant, it enters the second stage. According to the collected response, if one participant fails to prepare for response, it will send a rollback command to all participants, otherwise it will send a commit command.
This agreement is actually in line with normal thinking. Just like when we call for a class in a university, the teacher is actually the coordinator, and we are all participants.
The teacher called one by one, and we shouted one by one. Finally, the teacher started today's lecture after receiving all the students' arrival.
The difference from roll call is that the teacher can still continue the class when a certain number of students are absent, and our affairs do not allow this.
The transaction coordinator does not receive a response from individual participants in the first stage, and after waiting for a certain period of time, the transaction will be considered as a failure and a rollback command will be sent. Therefore, the transaction coordinator has a timeout mechanism in 2PC.
Let's analyze the advantages and disadvantages of 2PC.
The advantage of 2PC is that it can use the database's own functions to commit and roll back local transactions, which means that the actual operations of commit and rollback do not need to be implemented by us, and the business logic is not invaded by the database. After explaining the TCC, I believe everyone is concerned about this Point will have some experience.
2PC has three main disadvantages: synchronization blocking, single point of failure, and data inconsistency.
Synchronous blocking
It can be seen that after the preparation command was executed in the first stage, each of our local resources was locked because we did everything except the commit of the transaction.
So at this time, if other local requests want to access the same resource, for example, if you want to modify the data of the product table id equal to 100, then it is blocked at this time, and you must wait for the completion of the previous transaction and receive the commit/rollback command. After executing the release of resources, the request can be continued.
So suppose that this distributed transaction involves many participants, and then some participants are particularly complex and slow to process, then those nodes that process fast have to wait, so the efficiency is a bit low.
Single point of failure
It can be seen that this single point is the coordinator, and if the coordinator hangs up the entire transaction, it cannot be executed.
If the coordinator hangs up before sending the prepare command, after all, every resource has not executed the command, then the resource is not locked.
The terrible thing is that it hangs after sending the preparation command. At this time, every local resource is executed and locked, and it is very stiff. If a hot resource is blocked, it is estimated to be It's GG.
Data inconsistency
Because the communication between the coordinator and the participants is through the network, and the network sometimes convulsions or local network abnormalities occur.
Then it may cause some participants to fail to receive the request from the coordinator, and some to receive it. For example, a request is submitted, and then those participants who received the order submit the transaction. At this time, the problem of data inconsistency occurs.
To summarize 2PC
So far, let's summarize some 2PC, which is a two-phase commit protocol with synchronous blocking and strong consistency, which is the preparation phase and the commit/rollback phase.
The advantage of 2PC is that there is no intrusion into the business, and the database itself can be used to commit and roll back transactions.
Its disadvantages: it is a synchronous blocking protocol, which will cause high latency and performance degradation, and there will be a single point of failure of the coordinator, and in extreme cases there will be data inconsistency.
Of course, this is just an agreement, and the specific implementation can still be modified. For example, if the coordinator has a single point, I will make a master and implement the coordinator, right.
2PC improved model of distributed database
Maybe some people are not familiar with distributed databases, it doesn't matter, what we mainly learn is thinking, look at other people's ideas.
Let me briefly talk about the Percolator model. It is a model based on the distributed storage system BigTable. It doesn't matter if BigTable is a student who doesn't know what it is.
Let's take the example of transfer. I have 200 yuan now, and you now have 100 yuan. In order to highlight the key points, I don't draw this table according to the normal structure.
Then I want to transfer 100 yuan to you.
At this time, the transaction manager initiates a preparation request, and then the money on my account is less, and your account has more money, and the transaction manager also records the log of this operation.
The data at this time is still a private version. Whether other transactions are not available, simply understand that if there is a value on Lock, it is still private.
You can see that my record Lock is marked with a PK, and your record is marked with a pointer to my record. This PK is randomly selected.
Then the transaction manager will initiate a commit command to the record selected as the PK.
At this time, the lock of my record will be erased, which means that my record is no longer a private version, and other transactions can be accessed.
Is there a lock on your record? No need to update?
Hehe does not need to be updated in time, because when accessing your record, you will find my record based on the pointer and find that the record has been submitted so your record can be accessed.
Some people say that the efficiency is not bad, you have to look for it every time, don't worry.
There will be a thread in the background to scan, and then update the lock record.
Isn't this stable?
Improved compared to 2PC
First of all, Percolator does not need to interact with all participants in the submission phase, the main needs to deal with one participant, so this submission is atomic! Solve the problem of data inconsistency.
Then the transaction manager will record the operation log, so that when the transaction manager hangs up, the elected new transaction manager can learn the current situation through the log and continue to work, solving the single point of failure problem.
And Percolator will also have a background thread that will scan the transaction status, and will roll back the transactions on each participant after the transaction manager goes down.
It can be seen that compared to 2PC, many improvements have been made, and they are also clever.
In fact, there are other transaction models for distributed databases, but I am not too familiar with it, so I won't beep too much, and interested students can learn about it by themselves.
It can still broaden the mind.
XA specification
Let us come back to 2PC. Now that we talked about 2PC, I also briefly mentioned the XA specification. The XA specification is based on two-phase submission, which implements the two-phase submission protocol.
Before talking about the XA specification, we have to mention the DTP model, namely Distributed Transaction Processing, which regulates the model design of distributed transactions.
The XA specification restricts the interaction between the transaction manager (TM) and the resource manager (RM) in the DTP model. Simply put, the two of you must communicate in accordance with a certain format specification!
Let's first look at the DTP model under the XA constraint.
- The AP application is our application, the initiator of the transaction.
- RM resource manager, simply think of it as a database, with transaction commit and rollback capabilities, corresponding to our above 2PC is a participant.
- The TM transaction manager is the coordinator, communicating with each RM.
Simply put, AP uses TM to define transaction operations. TM and RM communicate through the XA specification and perform two-phase commit. AP's resources are taken from RM.
From the perspective of the model, there are three roles, but the actual implementation can be implemented by one role to achieve two functions, such as AP to achieve the function of TM, TM does not need to be deployed separately.
MySQL XA
After knowing DTP, let's take a look at how XA works in MySQL, but only InnoDB supports it.
Simply put, it is necessary to define a globally unique XID first, and then inform each transaction branch to perform the operation.
You can see that two operations have been performed in the figure, namely changing the name and inserting the log, which is equivalent to the first thing to do under registration. The SQL to be executed is wrapped by XA START XID and XA END XID.
Then you need to send a prepare command to execute the first stage, which is the stage where everything except the commit of the transaction is done.
Then according to the situation of preparation to choose to execute the commit transaction command or roll back the transaction command.
Basically it is such a process, but the performance of MySQL XA is not high.
It can be seen that although 2PC has shortcomings, there are still implementations based on 2PC. The introduction of 3PC is to solve some of the shortcomings of 2PC, but it is more expensive as a whole and cannot solve the problem of network partitioning. I have not found The implementation of 3PC.
But let me mention it a little bit, just know it, pure theory.
3PC
The introduction of 3PC is to solve 2PC synchronization blocking and reduce data inconsistency.
3PC is an additional stage, an inquiry stage, which is the three stages of preparation, pre-submission and submission.
The preparation stage is simply the coordinator visiting the participants. Is it okay to be similar to you? Can accept the request.
Pre-commit is actually the preparation phase of 2PC, except for transaction commit.
The submission phase is the same as that of 2PC.
In fact, 3PC has an additional stage to confirm whether the participant is normal before executing the transaction, preventing individual participants from being abnormal, and other participants performing the transaction and locking resources.
The starting point is good, but in most cases it is definitely normal, so it is not cost-effective to have an additional interactive phase every time.
Then 3PC also introduced a timeout mechanism at the participants, so that if the coordinator hangs up, if the commit phase has been reached, the participants will automatically commit the transaction if they wait for a long time without receiving the coordinator.
But what if the coordinator issues a rollback command? You see this is wrong, the data is inconsistent.
There is also Wikipedia that after the 2PC participant preparation phase, if the coordinator hangs up, the participants will not be able to know the overall situation, because the overall situation is controlled by the coordinator, so they are not clear about the situation between the participants.
3PC has passed the first stage of confirmation. Even if the coordinator hangs up the participants, they know that they are in the pre-commit stage because they have been approved by all participants in the preparation stage.
Simply put, it is like adding a fence to make the status of each participant unified.
Summary 2PC and 3PC
It has been known from the above that 2PC is a strongly consistent synchronous blocking protocol, and its performance is already relatively poor.
The starting point of 3PC is to solve the shortcomings of 2PC, but one more stage will increase the communication overhead, and it is useless communication in most cases.
Although the participant timeout is introduced to solve the blocking problem that the coordinator hangs, the data will still be inconsistent.
It can be seen that the introduction of 3PC has no actual breakthrough, and the performance is worse, so only the implementation of 2PC is actually implemented.
Again, both 2PC and 3PC are protocols, which can be considered as a guiding ideology, which is different from the actual implementation.
TCC
I don't know if you have noticed, whether it is 2PC or 3PC, it depends on the transaction commit and rollback of the database.
And sometimes some business involves not only the database, it may be sending a text message or uploading a picture.
Therefore, the commit and rollback of transactions must be promoted to the business level rather than the database level, and TCC is a two-phase commit at the business level or application level.
TCC is divided into three methods that refer to Try, Confirm, and Cancel, that is, three methods that need to be written at the business level. They are mainly used for data consistency issues in cross-database and cross-service business operations.
TCC is divided into two phases, the first phase is the resource check reservation phase, namely Try, the second phase is submission or rollback, if it is submitted, it is to perform real business operations, if it is rollback, it is to perform reserved resources Cancel and restore the initial state.
For example, if there is a deduction service, I need to write a Try method to freeze the deduction funds, and also need a Confirm method to perform the real deduction, and finally I need to provide Cancel to roll back the freezing operation, corresponding to a transaction All services need to provide these three methods.
It can be seen that there was one method, but now it needs to be expanded into three methods, so TCC has a great intrusion into the business, like if there is no frozen field, the structure needs to be changed.
Let's look at the process.
Although there is an intrusion to the business, TCC has no resource blockage. Each method submits the transaction directly. If an error occurs, it is compensated through the Cancel at the business level, so it is also called the compensatory transaction method.
Someone here said that if everyone tried successfully and executed Comfirm, but some Confirm failed, what should I do?
At this time, you can only retry and adjust the failed Confirm until it succeeds. If it really doesn't work, you can only record it, and then manual intervention.
Points to note for TCC
These points are very important, and you must pay attention to them when you implement them.
The problem of idempotence, because network calls cannot guarantee that the request will arrive, so there will be a readjustment mechanism. Therefore, the three methods of Try, Confirm, and Cancel need to be implemented idempotent to avoid errors caused by repeated execution.
The empty rollback problem means that the Try method did not receive a timeout due to network problems. At this time, the transaction manager will issue a Cancel command, so it is necessary to support Cancel to Cancel normally without executing Try.
Suspension problem, this problem also refers to the Try method due to the network congestion timeout triggered the transaction manager to issue a Cancel command, but after the Cancel command is executed, the Try request arrives, you are out of breath.
Cancel waits for you to have a Try. For the transaction manager, the transaction is over at this time, and the freezing operation is "suspended", so after the empty rollback, you have to record it to prevent the Try from being called again.
TCC variants
We are talking about the general-purpose TCC above. It needs to transform the previous implementation, but there is a situation that cannot be modified, that is, you are calling the interface of another company.
TCC without Try
For example, you need to transfer by plane, and the transfer is a different airline. For example, from A to B, then from B to C, it only makes sense to buy tickets for both A-B and B-C.
At this time, you don’t have to try, you can directly call the airline’s ticket purchase operation. When both airlines buy successfully, it will directly succeed. If a company fails to buy, then you need to call the cancel booking interface. .
That is, the entire business operation is executed directly in the first stage, so focus on the rollback operation. If the rollback fails, there is a reminder, and manual intervention is required.
This is actually the idea of TCC.
Asynchronous TCC
Can this TCC be asynchronous? In fact, it is also a kind of compromise. For example, some services are difficult to transform, and it will not affect the main business decision, that is, it is not so important and does not require timely execution.
At this time, reliable message services can be introduced, and individual services can be replaced by message services for Try, Confirm, and Cancel.
When trying, only the message is written, and the message cannot be consumed yet. Confirm is the operation to actually send the message, and Cancel is to cancel the sending of the message.
This reliable message service is actually similar to the transaction message to be mentioned later. This solution is equivalent to a combination of transaction message and TCC.
TCC summary
It can be seen that TCC implements transaction commit and rollback through business code, which is more intrusive to the business. It is a two-phase commit at the business level.
Its performance is higher than 2PC, because there will be no resource blocking, and the scope of application is larger than 2PC. Pay attention to the several points mentioned above in the implementation.
It is a commonly used method of distributed transaction implementation in the industry, and it can be known from the variants. It still depends on business flexibility. It does not mean that you must use TCC to rigidly transform all services into those three methods. .
Local message table
Local message is the use of local transactions, the local transaction message table will be stored in the database, and the local message insertion is added in the local transaction operation, that is, the execution of the business and the operation of putting the message into the message table are placed in the same Commit in transaction
In this way, if the local transaction is executed successfully, the message must be inserted successfully, and then other services are called. If the call is successful, the status of the local message is modified.
If it fails, it does not matter. There will be a background thread scanning, and it will always call the corresponding service when it finds these status messages. Generally, the number of retries will be set. If it does not work, it will be recorded specially and will be processed by manual intervention.
It can be seen that it is still very simple, and it is also a kind of best effort notification thought.
Transaction message
I actually wrote an article about transaction messages, analyzing the transaction message implementation of RocketMQ and Kafka from the source level, and the difference between the two.
I won't elaborate here, because the previous article is very detailed, about four to five thousand words. I will attach the link: transaction message
Seata's implementation
First of all, what is Seata, excerpt from the official website.
Seata is an open source distributed transaction solution dedicated to providing high-performance and easy-to-use distributed transaction services. Seata will provide users with AT, TCC, SAGA and XA transaction modes to create a one-stop distributed solution for users.
You can see that many modes are provided, let's take a look at the AT mode first.
AT mode
The AT mode is a two-phase submission. Earlier we mentioned the problem of synchronous blocking in the two-phase submission. The efficiency is too low. How does Seata solve it?
At the first stage, AT commits the transaction directly and directly releases the local lock. Is it so rashly and directly committed? Of course not, this is similar to the local message table, that is, using local transactions, the rollback log will be inserted during the actual transaction operation, and then committed in a transaction.
How did this rollback log come from?
Through some classes of the framework proxy JDBC, when SQL is executed, the SQL is parsed to obtain the data mirror before execution, and then the SQL is executed, and the data mirror after execution is obtained, and then the data is assembled into a rollback log.
The subsequent submission of this local transaction also inserts the rollback log into the UNDO_LOG table of the database (so the database needs an UNDO_LOG table).
This wave of operations can commit transactions without worries in one stage.
Then, if the first stage succeeds, the second stage can delete those rollback logs asynchronously, and if the first stage fails, the rollback log can be used to reverse compensation and recovery.
At this time, a careful classmate thought, what if someone changes this data? Your mirror image is wrong?
So there is also the concept of a global lock. You need to get a global lock (which can be understood as a lock on this data) before the transaction is committed, and then the local transaction can be successfully submitted.
If you can't get it, you need to roll back the local transaction.
The example on the official website is very good, so I won’t edit it myself. The following part is excerpted from the example on Seata’s official website:
At this time, there are two transactions, namely tx1, and tx2, which update the m field of table a, and the initial value of m is 1000.
tx1 starts first, opens the local transaction, gets the local lock, and updates m = 1000-100 = 900. Before the local transaction is committed, the global lock of the record is obtained first, and the local commit is released to release the local lock.
After tx2, start the local transaction, get the local lock, and update m = 900-100 = 800. Before the local transaction is committed, try to get the global lock of the record. Before tx1 is globally committed, the global lock of the record is held by tx1, and tx2 needs to retry and wait for the global lock.
You can see that the modification of tx2 is blocked, and after retrying to get the global lock, you can commit and release the local lock.
If tx1 rolls back globally in the second stage, tx1 needs to reacquire the local lock of the data, perform a reverse compensation update operation, and implement the branch rollback.
At this point, if tx2 is still waiting for a global lock for the data and holds a local lock at the same time, the branch rollback of tx1 will fail. The rollback of the branch will be retried until the locks such as the global lock of tx2 expire, the global lock is abandoned and the local transaction is rolled back to release the local lock, and the branch rollback of tx1 is finally successful.
Because the entire process is held by tx1 until the global location tx1 ends, the problem of dirty writing will not occur.
Then the AT mode defaults to the isolation level of read uncommitted globally. If the application is in a specific scenario, the global read must be required to be submitted, which can be delegated by the SELECT FOR UPDATE statement.
Of course, the premise is whether your local transaction isolation level has been committed and above.
AT mode summary
It can be seen that the front and back mirrors of the data are obtained without intrusion through the agent, and the rollback log is assembled into a rollback log and submitted with the local transaction, which solves the problem of two-phase synchronization blocking.
And use global locks to achieve write isolation.
For overall performance considerations, the default is the read uncommitted isolation level, and only the SELECT FOR UPDATE agent is used to perform read submitted isolation.
This is actually a variant implementation of the two-phase commit.
TCC mode
There is nothing fancy, that is, we need to do three methods analyzed above, and then incorporate custom branch transactions into the management of global transactions
I posted a picture of the official website that should be quite clear.
Saga mode
This Saga is a long transaction solution provided by Seata, which is suitable for the case of many and long business processes. In this case, if you want to implement a general TCC or something, you may have to nest multiple transactions.
And some systems cannot provide the three interfaces of TCC, such as old projects or other companies' companies, so they have developed a Saga mode. This Saga was proposed in a paper published by Hector & Kenneth in 1987.
How does Saga do it? Take a look at this picture.
Assuming there are N operations, directly starting from T1 is to directly execute the commit transaction, and then execute T2, you can see that it is a direct commit without lock. At T3, it is found that the execution has failed, and then it enters the Compenstaing phase, and starts to compensate one by one. Up.
The idea is to cover your head and do it at first, don't be persuaded, if something goes wrong, we will change back one by one.
It can be seen that this situation does not guarantee the isolation of things, and Saga also has the same attention points of TCC, which requires air compensation, anti-hanging and idempotence.
And in extreme cases, the data cannot be rolled back because the data has been changed. For example, in the first step, I called me 20,000 yuan and I took it out and spent it. At this time, you roll back and the balance on my account is already 0. What do you think? Could it be that it's not enough for me to fail?
This situation can only be started in the business process. In fact, I have always written the code like this. Take the scenario of buying a skin, I always deduct the money before giving the skin.
Suppose if you fail to deduct money from the skin first, won’t you give it out for nothing? Can you make up this money? Do you think users will give feedback that the skin has not deducted the money?
Maybe a little clever ghost said that I will change my skin back then, hehe, this kind of thing has indeed happened, tusk, it's terrible to be scolded.
So the correct process should be to first deduct the money and then give the skin. The money is in your pocket first. If the skin is not given to the successful user, the user will naturally find it. Then give it to him. Although you may have written a BUG, it is not a good thing. BUG given for nothing.
So this point must be paid attention to when coding.
At last
It can be seen that distributed transactions still have various problems, and the implementation of general distributed transactions can only achieve final consistency.
In extreme cases, manual intervention is still necessary, so good log records are critical.
There are also coding business processes. Write in a direction that is beneficial to the company. For example, first get the user’s money and then give the user something. Remember.
Before going to distributed transactions, think about it, is it necessary, can you change it to avoid distributed transactions?
To be more extreme, does your business need to go to business?