Distributed Transaction [7] flexible services: reliable sources eventual consistency

Flexible services: reliable sources eventual consistency

Consistency messaging: means consistent with the operation of the business message transmitted message generation. That is, if the business operation is successful, the message of this business generated by the operation must be successfully delivered out (usually sent to kafka, rocketmq, rabbitmqand other messaging middleware), otherwise lose the message.

    Flexible affairs, reliable sources eventual consistency, asynchronous ensure that sexual

 The following demonstrates message transmission and delivery unreliability pseudocode:

1, database operations first, and then send a message

1. public void test1(){
2. // 1 database operations
3. 2 // send message MQ
4. }

In this case it can not guarantee the consistency of database operations and sending a message, because there may be a database operation is successful, failed to send a message

2, to send a message, then operation of the databasepublic void test1(){

1. // send a message MQ
2. // 2 database operations
3. }

In this case can not guarantee the consistency of database operations and sending a message, because it may successfully sent messages, database operation failed

3, the database transaction, the first message sent after the operation of the database

1. @Transactional
2. public void test1(){
3. // send a message MQ
4. // 2 database operations
5. }

    As used herein spring of @Transactionalannotations, which methods of operations in a transaction. The same can not guarantee consistency, because the message is sent successfully, and if the database operation fails, the database operations are rolled back, but the MQ message can not be rolled back.

4, in the transaction database, the first database operation, after sending message

1. @Transactional
2. public void test1(){
3. // 1 database operations
4. 2 // send message MQ
5. }

    In this case, seemingly no problem, if the message is sent MQ failure, throws an exception, the transaction will be rolled back (after adding a @Transactional annotation, spring method after an exception is thrown, it will automatically rollback).

    It's just an illusion, because the sending MQ messages may in fact have succeeded, if it is abnormal response timeout caused. This time, still roll back the database operations, but MQ message has actually been sent successfully, leading to inconsistencies.

5, using the JTA transaction manager

    By @Transactional spring in front of the notes plus method to open the transaction. In fact, there is a condition not explicitly say it, is that we have configured Affairs Manager is DataSourceTransactionManager.

    In fact, Spring also provides another a distributed transaction manager JtaTransactionManager. This is using the XA two-phase commit to ensure the consistency of the transaction. Of course the premise is that your messaging middleware is to achieve a JMS API specification related transaction message when (JTA specification review in front of us, mentioned DB, MQ are just the resource manager RM, for transaction manager, the two are equivalent).

    So if you meet two conditions: 1, the use of JtaTransactionManager 2, DB, MQ API were realized in two phases RM provisions of JDBC, JMS specification submitted should be implemented, we can ensure the consistency of the message sent.

    DB as RM, generally support two-phase commit. However, some MQ middleware does not support, so you have to find support two-phase commit MQ middleware. In addition, JtaTransactionManager just a proxy, you need to provide a real transaction manager (TM) implementation. As previously mentioned atomikos company, there is such a product.

    But I still do not recommend, so to play. XA two-phase commit because of low performance, we use asynchronous messaging middleware is to decouple this case, although to ensure the consistency, but the response time is greatly increased system availability decreases.

    So how to ensure consistency of database operations and messages sent by it?

Two options: one is based on MQ transaction message, the following shows RocketMQ transaction message mechanism.

Logical transaction message, performed by the sender to ensure Producer (without regard to the consumer side)

    First, it sends a transaction message, this time, RocketMQ the message status is marked as Prepared, note that at this time this news consumers are unable to consume.

    Next, execute the business logic of the code, the database may be a local transaction operation

    Finally, make sure to send a message, this time, RocketMQ status messages marked as consumption, this time consumers, in order to truly guarantee the consumer to this data.

    If the confirmation message fails how to do? RocketMQ periodically scans the transaction message is a message in the cluster, if we find Prepared message, it will confirm the message the sender (producer). RocketMQ will be to decide whether to roll back or continue to send a confirmation message according to policies set by the sender. This ensures that messages sent with the local transaction succeed or fail.

    If the consumer fails how to do? Ali provide our solution is: artificial resolved.

    

    Another realization that not all mq support the transaction message. Once the message is sent to a message queue, it can consume to consumers immediately. At this point you can use a separate message service, or the local transaction table.

    You can see, in fact, a message is sent first to write ourselves an "independent news service" applications, beginning in prepare state, after the success of the business logic processing, send an acknowledgment message, this time "independent news service" will the real message queue to send the message. After the success of consumer spending, ack when, in addition to the message queue ack (not shown), for independent news service must also be ack, "the independent news service" generally delete these messages. Scheduled scanning prepare the message status, message acknowledgment to the transmitting side (producer) work done by independent message service.

    For the "local transaction table", and in fact, the role of "independent news service" similar, but "independent news service" is the need to deploy independent, and "local transaction table" is the "Independent news service" function built into the application in.