RabbitMQ message queue-based distributed transaction solutions - MQ distributed messaging middleware combat

1 Speed ​​understand MQ

Introduction Rabbitmg for solving the distributed transaction must master the five core concepts

A distributed messaging middleware, high concurrency processing power based on erlang language development, with the language level. And the Spring Framework is the same company.
Support persistence, high availability

Five core concepts:

1. Queue: Local real store data
2. Exchange: receiving a request, the data dump
3. Bind: After receiving the request where to store
4. Application data is transmitted: message producer
5. Message consumer: Remove the application data processing
   

2, distributed transaction issues

Distributed transaction is a business issue, not be divorced from the specific scene.

Several solutions 2.1 distributed transactions

● based database XA / JTA way of an agreement
requires the database vendor support; JAVA component has atomikos etc.
● asynchronous proofread the way data
Alipay, micro-channel pay initiative to check payment status, in the form of bills;
● solutions are based on reliable sources (MQ) of
asynchronous scene; universal strong; expansion of higher
● TCC programming solutions
carefully selected, Ali, ants gold dress own package DTX

This article goal: for all the people, learn to solve the problem of distributed transactions based on reliable sources.
Solutions for distributed transaction, highly targeted business, it is important ideas instead of copying

• US Mission comment system architecture
  

Distributed transaction issues between multiple systems 2.2

• Single user generated orders
  
• Orders need to pass data, thereby generating two transactional consistency issues
  

Wrong Case

When the interface call fails, the transaction is rolled back order system, the user is prompted operation fails

误以为这样的接口调用写法，就不会有分布式事务问题

Interface call success or failure, will have a distributed transaction issues:

1. Interface call is successful, the order system database transaction commit fails, the system does not roll back the waybill, generate data
2. Interface calls a timeout, the order system database transaction rollback, waybill system interface continues to generate data

In both cases, it can lead to data inconsistencies

3, to achieve a distributed transaction - Five-Step

Resolve distributed transactions via MQ 5 steps, as well as local distributed transaction processing to note

• Before ordering system to send all HTTP request interface waybill system, a problem!
• Therefore, we consider a message to MQ, asynchronous scratch!
  

3.1 overall design concept

After orders catering, slowly waiting waybill generating data center, is not mandatory simultaneity

1. Reliable production: to ensure that the message must be sent to the service Rabitmq
2. Reliable consumption: to ensure that the message must take out proper consumed

And finally to multi-party data to reach consensus.

3.2 Step 1 - producing reliable information recorded message

• There are hidden dangers - possible messages failed to send it!
  

To ensure the success of certain data is sent to the MQ.
In the same transaction, a record increase in the operating table, the record 每一条发往MQ的数据以及它的发送状态
so we add a local information table in the order system

So in practice the code, not by the HTTP interface call billing systems interfaces, but the use of MQ

When generating orders, but also save the local information table

3.3 Step 2 - Production Reliable Messaging (modification message transmission state)

• Use RabbitMQ transactional publication acknowledgment mechanism (confirm)
  after opening, MQ accurate acceptance message Return Receipt

• Then be able to know how to update the local information table
  

- Confirm mechanism to ensure open in SB

• If the acknowledgment is not received, the message modification failed state and other special circumstances
  兜底方案:定时检查消息表，超时没发送成功，再次重发

3.4 Step 3 - reliable messaging processing (normal processing)

• After waybill system receives the message data suddenly goes down, or visit the waybill when DB, DB suddenly goes down, the message data is not lost on you !!!
  

So we need the following characteristics:

Idempotent
prevent duplicate processing of the message data, a user operation, data processing time corresponding to only

open 手动ACK模式
by the consumer control message retransmission / remove / discard

3.5 Step 4 - reliable messaging (message retransmission)

Consumers processing fails, we need to re-MQ again to the consumer.
Abnormalities usually retry several times by consumers themselves record number of retries, and the number of controls (not always retry!)

Step Five 3.6 - reliable messaging (message discard)

Consumer handling failure, or discarded directly transferred to a dead letter queue (the DLQ)
重试次数过多、消息内容格式错误等情况，通过线上预警机制通知运维人员

4 Summary and extensions

Pros and Cons of 4.1 MQ program

Opening advantage

1. Versatility
2. Development and strong
3. Mature program

Mouth shortcomings

1. Message-based middleware, only for asynchronous scenarios
2. Message processing will be delayed, need to be able to tolerate on business

Try to avoid distributed transaction;
try to make asynchronous non-core transactions;

4.2 expansion

Theoretical basis for distributed transaction solutions

CAP theory
BASE theory
2PC protocol
3PC protocol
Paxos algorithm.
Raft coherence protocol