1 Interview questions
How to ensure the order of messages?
2 Test site analysis
MQ must ask topics
- Check whether you understand sequentiality
- Check if you have a way to ensure the order of messages, because this is a common problem in production systems.
3 Detailed
3.0 Case
A MySQL binlog synchronization system, with daily synchronization data reaching hundreds of millions.
增删改A piece of data in MySQL- That is to add, delete and modify 3 binlogs
- Then these three binlogs are sent to MQ
- Consumed and executed sequentially
It should be ensured that the messages are executed in order!
Otherwise , it was originally: add -> modify -> delete.
You were shocked to change the order to execute: delete -> modify -> add,
all wrong!!!
The data was synchronized, and it was supposed to be deleted in the end. As a result, you made a mistake in the order, but in the end it was retained, and there was an error in data synchronization!
3.1 Out-of-order scenes
3.1.1 rabbitmq
One queue, multiple consumers, this is not obviously messed up
3.1.2 kafka
A topic, a partition, a consumer, internal multi-threading, this is also obviously messed up
3.2 Ensure the order of messages
3.2.1 rabbitmq
Splitting multiple queues, each queue with a consumer
is just a few more
queues , it is really troublesome or just one queue but corresponding to one consumer, and then this consumer uses a memory queue internally for queuing, and then distributes it to different workers at the bottom for processing
3.2.2 kafka
One topic, one partition, one consumer, internal single-thread consumption, write N memory queues, and then N threads consume one memory queue respectively
