Sequence of messages
How to ensure sequence of messages?
The sequence of the message queue sent from the producer to the broker is FIFO, so the sending is sequential, and the messages in a single queue are sequential. Consumption of multiple Queues at the same time cannot absolutely guarantee the orderliness of messages. Therefore, for the same topic and the same queue, when a message is sent, a thread sends a message, and when a message is consumed, a thread consumes a message in a queue.
Follow-up: How to ensure that messages are sent to the same queue?
RocketMQ provides us with the MessageQueueSelector interface, you can rewrite the interface inside to implement your own algorithm, such as judging i%2==0, then send the message to queue1 or send it to queue2.
Message filtering
How to achieve message filtering?
There are two solutions. One is to filter according to the consumer's deduplication logic on the broker side. The advantage of this is to avoid the transmission of useless messages to the consumer side. The disadvantage is that it increases the burden on the broker and is relatively complicated to implement. The other is filtering on the Consumer side, such as deduplication according to the tag set by the message. The advantage of this is that it is simple to implement, but the disadvantage is that a large number of useless messages reach the Consumer side and can only be discarded without processing.
Message deduplication
If multiple duplicate messages are delivered to the Consumer end due to network and other reasons, how do you perform message deduplication?
We must first talk about the principle of idempotence of messages: that is, the results of multiple requests initiated by the user for the same operation are the same, and different results will not be produced due to multiple operations. As long as the idempotence is maintained, no matter how many messages come, the final processing result will be the same, which needs to be implemented on the Consumer side.
Deduplication scheme: Because each message has a MessageId, it is guaranteed that each message has a unique key, which can be the primary key or unique constraint of the database, or the key in the Redis cache. When a message is consumed, check first Whether the unique key exists in the database or cache, if it exists, the message will not be processed again. If the consumption is successful, it must be ensured that the unique key is inserted into the deduplication table.
Distributed Transaction Message
Do you know half message? How does RocketMQ implement distributed transaction messages?
Semi-message: refers to a message that cannot be consumed by the Consumer for the time being. The Producer successfully sends a message to the broker, but this message is marked as "temporarily undeliverable", only after the Producer finishes executing the local transaction after a second confirmation , Consumer can consume this message.