How to ensure the message queue messages can be hundred percent successful consumer
The most commonly used message queues there are many, such as RabbitMQ, ActiveMQ, Kafka ... below to RabbitMQ for example in terms of how to ensure the information in the message queue can be consumed hundred percent.
Consumer queue which works as follows:
We can add a mechanism for increasing a confirmation mechanism:
Process explained:
1) Prior to re-order service producers to deliver the messages, first message is persisted to the DB Redis or recommended redis, high performance. Status message for transmission.
2) confirm whether the mechanism listen for messages sent successfully? As ack success message, delete redis in this message.
3) If unsuccessful nack message, this can choose whether to retransmit the message according to their business. You can also delete the message, determined by their business.
4) here added a timed task to pull a certain time interval, or to send the message status in this state to indicate that the service is not received orders ack success message.
5) for compensating the timing task message delivery. This time if MQ callbacks ack successfully received, then delete this message in redis
This program is actually a plus compensation mechanism, regardless of MQ have not really received, as long as the cache redis messages sent or state, it means that the message is not delivered successfully go out and have not been spending, timed task It will resend startup.
Of course, the timing task over there we can also add a number of compensation, if more than three times, or did not receive ack message, then the message directly to the status set to [Failed], by hand to the investigation in the end why?
But this program, there will be possible to send the same message many times, MQ is likely to have received the news that a network failure occurs when ack message callback, so that producers do not receive. It would have required consumers must guarantee Idempotence at the time of consumption.