At present, it is necessary to select a suitable message queue for data transmission in business, so I specially investigated the characteristics of the current mainstream message queues:
There are three elements of message middleware: producer, message, and consumer.
Metrics: The interaction of producers, messages, and consumers.
1. Message routing: how messages reach consumers through message middleware.
2. Message reliability:
2.1. Do not allow message loss
2.2. Allow message loss, performance requirements outweigh reliability
3. Message replay: Whether the messages that have been consumed can be re-consumed after a certain time interval (applicable to importing old data into a new system to prevent data loss)
4. Message accumulation: During the peak traffic period, the message middleware may have problems when delivering messages with high concurrency, so the message is temporarily stored in the middleware, and then sent to the downstream business when the traffic peak passes.
5. Message priority:
5.1. The message delivery order is consistent with the consumption order
5.2. The consumption priority of certain messages can be set
6. Performance and Scaling
6.1. Performance: mainly refers to tps, qps and the number of concurrent connections
6.2. Expansion: Increase the consumption rate and the upper limit of the capacity of the message middleware by adding consumers
In view of the fact that our business is currently doing data transmission, we mainly need to ensure the integrity of the data, and we chose RabbitMQ for a comprehensive comparison.