When a consumer encounters an exception and fails to consume a message, Apache RocketMQ will re-deliver the message according to the consumption retry policy for failure recovery. This article introduces the principle, version compatibility and usage suggestions of the consumption retry mechanism.
Application scenarios
Apache RocketMQ's consumption retry mainly solves the consumption integrity problem caused by the failure of business processing logic. It is a strategy to protect the business and should not be used for business process control. It is recommended to use the retry mechanism in the following consumption failure scenarios:
The recommended scenarios for using message retry are as follows:
-
The business processing failed, and the reason for the failure is related to the current message content. For example, the transaction status corresponding to the message has not been obtained, and it is expected to be executed successfully after a period of time.
-
The reason for consumption failure will not lead to continuity, that is, the current message consumption failure is a small probability event, not a normal failure, and subsequent messages will be consumed successfully with a high probability. At this time, you can retry the current message to avoid process blocking.
Typical error usage scenarios are as follows:
-
It is unreasonable to use consumption failure to divert the results of conditional judgments in the consumption processing logic, because the processing logic has foreseen that this judgment branch will definitely appear in large numbers.
-
It is unreasonable to use consumption failure to limit the processing rate during consumption processing. The purpose of current limiting is to temporarily accumulate messages that exceed the flow rate in the queue to achieve peak shaving effect, rather than allowing the messages to enter the retry link.
Application purpose
A typical problem when using message middleware for asynchronous decoupling is how to ensure the integrity of the entire call link if the downstream service fails to process the message event. As a financial-level reliable business message middleware, Apache RocketMQ naturally supports reliable transmission strategies in the design of the message delivery processing mechanism, and ensures that each message is processed according to business expectations through a complete confirmation and retry mechanism.
Understanding Apache RocketMQ’s message acknowledgment mechanism and consumption retry strategy can help you analyze the following issues:
-
How to ensure that the business processes messages completely: Understanding the consumption retry strategy can ensure the integrity of each message processing when designing and implementing consumer logic, and prevent some messages from being ignored when exceptions occur, resulting in inconsistent business status.
-
How to restore the status of messages being processed when the system is abnormal: Helps you understand how to restore the status of messages being processed when the system is abnormal (downtime failure) and other scenarios, and whether there will be status inconsistencies.
Overview of consumption retry strategy
Consumption retry means that after the consumer fails to consume a certain message, the Apache RocketMQ server will re-consume the message according to the retry policy. If the message is not consumed successfully after a certain number of times, the message will not be retried. , is sent directly to the dead letter queue.
Trigger conditions for message retry
-
Consumption failure includes the consumer returning a message failure status identifier or throwing an unexpected exception.
-
Message processing timeouts, including queuing timeouts in PushConsumer.
Main behaviors of message retry strategy
-
Retry process state machine: Control the status and change logic of messages in the retry process.
-
Retry interval: The interval between retrying consumption after the last consumption failed or timed out.
-
Maximum number of retries: The maximum number of times a message can be retried for consumption.
Message retry strategy differences
Depending on the consumer type, the specific internal mechanism and setting method of the message retry policy are different. The specific differences are as follows
| consumer type | Retry process state machine | Retry interval | Maximum number of retries |
| PushConsumer | Ready, processing, pending retry submission* Dead letter | Metadata control when creating consumer groups. Unsequenced messages: stepped intervals Sequential messages: fixed intervals | Metadata control when creating consumer groups. |
| SimpleConsumer | Dead letter submission is ready for processing | Modify the invisible time when getting messages through the API. | Metadata control when creating consumer groups. |
For specific retry strategies, see PushConsumer consumption retry strategy and SimpleConsumer consumption retry strategy below .
PushConsumer consumption retry strategy
Retry state machine
When PushConsumer consumes a message, the main states of the message are as follows:
-
Ready: Ready status. The message is ready on the Apache RocketMQ server and can be consumed by consumers.
-
Inflight: Processing status. The message is obtained by the consumer client and is in a state of consumption but the consumption result has not yet been returned.
-
WaitingRetry: pending retry status, a status unique to PushConsumer. When consumer message processing fails or consumption times out, consumption retry logic will be triggered. If the current number of retries does not reach the maximum number, the message becomes the pending retry state. After the retry interval, the message will become ready again and can be consumed again. Between multiple retries, the retry interval can be extended to prevent invalid high-frequency failures.
-
Commit: commit status. In the status of successful consumption, the consumer returns a successful response to end the state machine of the message.
-
DLQ: Dead letter status. The final fallback mechanism for consumption logic. If a message continues to fail to be processed and retried continuously until the maximum number of retries is exceeded without success, the message will not be retried at this time and will be delivered to the dead letter queue. You can restore your business by consuming messages from the dead letter queue.
During the message retry process, the message status of each retry will change from Ready > Processing > Pending Retry. The interval between two consumptions is actually controlled by the consumption time and retry interval, and the maximum upper limit of consumption time is It is controlled by server system parameters and generally should not exceed the upper limit time.
Maximum number of retries
The maximum number of retries of PushConsumer is controlled by the metadata when the consumer group is created. For specific parameters, see Consumer Grouping .
For example, if the maximum number of retries is 3, the message can be delivered up to 4 times, 1 time is the original message, and 3 times is the number of retries.
Retry interval
-
Unsequential messages (non-sequential messages): The retry interval is the ladder time, the specific time is as follows:
If the number of retries exceeds 16, the interval between subsequent retries will be 2 hours.
- Sequential messages: The retry interval is a fixed time. For specific values, please see parameter restrictions .
Usage example
SimpleConsumer only needs to wait to trigger message retry.
SimpleConsumer simpleConsumer = null;
//消费示例：使用PushConsumer消费普通消息，如果消费失败返回错误，即可触发重试。
MessageListener messageListener = new MessageListener() {
@Override
public ConsumeResult consume(MessageView messageView) {
System.out.println(messageView);
//返回消费失败，会自动重试，直至到达最大重试次数。
return ConsumeResult.FAILURE;
}
};
Suggestions for use
Reasonable retry to avoid consumption retries triggered by demands such as current limiting.
As mentioned in the above application scenarios , message retry is suitable for scenarios where business processing fails and current consumption is a low-probability event. It is not suitable for use in scenarios where continuity fails, such as consumption current limiting scenarios.
-
Error example: If the current consumption rate is too high and current limiting is triggered, consumption failure will be returned and the user will wait for consumption again next time.
-
Correct example: If the current consumption rate is too high and triggers current limiting, delay obtaining the message and consume it later.
Reasonably control the number of retries to avoid infinite retries
Although Apache RocketMQ supports customizing the number of consumption retries, it is recommended to reduce the system pressure by reducing the number of retries + extending the retry interval to avoid infinite retries or a large number of retries.