RocketMQ (5) --- RocketMQ retry mechanism

RocketMQ retry mechanism

Retry message is divided into two: Producer retry sending the message and Consumer consumption retry message .

A, Producer retry end

Producer end retry means: Producer to the MQ message not sent, such as the network sends a message to cause the producer MQ failure.

Look at the code:

@Slf4j
public class RocketMQTest {
    /**
     * 生产者组
     */
    private static String PRODUCE_RGROUP = "test_producer";
  
    public static void main(String[] args) throws Exception {
        //1、创建生产者对象
        DefaultMQProducer producer = new DefaultMQProducer(PRODUCE_RGROUP);
        //设置重试次数(默认2次）
        producer.setRetryTimesWhenSendFailed(3000);
        //绑定name server
        producer.setNamesrvAddr("74.49.203.55:9876");
        producer.start();
        //创建消息
        Message message = new Message("topic_family", ("小小今年3岁" ).getBytes());
        //发送 这里填写超时时间是5毫秒 所以每次都会发送失败
        SendResult sendResult = producer.send(message,5);
        log.info("输出生产者信息={}",sendResult);
    }
}

超时重试 For the Internet, said a timeout exception will retry statement is wrong, think about all that terrible, so I checked the article says will retry timeout exception, do so many people did not go to see a test or source.

I found the problem, because I am above timeout to 5 milliseconds, according to the normal certainly will be reported timeout exception, but I set a retry and 3000 retry, although eventually reported the following exception, but the output error time report

Obviously it should not be a level. But tests found that the number of retries regardless of how many times I set, reported abnormal almost all the time.

org.apache.rocketmq.remoting.exception.RemotingTooMuchRequestException: sendDefaultImpl call timeout

For after this doubt, I went to the source, did he realize.

   /**
     * 说明 抽取部分代码
     */
    private SendResult sendDefaultImpl(Message msg, final CommunicationMode communicationMode, final SendCallback sendCallback, final long timeout) {
        
        //1、获取当前时间
        long beginTimestampFirst = System.currentTimeMillis();
        long beginTimestampPrev ;
        //2、去服务器看下有没有主题消息
        TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());
        if (topicPublishInfo != null && topicPublishInfo.ok()) {
            boolean callTimeout = false;
            //3、通过这里可以很明显看出 如果不是同步发送消息 那么消息重试只有1次
            int timesTotal = communicationMode == CommunicationMode.SYNC ? 1 + this.defaultMQProducer.getRetryTimesWhenSendFailed() : 1;
            //4、根据设置的重试次数，循环再去获取服务器主题消息
            for (times = 0; times < timesTotal; times++) {
                MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName);
                beginTimestampPrev = System.currentTimeMillis();
                long costTime = beginTimestampPrev - beginTimestampFirst;
                //5、前后时间对比 如果前后时间差 大于 设置的等待时间 那么直接跳出for循环了 这就说明连接超时是不进行多次连接重试的
                if (timeout < costTime) {
                    callTimeout = true;
                    break;

                }
                //6、如果超时直接报错
                if (callTimeout) {
                    throw new RemotingTooMuchRequestException("sendDefaultImpl call timeout");
                }
        }
    }

Obviously this can be seen by following the source

1. If it is 异步发送then the number of retries only 1
2. For 超时异常也是不会再去重试synchronization, .
3. If the retry retry occurs in a for loop to go, so it is retried immediately, rather than from time to time to try again.

Really Practice makes perfect! ! !

Two, Consumer end retry

Consumer side is more interesting, but also in the actual development process, we should consider is the consumer side and try again.

Failed consumer side is divided into two kinds of situations, Exceptionand Timeout.

1、Exception

@Slf4j
@Component
public class Consumer {
    /**
     * 消费者实体对象
     */
    private DefaultMQPushConsumer consumer;
    /**
     * 消费者组
     */
    public static final String CONSUMER_GROUP = "test_consumer";
    /**
     * 通过构造函数 实例化对象
     */
    public Consumer() throws MQClientException {
        consumer = new DefaultMQPushConsumer(CONSUMER_GROUP);
        consumer.setNamesrvAddr("47.99.203.55:9876;47.99.203.55:9877");
        //订阅topic和 tags（ * 代表所有标签)下信息
        consumer.subscribe("topic_family", "*");
        //注册消费的监听 并在此监听中消费信息，并返回消费的状态信息
        consumer.registerMessageListener((MessageListenerConcurrently) (msgs, context) -> {
            //1、获取消息
            Message msg = msgs.get(0);
            try {
                //2、消费者获取消息
                String body = new String(msg.getBody(), "utf-8");
                //3、获取重试次数
                int count = ((MessageExt) msg).getReconsumeTimes();
                log.info("当前消费重试次数为 = {}", count);
                //4、这里设置重试大于3次 那么通过保存数据库 人工来兜底
                if (count >= 2) {
                    log.info("该消息已经重试3次,保存数据库。topic={},keys={},msg={}", msg.getTopic(), msg.getKeys(), body);
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
                }
                //直接抛出异常
                throw new Exception("=======这里出错了============");
                //return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            } catch (Exception e) {
                e.printStackTrace();
                return ConsumeConcurrentlyStatus.RECONSUME_LATER;
            }
        });
        //启动监听
        consumer.start();
    }
}

Here's the code meaning is clear: take the initiative to throw an exception, and if more than three times, then do not continue to retry it, but the record is saved to the database by hand to reveal all the details.

Look at operating results

注意 Retry consumers and producers there are differences, there are two main

1, the default number of retries: Product default is 2 times, and Consumer default is 16 .

2, the retry interval: Product is immediately retried, and Consumer there is a certain time interval . It is as 1S,5S,10S,30S,1M,2M····2Ha retry.

2、Timeout

说明Here timeout timeout exception is not in the true sense, it refers to refers to the acquisition news, for some reason it did not return to the state of consumption RocketMQ, ie no return ConsumeConcurrentlyStatus.CONSUME_SUCCESSor return ConsumeConcurrentlyStatus.RECONSUME_LATER.

So RocketMQ think that the message is not sent, would have been sent . Because it will simply not believe the message is sent to the consumer, it is certainly not the consumer.

This test is very simple to do.

        //1、消费者获得消息
        String body = new String(msg.getBody(), "utf-8");
        //2、获取重试次数
        int count = ((MessageExt) msg).getReconsumeTimes();
        log.info("当前消费重试次数为 = {}", count);
        //3、这里睡眠60秒
        Thread.sleep(60000);
       log.info("休眠60秒 看还能不能走到这里。topic={},keys={},msg={}", msg.getTopic(), msg.getKeys(), body);
        //返回成功
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;

When obtaining the number of retries current consumption = 0 after turn off the process . And then restart the process, it is still able to acquire the piece of news

consumer消费者  当前消费重试次数为 = 0
休眠60秒 看还能不能走到这里。topic=topic_family,keys=1a2b3c4d5f,msg=小小今年3岁

只要自己变优秀了，其他的事情才会跟着好起来（上将2）