RocketMq learning (III retry mechanism)

Retry strategy rocketmq generally divided into two types: one producer MQ retry to send a transmission to the consumer MQ retry.

A producer retry

package com.wk.test.rocketmqTest;

import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;

public class Producer {
    public static void main(String[] args) {
        //定义生产者名称
        DefaultMQProducer producer = newDefaultMQProducer ( "quickstart_product" );
         // connected rocketMQ of namesrv address (here a cluster) 
        producer.setNamesrvAddr ( "10.32.16.179:9876" );
         // send failure retry 3 
        producer.setRetryTimesWhenSendFailed (3);
         the try { 
            Producer .start (); 
            // 1. theme, usually in the server set up, not from the new code. 2. Label. 3. Send content. 
            Message = the Message new new the Message ( "TopicQuickStart", "Tag1", ( "producer Retry" ) .getBytes ());
             // send timeout time of 10 seconds is provided 
            SendResult sendResult = producer.send (Message, 10000 ); 
            the System. Out.println (sendResult); 
        } the catch (MQClientException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (RemotingException e) {
            e.printStackTrace();
        } catch (MQBrokerException e) {
            e.printStackTrace();
        }finally {
            producer.shutdown();
        }
    }
}

Not sent within 10 seconds, the maximum number of retries is 3

Second, consumers retry

Consumers retry divided into two cases: 1. The consumer receives a message thrown exception exception. 2. The consumer does not receive the message, MQ send out.

I.

package com.wk.test.rocketmqTest;

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;

public class Consumer {
    public static void main(String[] args) throws MQClientException {
        //Definition of consumer names, MQ consumer to push 
        DefaultMQPushConsumer Consumer = new new DefaultMQPushConsumer ( "quickstart_consumer" );
         // connection rocketMQ of namesrv address (the cluster) 
        consumer.setNamesrvAddr ( "10.32.16.179:9876" );
         // new Subscribe to the group first started, from the beginning to the end consumer, the consumer continues to follow the progress of consumption from the previous 
        consumer.setConsumeFromWhere (ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);
         // subscribe to topics and tags (* represents all tags) 
        Consumer.subscribe ( "TopicQuickStart" , "Tag1 || Tag2" );
         // consumer listening 
        consumer.registerMessageListener ((MessageListenerConcurrently) (Msgs, context) -> { 
            MessageExt msg = msgs.get (0);
             The try { 
                String Topic = msg.getTopic (); 
                String msgBody = new new String (msg.getBody (), "UTF-. 8" ); 
                String Tag = msg.getTags (); 
                System.out.println ( "Topic: "+ + Topic" msgBody: "+ + msgBody" Tag: "+ Tag);
                 // doSomething service processing ... 
            } the catch (Exception E) { 
                e.printStackTrace (); 
                // throw three retries are unsuccessful does not continue to retry 
                IF (msg.getReconsumeTimes () ==. 3) { 
                    // logging or persistence operations. 
                    return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; 
                }
                 // the MQ transmission failure retry mechanism, 5S lS 10s 30s 5m 1M 2M 4M 6M 3M 8M 7M 9m 10m 20m 30m IH 2H 
                return ConsumeConcurrentlyStatus.RECONSUME_LATER; 
            } 
            // message processing is successful 
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS; 
        } ); 
        consumer.start (); 
    } 
}

After receiving the message, for processing consumer side, if an exception is thrown, then the three unsuccessful retries for message recording.

Scenario 2

Comsumer case where the cluster is, when sending a message to the MQ c1, c1 if the received message is down, the same message is sent for processing to c2. Provided that c1 and c2 must be for the same consumer groups,

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("quickstart_consumer");

To the same.

And here there will be a problem is if they have been restarted after c1, c2 may simultaneously sent the same message, has been handled by c1, which appeared in the same situation a message is spending more than consumers, case we need to maintain business idempotency.

Solutions web search are:

1, the consumer-side business logic processing message retention idempotency
2, ensure that each message has a unique ID and the message log to ensure successful treatment with the simultaneous de-duplication table

Article 1 well understood, just keep idempotency, no matter how many duplicate messages, but it finally treated the same. Article 2 principle is to use a log table to record the ID has successfully processed the message, if newly arrived message ID is already in the log table, it will no longer deal with this message.

We can see article 1 of the solution, it is clear that should be implemented at the consumer side, not part of the message system functions to be implemented. Article 2 may be implemented messaging system, the business end can be achieved. Duplicate Messages probability under normal circumstances is not necessarily large, and implemented by the messaging system, it will certainly be highly available messaging system throughput and influence, so best to deal with their own messages from the business end of duplicate questions, and this is RocketMQ the reason does not solve the problem of duplicate messages.

RocketMQ is no guarantee that the message does not repeat, if your business does not need to be rigorously duplicate messages, you need to go heavy on the business side.