SpringBoot + RabbitMQ, to ensure successful delivery message and a 100% consumption (with source)

First, the first to throw a map

Description:
This article covers the RabbitMQ knowledge about many aspects, such as:

1. Send a message confirmation mechanism

2. Consumer acknowledgment mechanism

3. Rerouted message

4. Consumer idempotency, etc.

These are deployed around the entire flowchart goes on the above, it is necessary to first posted, see Figure EENOW

Second, the realization of ideas

1. 163 E-mail a brief description of obtaining the authorization code

2. Send e-mail writing tools

3. Write RabbitMQ profile

4. Producers initiate calls

5. Consumers send a message

6. The timing of the timing pull task message delivery failure, rerouted

7. Testing and validation of the various anomalies

8. Development: dynamic proxies to achieve consumer end idempotency verification and message acknowledgment (ack)

Third, Projects

1. springbootVersion 2.1.5.RELEASE, the old version may be some configuration attributes can not be used, need to be configured in code

2. RabbitMQversion3.7.15

3. MailUtil: Send Mail Tools

4. RabbitConfig: Rabbitmq configuration

5. TestServiceImpl: Manufacturer, sending a message

6. MailConsumer: Consumer, consumer news, send e-mail

7. ResendMsg: Regular tasks, send a failure message redelivery

Note: The above is the core code MsgLogService mapper xml, were not posted, I can refer to the complete code GitHub, welcome fork, https://github.com/wangzaiplus/springboot/tree/wxw

Fourth, code implementation

1. Mailbox 163 obtain authorization codes, as shown:

   
   

   The authorization code is profile spring.mail.passwordpassword required
   

2.pom

3 rabbitmq, mailbox configuration

Description:  passwordThat authorization code,  usernameand fromto be consistent

Table 4. Structure

Description: The  exchange routing_keyfield is in the timed task rerouted messages need to use

5. MailUtil

6.RabbitConfig

@Configuration@Slf4jpublic class RabbitConfig {

    @Autowired
    private CachingConnectionFactory connectionFactory;

    @Autowired
    private MsgLogService msgLogService;

    @Bean
    public RabbitTemplate rabbitTemplate() {
        RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setMessageConverter(converter());

        // 消息是否成功发送到Exchange
        rabbitTemplate.setConfirmCallback((correlationData, ack, cause) -> {
            if (ack) {
                log.info("消息成功发送到Exchange");
                String msgId = correlationData.getId();
                msgLogService.updateStatus(msgId, Constant.MsgLogStatus.DELIVER_SUCCESS);
            } else {
                log.info("消息发送到Exchange失败, {}, cause: {}", correlationData, cause);
            }
        });

        // 触发setReturnCallback回调必须设置mandatory=true, 否则Exchange没有找到Queue就会丢弃掉消息, 而不会触发回调
        rabbitTemplate.setMandatory(true);
        // 消息是否从Exchange路由到Queue, 注意: 这是一个失败回调, 只有消息从Exchange路由到Queue失败才会回调这个方法
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
            log.info("消息从Exchange路由到Queue失败: exchange: {}, route: {}, replyCode: {}, replyText: {}, message: {}", exchange, routingKey, replyCode, replyText, message);
        });

        return rabbitTemplate;
    }

    @Bean
    public Jackson2JsonMessageConverter converter() {
        return new Jackson2JsonMessageConverter();
    }

    // 发送邮件
    public static final String MAIL_QUEUE_NAME = "mail.queue";
    public static final String MAIL_EXCHANGE_NAME = "mail.exchange";
    public static final String MAIL_ROUTING_KEY_NAME = "mail.routing.key";

    @Bean
    public Queue mailQueue() {
        return new Queue(MAIL_QUEUE_NAME, true);
    }

    @Bean
    public DirectExchange mailExchange() {
        return new DirectExchange(MAIL_EXCHANGE_NAME, true, false);
    }

    @Bean
    public Binding mailBinding() {
        return BindingBuilder.bind(mailQueue()).to(mailExchange()).with(MAIL_ROUTING_KEY_NAME);
    }}

7.TestServiceImpl生产消息

8.MailConsumer消费消息, 发送邮件

说明: 其实就完成了3件事: 1.保证消费幂等性, 2.发送邮件, 3.更新消息状态, 手动ack

9.ResendMsg定时任务重新投递发送失败的消息

五、基本测试

OK, 目前为止, 代码准备就绪, 现在进行正常流程的测试

1. 发送请求:

   
   

1. 后台日志:

1. 数据库消息记录:

   
   

2. 状态为3, 表明已消费, 消息重试次数为0, 表明一次投递就成功了
   

3. 查看邮箱

   
   

发送成功

六、各种异常情况测试

步骤一罗列了很多关于RabbitMQ的知识点, 很重要, 很核心, 而本文也涉及到了这些知识点的实现, 接下来就通过异常测试进行验证(这些验证都是围绕本文开头扔的那张流程图展开的, 很重要, 所以, 再贴一遍)

1. 验证消息发送到Exchange失败情况下的回调, 对应上图P -> X

如何验证? 可以随便指定一个不存在的交换机名称, 请求接口, 看是否会触发回调

发送失败, 原因: reply-code=404, reply-text=NOT_FOUND - no exchange 'mail.exchangeabcd' in vhost '/', 该回调能够保证消息正确发送到Exchange, 测试完成

1. 验证消息从Exchange路由到Queue失败情况下的回调, 对应上图X -> Q

同理, 修改一下路由键为不存在的即可, 路由失败, 触发回调

发送失败, 原因: route: mail.routing.keyabcd, replyCode: 312, replyText: NO_ROUTE

1. 验证在手动ack模式下, 消费端必须进行手动确认(ack), 否则消息会一直保存在队列中, 直到被消费, 对应上图Q -> C

将消费端代码channel.basicAck(tag, false);// 消费确认注释掉, 查看控制台和rabbitmq管控台

可以看到, 虽然消息确实被消费了, 但是由于是手动确认模式, 而最后又没手动确认, 所以, 消息仍被rabbitmq保存, 所以, 手动ack能够保证消息一定被消费, 但一定要记得basicAck

1. 验证消费端幂等性

接着上一步, 去掉注释, 重启服务器, 由于有一条未被ack的消息, 所以重启后监听到消息, 进行消费, 但是由于消费前会判断该消息的状态是否未被消费, 发现status=3, 即已消费, 所以, 直接return, 这样就保证了消费端的幂等性, 即使由于网络等原因投递成功而未触发回调, 从而多次投递, 也不会重复消费进而发生业务异常

1. 验证消费端发生异常消息也不会丢失

很显然, 消费端代码可能发生异常, 如果不做处理, 业务没正确执行, 消息却不见了, 给我们感觉就是消息丢失了, 由于我们消费端代码做了异常捕获, 业务异常时, 会触发: channel.basicNack(tag, false, true);, 这样会告诉rabbitmq该消息消费失败, 需要重新入队, 可以重新投递到其他正常的消费端进行消费, 从而保证消息不被丢失

测试: send方法直接返回false即可(这里跟抛出异常一个意思)

可以看到, 由于channel.basicNack(tag, false, true), 未被ack的消息(unacked)会重新入队并被消费, 这样就保证了消息不会走丢

1. 验证定时任务的消息重投

实际应用场景中, 可能由于网络原因, 或者消息未被持久化MQ就宕机了, 使得投递确认的回调方法ConfirmCallback没有被执行, 从而导致数据库该消息状态一直是投递中的状态, 此时就需要进行消息重投, 即使也许消息已经被消费了

定时任务只是保证消息100%投递成功, 而多次投递的消费幂等性需要消费端自己保证

我们可以将回调和消费成功后更新消息状态的代码注释掉, 开启定时任务, 查看是否重投

You can see, the message will re-enter 3 times, more than three times to give up, the message status is set to deliver a failed state, this abnormal situation occurs, we need to troubleshoot the cause of human intervention

Seven expansion: the use of dynamic proxy to achieve consumer end power consumption such as verification and acknowledgment (ack)

Do not know if you noticed, in MailConsumerthe real business logic is actually just send e-mail mailUtil.send(mail)only, but we have to call sendbefore checking method idempotency consumer, after send, but also updates the message status is "consumption" state, and manual ack, the actual project, there may be many producer - consumer application scenarios, such as logging, send text messages, etc., need rabbitmq, if every write these duplicate common code is not necessary, it is difficult to maintain Therefore, we can be pulled out common code, so that the core business logic only care about their realization, without doing other operations, in fact, AOP

To achieve this, there are many ways you can use spring aop, you can use interceptors, you can use a static proxy, you can also use dynamic proxies, where I use dynamic proxies

Directory structure is as follows:

The core code is the implementation agency, there is not all the code stickers out, just a thought, as much as possible we want to write the code more simple and more elegant

Eight, summary

Send e-mail is actually very simple, but in fact there is a lot to get to the bottom up attention and perfect point, a seemingly small knowledge can also come out of a lot of problems, even involving all aspects, these need to step on his own pit, of course, my Code certainly there are many imperfections and the need to optimize the point, I hope small partners speak up and recommendations

My code is validated through a self-test, drawing little by little, they are also painted himself or seriously cut, hoping that partners can learn a little something, a passing point or points followers chanting praise, thank you

Project Github , welcome fork
https://github.com/wangzaiplus/springboot/tree/wxw