Teach you how to build a SpringCloud project (16) Integrated Stream message driver

Continue to update, welcome to like and follow!

1. Overview of message-driven

1. What is message-driven?

There are a lot of message middleware in practical applications, such as ActiveMQ, RabbitMQ, RocketMQ, Kafka, etc., which are commonly used in enterprises. Learning all these message middleware will undoubtedly take a lot of time and cost. Is there a technology that makes us no longer Need to pay attention to the details of specific message middleware, but only need to use an adaptive binding method to automatically switch among various message middleware? Message-driven is such a technology, which can shield the differences of underlying message middleware, reduce switching costs, and unify the programming model of messages.

Spring Cloud Stream is a framework for component message-driven microservices. The application interacts with the binder (binder) object in SpringCloud Stream through inputs and outputs, binding through configuration, and the binder object of SpringCloud Stream is responsible for interacting with the message middleware, so we only need to figure out how Interacting with Spring Cloud Stream makes it easy to use a message-driven approach. But as of now, SpringCloud Stream currently only supports RabbitMQ and Kafka.

2. Design thinking

In a classic message queue, the producer/consumer transmits information content through the message medium, and the message must go through a specific channel Message Channel. The sub-interface Subscribable Channel in the message channel consumes the message, and the MessageHandler is responsible for sending and receiving.

In Spring Cloud Stream, the isolation between the application and the details of the message middleware is realized by defining the binder as the middle layer. In the message binder, INPUT corresponds to the consumer, OUTPUT corresponds to the producer, and the message communication method in the Stream follows the publish-subscribe mode: broadcast with Topic (the corresponding exchange switch in RabbitMQ, and that in Kafka Topic).

3. Spring Cloud Stream encoding API and common annotations

insert image description here

2. Spring Cloud Stream case practice

Create three new sub-modules corresponding to message producers and consumers:

Module name Microservice function cloud-stream-rabbitmq-provider8801 producer, send message module cloud-stream-rabbitmq-consumer8802 consumer, receive message module cloud-stream-rabbitmq-consumer8803 consumer, receive message module

1. Message-driven message producer

Create a new Module: cloud-stream-rabbitmq-provider8801 is used as a message producer to send messages. In its POM file, in addition to introducing necessary starters such as web, actuator, and eureka-client, it is also necessary to introduce Spring Cloud Stream to implement RabbitMQ startup Device dependencies:

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-stream-rabbit</artifactId>
</dependency>

Write its configuration file application.yml:

server:
  port: 8801
 
spring:
  application:
    name: cloud-stream-provider
  cloud:
    stream:
      binders: # 在此处配置要绑定的rabbitmq的服务信息；
        defaultRabbit: # 表示定义的名称，用于于binding整合
          type: rabbit # 消息组件类型
          environment: # 设置rabbitmq的相关的环境配置
            spring:
              rabbitmq:
                host: localhost
                port: 5672
                username: guest
                password: guest
      bindings: # 服务的整合处理
        output: # 这个名字是一个通道的名称，OUTPUT表示这是消息的发送方
          destination: testExchange # 表示要使用的Exchange名称定义
          content-type: application/json # 设置消息类型，本次为json，文本则设置“text/plain”
          binder: defaultRabbit # 设置要绑定的消息服务的具体设置
 
eureka:
  client: # 客户端进行Eureka注册的配置
    service-url:
      defaultZone: http://eureka7001.com:7001/eureka,http://eureka7002.com:7002/eureka
  instance:
    lease-renewal-interval-in-seconds: 2 # 设置心跳的时间间隔（默认是30秒）
    lease-expiration-duration-in-seconds: 5 # 如果现在超过了5秒的间隔（默认是90秒）
    instance-id: send-8801.com  # 在信息列表时显示主机名称
    prefer-ip-address: true     # 访问的路径变为IP地址

After writing its main startup class, then write the business class. In the business, write the sending message interface and its implementation class respectively, and add **@EnableBinding annotation to the push pipeline for binding the message in the implementation class of sending the interface message , the message push pipeline bound by the message producer is org.springframework.cloud.stream.messaging.Source**:

package cn.sher6j.sprincloud.service;
 
/**
 * 发送消息接口
 * @author sher6j
 * @create 2020-05-25-12:20
 */
public interface IMessageProvider {
    
    
    public String send();
}
----------------------------------------------------------------------
package cn.sher6j.sprincloud.service.impl;
 
import cn.sher6j.sprincloud.service.IMessageProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.messaging.Source;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.support.MessageBuilder;
import java.util.UUID;
 
/**
 * 发送消息接口实现类
 * @author sher6j
 * @create 2020-05-25-12:21
 */
@EnableBinding(Source.class) //定义消息的推送管道
public class MessageProviderImpl implements IMessageProvider {
    
    
 
    @Autowired
    private MessageChannel output; //消息发送管道
 
    @Override
    public String send() {
    
    
        String serial = UUID.randomUUID().toString();
        output.send(MessageBuilder.withPayload(serial).build());//发送消息
        System.out.println("========serial:" + serial);
        return null;
    }
}

Note that we no longer need the @Service annotation in the service implementation class, because this service is no longer a service that interacts with Controller, DAO data, etc. in the traditional sense, but a service that needs to be bound with a binder.

Then write the Controller of its business layer:

@RestController
public class SendMessageController {
    
    
    @Autowired
    private IMessageProvider messageProvider;
 
    @GetMapping("/sendMessage")
    public String sendMessage() {
    
    
        return messageProvider.send();
    }
}

After starting the service registration center and RabbitMQ, start the message producer microservice. We can see an additional switch named testExchange in the RabbitMQ control panel. This switch is exactly the name of the switch we configured in the configuration file before:
insert image description here

Then we visit http://localhost:8801/sendMessage and use the message producer microservice to send a message. In the background of its microservice, we see the printed message:
insert image description here

In the control panel of RabbitMQ, we also saw that the message was indeed sent.
insert image description here

2. Message-driven message consumer

Create a new Module: cloud-stream-rabbitmq-consumer8802/8803 is used as a message producer to receive messages. The initiator dependency introduced in its POM file is almost the same as the dependency of the message producer microservice, and then write its configuration file application.yml , the writing of its configuration file is almost the same as that of the message producer. It should be noted that the channel used by the message producer microservice is OUTPUT, while the channel used by the message consumer microservice is INPUT. Other configuration file information You only need to pay attention to the difference between the port number and the registered service name:

spring:
  cloud:
      bindings: 
        input: # 这个名字是一个通道的名称，INPUT表示消息消费者

After writing the main startup class, write the business class of the message consumer. Since it is a consumer, you only need to write its Controller. On its Controller, you also need to add the **@EnableBinding annotation to bind the push pipeline of the message, the message The message push pipeline bound by the consumer is import org.springframework.cloud.stream.messaging.Sink. In the method of receiving messages, @StreamListner** annotation needs to be used to monitor the bound message push pipeline:

package cn.sher6j.springcloud.controller;
 
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;
import org.springframework.messaging.Message;
import org.springframework.stereotype.Component;
 
/**
 * @author sher6j
 * @create 2020-05-25-12:58
 */
@Component
@EnableBinding(Sink.class)
public class ReceiveMessageController {
    
    
    
    @Value("${server.port}")
    private String serverPort;
    
    @StreamListener(Sink.INPUT)
    public void input(Message<String> message) {
    
    
        System.out.println("消费者" + serverPort + "号，收到消息：" + message.getPayload());
    }
}

Then start the message sending consumer service, use the producer to send the message, we can find that the message can be successfully received on the consumer side:
insert image description here

3. Group consumption and persistence

1. Repeat consumption problem

When the producer sends a message, our consumers at this time all accept the message and consume it, that is to say, the same message is consumed by multiple message consumers:
insert image description here

The above problem is the problem of repeated consumption of messages, so why is this problem so important? In fact, the problem of repeated consumption itself is not terrible. What is terrible is how to ensure idempotence after repeated consumption is not considered. (Idempotency, in layman's terms, is repeated many times for one data or one request, and it is necessary to ensure that the corresponding data will not change and cannot make mistakes). In order to achieve high availability and load balancing in distributed microservice applications, in fact, multiple specific service instances will be deployed for services with the same function. For example, suppose there is a system with a message requesting to insert a piece of data into the database. If the message is consumed twice, the result is that two pieces of data are inserted into the database. In this way, the data will be wrong, which violates idempotence. However, if the message is consumed for the second time, it can be judged that it has already been consumed, and then the message is discarded directly. Once it is actually consumed, only one piece of data is inserted into the database, which ensures the idempotency of the system.

The above briefly introduces the problem of repeated consumption of messages, so how to solve this problem of repeated consumption, then we need to perform grouping and persistent attribute group operations, and use message grouping in Spring Cloud Stream to solve this problem. It should be noted that Multiple message consumers in the same group in the Stream are in a competitive relationship, which means that the same message sent by the producer is guaranteed to be consumed only once by one of the consumers. Consumers in different groups can fully consume (repeatedly consume) messages, and only within the same group will there be competition.

In RabbitMQ, the default grouping is different, the group serial number is different, it is considered as a different group, we check the testExchange switch, we can find that the two message consumers 8802 and 8803 are in different groups, so the message sent by the 8801 message producer can be Consumed repeatedly by these two consumers:
insert image description here

2. Group to solve the problem of repeated consumption

The group serial number we saw above in the RabbitMQ control panel is randomly assigned by the system, which is undoubtedly difficult to control, so we should customize the configuration group and divide the two message consumer microservices 8802/8803 into the same group. In order to solve the problem of repeated consumption of messages, let's first demonstrate how to customize grouping.

Add group name attributes to the configuration files in the 8802/8803 microservices:

spring:
  cloud:
    stream:
      bindings:
        input:
          group: A/B ## 分组名称

Here we set 8802 as group A and 8803 as group B, then we restarted the two microservices of the message consumer, and we checked the group serial number again, and found that it was no longer a long random group serial number, but changed to Became our custom grouping:
insert image description here

At this time, since 8802/8803 are located in two different groups, there is no competition. After the message producer sends the message, it can still be consumed repeatedly.

Next, we divide the two message consumer microservices into the same consumer group, so that there is only one consumer at a time, and the message sent by the message producer can only be received by one of 8802 or 8803, thus avoiding duplication For consumption, change the group names of 8802 and 8803 to A, and restart the two message consumer microservices again. At this time, we can see that there are already two consumers under group A:
insert image description here

Then use the producer to send 5 messages, we found that 8802/8803 consumed 3 and 2 different messages respectively, without the problem of repeated consumption:

insert image description here

Due to time reasons, other friends Liujia Hengbao forwarded the article, click to view the original text !

We have finished learning Spring Cloud Stream here. In the next article, we will learn Spring Cloud Stream distributed link tracking, continue to learn and update, and the next section will be more exciting! Friends are welcome to like and follow! grateful! insert image description here