Spring Cloud Stream binder architecture analysis and development

Spring Cloud Stream binder architecture analysis and development

According to different usage scenarios, we usually choose the appropriate messaging middleware. For example, we may choose to use Kafka for the log collection scenario, and usually choose RocketMQ for the order scenario. Different messaging middleware clients are different, and we need to target different Message middleware writes different codes. For TOC products, we can choose the appropriate message middleware according to the business scenario, but for TOB products, different customers have different requirements for message middleware, we need to switch message middleware conveniently, and do not need to refactor the code

Spring Cloud Stream can help us solve the above problems very well. Decoupling specific message middleware through Binder, shielding the differences in the use of different message middleware, and providing a unified configuration and usage method based on Binder Spring Cloud Stream for sending and consuming messages. For the user, it only needs to understand how to use Spring Cloud Stream to send and receive messages without caring about the kind of message middleware used. Binder needs to be responsible for how to send and receive messages and other advanced features such as transactions, etc. without communicating with the message middleware. Therefore, we can see binders for unreachable message middleware, such as spring-cloud-stream-binder-kafka, spring-cloud-stream-binder-rabbit, etc. We can also develop binders that meet our own needs according to the specifications, such as for Users with a small amount of messages can use redis-based message middleware, and we can develop a corresponding Binder to connect to Spring Cloud Stream. Below I will introduce the architecture of Binder and how to develop a Binder

Binder architecture introduction and source code analysis

From the perspective of source code, let's see how Spring Cloud Stream interacts with Binder, and what specific things Binder does

Project begining

When the project starts, it will first scan all dependent binders under the project, and store the binder information in the memory object. Specifically, the following things are done:

• Scan META-INF/spring.binders to obtain the binder type and the corresponding configuration loading class, which is used to create the BinderTypeRegistry object
  

• Build a BinderFactory based on a BinderTypeRegistry. BinderFactory contains both binder information and configuration information related to spring cloud steam configured under the scanning yaml file
  

Send a message

After the project is started, the preparatory work has been completed. From the perspective of the message sending process, if Spring Cloud Stream cooperates with Binder, the message sending is completed.

BinderFactory gets the binder instance

When starting the project, we put all the information of the binder into the BinderFactory, so the first step in sending a message is to create a binder instance according to the information of the binder, and complete the initial work of the binder

Load the binder configuration class

Spring Cloud Stream loads the configuration class corresponding to the binder through the initializeBinderContextSimple method of DefaultBinderFactory, and creates the AnnotationConfigApplicationContext
context object

. After loading the configuration class corresponding to the binder, the initialization of the binder will be triggered

• Initialize the Binder configuration class, initialize the Binding configuration class
  
• Initialize the binder target provider
  
• Initialize the channel binder
  

Get a binder instance

The binder instance can be obtained through the context object created in the previous step, and the obtained binder instance is cached in memory

Use a binder to bind the output channel to a specific message middleware

The output channel is the concept of Spring Cloud Stream. We need to associate the output channel with a specific client for sending messages, and the binder instance has a method for constructing a client for sending messages. Therefore, this step needs to create a specific sending message client and associate it with the output channel. The details are as follows:

• Get extended producer configuration information and merge with Spring Cloud Stream producer properties
  
  
• Get producer send target provider
  
  
• Create a send message instance and complete the send message object initialization
  
  
• Bind the output channel to the sending message instance
  
• Create a binding and cache it to the sending channel for subsequent operations such as unbinding
  
  

send message via send channel

After completing the binding of the sending channel and the specific sending client, the message can be sent. In the following method, the client of the specific message middleware will be used to complete the message sending

Summarize

Through the above source code analysis, we can clearly see that Spring Cloud Stream provides users with unified management of sending messages, consuming messages, and configuration parameters. However, when sending and consuming are actually executed, it is done by the producer instance and consumer instance created by the binder. These producer instances and consumer instances created by the binder are associated with specific message middleware, so different message middleware will correspond to different binders. The binder serves as a bridge between Spring Cloud Steam and specific message middleware

Development example

The following will specifically explain how we write our own binder

create spring.binders

Create resources/META-INF/spring.binders, specify the binder type and the corresponding configuration class

Create a configuration class

The configuration class needs to scan our extended attribute classes, including the configuration parameter class of the binder, the configuration parameter class of Binding, the registered target provider object, and the message channel binder object

Binder parameter class

The binder parameter class is used to store information related to specific message middleware connections, such as address, user name, password, port and other information

Binding extended attribute configuration class

This includes producer configuration properties and consumer configuration properties. Special parameters required for sending and consuming can be placed here. For example, using redis as message middleware, we need to specify the number of DBs for sending and consuming
, so we need to put this parameter in the production and consumption configuration class

• Create RedisMQExtendedBindingProperties and inherit AbstractExtendedBindingProperties to specify consumption attribute classes, send message attribute classes, and Binding attribute classes
  

• Create RedisMQConsumerProperties consumption message configuration class
  

• Create RedisMQProducerProperties sending message configuration class

• To create a RedisMQBindingProperties configuration class, you need to implement BinderSpecificPropertiesProvider. The attributes are both the consumption message attribute class and the sending message attribute class
  

provider

Once the parameters are processed, we need to create a provider. The provider records the topics and partitions that we need to send or consume messages. Providers need to implement ProvisioningProvider

message channel binder

The message channel binder needs to provide Classes for creating sending message instances, consuming message instances, obtaining extended producer attribute classes, extended consumer attribute classes, and Binding attribute classes. Inherit AbstractMessageChannelBinder to implement ExtendedPropertiesBinder

Concrete producer implementation

The specific producer implementation class will complete the specific message connection initialization work, specific message sending and other operations. Need to inherit AbstractMessageHandler to implement Lifecycle

Concrete consumer implementation

To complete specific message consumption and stop message consumption, you need to inherit MessageProducerSupport