1, RocketMq four core components:
In RocketMQ, there are several core modules: Producer , Consumer , Broker , NameSrv . The relationship between them is as follows
First a quick look at the function of each module, the following will describes the functions of each module.
Producer and Consumer well understood, as the name suggests is the producer and consumer, producer in charge of production news, the consumer is responsible for consumer news, which are two logical defined by the business user.
Broker is the heart of RocketMQ, Broker achieve storing messages, pulling and other functions. Broker usually start the cluster approach, and the master-slave configuration to provide services on the specified topic on each Broker. Understand the principles Broker, as well as other services and interactive way to understand the basic principle of the whole messaging middleware.
NameSrv is a stateless name service, you can cluster deployment. All Broker will start when registering their information to NameSrv. Producer will get the target topic from NameSrv arrived at the designated Broker routing information, Consumer empathy.
For the Producer end RocketMQ using a polling way to ensure load balancing, Consumer end usually cluster cluster approach consumer information, we can define your own news distribution at the end of the message. In addition, MQ also provides the characteristics of the order message, simply look at the characteristics of the MQ to provide specific implementation will be explained in later chapters.
2, RocketMQ packet structure detailed analysis:
Source directory structure introduced RocketMQ source code into the following package:
Broker-rocketmq : the entire core mq, he was able to accept the request of producer and consumer, and call the store level service processes the message. HA service basic unit supports simultaneous dual-write , asynchronous dual-write modes.
CLIEN-rocketmq :: mq client implementation, the current official version of the only open source java mq client , c ++, Go client has open source community contributions.
Common-rocketmq : Some common inter-module function class , such as configuration files, constant.
Example-rocketmq : official of example , typical functions such order message, push consumer, pull consumer usage have been demonstrated.
filtersrv-rocketmq : message filtering service, equivalent broker and a filter joined intermediate proxy consumer.
Remoting-rocketmq : Based on the bottom of the netty communication achieve interaction between all services are based on this module.
srvut-rocketmq : parsing command line tools.
Store-rocketmq : a storage layer, it includes both indexing services, high availability HA service implementation.
Tools-rocketmq : MQ cluster management tool that provides information query.
3, Rocketmq-remoting telecommunications Resolution:
rocketmq-remoting remoting module communication layer is introduced on the basis mq communication module, the communication layers in understanding the principles of the interaction between helpful understanding module.
For an actual request, MQ is how to encode and decode and distribute requests it? Two important classes include NettyRemotingClient and NettyRemotingServer, here as an example to look at it NettyRemotingServer start
You can see ch.pipeline (). AddLast is added to the pipe processing logic data, you first need to know an event handler for each event handler, he can handle include the following (covering the parent class method can be realized) , event handling method as long as the condition data will go through each corresponding to a handler:
channelActive , channelInactive : connection establishment and the connection is closed when the correction will be.
channelRead : When the channel data is read when the callback function will be. It is from this mq distribution function for processing the request to the back end of the thread.
exceptionCaught : the occurrence of an exception callback.
userEventTriggered : When the above events do not meet their needs, users can inside this custom event handler method.
** ** Pipeline pipe handler defined mq of the following meanings: _ - `NettyEncoder`,` NettyDecoder` : an encoder and a decoder corresponding to the logical mq, they were covered and ** ** ** encode parent class decode ** method. - `IdleStateHandler`: Netty own heartbeat manager -` NettyConnetManageHandle`: Connection Manager, he was responsible for capturing new connection, disconnection, abnormal events and then unified to NettyEventExecuter processor. - `NettyServerHandler`: When a message after a preceding decoding step, and then dispatched to channelRead0 method, and then distributed according to the type of the message
continues to track ` NettyServerHandler` Code:
_ Processing logic code into the next process request message and a response message.
_ ** (a) processing the request message processRequestCommand **
First see a member of the class of `NettyRemotingAbstract`:` `` HashMap> processorTable ` ` ` annotation can be seen, showing the key` request code`, mq may be different different types of processor request code specifies `Processor` process, it is to be noted that the actual message is not processed in the current thread, but is encapsulated into the task thread pool corresponding to` Processor`:
final RequestTask requestTask = new RequestTask(run, ctx.channel(), cmd); pair.getObject2().submit(requestTask);
In RocketMQ can be seen in many places are like this deal, this design to the greatest degree of assurance asynchronous ensure that each thread dedicated processing what they are responsible. The following are the Processor implementation:
Finally, processRequestCommand the overall processing logic function as follows:
In addition, we should note, the second step build time task, using a template design patterns , adding a before and after the mission hook: we can use this hook to do some extra work, such as encryption decrypting messages.
rpcHook.doBeforeRequest(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd);Processor.processRequest()rpcHook.doAfterResponse(RemotingHelper.parseChannelRemoteAddr(ctx.channel()), cmd, response);
** (b) processing the response message processResponseCommand **
In fact, this part of the process is not too difficult, to first understand the following structure:
protected final ConcurrentHashMap<Integer /* opaque */, ResponseFuture> responseTable
opaque party initiates a connection request represents a request for a different identification code on the same connection, each transmitting a message, he can choose to sync mode and blocking non-blocking asynchronous manner, Either way, he will save operation code to ResponseFuture mapping. Key explain ResponseFuture this class, the more important members include a callback function the invokeCallback , and a semaphore semaphore .
- For synchronization message , these two parameters are usually a null.
For asynchronous messages , the invokeCallback action is possible when it receives a response message according responseTable find the opcode corresponding to a callback function; semaphore main function is to act as flow control , when a plurality of threads simultaneously write data to a connection may be through a signal volume control permit at the same time to write the number of licenses.
In simple terms, the overall process is as follows:
- Of course, the flowchart does not include an operation amount of resources further comprises a release signal, and clearing responseTable table key-related information and other operations.
`NettyRemotingClient` treatment is actually consistent with · NettyRemotingServer · treatment, the only difference is Netty pipeline in connection management ** ** additional relevant handler also handles the` `connect event, the event in the client actively connected to the remote after the success callback.
https://mp.weixin.qq.com/s/vkvYJnKfQyuUeD_BDQy_1g
For more learning materials, can be added to the group: 473 984 645 or under the Fanger Wei code scanning
