Detailed explanation of PushConsumer core parameters
consumerFromWhere: Indicates where the consumer starts to consume
allocateMessageQueueStrategy: Message distribution strategy, used in cluster mode, evenly distributed to all clients in cluster mode
subscription:subscription
offsetStore: Store the actual offset
consumeThreadMin/consumeThreadMax：Automatically adjust configuration items of thread pool, adjust the thread pool on the consumer side, and assist consumer consumption
consumeConcurrentlyMaxSpan/pullThresholdForQueue: Flow control, the first one indicates the parallel span of a single queue, and the second one indicates the maximum number of messages in a queue
pullinterval/pullBatchSize：Pull mode message pull, the first is mainly the time interval of message pull, the second is how many pieces of data are pulled at one time
consumerMessageBatchMaxSize: The default is 1, which means how many pieces of data are pulled at one time
PushConsumer consumption mode-cluster mode
Clustering mode (default)
GroupName is used to organize multiple Consumers together
Consumers of the same GroupName only consume part of the subscribed messages, in order to achieve a natural load balancing mechanism
PushConsumer consumption mode-broadcast mode
Broadcasting mode (broadcast mode)
Consumers in the same ConsumerGroup all consume and subscribe to all the Topic information, that is, a message will be consumed by each Consumer
setMessageModel method
Message storage core-Offset
Offset is the core of message consumption progress
Offset refers 某个topic下的一条消息to the position in a MessageQueue
This message can be located through Offset
Offset storage implementation is divided into 远程文件类型和本地文件two types, the following analysis of these two modes
Cluster mode-RemoteBrokerOffsetStore analysis
By default 集群模式Clustering, use remote file storage Offset
Essentially because of the multi-consumption model, each Consumer consumes part of the subscribed topic
In this case, Broker needs to control the value of offset, using RemoteBrokerOffsetStore
Broadcast mode-LocalFileOffsetStore analysis
广播模式Down, because each Consumer will receive the message and consume
There is no interference between each Consumer, independent thread consumption
So use LocalFileOffsetStore, that is, store the Offset locally
Analysis of Consumer Long Polling Mode
DefaultPushConsumer is implemented using long polling mode
Usually the mainstream message acquisition mode:Push消息推送模式 & Pull消息拉取模式
Push message push mode: Broker端主动把消息推送到消费端消费This method has many disadvantages. Firstly, it increases the workload of the Broker and requires additional threads of the Broker to interact with the consumer; secondly, the processing capabilities of the client are different, some are high, some are low, and some have processing capabilities. It won't work, so it is uncontrollable for Broker to push messages. For example, if the Client consumes too slowly, messages will accumulate, causing memory overflow. This is the message push mechanism.
Pull message pull mode: The 主动权掌握在Consumer手里client can pull as much as it consumes according to its own consumption capacity, but there are also problems, such as the time interval of each pull message, such as a message in the queue, but the pull time is one minute Pulling once is basically meaningless for most of the time; on the other hand, the pulling time is too long and the message will not be processed in the first time.
Long polling mechanism: RocketMQ is neither a Push nor Pull mechanism. It uses a long polling mechanism; in fact, it is also a consumer who initiates a request to the Broker. It is different from Pull. After the request is sent to the Broker Broker不让请求直接响应，它会做一个长时间的阻塞, The default blocking time is 15s. If there is a message, it will return directly. If there is no message, it will block and wait.