Apache RocketMQ architecture and core concepts

Overview

RocketMQ is a message and stream processing platform with low latency, high performance, high reliability, trillion-level message storage, and flexible scalability. The system architecture diagram is as follows: RocketMQ system architecture diagramRocketMQ is mainly composed of four core parts: name servers, Brokers, producers and consumers, each part can be deployed in clusters to avoid single point of failure.

Core idea

Producer

Message producers send messages to brokers through business application systems. RocketMQ provides a variety of message sending paradigms: synchronous (synchronous), asynchronous (asynchronous) and one-way (one-stop).
The Producer establishes a persistent connection with one of the nodes (randomly selected) in the Name Server cluster, periodically obtains topic routing information from the Name Server, establishes a persistent connection to the Master that provides the Topic service, and regularly sends a heartbeat to the Master. Producer is completely stateless and can be deployed in clusters.

Producer Group

Identifies the Producer that sends the same type of message, usually with the same sending logic.

Consumer

Message consumers get messages from brokers for processing. RocketMQ provides two consumer implementations: PullConsumer and PushConsumer.
The Consumer establishes a persistent connection with one of the nodes (randomly selected) in the Name Server cluster, periodically obtains topic routing information from the Name Server, establishes persistent connections to the Master and Slave that provide Topic services, and regularly sends heartbeats to the Master and Slave. The Consumer can subscribe to messages from the Master or from the Slave. The subscription rules are determined by the Broker configuration.

PullConsumer

A consumer of pull mode. The application actively controls the timing of pulling, how to pull, how to consume, etc. The initiative is higher, but it must handle various scenarios by itself.

PushConsumer

Consumers in push mode. RocketMQ uses long polling. The essence of this mode is to encapsulate message pulling, consumption processing and other maintenance logic internally, and reserve callback interfaces for users to process the logic after the message arrives.

Consumer Group

Identifies Consumers that process the same type of message, usually with the same processing logic. For consumer instances of the same consumer group, the topics subscribed to by them must be exactly the same.

Topic

It can be understood as the granularity of the message grouping, the logical management unit of the message. Regardless of message production or consumption, Topic needs to be specified.

Message

The amount of information being sent. A Message must contain a topic, as well as optional attribute tags and custom key-value pair information. For example, you can put the business primary key into the Message to facilitate business processing when consuming messages later.

Message Queue

The message queue, the physical management unit of the message, is understood as a subset partition of the topic.

Tag

message label. RocketMQ supports tagging topics when sending, so different logic processing is performed according to different tags at the time of consumption.

Broker

Broker is the main component of RocketMQ. It receives messages from producers, stores messages, and processes requests from consumers to pull messages. It also saves message metadata, consumer groups, consumer process metrics, and topic/queue information.

Name Server

The Name server acts as the routing information provider, and the Producer/Consumer client looks up the topic to obtain the corresponding broker list information.

Message Model

  • Clustering (cluster, messages are consumed by a Customer)
  • Broadcasting (broadcasting, messages are consumed by all customers)

Message Order

Message ordering means that when a type of message is consumed, it can be consumed in the order in which it was sent.

Guess you like

Origin http://43.154.161.224:23101/article/api/json?id=325235184&siteId=291194637