1. Architecture diagram and roles

1. Architecture diagram

Insert picture description here

2. Role

2.1、Broker

Understand as RocketMQ itself
Broker is mainly used for producer and consumer to receive and send messages
The broker regularly submits its information to the nameserver
It is the message storage and forwarding server of message middleware
When each Broker node is started, it will traverse the NameServer list, establish a long connection with each NameServer, register its own information, and then report it regularly

2.2、Nameserver

Understand the effect of zookeeper, but he didn’t use zk, instead he wrote a nameserver instead of zk.
The bottom layer is implemented by netty, which provides routing management, service registration, and service discovery functions. It is a stateless node
The nameserver is the service discoverer. Each role in the cluster (producer, broker, consumer, etc.) needs to report their status to the nameserver regularly in order to discover each other. If the timeout does not report, the nameserver will remove it from the list.
Multiple nameservers can be deployed. When multiple nameservers exist, other roles report information to them at the same time to ensure high availability.
NameServer clusters do not communicate with each other, and there is no concept of active and standby
Nameserver memory storage, the broker, topic and other information in the nameserver will not be persisted by default, so it is a stateless node

2.3、Producer

Message producer
Randomly select one of the NameServer nodes to establish a long connection to obtain Topic routing information (including queues under topic, which brokers these queues are distributed on, etc.)
Next, establish a long connection to the master providing topic services (because only the master can write messages in rocketmq), and send heartbeats to the master regularly

2.4、Consumer

Message consumer
Obtain Topic routing information through the NameServer cluster, and connect to the corresponding Broker to consume messages
Since both Master and Slave can read messages, Consumer will establish a connection with both Master and Slave to consume messages

3. The core process

Brokers are registered on the nameserver
When the Producer sends a message, it will obtain the topic information of the message from the name server
Producer establishes long connections to all masters that provide services, and sends heartbeats to masters regularly
Consumer obtains Topic routing information through the NameServer cluster
The Consumer will establish connections with all Masters and all Slaves to monitor new messages

Two, the core concept

1、Message

Message carrier. Topic must be specified when Message is sent or consumed. Message has an optional Tag item for filtering messages, and additional key-value pairs can be added.

2、topic

Logical classification of the message. Before sending a message, you must specify a topic to send, that is, the message is sent to this topic. Specify this topic for consumption when consuming messages. It is logical classification.

3、queue

A Topic will be divided into N Queues, and the number is configurable. The message itself is actually stored on the queue, and what consumers consume is also the message on the queue. Let me talk more about it. For example, if there are 1 topic and 4 queues, and 5 consumers are consuming this topic, then one consumer will be wasted. Because of the load balancing strategy, each consumer consumes 1 queue, 5>4, overflow 1 Yes, this will not work.

4、Tag

Tag is a further subdivision of Topic, as the name implies, tag. Each message can be tagged when it is sent, and it can be filtered according to the tag when it is consumed, and it can be selectively consumed.

5、Message Model

Message model: Clustering and Broadcasting

6、Message Order

Message order: order (Orderly) and concurrent (Concurrently)

7、Producer Group

Message Producer Group

8、Consumer Group

Message consumer group

Three, ACK

The first thing to be clear is that the ACK mechanism occurs on the Consumer side, not on the Producer side . That is to say, after the Consumer consumes the message, it must confirm the ACK. If it is not confirmed, it means that the consumption has failed. At this time, the Broker will carry out a retry strategy (only the cluster mode will retry). ACK means: Consumer said: ok, I succeeded in consumption. Mark this message as consumed.

4. Consumption model

1. Cluster mode (Clustering)

1.1 Diagram

Insert picture description here

1.2. Features

Each message only needs to be processed once, and the broker will only send the message to one consumer in the consumer cluster
When the message is reposted, it is not guaranteed to be routed to the same machine
The consumption status is maintained by the broker

The core concept and architecture diagram of RocketMQ

1. Architecture diagram and roles