Overview

Apache RocketMQ and distributed message stream is a platform, which characteristics include low latency, high performance, high reliability, and the elastic expansion capacity of one trillion. Its architecture includes four parts: Naming Service (name servers), agency services (brokers), producer (Producers), consumers (Consumers). Wherein each portion is extended horizontally to avoid single point of failure. as the picture shows:

Here Insert Picture Description ## Naming Service Cluster
Naming Service provides a lightweight service for finding and routing. Each naming service are recorded full amount of routing information, provides read and write consistent service and support rapid storage expansion.

Agency services cluster

Agency services using the lightweight topic (TOPIC) and queues (QUEUE) mechanism to store messages. Push proxy service support (the Push) and pull (the Pull) mode, to support fault tolerance by using multiple copies (2 copies of parts or 3 minutes), to provide a strong peak in the original time-sequentially filled and stacked trillion magnitude message. And agency services also have some traditional messaging systems do not have, such as disaster recovery, multi-dimensional statistics, early warning mechanism.

Producers Cluster

Producer support distributed deployment. Producers will be distributed through load balancing information is sent to the proxy service cluster, the sending process to support rapid failure (fail fast) [^ 1] and low latency.

Consumers cluster

Consumers can also support distributed deployments in push-pull mode, and supports clustering consumption, and message broadcasting. In this way also provides real-time news subscription mechanism to meet the needs of most consumers. RocketMQ official website also provides relevant pages to interested users.

Naming Service

Naming Service is purely functional server, it contains two main features:

Agency management, naming service to accept registered agent services cluster, while the agent to check the validity of a single heartbeat mechanism
Routing management, each routing services have saved the proxy cluster complete routing information and queue information query client needs. RocketMQ client (producer / consumers) will get routing information from the queue naming service, but how the client obtains naming service address?

Here are four methods naming service address to the client pushed flows:

Hard coded, such as producer.setNamesAddr("ip:port").
Java operating parameters, use rocketmq.namesrv.addr.
Environment variables, use NAMESRV_ADDR.
HTTP endpoint (Endpoint).

Click here to see more details about the document naming service.

Agency services

Proxy service is responsible for message storage and distribution, information queries, high availability, and so on.
As shown below, the proxy service includes several important sub-modules:

Remote modules, the proxy service entrance, the processing request sent by the client.
Client management, managing client (producer / consumer) and to safeguard consumer subscription.
Storage services, or provide simple query interface to save messages are stored on a physical hard disk.
High availability service, the main agents and providers directly from proxy data synchronization.
Indexing Service, the message constructed according to a specific index key, a message to provide fast retrieval.

Here Insert Picture Description [^ 1]: When multiple threads operating on the same set of content, it may generate fail-fast event.
For example: When a thread A process iterator to traverse through a collection of content if the set is changed by the other thread; then thread A accesses the collection, an exception will be thrown ConcurrentModificationException generate fail-fast event

RocketMQ Infrastructure

Article Directory