Understand RocketMQ message queue in three minutes

Language 2023-07-16 08:17:24 views: null

Article directory


RocketMQ is a distributed message middleware with strong performance and high reliability, which has been widely used in large distributed systems. This article will briefly explain the principle, architecture, usage and practical application of RocketMQ from entry to principle introduction.
Since its birth, Apache RocketMQ has been widely adopted by many enterprise developers and cloud vendors because of its simple architecture, rich business functions, and strong scalability. After more than ten years of large-scale scene polishing, RocketMQ has become the industry's consensus first-choice solution for financial-level reliable business messages, and is widely used in business scenarios in the fields of the Internet, big data, mobile Internet, and the Internet of Things.
insert image description here

basic concept

insert image description here
Before learning RocketMQ, you need to understand some basic concepts:

  • Producer: The producer of the message is responsible for sending the message to the Broker.
  • Broker: The middleware of the message, responsible for storing and delivering the message.
  • Consumer: The consumer of the message is responsible for consuming messages from the Broker.
  • Topic: The topic of a message is a logical concept of a message and is used to identify a type of message.
  • Message: The entity of the message, including message content, tags, attributes and other information.
  • NameServer: RocketMQ's registration center, used to save Broker's metadata information.

detailed introduction

This article introduces the basic concepts of Apache RocketMQ so that you can better understand and use Apache RocketMQ.

Topic

The top-level container for message transmission and storage in Apache RocketMQ, used to identify messages of the same type of business logic. Topics are uniquely identified and distinguished by TopicName. See Topic for more information .

Message Type (MessageType)

The classification defined in Apache RocketMQ according to the different characteristics of message transmission is used for type management and security verification. The message types supported by Apache RocketMQ are ordinary messages, sequence messages, transaction messages and timing/delay messages.

Starting from version 5.0, Apache RocketMQ supports mandatory verification of message types, that is, each topic Topic is only allowed to send messages of one message type, which can better maintain and manage the production system and avoid confusion. But at the same time, it is guaranteed to be backward compatible with the behavior of version 4.x. The mandatory verification function is disabled by default. It is recommended to manually enable the verification through the server parameter enableTopicMessageTypeCheck.

Message Queue (MessageQueue)

The queue is the actual container for message storage and transmission in Apache RocketMQ, and it is also the smallest storage unit for messages. All topics of Apache RocketMQ are composed of multiple queues, so as to realize the horizontal splitting of the number of queues and the streaming storage inside the queues. Queues are uniquely identified and differentiated by QueueId. For more information, see Queues (MessageQueue) .

Message

A message is the smallest unit of data transfer in Apache RocketMQ. The producer packs the load and extended attributes of the business data into a message and sends it to the server, and the server delivers the message to the consumer for consumption according to the relevant semantics. See Message for more information .

Message View (MessageView)

The message view is a message read-only interface provided by Apache RocketMQ for the development perspective. Through the message view, you can read multiple attributes and payload information inside the message, but you cannot make any modifications to the message itself.

Message Tag (MessageTag)

Message tags are fine-grained message classification attributes provided by Apache RocketMQ, which can be subdivided under the topic level. Consumers implement fine-grained filtering by subscribing to specific tags. For more information, see Message Filtering .

Message position (MessageQueueOffset)

Messages are stored in multiple queues of a specified topic in the order in which they arrive at the Apache RocketMQ server. Each message has a unique Long type coordinate in the queue, which is defined as a message location. For more information, see Consumption Progress Management .

Consumer Offset (ConsumerOffset)

After a message is consumed by a consumer, it will not be deleted from the queue immediately. Apache RocketMQ will record the location of the latest message consumed based on each consumer group, that is, the consumption location. For more information, see Consumption Progress Management .

Message Index (MessageKey)

Message index is a message-oriented index property provided by Apache RocketMQ. You can quickly find the corresponding message content through the set message index.

Producer

A producer is a running entity in the Apache RocketMQ system used to construct and transmit messages to the server. The producer is usually integrated in the business system, encapsulates business messages into messages as required and sends them to the server. See Producer for more information .

Transaction Checker (TransactionChecker)

A listener used by producers in Apache RocketMQ to perform local transaction checking and abnormal transaction recovery. The transaction checker should check and judge the status of the transaction message through the status of the business side data. For more information, see Transaction Messages .

Transaction Status (TransactionResolution)

In the process of sending a transaction message in Apache RocketMQ, the status of the transaction submission is identified, and the server controls whether the transaction message should be submitted and delivered through the transaction status. Transaction states include transaction commit, transaction rollback, and transaction pending. For more information, see Transaction Messages .

Consumer Group (ConsumerGroup)

A consumer group is a load-balancing group in the Apache RocketMQ system that carries multiple consumers with consistent consumption behavior. Unlike a consumer, a consumer group is not a running entity, but a logical resource. In Apache RocketMQ, the horizontal expansion of consumption performance and high-availability disaster recovery are realized by initializing multiple consumers in the consumer group. For more information, see Consumer Group (ConsumerGroup) .

Consumer

Consumers are running entities in Apache RocketMQ that receive and process messages. Consumers are usually integrated in the business system, obtain messages from the server, and convert the messages into business-understandable information for business logic processing. See Consumer for more information .

Consume Result (ConsumeResult)

The processing result returned by the PushConsumer consumer listener in Apache RocketMQ after processing the message is used to identify whether the message is processed correctly. The consumption result includes consumption success and consumption failure.

Subscription relationship (Subscription)

The subscription relationship is the rules and status configurations for consumers to obtain and process messages in the Apache RocketMQ system. The subscription relationship is dynamically registered to the server system by the consumer group, and in the subsequent message transmission, message matching and consumption progress maintenance are performed according to the filtering rules defined by the subscription relationship. See Subscription for more information .

message filtering

Consumers can filter messages by subscribing to a specified message tag (Tag) to ensure that only the filtered message collection is finally received. The calculation and matching of filtering rules
are done on the server side of Apache RocketMQ. For more information, see Message Filtering .

Reset consumption point

Taking the time axis as the coordinate, within the time range of persistent message storage, reset the consumption progress of the consumer group on the subscribed topic. After the setting is completed, the consumer will receive the set time point, and the producer will send it to the Apache RocketMQ service terminal news. For more information, see Resetting Consumption Sites .

message track

In the process of sending a message from the producer to receiving and processing by the consumer, the complete link information is gathered from the time, location and other data of each related node. Through the message trace, you can clearly locate the complete link where the message is sent from the producer and delivered to the consumer through the Apache RocketMQ server, which is convenient for locating and troubleshooting.

news accumulation

The producer has sent the message to the Apache RocketMQ server, but due to the limited consumption capacity of the consumer, it cannot consume all the messages correctly in a short time. At this time, the unconsumed message is saved on the server. That is, message accumulation.

transactional message

Transactional messages are an advanced message type provided by Apache RocketMQ, which supports eventual consistency of message production and local transactions in distributed scenarios.
The appeal of distributed transactions
insert image description here
The characteristic of distributed system calls is the execution of a core business logic, and at the same time, multiple downstream services need to be called for processing. Therefore, how to ensure that the execution results of the core business and multiple downstream businesses are completely consistent is the main problem that distributed transactions need to solve. Business message request

Taking the e-commerce transaction scenario as an example, the core operation of user payment for orders will also involve changes in multiple subsystems such as downstream logistics delivery, point changes, and shopping cart status clearing. The processing branches of the current business include:

Status update of the main branch order system: changed from unpaid to successfully paid.

Added logistics system status: Add logistics records to be shipped, create order logistics records.

Status change of points system: change user points, update user points table.

The status of the shopping cart system changes: clear the shopping cart and update the user's shopping cart record.

Timed/Delayed Messages

Timing/delay message is an advanced message type provided by Apache RocketMQ. After the message is sent to the server, it can be consumed by consumers after a specified time. By setting a certain timing time, the delayed scheduling trigger effect of the distributed scene can be realized.
Task timeout processing Overtime task processing
insert image description here
Take the e-commerce transaction scenario as an example. After the order is placed, the payment has not yet been made. At this time, the order cannot be closed directly, but it needs to wait for a period of time before closing the order. Using Apache RocketMQ timing messages can realize the checking and triggering of timeout tasks.

The overtime task processing based on timing messages has the following advantages:
high precision and low development threshold: there is no timing step interval based on the message notification method. Arbitrary-precision event triggering can be easily implemented without deduplication of services.

High-performance and scalable: The traditional database scanning method is relatively complex, requiring frequent calls to interface scanning, which is prone to performance bottlenecks. Apache RocketMQ's timed messages have high concurrency and horizontal expansion capabilities.

sequential message

Sequential message is an advanced message type provided by Apache RocketMQ, which supports consumers to obtain messages in the order in which they are sent, so as to realize sequential processing in business scenarios.
In scenarios such as orderly event processing, matching transactions, and real-time incremental data synchronization, heterogeneous systems need to maintain strong and consistent state synchronization, and upstream event changes need to be transmitted to downstream for processing in order. In such scenarios, using Apache RocketMQ's sequential messages can effectively guarantee the sequentiality of data transmission.

Typical Scenario 1: Matchmaking Transaction Matchmaking
insert image description here

Taking securities and stock trading as an example, for transaction orders with the same bid, the principle of first bid first trade is adhered to, and the downstream order processing system needs to process orders strictly in accordance with the order of bids.

Typical Scenario 2: Real-time Incremental Data Synchronization

general newsinsert image description here

sequential message sequential message
insert image description here

Taking the database change incremental synchronization scenario as an example, the upstream source database performs addition, deletion, and modification operations as needed, and the binary operation log is used as a message, which is transmitted to the downstream search system through Apache RocketMQ, and the downstream system restores the message data in order to realize the state data order refresh. If it is an ordinary message, it may cause state confusion, which is inconsistent with the expected operation result. Based on the sequential message, the downstream state can be consistent with the upstream operation result.

Functional principle

what is sequential message

Sequential message is an advanced message type provided by Apache RocketMQ, which supports consumers to obtain messages in the order in which they are sent, so as to realize sequential processing in business scenarios. Compared with other types of messages, sequential messages place more emphasis on the sequential relationship between multiple messages during the process of sending, storing, and delivering.

The order relationship of Apache RocketMQ sequential messages is judged and identified by the message group (MessageGroup). There is no sequence involved between the messages of the message group and the non-message group.

Based on the sequence judgment logic of the message group, it supports fine-grained splitting according to the business logic, which can improve the parallelism and throughput of the system on the premise of satisfying the partial order of the business.

How to ensure the order of messages

The sequence of Apache RocketMQ messages is divided into two parts, production sequence and consumption sequence.

  • Production sequence :

    Apache RocketMQ guarantees that a single producer sends messages serially through the agreement between the producer and the server, and stores and persists them sequentially.

    To ensure the order of message production, the following conditions must be met:

    • Single producer: The sequence of message production only supports a single producer. Different producers are distributed in different systems. Even if the same message group is set, the sequence of messages generated between different producers cannot be determined.
    • Serial sending: The Apache RocketMQ producer client supports multi-thread security access, but if the producer uses multiple threads to send in parallel, the order of messages generated between different threads cannot be determined.

    Producers that meet the above conditions, after sending sequential messages to Apache RocketMQ, will ensure that messages with the same message group are set and stored in the same queue in the order of sending. The sequential storage logic of the server is as follows:

    • Messages of the same message group are stored in the same queue in sequence.
    • Messages from different message groups can be mixed in the same queue, and continuity is not guaranteed.

sequential storage logic

As shown in the figure above, the messages of message group 1 and message group 4 are mixed and stored in queue 1. Apache RocketMQ ensures that the messages G1-M1, G1-M2, and G1-M3 in message group 1 are stored in the order of sending, and the message group The messages G4-M1 and G4-M2 of 4 are also stored in sequence, but the messages in message group 1 and message group 4 do not involve sequence relations.

  • Consumption sequence :

    Apache RocketMQ guarantees that message consumption is processed strictly according to the order of storage through the agreement between the consumer and the server.

    To ensure the order of message consumption, the following conditions must be met:

    • delivery order

      Apache RocketMQ guarantees that messages are delivered according to the storage order of the server through the client SDK and server-side communication protocol. However, when the business party consumes messages, it needs to process messages strictly according to the semantics of receiving-processing-response to avoid message disorder due to asynchronous processing.

      Remark

      When the consumer type is PushConsumer, Apache RocketMQ guarantees that the messages are delivered to the consumer one by one according to the storage order. If the consumer type is SimpleConsumer, the consumer may pull multiple messages at a time. At this time, the sequence of message consumption needs to be guaranteed by the business side. For specific information on consumer types, see Consumer Classification .

    • limited retries

      Apache RocketMQ sequential message delivery is only within the limit of the number of retries, that is, if a message fails to be retried all the time, it will not be retried after exceeding the maximum number of retries, skipping the consumption of this message, and will not always block subsequent message processing.

      For scenarios where the order of consumption needs to be strictly guaranteed, please set a reasonable number of retries to avoid message disorder caused by unreasonable parameters.

Combination of production sequence and consumption sequence

If messages need to be processed in strict accordance with the first-in-first-out (FIFO) principle, that is, those sent first are consumed first, and those sent later are consumed later, then both production sequence and consumption sequence must be satisfied.

In general business scenarios, the same producer may connect to multiple downstream consumers. Not all consumer businesses require sequential consumption. You can differentiate the combination of production sequence and consumption sequence and apply it to different business scenarios . For example, send sequential messages, but use non-sequential concurrent consumption to improve throughput. More combinations are shown in the table below:

production sequence consumption order sequential effect
Set the message group to ensure that the messages are sent in order. sequential consumption According to the message group granularity, the order of messages is strictly guaranteed. The consumption order and sending order of messages in the same message group are exactly the same.
Set the message group to ensure that the messages are sent in order. concurrent consumption Concurrent consumption should be processed in chronological order as much as possible.
No message group is set, messages are sent out of order. sequential consumption Store granularity by queue, in strict order. Based on the properties of Apache RocketMQ's own queue, the order of consumption is consistent with the order of queue storage, but it is not guaranteed to be consistent with the order of sending.
No message group is set, messages are sent out of order. concurrent consumption Concurrent consumption should be processed in chronological order as much as possible.

Sequential message life cycle life cycle

  • Initialization: The message is constructed and initialized by the producer, and is ready to be sent to the server.
  • To be consumed: The message is sent to the server, visible to the consumer, and waiting for the consumer to consume.
  • Consuming: The process in which the message is acquired by the consumer and processed according to the consumer's local business logic. At this time, the server will wait for the consumer to complete the consumption and submit the consumption result. If no response is received from the consumer after a certain period of time, Apache RocketMQ will retry the message. For details, see Consumption Retries .
  • Consumption submission: The consumer completes the consumption process and submits the consumption result to the server, and the server marks that the current message has been processed (including consumption success and failure). Apache RocketMQ supports retaining all messages by default. At this time, the message data will not be deleted immediately, but the logical mark has been consumed. Before the message is deleted when the storage time expires or the storage space is insufficient, the consumer can still go back and consume the message again.
  • Message deletion: Apache RocketMQ scrolls and cleans up the earliest message data according to the message preservation mechanism, and deletes the message from the physical file. For more information, see Message Storage and Cleanup Mechanisms .

parameter constraints

There are many custom parameters and resource names in the Apache RocketMQ system. When using Apache RocketMQ, it is recommended to refer to the following instructions to standardize the system settings, so as to avoid unreasonable setting of some specific parameters and cause abnormalities in the application.

parameter Recommended range illustrate
Topic name Suggested characters: letters a~z or A~Z, numbers 0~9, underscore ( ), dash (-) and percent sign (%).
Suggested length: 1~64 characters.
System reserved characters: Topic names are not allowed to use the following reserved characters or characters with special prefixes.
Reserved characters: TBW102 * BenchmarkTest * SELF_TEST_TOPIC * OFFSET_MOVED_EVENT * SCHEDULE_TOPIC_XXXX * RMQ_SYS_TRANS_HALF_TOPIC * RMQ_SYS_TRACE_TOPIC * RMQ_SYS_TRANS_OP_HALF_TOPIC
Special prefix: * rmq_sys %RETRY%_ %DLQ%_ rocketmq-broker-		 Topic naming should try to use short and commonly used characters and avoid using special characters. Special characters will cause system parsing exceptions, and characters that are too long may cause message sending and receiving to be rejected.
ConsumerGroupName Suggested characters: letters a~z or A~Z, numbers 0~9, underscore (_), dash (-) and percent sign (%) are supported.
Suggested length: 1~64 characters.
System reserved characters: ConsumerGroup does not allow the following reserved characters or characters with special prefixes to be named.
Reserved characters: * DEFAULT_CONSUMER * DEFAULT_PRODUCER * TOOLS_CONSUMER * FILTERSRV_CONSUMER * _ MONITOR_CONSUMER * CLIENT_INNER_PRODUCER * SELF_TEST_P_GROUP * SELF_TEST_C_GROUP * CID_ONS-HTTP-PROXY * CID_ONSAPI_PERMISSION * C ID_ONSAPI_OWNER * CID_ONSAPI_PULL * CID_RMQ_SYS_TRANS * special characters * CID_RMQ_SYS * CID_HOUSEKEEPING		 none.
ACL Credentials Suggested characters: AK (AccessKey ID), SK (AccessKey Secret) and Token only support letters a~z or A~Z, numbers 0~9.
Recommended length: no more than 1024 characters.		 none.
request timeout Default: 3000 milliseconds.
Value range: This parameter is the local behavior of the client, and the value range is recommended not to exceed 30000 milliseconds.		 The request timeout is the waiting time of the client's local synchronous call. Please set a reasonable value according to the actual application to avoid excessive thread blocking time.
message size Default: no more than 4 MB. No message compression is involved, only the size of the message body is calculated.
Value range: no more than 4 MB is recommended.		 Message transmission should be compressed and the payload size should be controlled as much as possible to avoid oversized file transmission. If the size of the message does not meet the limit requirements, you can try to split the message or use OSS to store it, and use the message to transmit the URL.
Message custom properties Character limit: All visible characters.
Suggested length: The total length of the Key and Value of an attribute should not exceed 16 KB.
System reserved attributes: The following reserved attributes are not allowed to be used as keys of custom attributes. Reserved property Key		 none.
MessageGroup Character limit: All visible characters.
Suggested length: 1~64 bytes.		 MessageGroup is the grouping identifier for sequential messages. Generally, it is set to a set of message identifiers that need to guarantee the order, such as order ID, user ID, etc.
Message sending retries Default value: 3 times.
Value range: unlimited.		 Message sending retry is a built-in retry strategy of the client SDK, which is invisible to the application. It is recommended that the value not be too large to avoid blocking the business thread. If the message has not been sent successfully after reaching the maximum number of retries, it is recommended that the business side do a good job of handling it to ensure the reliability of the message.
Number of message consumption retries Default value: 16 times. The number of consumption retries should be set to a reasonable parameter value according to actual business needs to avoid infinite triggering using retries. Too many retries can easily cause an excessive increase in system pressure.
Transaction exception check interval Default: 60 seconds. The transaction exception check interval means that if the semi-transaction message is not submitted due to system restart or abnormal situation, the producer client will check back the transaction status according to this interval. It is not recommended to set the interval too short, otherwise frequent checkback calls will affect system performance.
The time of the first lookup of semi-transactional messages Default value: The value is equal to [Transaction exception check interval] * Maximum limit: No more than 1 hour. none.
The maximum timeout period for semi-transactional messages Default: 4 hours. * Value range: Custom modification is not supported. If the half-transaction message is not submitted due to system restart or abnormal conditions, the producer client will check back according to the transaction exception check interval. If the half-transaction message does not return a result after the timeout period of the half-transaction message, the half-transaction message will be forcibly rolled back. You can avoid unusual transactions by monitoring this metric.
PushConsumer local cache 默认值：
* 最大缓存数量：1024条。
* 最大缓存大小：64 M。
取值范围：支持用户自定义设置，无限制。		 消费者类型为PushConsumer时，为提高消费者吞吐量和性能，客户端会在SDK本地缓存部分消息。缓存的消息的数量和大小应设置在系统内存允许的范围内。
PushConsumer重试间隔时长 默认值：
* 非顺序性投递：间隔时间阶梯变化，具体取值，请参见PushConsumer消费重试策略。
* 顺序性投递：3000毫秒。		 无。
PushConsumer消费并发度 默认值：20个线程。 无。
获取消息最大批次 默认值：32条。 消费者从服务端获取消息时，一次获取到最大消息条数。建议按照实际业务设置合理的参数值，一次获取消息数量过大容易在消费失败时造成大批量消息重复。
SimpleConsumer最大不可见时间 默认值：用户必填参数，无默认值。
取值范围建议：最小10秒；最大12小时。

RocketMQ 的架构

insert image description here

RocketMQ 的架构分为两部分：消息存储和消息传输。

消息存储部分包括：

  • NameServer：用于保存 Broker 的元数据信息，例如 Broker 的地址、主题等。
  • Broker：用于存储和传递消息，包括消息的存储、消息的订阅和推送等功能。

消息传输部分包括：

  • Producer：消息的生产者，负责向 Broker 发送消息。
  • Consumer：消息的消费者，负责从 Broker 消费消息。

RocketMQ 的消息传输采用的是异步传输方式，生产者将消息发送到 Broker 后立即返回，不需要等待消息被消费者消费。消费者从 Broker 拉取消息后再进行消费，消费完成后向 Broker 发送确认信息，告诉 Broker 该消息已经被消费。

RocketMQ 的使用方法

RocketMQ 的使用方法分为两部分：生产者和消费者。

生产者

生产者负责将消息发送到 Broker，其主要步骤如下：

  1. 创建 DefaultMQProducer 对象，并设置 NameServer 的地址。
  2. 设置 Producer 的 Group 名称，用于区分不同的 Producer。
  3. 启动 Producer。
  4. 创建 Message 对象，设置消息的主题、标签和内容等信息。
  5. 调用 send 方法发送消息。
DefaultMQProducer producer = new DefaultMQProducer("producer_group");
producer.setNamesrvAddr("localhost:9876");
producer.start();

Message message = new Message("topic", "tag", "Hello World".getBytes());
SendResult result = producer.send(message);

producer.shutdown();

消费者

消费者负责从 Broker 消费消息，其主要步骤如下：

  1. 创建 DefaultMQPushConsumer 对象，并设置 NameServer 的地址。
  2. 设置 Consumer 的 Group 名称，用于区分不同的 Consumer。
  3. 注册 MessageListener 监听器，用于处理消费到的消息。
  4. 启动 Consumer。
  5. 订阅感兴趣的消息主题。
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumer_group");
consumer.setNamesrvAddr("localhost:9876");
consumer.registerMessageListener(new MessageListenerConcurrently() {
    
    
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
    
    
        for (MessageExt msg : msgs) {
    
    
            System.out.println(new String(msg.getBody()));
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();

consumer.subscribe("topic", "*");

RocketMQ 的实战应用

RocketMQ 在实际应用中有很多场景，例如：

  • 日志收集：将分布式系统的日志收集到一个中心化的地方进行分析和处理。
  • 消息通知：例如订单状态变更、支付成功等消息通知。
  • 分布式事务：RocketMQ 提供了事务消息的功能，支持分布式事务的一致性。

以消息通知为例，实现步骤如下：

  1. 创建消息生产者，向指定主题发送消息。
  2. 创建消息消费者，订阅指定主题，并注册消息监听器。
  3. 在消息监听器中处理消费到的消息，例如发送邮件、短信等通知。

示例代码：

// 生产者
DefaultMQProducer producer = new DefaultMQProducer("notification_producer_group");
producer.setNamesrvAddr("localhost:9876");
producer.start();

Message message = new Message("notification_topic", "order_status_changed", "Order #123 status changed to shipped".getBytes());
SendResult result = producer.send(message);

producer.shutdown();

// 消费者
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("notification_consumer_group");
consumer.setNamesrvAddr("localhost:9876");
consumer.registerMessageListener(new MessageListenerConcurrently() {
    
    
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
    
    
        for (MessageExt msg : msgs) {
    
    
            // 处理消息，例如发送邮件、短信等通知
            System.out.println(new String(msg.getBody()));
        }
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();

consumer.subscribe("notification_topic", "order_status_changed");

以上示例代码仅为演示代码，实际应用中还需要进行错误处理、重试机制等。同时，RocketMQ 还支持集群部署、负载均衡、消息过滤等功能，需要根据具体业务场景进行配置和使用。

总的来说，RocketMQ 的使用方法较为简单，但在实际应用中需要考虑到各种场景和问题，需要进行深入的学习和实践。

发送普通消息

package com.aliyun.openservices;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;
import org.apache.rocketmq.client.apis.message.MessageBuilder;

public class Demo {
    
    
    /**
     * 实例接入点，从控制台实例详情页的接入点页签中获取。
     * 如果是在阿里云内网 VPC 访问，建议填写 VPC 接入点。
     * 如果是在本地公网访问，或者是线下 IDC 环境访问，可以使用公网接入点。使用公网接入点访问，必须开启实例的公网访问功能。
     */
    public static final String ENDPOINT = "rmq-cn-pe33aqsu40k-vpc.cn-hangzhou.rmq.aliyuncs.com:8080";
    public static final String TOPIC_NAME = "rmqfctopic_nomal_retry";
    public static final String TAG = "";
    public static final String KEY = "";
    public static final String BODY = "";

    public static void main(String[] args) throws ClientException, IOException {
    
    
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder configBuilder = ClientConfiguration.newBuilder().setEndpoints(ENDPOINT);

        /**
         * 如果是使用公网接入点访问，configuration 还需要设置实例的用户名和密码。用户名和密码在控制台实例详情页获取。
         * 如果是在阿里云内网 VPC 中访问，无需填写该配置，服务端会根据内网 VPC 信息智能获取。
         */
        // configBuilder.setCredentialProvider(
        //  new StaticSessionCredentialsProvider("Instance UserName", "Instance Password")
        // );
        ClientConfiguration configuration = configBuilder.build();

        /**
         * 初始化 Producer 时直接配置需要使用的 Topic 列表，实现提前检查错误配置、拦截非法配置启动。
         * 针对非事务消息 Topic，也可以不配置，服务端会动态检查消息的 Topic 是否合法。
         * 注意！！！事务消息 Topic 必须提前配置，以免事务消息回查接口失败，具体原理请参见事务消息。
         */
        Producer producer = provider.newProducerBuilder()
                .setClientConfiguration(configuration)
                .setTopics(TOPIC_NAME)
                .build();

        MessageBuilder builder = provider.newMessageBuilder()
                // 为当前消息设置 Topic。
                .setTopic(TOPIC_NAME)
                // 消息体。
                .setBody(BODY.getBytes(StandardCharsets.UTF_8));

        if (!KEY.isEmpty()) {
    
    
            // 设置消息索引键，可根据关键字精确查找某条消息。
            builder.setKeys(KEY);
        }

        if (!TAG.isEmpty()) {
    
    
            // 设置消息 Tag，用于消费端根据指定 Tag 过滤消息。
            builder.setTag(TAG);
        }

        // 配置消息的自定义属性
        // builder.addProperty("key", "value");

        try {
    
    
            // 发送消息，需要关注发送结果，并捕获失败等异常。
            final SendReceipt sendReceipt = producer.send(builder.build());
            System.out.println("Send mq message success! Topic is:" + TOPIC_NAME + " msgId is: "
                    + sendReceipt.getMessageId().toString());
        } catch (Throwable t) {
    
    
            System.out.println("Send mq message failed! Topic is:" + TOPIC_NAME);
            t.printStackTrace();
        }
        
        // 如果不需要再使用，可关闭该进程。
        producer.close();
    }
}

PushConsumer 方式消费

package com.aliyun.openservices;

import java.util.Collections;
import java.io.IOException;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.consumer.FilterExpressionType;
import org.apache.rocketmq.client.apis.consumer.FilterExpression;
import org.apache.rocketmq.client.apis.consumer.ConsumeResult;
import org.apache.rocketmq.client.apis.consumer.PushConsumer;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;

public class Demo {
    
    
    /**
     * 实例接入点，从控制台实例详情页的接入点页签中获取。
     * 如果是在阿里云内网 VPC 访问，建议填写 VPC 接入点。
     * 如果是在本地公网访问，或者是线下 IDC 环境访问，可以使用公网接入点。使用公网接入点访问，必须开启实例的公网访问功能。
     */
    public static final String ENDPOINT = "rmq-cn-pe33aqsu40k-vpc.cn-hangzhou.rmq.aliyuncs.com:8080";
    public static final String TOPIC_NAME = "rmqfctopic_nomal";
    public static final String FILTER_EXPRESSION = "*";
    public static final String CONSUMER_GROUP_ID = "";

    public static void main(String[] args) throws ClientException, IOException, InterruptedException {
    
    
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder configBuilder = ClientConfiguration.newBuilder().setEndpoints(ENDPOINT);

        /**
         * 如果是使用公网接入点访问，configuration 还需要设置实例的用户名和密码。用户名和密码在控制台实例详情页获取。
         * 如果是在阿里云内网 VPC 中访问，无需填写该配置，服务端会根据内网 VPC 信息智能获取。
         */
        // configBuilder.setCredentialProvider(
        //  new StaticSessionCredentialsProvider("Instance UserName", "Instance Password")
        // );
        ClientConfiguration configuration = configBuilder.build();

        // 订阅消息的过滤规则。* 代表订阅全部消息。
        FilterExpression filterExpression = new FilterExpression(FILTER_EXPRESSION, FilterExpressionType.TAG);

        PushConsumer pushConsumer = provider.newPushConsumerBuilder()
                .setClientConfiguration(configuration)
                // 为消费者指定所属的消费者分组，Group 需要提前在控制台创建，如果不创建直接使用会返回报错。
                .setConsumerGroup(CONSUMER_GROUP_ID)
                // 指定需要订阅哪个目标 Topic，并设置预绑定的订阅关系。Topic 需要提前在控制台创建，如果不创建直接使用会返回报错。
                .setSubscriptionExpressions(Collections.singletonMap(TOPIC_NAME, filterExpression))
                .setMessageListener(messageView -> {
    
    
                    // 处理消息并返回消费结果。
                    System.out.println("Consume message=" + messageView.toString());
                    return ConsumeResult.SUCCESS;
                })
                .build();
        Thread.sleep(Long.MAX_VALUE);
        // 如果不需要再使用 PushConsumer，可关闭该进程。
        // pushConsumer.close();
    }
}

延时消息（定时消息）

package com.aliyun.openservices;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;
import org.apache.rocketmq.client.apis.message.MessageBuilder;

public class Demo {
    
    
    /**
     * 实例接入点，从控制台实例详情页的接入点页签中获取。
     * 如果是在阿里云内网 VPC 访问，建议填写 VPC 接入点。
     * 如果是在本地公网访问，或者是线下 IDC 环境访问，可以使用公网接入点。使用公网接入点访问，必须开启实例的公网访问功能。
     */
    public static final String ENDPOINT = "rmq-cn-pe33aqsu40k-vpc.cn-hangzhou.rmq.aliyuncs.com:8080";
    public static final String TOPIC_NAME = "rmqfctopic_delay";
    public static final String TAG = "";
    public static final String KEY = "";
    public static final String BODY = "";
    public static final String DELAY_SECONDS = "10";
    public static final String DELIVERY_TIMESTAMP = "";

    public static void main(String[] args) throws ClientException, IOException {
    
    
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder configBuilder = ClientConfiguration.newBuilder().setEndpoints(ENDPOINT);

        /**
         * 如果是使用公网接入点访问，configuration 还需要设置实例的用户名和密码。用户名和密码在控制台实例详情页获取。
         * 如果是在阿里云内网 VPC 中访问，无需填写该配置，服务端会根据内网 VPC 信息智能获取。
         */
        // configBuilder.setCredentialProvider(
        //  new StaticSessionCredentialsProvider("Instance UserName", "Instance Password")
        // );
        ClientConfiguration configuration = configBuilder.build();

        /**
         * 初始化 Producer 时直接配置需要使用的 Topic 列表，实现提前检查错误配置、拦截非法配置启动。
         * 针对非事务消息 Topic，也可以不配置，服务端会动态检查消息的 Topic 是否合法。
         * 注意！！！事务消息 Topic 必须提前配置，以免事务消息回查接口失败，具体原理请参见事务消息。
         */
        Producer producer = provider.newProducerBuilder()
                .setClientConfiguration(configuration)
                .setTopics(TOPIC_NAME)
                .build();

        MessageBuilder builder = provider.newMessageBuilder()
                // 为当前消息设置 Topic。
                .setTopic(TOPIC_NAME)
                // 消息体。
                .setBody(BODY.getBytes(StandardCharsets.UTF_8));

        if (!KEY.isEmpty()) {
    
    
            // 设置消息索引键，可根据关键字精确查找某条消息。
            builder.setKeys(KEY);
        }

        if (!TAG.isEmpty()) {
    
    
            // 设置消息 Tag，用于消费端根据指定 Tag 过滤消息。
            builder.setTag(TAG);
        }

        if (!DELIVERY_TIMESTAMP.isEmpty()) {
    
    
            // 设置您期望的投递消息的时间的 Unix 时间戳。
            builder.setDeliveryTimestamp(Long.parseLong(DELIVERY_TIMESTAMP));
        } else if (!DELAY_SECONDS.isEmpty()) {
    
    
            // 设置您期望的投递消息的时间的 Unix 时间戳。
            builder.setDeliveryTimestamp(
                    System.currentTimeMillis() + Duration.ofSeconds(Long.parseLong(DELAY_SECONDS)).toMillis());
        }

        // 配置消息的自定义属性
        // builder.addProperty("key", "value");

        try {
    
    
            // 发送消息，需要关注发送结果，并捕获失败等异常。
            final SendReceipt sendReceipt = producer.send(builder.build());
            System.out.println("Send mq message success! Topic is:" + TOPIC_NAME + " msgId is: "
                    + sendReceipt.getMessageId().toString());
        } catch (Throwable t) {
    
    
            System.out.println("Send mq message failed! Topic is:" + TOPIC_NAME);
            t.printStackTrace();
        }

        // 如果不需要再使用，可关闭该进程。
        producer.close();
    }
}

事务消息

package com.aliyun.openservices;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.apache.rocketmq.client.apis.producer.Transaction;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;
import org.apache.rocketmq.client.apis.message.MessageBuilder;
import org.apache.rocketmq.client.apis.producer.TransactionResolution;

public class Demo {
    
    
    /**
     * 实例接入点，从控制台实例详情页的接入点页签中获取。
     * 如果是在阿里云内网 VPC 访问，建议填写 VPC 接入点。
     * 如果是在本地公网访问，或者是线下 IDC 环境访问，可以使用公网接入点。使用公网接入点访问，必须开启实例的公网访问功能。
     */
    public static final String ENDPOINT = "rmq-cn-pe33aqsu40k-vpc.cn-hangzhou.rmq.aliyuncs.com:8080";
    public static final String TOPIC_NAME = "rmqfctopic_tx";
    public static final String TAG = "";
    public static final String KEY = "";
    public static final String BODY = "";

    public static void main(String[] args) throws ClientException, IOException {
    
    
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder configBuilder = ClientConfiguration.newBuilder().setEndpoints(ENDPOINT);

        /**
         * 如果是使用公网接入点访问，configuration 还需要设置实例的用户名和密码。用户名和密码在控制台实例详情页获取。
         * 如果是在阿里云内网 VPC 中访问，无需填写该配置，服务端会根据内网 VPC 信息智能获取。
         */
        // configBuilder.setCredentialProvider(
        //  new StaticSessionCredentialsProvider("Instance UserName", "Instance Password")
        // );
        ClientConfiguration configuration = configBuilder.build();

        /**
         * 初始化 Producer 时直接配置需要使用的 Topic 列表，实现提前检查错误配置、拦截非法配置启动。
         * 针对非事务消息 Topic，也可以不配置，服务端会动态检查消息的 Topic 是否合法。
         * 注意！！！事务消息 Topic 必须提前配置，以免事务消息回查接口失败，具体原理请参见事务消息。
         */
        Producer producer = provider.newProducerBuilder()
                .setClientConfiguration(configuration)
                .setTopics(TOPIC_NAME)
                /**
                 * 事务检查器，用于检查确认异常半事务的中间状态。事务检查器一般是根据业务的 ID 去检查本地事务是否正确提交还是回滚。
                 * 例如，如果在订单表找到了这个订单，说明本地事务插入订单的操作已经正确提交；如果订单表没有订单，说明本地事务已经回滚。
                 */
                .setTransactionChecker(messageView -> {
    
    
                    System.out.println("Receive transactional message check, message=" + messageView.toString());
                    return TransactionResolution.COMMIT;
                })
                .build();

        // 开启事务分支。
        final Transaction transaction = producer.beginTransaction();

        MessageBuilder builder = provider.newMessageBuilder()
                // 为当前消息设置 Topic。
                .setTopic(TOPIC_NAME)
                // 消息体。
                .setBody(BODY.getBytes(StandardCharsets.UTF_8));

        if (!KEY.isEmpty()) {
    
    
            // 设置消息索引键，可根据关键字精确查找某条消息。
            builder.setKeys(KEY);
        }

        if (!TAG.isEmpty()) {
    
    
            // 设置消息 Tag，用于消费端根据指定 Tag 过滤消息。
            builder.setTag(TAG);
        }

        // 一般事务消息都会设置一个本地事务关联的唯一 ID，用来做本地事务回查的校验。
        builder.addProperty("OrderId", "xxx");

        // 配置消息的自定义属性
        // builder.addProperty("key", "value");

        try {
    
    
            // 发送半事务消息
            final SendReceipt sendReceipt = producer.send(builder.build(), transaction);
            System.out.println("Send mq message success! Topic is:" + TOPIC_NAME + " msgId is: "
                    + sendReceipt.getMessageId().toString());
        } catch (Throwable t) {
    
    
            System.out.println("Send mq message failed! Topic is:" + TOPIC_NAME);
            t.printStackTrace();
        }

        /**
         * 执行本地事务，并确定本地事务结果。
         * 1. 如果本地事务提交成功，则提交消息事务。
         * 2. 如果本地事务提交失败，则回滚消息事务。
         * 3. 如果本地事务未知异常，则不处理，等待事务消息回查。
         *
         */
        transaction.commit();
        // transaction.rollback();

        // 如果不需要再使用，可关闭该进程。
        producer.close();
    }
}

顺序消息

package com.aliyun.openservices;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;
import org.apache.rocketmq.client.apis.message.MessageBuilder;

public class Demo {
    
    
    /**
     * 实例接入点，从控制台实例详情页的接入点页签中获取。
     * 如果是在阿里云内网 VPC 访问，建议填写 VPC 接入点。
     * 如果是在本地公网访问，或者是线下 IDC 环境访问，可以使用公网接入点。使用公网接入点访问，必须开启实例的公网访问功能。
     */
    public static final String ENDPOINT = "rmq-cn-pe33aqsu40k-vpc.cn-hangzhou.rmq.aliyuncs.com:8080";
    public static final String TOPIC_NAME = "rmqfctopic_fifo";
    public static final String TAG = "";
    public static final String KEY = "";
    public static final String BODY = "";
    public static final String MESSAGE_GROUP = "";

    public static void main(String[] args) throws ClientException, IOException {
    
    
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder configBuilder = ClientConfiguration.newBuilder().setEndpoints(ENDPOINT);
        /**
         * 如果是使用公网接入点访问，configuration 还需要设置实例的用户名和密码。用户名和密码在控制台实例详情页获取。
         * 如果是在阿里云内网 VPC 中访问，无需填写该配置，服务端会根据内网 VPC 信息智能获取。
         */
        // configBuilder.setCredentialProvider(
        //  new StaticSessionCredentialsProvider("Instance UserName", "Instance Password")
        // );
        ClientConfiguration configuration = configBuilder.build();
        /**
         * 初始化 Producer 时直接配置需要使用的 Topic 列表，实现提前检查错误配置、拦截非法配置启动。
         * 针对非事务消息 Topic，也可以不配置，服务端会动态检查消息的 Topic 是否合法。
         * 注意！！！事务消息 Topic 必须提前配置，以免事务消息回查接口失败，具体原理请参见事务消息。
         */
        Producer producer = provider.newProducerBuilder()
                .setClientConfiguration(configuration)
                .setTopics(TOPIC_NAME)
                .build();

        MessageBuilder builder = provider.newMessageBuilder()
                // 为当前消息设置 Topic。
                .setTopic(TOPIC_NAME)
                // 消息体。
                .setBody(BODY.getBytes(StandardCharsets.UTF_8));

        if (!KEY.isEmpty()) {
    
    
            // 设置消息索引键，可根据关键字精确查找某条消息。
            builder.setKeys(KEY);
        }

        if (!TAG.isEmpty()) {
    
    
            // 设置消息 Tag，用于消费端根据指定 Tag 过滤消息。
            builder.setTag(TAG);
        }

        if (!MESSAGE_GROUP.isEmpty()) {
    
    
            // 顺序消息的顺序关系通过消息组来判定和识别，发送顺序消息时需要为每条消息设置归属的消息组。
            builder.setMessageGroup(MESSAGE_GROUP);
        }

        // 配置消息的自定义属性
        // builder.addProperty("key", "value");

        try {
    
    
            // 发送消息，需要关注发送结果，并捕获失败等异常。
            final SendReceipt sendReceipt = producer.send(builder.build());
            System.out.println("Send mq message success! Topic is:" + TOPIC_NAME + " msgId is: "
                    + sendReceipt.getMessageId().toString());
        } catch (Throwable t) {
    
    
            System.out.println("Send mq message failed! Topic is:" + TOPIC_NAME);
            t.printStackTrace();
        }

        // 如果不需要再使用，可关闭该进程。
        producer.close();
    }
}

SDK相关

The above examples in this article are provided based on the RockMQ cloud message service.
The compatibility between the server-side version and the client-side SDK version is as follows. In order to obtain more complete product functions and higher performance, it is recommended that you use the 5.x version instance, and access the server through the RocketMQ 5.x series SDK first.
https://help.aliyun.com/document_detail/441918.html
insert image description here

Guess you like

Origin blog.csdn.net/wangshuai6707/article/details/131633086
Recommended
Arc Browser for Windows 1.0 officially GA
A programmer born in the 1990s developed a video porting software and made over 7 million in less than a year. The ending was very punishing!
Ranking
1. Select Sort
Create a thread thread
3 press to play ball that reach 6
Programmation CUDA (4) : gestion de la mémoire
SpringBoot database connection pool Druid error
E Diudiu App redesign summary
4EVERLAND Hosting now supports SNS+IPFS
About HTTPS
[vue3+vite+ts+element-plu+sass] uses bug records in sass
Interpretation of HUAWEI CLOUD GaussDB (for Influx): Best Practice Data Modeling
Daily
More
2024-05-03(8)
2024-05-02(0)
2024-05-01(4)
2024-04-30(36)
2024-04-29(5)
2024-04-28(12)
2024-04-27(29)
2024-04-26(22)
2024-04-25(32)
2024-04-24(30)

Code World

© 2018 codetd.com