introduce
RocketMQ is an open source distributed messaging middleware developed by Alibaba. It was developed in 2012. It has the characteristics of high availability, high performance, strong reliability, and rich functions, and is widely used in e-commerce, finance, logistics, Internet, big data and other fields. RocketMQ supports load balancing, data sharding, data retry, data expiration and other features for producers and consumers. It also supports advanced features such as message transactions, sequential consumption, scheduled consumption, and broadcast consumption. RocketMQ's message storage adopts a CommitLog-based method to ensure reliable storage and efficient reading of messages. The overall architecture of RocketMQ is divided into three components: Producer, Broker and Consumer. Producer is responsible for message production, Broker is responsible for message storage and forwarding, and Consumer is responsible for message consumption.
main feature
可靠性: RocketMQ uses persistence mechanism and release confirmation mechanism to ensure the reliability of messages. When a message fails to be sent, it can be retried automatically or manually to ensure that the message will not be lost.
高性能: RocketMQ adopts an efficient communication protocol and excellent algorithms, making it high-performance. When producers send messages to Brokers, RocketMQ uses batch sending and compression mechanisms to improve message transmission efficiency.
分布式: RocketMQ naturally supports distribution. Producer, Broker and Consumer can all be deployed in a distributed manner, making it easy to cope with large-scale concurrent access.
多种消息模型: RocketMQ supports queue model and publish-subscribe model, as well as broadcast and point-to-point messaging modes.
灵活的路由规则: RocketMQ supports flexible routing rules, which can route messages to specified queues based on message attributes or tags.
多种消费模式: RocketMQ supports multiple consumption modes, including pull mode, push mode and automatic submission mode.
事务消息: RocketMQ supports transaction messages to ensure the reliability and consistency of messages.
丰富的API: RocketMQ provides a rich API to facilitate developers to carry out customized development.
可扩展性强: Each component of RocketMQ can be expanded horizontally and has strong scalability.
社区活跃: RocketMQ has an active community, with a large number of open source projects and developers using and contributing code.
scenes to be used
The usage scenarios of RocketMQ include but are not limited to the following:
Cutting peaks and filling valleys: in situations such as flash sales, grabbing red envelopes, good business starts, etc. During large-scale events, system traffic pulses are high. If no corresponding protection is provided, the system may be overloaded or even crash. Using RocketMQ's high-performance message processing capabilities, peak-shaving and valley-filling services can be provided to solve this problem.
Asynchronous decoupling: The loose coupling design of the upstream and downstream business systems is completed through RocketMQ. Even if some service nodes are abnormal, it will not affect the normality of the core trading system. operation, and at the same time, exception retry processing can be implemented through RocketMQ's retry queue.
Sequential messages: The sequential messages (partition order) provided by RocketMQ can ensure the first-in-first-out of messages and are applied to order creation, payment, and Refund and other processes.
Distributed transaction messages: Introducing RocketMQ’s distributed transactions in distributed business scenarios can achieve decoupling between systems while ensuring the final data consistency.
Component installation steps
The detailed steps for RocketMQ installation are as follows:
- Download RocketMQ. Download from the apache file library. Here we choose the latest version. After downloading, unzip it. The path is /home/rocketmq-4.9.3.
- Modify startup configuration. Find the configuration file in the /home/rocketmq-4.9.3/bin directory and modify the memory configuration in runserver.sh and runbroker.sh respectively. Modify the original memory configuration to JAVA_OPT="${JAVA_OPT}-server -Xms256m -Xmx256m -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m".
- Start RocketMQ. First configure the system environment variables, set ROCKETMQ_HOME to /home/rocketmq-4.9.3, and PATH include /home/rocketmq-4.9.3/bin. Then start NameServer and Broker.
- Test RocketMQ. Turn off the firewall and test.
Main advantages and disadvantages
RocketMQ is a widely used messaging middleware that has the following advantages:
- Flexible scalability: RocketMQ naturally supports clusters, and each of its four core components (NameServer, Broker, Producer, Consumer) can operate without a single point of failure. horizontal expansion.
- Has the ability to accumulate massive messages: RocketMQ uses the zero-copy principle to achieve the ability to accumulate extremely large amounts of messages. It is said that a single machine has supported the accumulation of billions of messages, and it has accumulated so many messages. Keep write latency low.
- Support sequential messages: RocketMq can keep message consumers to consume messages in the order in which they are sent. Sequential messages are divided into global ordered messages and locally ordered messages. It is generally recommended to use locally ordered messages, that is, the producer sends a certain type of messages to the same queue in order.
- Supports multiple message filtering methods: Message filtering is divided into server-side filtering and consumer-side filtering. When filtering on the server side, filtering can be performed according to the requirements of message consumers. The advantage is that unnecessary message transmission is reduced. The disadvantage is that it increases the burden on the message server and is relatively complicated to implement. Consumer-side filtering is completely customized by specific applications, which is more flexible.
However, RocketMQ also has some disadvantages:
- Relatively little community support: Compared with some other popular messaging middleware, RocketMQ has relatively little community support, which may affect users when they encounter problems. Convenience of getting help.
- Incomplete documentation and tutorials: Although RocketMQ’s official documentation has been improved, there are still incomplete documentation and tutorials for some complex functions and configurations. This may cause certain learning difficulties for users.
- There are compatibility issues when integrating with some other technologies: In some specific scenarios, RocketMQ may have compatibility issues when integrating with some other technologies, which may limit its use certain applications.
RocketMQ has some significant advantages, but also some disadvantages. When choosing to use it, you need to comprehensively consider it based on actual needs and scenarios.
The difference between RocktMq and RabbitMq
RocketMQ and RabbitMQ are two widely used message queue (MQ) technologies, they have some differences in the following aspects:
- Development language and ecology: RabbitMQ is developed based on Erlang, uses the AMQP protocol, and supports multiple programming languages, such as Java, Python, Ruby, etc.; while RocketMQ is developed based on the Java language and integrates a large number of Java ecosystem tools.
- Message model: RabbitMQ uses the ack mechanism to confirm the completion of consumption, while RocketMQ uses the pull mode, where consumers actively pull messages.
- High availability: RabbitMQ has high availability and reliability, and supports multiple high-availability deployment methods such as master-slave mode and mirror queue. RocketMQ also supports master-slave mode and multi-copy synchronous replication, but in some cases message duplication or loss may occur.
- Performance: RocketMQ has excellent performance when processing a large number of messages, and can reach hundreds of thousands of data volumes. RabbitMQ is superior in terms of community activity and visual interface.
RocketMQ and RabbitMQ each have their own advantages. Which technology to choose depends on the specific usage scenarios and needs.
Java code example
The following is a simple Java code example that demonstrates how to send and receive messages using RocketMQ:
Send a message:
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.common.message.Message;
public class Producer {
public static void main(String[] args) throws Exception {
// 创建消息生产者，并指定生产者组名
DefaultMQProducer producer = new DefaultMQProducer("producer-group");
// 指定NameServer的地址
producer.setNamesrvAddr("localhost:9876");
// 启动生产者
producer.start();
for (int i = 0; i < 100; i++) {
// 创建消息，并指定Topic、Tag和消息体
Message msg = new Message("TopicTest", "TagA", ("Hello RocketMQ " + i).getBytes());
// 发送消息到指定的队列
producer.send(msg, "queueA");
}
// 关闭生产者
producer.shutdown();
}
}
Receive message:
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.MessageExt;
import java.util.List;
public class Consumer {
public static void main(String[] args) throws MQClientException, InterruptedException {
// 创建消费者，并指定消费者组名
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumer-group");
// 指定NameServer的地址
consumer.setNamesrvAddr("localhost:9876");
// 订阅一个或多个Topic，以及Tag来过滤需要消费的消息
consumer.subscribe("TopicTest", "*");
// 注册回调实现类来处理从broker拉取回来的消息
consumer.registerMessageListener((List<MessageExt> msgs, ConsumeConcurrentlyContext context) -> {
System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
// 标记该消息已经被成功消费
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
});
// 启动消费者实例，开始消费消息
consumer.start();
System.out.printf("Consumer Started.%n");
}
}