Introduction
The message queue RocketMQ version (TDMQ for RocketMQ, referred to as TDMQ RocketMQ version) is officially beta today! TDMQ RocketMQ is a distributed high-availability message queue service in the TDMQ series products. It is compatible with various components and concepts of Apache RocketMQ. The clients of RocketMQ 4.6 and above have almost zero modification and access. Welcome to scan the QR code at the end of the article to experience it!
Background of TDMQ RocketMQ Edition
As a typical business processing message queue, RocketMQ is mainly used to process orders, payments, points and other business types. It has extremely high requirements for the accuracy of information exchange. Its core architecture: producer side, server side (NameServer and Broker services) instance), the consumer side satisfies this very well.
TDMQ RocketMQ version (Tencent Distributed Message Queue for RocketMQ) is a distributed message queuing product created by Tencent, featuring strong consistency, high reliability, and high performance. Compatible with clients of open source RocketMQ 4.6 and above, it supports native capabilities such as sequential messages, transaction messages, cluster/broadcast consumption modes, and has the underlying architecture of separation of computing and storage, flexible expansion and contraction, and is widely used in various e-commerce, finance, Games, government affairs and other industries.
TDMQ RocketMQ version product advantages
TDMQ RocketMQ version can better cope with the traffic impact brought by various marketing activities, it is very suitable for scenarios with high sequence and transaction requirements, and has a very wide range of applications in e-commerce transactions, financial settlement and other fields.
- Compatible with open source
It is compatible with various components and concepts of Apache RocketMQ, supports zero-modification access of clients of RocketMQ 4.6 and above, and has the underlying advantages of separation of computing and storage, and flexible expansion and contraction.
- Resource isolation
In order to better adapt to the scenarios used on the cloud, the TDMQ RocketMQ version has a multi-level resource structure, not only virtual isolation based on namespaces, but also physical isolation in the cluster dimension. It supports configuring permission verification for clients in the namespace dimension, distinguishing clients in different environments, which is convenient and flexible.
- Sharded storage
Compared with the open source version, TDMQ RocketMQ uses sharding to persist message data, which is not easy to cause problems such as data skew. When nodes are added or deleted due to capacity expansion, machine failure, etc., rebalancing will not be triggered, resulting in a sharp drop in the throughput of the entire cluster.
- Rich message types
It supports various message types such as ordinary messages, sequential messages, delayed messages, and distributed transaction messages, and supports message retry and dead letter mechanisms to meet various business scenarios.
- high performance
A single machine can support up to tens of thousands of production and consumption throughput, distributed architecture, stateless services, and horizontal expansion to enhance the throughput of the entire cluster.
- Easy to use and free of operation and maintenance
Provides API access interface and supports open source SDKs in all languages and versions. Provide a complete set of operation and maintenance services of Tencent Cloud Platform, monitor alarms in real time, help users quickly find and solve problems, and ensure service availability.
Main capabilities of TDMQ RocketMQ version
normal message
A common message is a basic message type. After it is delivered to a specified topic by production, it is consumed by consumers subscribed to the topic. The non-sequential concept in the topic of ordinary messages can use multiple partitions to improve the efficiency of message production and consumption, and its performance is best when the throughput is huge.
sequential message
Local sequential message : Compared with ordinary message type, local sequential message has one more local sequential feature. That is, under the same partition, when consumers consume messages, they consume messages in strict accordance with the order in which the producers deliver to the partition. Local sequential messages ensure a certain order while retaining the partitioning mechanism to improve performance. But locally ordered messages do not guarantee order between different partitions.
Global sequential messages : The biggest feature of global sequential messages is that messages are strictly guaranteed to be consumed in the order delivered by producers. Therefore, it uses a single partition to process messages, and the user cannot customize the number of partitions. Compared with the first two message types, this type of message has lower performance.
dead letter message
Dead letter messages are messages that cannot be consumed normally. TDMQ will automatically create a dead letter queue for processing this kind of message when a new subscription is created (the consumer determines the subscription relationship with a topic).
retry queue
A retry queue is a queue designed to ensure that messages are consumed normally. When some messages are consumed by consumers for the first time and do not get a normal response, they will enter the retry queue. When the retry reaches a certain number of times, the retry will be stopped and delivered to the dead letter queue.
Due to some temporary and short-term problems (such as network jitter, service restart, etc.) that may exist in the actual scenario, the message cannot be processed in time, but it returns to normal after a short period of time. In this scenario, the retry mechanism of the retry queue can solve such problems very well.
dead letter queue
A dead letter queue is a special kind of message queue that is used to centrally process messages that cannot be consumed normally. When the message cannot be consumed normally after a certain number of retries in the retry queue, TDMQ will determine that the message cannot be consumed under the current situation, and will deliver it to the dead letter queue.
In actual scenarios, messages may not be consumed due to service downtime and network disconnection for a period of time. In this scenario, the message will not be discarded immediately, and the dead letter queue will persist the message for a long time. After finding the corresponding solution, the user can create a consumer to subscribe to the dead letter queue to complete the message that cannot be processed at that time. processing.
Delayed message
TDMQ RocketMQ version supports delay messages with a customized duration within a certain range. You only need to add specific delay parameters to the message properties of each message. Even if you use the open source version of RocketMQ client, you can easily Expand the use of this ability.
Cluster consumption
Cluster consumption is suitable for scenarios where each message only needs to be processed once.
broadcast consumption
Broadcast consumption is suitable for scenarios where each message needs to be processed by each consumer in the cluster.
Application scenarios
Asynchronous decoupling
Each transaction order data needs to be concerned by dozens of downstream business systems, including item pricing, delivery, integration, stream computing analysis, etc. The processing logic of multiple systems is inconsistent, and it is impossible for a single system to adapt to each association business. At this time, the TDMQ RocketMQ version can realize efficient asynchronous communication and application decoupling to ensure the continuity of the master station business.
shaving peaks and filling valleys
Some marketing activities held by enterprises from time to time, new product launches, red envelopes grabbing at festivals, etc., often bring temporary traffic peaks, which is a huge test for various back-end application systems. TDMQ RocketMQ version can assume the role of a buffer at this time, collect the sudden increase of upstream requests centrally, and downstream can consume request messages according to their actual processing capabilities.
Sequentially send and receive
The application of sequential messages appears in business scenarios. For example, the order creation, payment, refund and other processes in the process of purchasing and distributing goods in APP or games are executed in strict order. Similar to the principle of First In First Out (FIFO), TDMQ RocketMQ version provides a A sequential message function designed specifically for this situation, i.e. guaranteed message FIFO.
product experience entry
Tencent Cloud Message Queue TDMQ RocketMQ version has been open for public beta, and you are welcome to visit the official website to experience it.
Users who are in the public beta can move to the "Tencent Cloud Middleware" WeChat public account TDMQ RocketMQ public beta tweet to scan the code to enter the official user group of TDMQ RocketMQ. There are professional technicians from the Tencent Cloud message queue team in the group. Relevant questions can be directly responded in the group, and there will be professionals to answer for you.