Introduction
On December 1, 2021, Tencent Cloud distributed message queue TDMQ Pulsar version was officially commercialized.
Financial-grade distributed messaging middleware
The message queue TDMQ Pulsar is a financial-level distributed message middleware based on Apache Pulsar self-developed. It has the characteristics of high consistency, high reliability and high concurrency. It can provide asynchronous decoupling and peak shaving and valley filling capabilities for distributed application systems. At the same time, it also has the characteristics of massive message accumulation, high throughput, and reliable retry required by Internet applications. It is a message queue that has undergone the test of hundreds of billions of transactions for three years. It is also a cloud message queue that truly separates computing and storage. The architecture achieves cloud-native statelessness and caters to the trend of serverless.
Product advantages
TDMQ Pulsar version is compatible with many native advantages of Pulsar, and also adds some functions required for actual business scenarios, such as support for retry & dead letter messages, message traces, etc. TDMQ Pulsar Edition has the following product advantages:
-
Strong data consistency
The TDMQ Pulsar version uses the Quorum mechanism to achieve strong data consistency, writes the message data backup to different physical machines, and refreshes the disk synchronously. When a physical machine fails, the background data replication mechanism can quickly migrate data to ensure the availability of user data backup.
-
High performance and low latency
TDMQ Pulsar version can efficiently support the production and consumption of millions of messages and the accumulation of massive messages, supporting all scenarios of Tencent billing; in terms of performance, a single node can carry more than 100,000 QPS, and there is a protection mechanism in terms of time consumption to ensure low latency.
-
High scalability
TDMQ Pulsar version inherits the design of the separation of open source Pulsar computing and storage architecture. Broker nodes are stateless and can flexibly scale horizontally.
-
Rich message types
TDMQ Pulsar version provides rich message types, covering common messages, sequential messages (global order/partition order), timed messages, retry and dead letter messages, etc., to meet the advanced feature requirements in various severe scenarios.
-
Support multilingual access
The API of TDMQ Pulsar version supports Java, C++, Go and other languages, and supports HTTP protocol, which can expand access to more languages.
-
isolation control
TDMQ Pulsar version can isolate topics by namespace and set related flow control policies to ensure that different business topics do not affect each other, and message processing will not cause resource competition.
-
Flexible configuration permissions
TDMQ provides free and flexible permission configuration capabilities. It can set different namespace production or consumption permissions for roles, and achieve data isolation from the security mechanism level without affecting each other.
When users use the message queue TDMQ Pulsar version, there is no need to worry about the compatibility with the open source Apache Pulsar community. TDMQ Pulsar Edition now supports Pulsar Community Edition Go, Java, C++, Python, Node.js and other multi-language SDKs. Customers who use the Apache Pulsar open source client can quickly access TDMQ Pulsar Edition, easily access the cloud, and enjoy the services provided by Tencent Cloud. 7*24 hours of professional message queue service support, enjoy high availability message queue service, no need to pay attention to the details of operation and maintenance.
Application scenarios
-
Asynchronous decoupling
As the core system of Tencent's billing, the transaction engine requires the attention of dozens of downstream business systems for each transaction order data, including item pricing, item delivery, points, stream computing analysis, etc., and the processing logic of multiple systems for messages Inconsistent, it is impossible for a single system to adapt to every associated service. The message queue TDMQ Pulsar version can realize efficient asynchronous communication and application decoupling, and ensure the continuity of the main station business.
-
shaving peaks and filling valleys
Some marketing activities held by enterprises from time to time, such as new product launches, red envelope grabbing on festivals, etc., often bring temporary traffic peaks, which is a huge test for various back-end application systems. Bring a certain waste of resources. At this time, the TDMQ Pulsar version can assume the role of a buffer, collect upstream sudden requests, and downstream can consume request messages according to their actual processing capabilities.
-
Sequentially send and receive
The application of sequential messages appears in business scenarios. For example, the order creation, payment, refund and other processes in the transaction process are executed in strict order. Similar to the principle of First In First Out (FIFO), TDMQ Pulsar version provides a special order to deal with this situation. Message function, i.e. guaranteed message FIFO.
end
The commercialization of TDMQ Pulsar version provides developers with a unified messaging platform, further improves operational efficiency, reduces hardware and software costs, and has been widely used in finance, government, pan-Internet, medical and other business fields . At present, it has been applied in most scenarios of Tencent's billing, including the main payment path, real-time reconciliation, real-time monitoring, and real-time analysis of big data. As of the end of November 2021, the message queue TDMQ Pulsar version has hundreds of active enterprise users. The message queue TDMQ Pulsar version has greatly promoted the digital transformation of enterprises with its advanced design concept, efficient and stable service performance, sufficient operation and maintenance support, and perfect after-sales service, and continued to provide high-quality and efficient services for enterprises.