"Intelligent change is accelerating and industries are renewed." The 2023 Tencent Global Digital Ecosystem Conference came to a successful conclusion on September 7-8. The 40+ special events showcased Tencent's latest cutting-edge technologies, core products, and solutions.
Tencent Cloud message queue RocketMQ 5.x series of commercial products were launched at this conference as a new member of the TDMQ product family.
TDMQ RocketMQ version is a distributed high-availability message queue service that is compatible with various components and concepts of Apache RocketMQ, supports zero-modification access to open source clients, and has the underlying advantages of separation of computing and storage, and flexible expansion and contraction. Due to its excellent single-machine throughput, it is very suitable for large-scale online business scenarios with high reliability requirements.
The latest 5.x new architecture released by the RocketMQ community also brings us new surprises in terms of functionality and flexibility, especially the enhancement of cloud-native serverless capabilities, which enables RocketMQ to better adapt to business peak changes in real time. It can be said that for large-scale online messaging services, the first product we at Tencent Cloud recommend is RocketMQ.
As a new generation of commercial products, compared with the RocketMQ 4.x series, what are the advantages of the RocketMQ 5.x series?
RocketMQ 5.x series capability upgrade
Underlying technology advantages
- Cloud native architecture with separation of storage and computing
RocketMQ 5.x adopts an architecture that separates storage and computing. The Broker, which originally had to focus on both storage and computing, can finally hand over the computing function to the Proxy.
- Lightweight access
The new technical architecture has moved the original client load balancing mechanism to Proxy, so its client is more lightweight, which also creates conditions for the community to provide SDKs in more languages. At the same time, RocketMQ 5.x has added a more flexible POP consumption mode to facilitate developers to flexibly customize message pulling according to their own business needs.
- Reduce costs and increase efficiency
New technological advancements also help users better reduce resource costs. Taking storage elasticity as an example, RocketMQ 5.x provides users with the implementation of tiered storage, which can help users achieve separate storage of hot and cold data, further reducing storage costs. Resource costs and benefits to the majority of users.
- Standardized Ecology
RocketMQ 5.x provides integration with the standardized ecosystem on the cloud, such as providing more comprehensive performance monitoring indicators, and supporting the OpenTelemetry open source standard to provide richer trajectory indicators.
Function enhancement
Based on the excellent underlying technical architecture of RocketMQ 5.x , what functional enhancements has Tencent Cloud made ?
- In order to provide customers with more comprehensive problem location and troubleshooting capabilities, Tencent Cloud has further packaged it based on standardized indicators implemented on open source technology, providing richer indicators than open source, such as multiple Groups and consumer IDs under different topics. Drill-down of hierarchical indicators, and all indicators are connected to Tencent Cloud monitoring and alarm capabilities. It also provides some basic consumption problem troubleshooting and stack viewing capabilities.
- In terms of cloud support capabilities, Tencent Cloud provides customers with multiple trump card functions , such as cluster-free migration capabilities and cross-regional replication of messages. At the same time, all cloud APIs of Tencent Cloud TDMQ RocketMQ 5.x series have been connected to Terraform, ensuring that customers can switch freely between different cloud platforms.
- In terms of message life cycle management, Tencent Cloud supports setting message retention time according to different topics to meet customers' flexible business requirements. In the previous version , if the customer wanted to adjust the retention time of messages for a certain business, the entire cluster dimension needed to be adjusted, which resulted in a waste of resources. Tencent Cloud's ability can help customers reduce costs and adapt more flexibly. The retention time requirements of different services.
- In terms of security and performance optimization, in addition to elastic TPS and common cross-availability zone deployment, Tencent Cloud also supports the adjustment of the ratio of message sending and receiving within the cluster, and adjusts the ratio of sending and receiving message traffic according to business changes. In addition, Tencent Cloud has also optimized distributed current limiting capabilities this time, with multiple downgrade strategies and priority current limiting to protect customers' cluster security and stability.
Serverless enables ultimate flexibility and significant cost reduction
Perhaps the most popular feature at this conference is this! The serverless product form brings ultimate flexibility to RocketMQ 5.x.
In terms of computing, the RocketMQ 5.x series has launched the elastic TPS capability, as shown in the figure. After users purchase specific TPS specifications, Tencent Cloud allows customers to partially exceed the specification limits under specific conditions and avoid frequent The cumbersome operation of lifting and lowering the configuration.
To give a specific example, suppose you purchased a 6000 TPS cluster, but due to the boom in business during the big promotion period, the volume of messages increased dramatically. At this time, users do not need to worry about peak traffic being ruthlessly limited, nor do they need to rush to expand the capacity. , just calmly turn on the elastic TPS switch, and then the upper limit of the entire cluster will become 10,000 TPS.
In terms of storage elasticity, Tencent Cloud recently supports pay-as-you-go storage, and charges are based on the actual storage size occupied every hour. At the same time, there is no upper limit on storage, so theoretically, the storage is unlimited, and there is no need to always pay attention to the issue of upgrading or downgrading storage. Users do not need to consider the size of the storage when purchasing a cluster. In many scenarios, customers often do not increase the capacity immediately after purchasing an exclusive cluster. If they continue to charge according to the capacity limit during the period, the cost is not small. burden.
Okay, after introducing so much, you may want to ask, such a good product must be expensive, right? The fact is that not only is it not expensive, we have also broken down the specifications for different scenarios and customers.
RocketMQ 5.x series commercial product form
Four product forms
According to the computing specification TPS ladder, the RocketMQ 5.x series is divided into four specifications: experience version, basic version, professional version and platinum version. Each sub-category provides specific TPS specifications.
Trial version : The trial version is a separate specification we set up to lower the experience threshold. The upper limit of TPS is 500. However, through the trial version, you can experience almost all the new capabilities of RocketMQ 5.x at a very low cost. It is suitable for individual users or Enterprise customers in beta phase. The functionality is the same as other production-ready specs, but at a much lower cost. More importantly, the trial version supports the seamless and smooth upgrade of clusters to high-specification clusters, allowing users to truly "use it well before buying it", and is also in line with the product "first test in small steps and then use it on a large scale" Landing rules.
Basic version : The basic version is the smallest specification we recommend that can be used in a production environment. The underlying resources are also shared. The TPS specification supports a relatively small span, supporting 1000 to 6000 TPS, but the basic version is more cost-effective. , compared with customers’ self-built clusters, it has a great cost advantage and is suitable for medium-sized scenarios of small and medium-sized enterprises.
Professional Edition : In addition to underlying resource sharing, the Professional Edition is similar to the Platinum Edition in terms of user experience. The TPS specifications it supports are also relatively high, up to 150,000 TPS. It is mainly suitable for enterprise-level customers with large business scale but no special requirements for physical resource isolation.
Platinum version : The platinum version is the only one among the four specifications that is exclusive to the underlying resources. It supports the largest range of TPS specifications and can support up to one million TPS. Because of the isolation of underlying resources, its price is also the most expensive among the four specifications. It is suitable for large enterprise customers and large-scale business scenarios that require physical isolation of resources.
Dismantling of billing items
When we spend money, we have to spend it clearly. Let’s break down where the money is spent.
The four product forms of RocketMQ 5.x all support the independent opening and closing of secure public network connections. The public network bandwidth supports hourly and annual and monthly billing, and is charged after the public network is turned on.
In addition to the public network, the most basic charges are computing and storage. Storage billing items are charged the same for all specifications, and are charged by the hour, depending on the usage.
The calculated fee consists of two parts: the regular billing specification fee and the elastic TPS fee. The conventional calculation specification fee is the basic fee when purchasing different TPS specifications. It supports hourly and annual and monthly purchases.
Currently, only Professional and Platinum edition clusters support the elastic TPS switch. After the switch is turned on, you will be charged on an hourly basis based on the actual amount of elastic TPS used. In other words, even if you turn on the elastic switch, but it does not exceed the normal specifications and does not trigger the elastic TPS outside the specifications, there will still be no charge.
The future is promising
As the functions of RocketMQ 5. Scenarios, the ability to lightweight queues also has huge market and imagination in IoT scenarios.
At the same time, RocketMQ 5.x's Connect and EventBridge event integration capabilities also provide the Tencent Cloud Message Queuing team with a technical foundation to help customers achieve cloud data integration and ecological connectivity.
In addition to further developing the existing capabilities of the community, the Tencent Cloud Message Queue team is also combining the actual needs of cloud customers and learning the advantages of other message queue products to feed back the community and expand more functions for the community, such as message priority. Improve the management of producers and so on.
In terms of privatized output, the Tencent Cloud Message Queuing team is also exploring, such as the Broker Container capabilities and controller capabilities proposed by the community. When privatized output, while ensuring stability, customers can better enjoy open source technology. dividend.
This release is just a starting point. Tencent Cloud’s message queue team is also actively participating in product innovation and community building. More complete, more stable, and more cost-effective product capabilities are the goals that Tencent Cloud has always pursued. I believe that with the help of our customers and With the joint efforts of ecological partners, open source communities, and all colleagues at Tencent Cloud, the Tencent Cloud message queue team can polish the message queue product even better in the future.