Welcome to the cloud + community to get more Tencent's massive technical practice dry goods~
Author: Li Xiaohui
Recently, Tencent Cloud launched the cloud time series database CTSDB (Cloud Time Series Database). CTSDB is a distributed, scalable and highly reliable time series database, which is suitable for the Internet of Things, big data analysis and Internet monitoring with massive time series data. Scenes. CTSDB has now opened public beta applications in Guangzhou, Beijing and Shanghai ( click for details ), and it is planned to be officially launched in late March 2018.
The time series database CTSDB can provide high-performance read and write, low-cost storage, aggregation analysis, instance monitoring, and visualization of data query results. The whole system adopts the deployment method of multi-node and multi-copy, which effectively ensures the high availability and security of data. CTSDB has been successfully applied to more than 20 core businesses within Tencent (WeChat Lottery, Tenpay, Tencent Cloud Monitoring, Tencent Cloud Load Balancing, etc.). Among them, Tencent Cloud monitors the writing pressure of millions of data points per second and runs stably in the writing scenario of 20TB+ data volume per day, which is enough to prove that it can stably support various massive data scenarios. CTSDB is committed to helping enterprise users process massive data more easily and quickly, and efficiently improve business feedback.
What exactly is time series data? What is the significance of the time series database for the processing of time series data?
Time series data refers to time series data, which is a data column recorded by the same indicator in chronological order. For example: the time series data composed of the temperature data per minute of a certain machine on the factory production line, the time series data composed of the memory usage data per second of a module in the Internet log system, etc. Therefore, time series data has the main characteristics of timeliness, trend and mass.
With the rapid development of technologies such as IoT and big data analysis, traditional solutions for time series data based on relational databases or the Hadoop ecosystem have gradually encountered problems such as high storage costs and poor write and query performance. The time series database CTSDB can provide high-performance reading and writing, low-cost storage, powerful aggregation analysis capabilities, instance monitoring, and visualization of data query results, making it a powerful tool for users to process time series data.
Next, the editor will take you to learn more about the advantages and functions of Tencent Cloud Time Series Database CTSDB.
Distributed deployment, high read and write performance
Each instance is a distributed cluster composed of multiple nodes. The data in the cluster is managed according to time, and finally divided into multiple physical shards, each of which has multiple copies. This architecture not only improves the system throughput, but also fully guarantees the high availability of the system when a single node fails. In addition, the system adopts the strategy of writing data to memory first, and then periodically flushing it to disk, which greatly improves the response speed. Compared with the open source time series database InfluxDB, the performance is improved several times. During the internal testing phase, the write capacity provided by each instance can reach up to 50,000 points per second. After the official launch, instances with higher performance and more flexible configuration will be provided for users to choose from.
Powerful aggregation analysis capabilities
Supports rich aggregation query methods, not only supports common aggregation methods such as avg, min, max, count, etc., but also supports complex aggregation analysis such as Group By, interval, Geo, and nesting. The API provides nearly 47 aggregation methods.
Reasonable compression algorithm, high data compression ratio
The cluster adopts columnar storage, and on the basis of columnar storage, a flexible and reasonable coding and compression algorithm is used to improve the storage compression ratio. The cluster also provides the Rollup function, which aggregates raw data according to user-defined aggregation methods and saves coarse-grained aggregation results. Supports defining the validity period of data, regularly cleaning expired data, and reducing storage costs. Relative to the original data, the compression ratio can reach up to 90%.
Strict permission verification to ensure data security
CTSDB refers to the design of the MySQL permission system, and develops a fully functional user and permission management system, which supports user management (including viewing users, creating users, deleting users, and changing passwords) and permission management (including authorization, revocation) and other operations.
Strong monitoring and operation ability, simple and easy to use
The system provides nearly 49 monitoring indicators, and displays the indicators that users care about most in the console. At the same time, it provides a complete alarm system to monitor the health status of the cluster at any time. The console interface is simple and easy to operate. It can not only manage instances, but also query and analyze data. In addition to the console and API for querying data, CTSDB can also interface with some visual interfaces to facilitate analysis and display of user data. In addition, CTSDB has professional operation and management personnel to escort the user's business at any time ( click for details ).
Related Reading
Tencent's only time series database: CTSDB decryption
Overview of Time Series Databases
Hundreds of billions of server monitoring data storage practices
This article has been authorized by the author to publish in the Yunjia community, please indicate the source of the original text when reprinting