On July 4-5, 2023, the "2023 Trusted Database Development Conference" hosted by China Communications Standards Association and China Academy of Information and Communications Technology, organized by the Big Data Technology Standard Promotion Committee, and co-hosted by InfoQ was held at the Beijing International Convention Center.
With the theme of "Independent Innovation Leading", the conference invited nearly a hundred database academic experts and senior technical experts to discuss new practices, new achievements, and new trends in database development, and deeply explore the future direction of database technology. In the "Cloud Native and Open Source Database Sub-Forum" on the morning of the 5th, He Chenghan, chief engineer of openGauss open source database, introduced the features of openGauss 5.0.0 version in detail.
He Chenghan, chief engineer of openGauss open source database
He Chenghan interpreted the features of the latest version of openGauss in his keynote speech "Introduction to Features of OpenGauss 5.0.0 Version". OpenGauss has evolved to version 5.0.0 after three years since the release of version 1.0.0 in June 2020. In this version, openGauss continues to enhance its core capabilities in terms of high performance, high availability, high security, and high intelligence, and also introduces the DataPod+DataKit product portfolio solution.
Four high-core capability enhancements
In terms of high performance, openGauss continues to optimize basic operators such as scanning operators and aggregation operators, achieving a 37% end-to-end performance improvement in the TPC-H 100G data volume scenario. At the same time, the Parser layer, communication layer and other aspects have also been optimized, and good optimization results have been achieved.
In terms of high availability, openGauss implements active-standby dual-cluster deployment through streaming disaster recovery, which satisfies the cross-Region disaster recovery scenario of three centers in two places. New features such as defining detection scripts. In terms of the high-availability capability of the autonomous consensus implemented by the kernel based on the Paxos protocol, new features such as strategic majority and weighted master selection have been newly realized, and log replication performance has been optimized. Based on the ability to subscribe and publish, openGauss realizes the two-way data synchronization capability between clusters, and version 5.0.0 also enhances this function, realizing uninterrupted publishing and subscription after backup recovery, incremental synchronization seamlessly connecting basic data synchronization, publishing end New features such as automatic connection of subscribers during active/standby switchover.
In terms of high security, openGauss has added multiple security features, such as: support for user-level auditing, fine-grained Any permission enhancement, etc.
In terms of high intelligence, the query optimizer implements a multi-column query intelligent cardinality estimator, which is based on the Bayesian network algorithm in the library and implements intelligent statistics to improve the accuracy of multi-column cardinality estimation, thereby improving the quality of the generated execution plan. In addition, anomaly detection supports self-configuration, and users can add, suspend or delete corresponding anomaly detection items in real time before or during operation.
DataPod+DataKit Combination Solution
openGauss DataPod is a product solution that realizes memory pooling and storage pooling by adding DMS and DSS components on the basis of the kernel. At present, the DataPod cluster supports 1 master and 7 slaves. The master node supports reading and writing, and the read and write capabilities of the slave nodes are horizontally expanded to meet the scenario of writing less and reading more. The data between cluster nodes is consistent in real time, supporting the transparent expansion of data consistency-sensitive application loads from a single machine to multiple nodes. The data page of each node supports memory and SCM multi-level cache, achieving a 30% performance improvement at the same cost. When the master node of the cluster fails, the RTO is less than 10 seconds and seamlessly switches to the standby node, and the RPO=0.
Although DataPod currently only supports one read-write node for write operations, applications can access the database from any node, and the database kernel will forward write operations, including those within transactions, to the master node for processing. Read operations are still only accessed It is processed on the read node, which greatly facilitates service access.
Shared storage supports enterprise-level SAN storage and distributed storage. Multiple nodes share one data, and the storage space is reduced by more than 50%. The NDP near-number calculation based on shared storage can greatly reduce the network IO traffic of the storage layer and the computing layer, and make full use of the CPU resources of the storage layer.
针对DataPod，CM在现有的检测能力之上，增加了共享磁盘心跳检查，并将检查结果作为仲裁的依据。此外，基于SCSI的共享磁盘锁，有效应对脑裂场景，还有基于IO Fence，可以隔离故障节点IO，避免故障扩散。
openGauss DataKit是数据全生命周期生产力工具，它既是一个集数据库资源管理、安装部署、数据建模、数据迁移、数据开发、智能运维于一体的插件化的工具集，同时它也是一个工具开发平台，用户可以基于该平台开发自己特有的工具，并且可以利用平台和其他插件开放出来的接口集成平台和其他插件的能力。
DataKit下载地址：https://opengauss.org/zh/download/
5.0.0版本openGauss针对MySQL的兼容和迁移能力都有全面提升，数据迁移和校验的速度也有大幅度提升。openGauss迁移工具集已经完全集成到DataKit中，实现了图形化、一键式启动迁移全流程，同时也支持批量并行迁移。
大咖观点
本文分享自微信公众号 - openGauss（openGauss）。
如有侵权，请联系 [email protected] 删除。
本文参与“OSC源创计划”，欢迎正在阅读的你也加入，一起分享。