Traditional enterprise data processing is based on structured data, and database + BI application is the main mode. In the intelligent era, data explosively grows, database systems are getting larger and larger, and data processing is shifting from structured data to structured + unstructured diversified data processing. Traditional manual maintenance methods will face huge challenges. Therefore, Huawei redefines the database architecture and introduces AI technology into the database, which greatly improves the database automation management and optimization capabilities.
Wang Tao, Executive Director, Huawei, President of ICT Strategy and Marketing
According to reports, GaussDB is the industry's first AI-Native database and the industry's first enterprise-level database that supports ARM. It has two major innovations: First, it integrates artificial intelligence technology into the full life cycle of distributed databases for the first time to realize self-operation and maintenance, self-management, self-tuning, fault self-diagnosis and self-healing. In transaction, analysis and mixed load scenarios, based on optimization theory, the first self-tuning algorithm based on deep reinforcement learning, the tuning performance is improved by more than 60% than the industry; the second is to make full use of X86, ARM, and ARM through the heterogeneous computing innovation framework. GPU and NPU have multiple computing power advantages. When Huawei's laboratory uses the authoritative standard test set TPC-DS for testing, its performance is 50% higher than the industry, ranking first.
According to understanding, the AI-Native database is divided into five levels:
1. AI Advised, using the external AI engine to give database optimization suggestions to reduce the burden on database administrators;
2. AI Assisted, the database has a built-in AI engine to realize online auxiliary optimization and improve database management efficiency;
3. AI Enhanced, on the one hand, implants AI into database core components, core algorithms and data structures to realize database self-optimization;
4. AI Enhanced, on the one hand, implants AI into database core components, core algorithms and data structures to realize database self-optimization;
5. AI Designed, realizes the design, verification, and development of the database through AI technology to achieve the purpose of full-scenario intelligence and inclusive AI.
At the scene, Huawei demonstrated the powerful performance of AI-Native database GaussDB through case studies. For example, in OLAP analysis scenarios and OLTP transaction scenarios, using AI-Native intelligent tuning mode has an average performance improvement of 68% compared with other automatic tuning methods, which is better than DBA tuning. Excellent increase of 54%.
The AI-Native database proposed by Huawei, in addition to reconstructing the database core and algorithm with AI concepts, also considers how to use the computing power of AI chips such as GPU and NPU to greatly improve the AI algorithm when Moore's Law is on the verge of failure in general computing. Performance.
Currently, GaussDB supports multiple scenarios such as local deployment, private cloud, and public cloud. On Huawei Cloud, GaussDB provides full-featured, high-performance cloud data warehouse services for customers in the financial, Internet, logistics, education, and automotive industries. This is also a big difference from Huawei's conference. The moment the product is released, it has basically been successfully implemented.
Huawei's AI capabilities
Many developers must have heard of Huawei's AI capabilities. At last year’s HUAWEI CONNECT 2018 conference, Huawei launched the ModelArts full-process platform, which allows developers to get started with AI faster and accelerate model training and deployment; and HiLens, a visual AI application development platform that understands developers better, is equipped with Ascend 310, which can provide powerful computing power and storage space, meet the needs of larger image processing operations, and even achieve a processing capacity of one hundred frames per second and millisecond face detection; offline development kit Atlas 200, convenient Developers research and develop offline.
As we all know, Huawei has a "legendary" 2012 laboratory, which is Huawei's main battle force in exploring the future of science and technology no man's land. Previously, InfoQ was invited to visit the Noah’s Ark Lab, which is Huawei’s AI R&D Center. The highlight of the show is the “Artificial Intelligence Capability Demonstration”, such as computing vision for smart terminals, voice assistants, and recommended search; Supply chain brains, question and answer systems, etc.; Thermal technology laboratory focuses on highly integrated packaging, high-density air cooling, high-efficiency liquid cooling/phase change, and showcases low thermal resistance of devices, bionic radiators, phase change liquid cooling, acoustic noise reduction materials, and flow field visualization Testing and other technologies. The GaussDB released this time was developed under the leadership of the 2012 laboratory.
在 AI 领域，华为一直坚持从芯片开始往上做到应用框架，基本覆盖了人工智能的所有方向，换句话说，就是支持人工智能全栈全场景。截止 2019 年 3 月底，华为云共发布有 59 个服务，160+ 的功能。这很好得解释了为什么华为有能力开发业界首款 AI-Native 数据库 GaussDB，并获得如此高的性能提升。
为什么是云？
如华为方面所言，GaussDB 运行在云平台之上。其实，这个消息本身并不是很令人惊讶，即便是地位还算稳固的 Oracle 近年也一直在遵守云优先的策略，即功能更新会优先在云平台进行，这也可以理解为云业务拉新的一种手段。
传统的数据库体系结构已有 30 多年的历史，专为本地部署，本地存储和单机数据库实例而设计和优化，此后很长一段时间内，该体系结构未针对公有云环境进行优化。在公有云环境下，计算和存储通过网络分离以支持多租户数据库实例并 scale out 向两端扩展；并不要求单个节点的高可靠性。
举例来说，假设一个本地数据库的每个数据库实例都有本地存储，为了支持高可用性，通常部署一个备实例并使用数据库内部复制机制将数据库日志（MySQL binlog）发送到备实例重建数据库副本。目前，大多数云厂商都采用类似机制来支持云数据库高可用性。但是，这种内部复制机制并没有利用云存储已经通过存储层复制提供的可靠性，这导致大量存储和带宽浪费。实际上，备数据库实例可以通过云存储与主实例共享数据，不仅可以节省磁盘空间和带宽，还可以为云数据库服务提供快速 RTO。
很明显，在将传统数据库迁移上云时，需要重新设计和优化数据库系统。过去几年，主要云供应商，比如 AWS，已经做了部分工作。预计未来几年，更多云和数据库供应商将遵循这一趋势，构建某种形式的云原生数据库技术。此外，Gartner 预测，到 2019 年，90％的云数据库架构将支持计算和存储分离。
Therefore, it is obviously the most suitable way to design and provide AI-Native database GaussDB according to the cloud platform. In the past few years, Huawei has been working hard to develop the cloud computing business, mainly to provide servers and other equipment for the data center, but the company's R&D laboratory is also studying enterprise software products. According to Huawei’s annual report, revenue from its enterprise business unit (including servers and other products for enterprise customers) increased by 24% to approximately US$11 billion last year, accounting for approximately 10% of the company’s US$106 billion in total revenue.
It is understood that this is not the first time that Huawei Cloud has explored databases on a cloud platform. Since June 2016, Huawei has released a total of 13 database services in 4 categories. Among them, the multi-mode database GeminiDB can meet the needs of multiple databases through one service.
Prior to this, Huawei Cloud's self-developed relational database Taurus also made public appearances. As the latest generation of cloud-native distributed database self-developed by Huawei Cloud, Taurus is fully compatible with MySQL 8.0, adopts an architecture design that separates computing and storage, and logs as data. It supports 1 write and 15 reads, and its performance is 7 times that of native MySQL.