On April 20, 2023, the online conference of [Databend Cloud, a new generation of cloud-native data warehouse] was successfully held. The release was jointly organized by Beijing Databend Technology Co., Ltd. [Databend Labs] and Alibaba Cloud. Databend Cloud implements cloud-based and easy-to-use big data analysis scenarios with the help of cloud-native data warehouse Databend.
In the past year, the Alibaba Cloud storage team has carried out in-depth cooperation with the cloud-native data warehouse Databend Cloud team to support Databend through object storage OSS to realize the separation of storage and computing, and to move towards the future of the serverless architecture together. The following content comes from the sharing summary of Databend co-founder - Wang Yin, Databend Cloud platform leader - Li Yazhou, and Alibaba Cloud intelligent senior product operation expert - Cai Liangwei at this conference. This conference is divided into three parts:
Part 1: Wang Yin and Li Yazhou share "Data transformation on the cloud, Databend Cloud released"
The second part: Cai Liangwei shared "Alibaba cloud object storage OSS, building an enterprise-level data lake base"
Part III: How to activate Databend Cloud on the Alibaba Cloud platform
Wang Yin (left) and Cai Liangwei (right) jointly witnessed Databend Cloud obtain Alibaba Cloud Ecological Certification
Databend Cloud launch
Introduction to the main functions of Databend Cloud
Databend is a cloud-native data warehouse developed using Rust, fully oriented to the cloud architecture, and built on object storage.
Databend Cloud is an elastic cloud data warehouse based on Databend. It consists of three layers. The bottom layer is based on object storage, such as Alibaba Cloud OSS; the upper layer computing nodes use different specifications, just like buying clothes for individuals, with small, medium, and large sizes. Etc. Similarly, computing nodes also have small size 1x, medium size 2x, and large size 8x. You can choose different models according to your own computing needs; the top layer is our metadata and management clusters to achieve multi-tenant isolation and ensure user data Safety.
The Databend Cloud architecture is as follows:
Databend Cloud can provide users with:
- One-stop data analysis and management platform
- Rich tenant and organizational management capabilities
- Billing according to resource usage, zero management and zero operation and maintenance
- multi-cloud solution
- Databend core team provides support
How about Databend performance?
ClickBench is an analytical database performance test leaderboard initiated by ClickHouse. It contains the test results of more than 50 mainstream analytical databases such as Snowflake and ClickHouse. It uses an open standard to measure the performance of the database. Our import performance is tested on three machines All models rank first. In the hot run query, we have one model that ranks first, and the other two models rank second and third respectively.
The difference between Databend Cloud and building your own data warehouse, why choose Databend Cloud
At present, many users use cloud hosts to deploy data warehouse clusters in the public cloud. Usually, the traditional data warehouse integrated storage and computing architecture is adopted, and the underlying storage is constructed using hard disks. The bottom layer of Databend Cloud uses object storage to use as much as possible without considering the space capacity. The cost is 1/3 to 1/8 or even less of cloud hard drives. The computing resources used are also related to business needs. If you are running batch business, you only need Use at specified times. Overall, it can save more than 80% of the cost for the enterprise.
The difference between Databend Cloud and using public cloud data warehouses
The data warehouse in the public cloud is mainly an integrated storage and computing architecture, and the bottom layer is also built based on cloud hard disks. Compared with scenario 1, public cloud vendors usually provide PaaS services, and the entire cluster can be installed in the user's VPC with one click, but the user still needs time. Keep an eye on the cluster status. In terms of cost, the cluster is occupied for a long time, and the cost is even higher than that of the first scenario that users build by themselves. So on the whole, compared with Databend Cloud, it needs a certain operation and maintenance cost, and the overall cost is more than 4-10 times higher.
Compared with the data warehouse service in the public cloud, Databend Cloud is a more open company that can provide you with multi-cloud lock-free services. At the same time, Databend Cloud is also developed based on Databend, and Databend can also realize private deployment on all public clouds in the market today. It allows users to achieve zero worry about being locked by the cloud.
Databend Cloud is suitable for the following business scenarios
Databend Cloud is mainly positioned in the storage and analysis of big data on the cloud. Judging from the current actual user usage scenarios, Databend Cloud provides users with the following functions:
Based on the separation of storage and calculation, independent expansion and contraction of the computing layer can be realized
Based on object storage, it helps users realize structured and semi-structured high-compression storage and
provides a unified service experience on multiple clouds
Strong computing and analysis capabilities, user data is not only stored, but also has flexible computing capabilities
Q&A application based on AI, intelligent customer service, etc.
some users
How Alibaba Cloud Object Storage Helps Cloud Native Data Warehouse Build a Unified Data Base
Alibaba Cloud Object Storage has provided tens of thousands of customers with support for data lake and data warehouse analysis services on the cloud. During this process, Object Storage has continuously improved and evolved its data lake storage capabilities. From version 1.0 to version 3.0, its ultimate vision is Provide a unified, multi-protocol access storage base for the ecological application of upper-layer analysis.
In the past year, the Alibaba Cloud storage team has carried out in-depth cooperation with the cloud-native data warehouse Databend Cloud team to support Databend through object storage OSS to realize the separation of storage and computing, and to move towards the future of the serverless architecture together.
During this process, the two parties have brought the following value to customers in terms of data storage:
- Stable and reliable: Adopt a variety of high-reliability technologies, such as verification, multiple availability zones, and cross-region replication to ensure data is not lost and ensure continuous access to data.
- Safe and trustworthy: Through various data encryption technologies and perfect authority management and control capabilities, full-link access security and storage security are realized.
- Elastic scaling: In a service-oriented manner, pay as you go, providing scalability from zero to trillions of objects and EB-level storage space.
- Extreme performance: It can provide customers with Tbps-level throughput bandwidth, optimize the reading and writing of massive small files, and support 5 billion files in a single namespace.
How to activate Databend Cloud
Databend Cloud is now available on the Alibaba Cloud market, and it is also more convenient for Alibaba Cloud users to activate Databend. For the activation method, you can click on the original text at the end of the article to watch the video.
In addition, it can also be activated through the Alibaba Cloud OSS console:
About Databend Cloud
Databend Cloud is a cloud-native data warehouse SAAS product based on the implementation of Databend. It provides a one-stop SaaS cloud data platform. It has: no installation, no operation and maintenance, you can use it after registering an account, and pay according to your SQL query. , It really achieves the characteristics of paying only when you use it, and no payment when you don’t inquire or use it. It provides users with:
- Separation of storage and calculation, or further separation of calculation: resource flexibility, pay-as-you-go
- The same data supports multiple computing nodes to access together
- Data encryption based on the application layer
- Multi-level storage, transparent to users
- Support data time travel, allowing users to easily access data at any point in time
- Multi-cloud, no lock-in, big data solution for unified user experience
Based on the above conditions, Databend Cloud customizes a multi-cloud, lock-free, highly elastic, high-performance, and low-cost cloud big data solution for users.
Overseas official website: https://www.databend.com
Domestic official website: https://www.databend.cn
Open source community website: https://databend.rs
Github：https://github.com/datafuselabs/databend
Station B: Search Databend, technology sharing video
Data revolution on the cloud: Databend Cloud is officially released