Databend is a modern cloud data warehouse. Designed for flexibility and efficiency to support your large-scale analysis needs. Free and open source. Experience the cloud service now: https://app.databend.cn .
What's On In Databend
Explore Databend's new developments this week and encounter Databend that is closer to your heart.
MERGE INTO now supports distributed execution
The MERGE INTO statement inserts, updates, or deletes rows in a target table using data from a specified source based on specified conditions and matching criteria.
Now MERGE INTO in Databend already supports distributed execution, which will bring more efficient data integration capabilities. Its physical plan is shown in the figure below:
If you'd like to learn more, feel free to contact the Databend team or check out the resources listed below.
Code Corner
Let’s explore code snippets or projects in Databend and the surrounding ecosystem.
Embed data versioning into your analytics workflow with LakeFS
lakeFS is committed to providing solutions for sharing and collaborative data processing, empowering object storage with Git-like operating logic, using a versioning solution to provide a logically consistent view of data, and embedding meaningful branch names and commit information for modern data workflows. And provide solutions for the integration of data and documents.
Databend provides a high-performance query engine that is distributed, elastically scalable, and easy to operate and maintain. It also supports multi-version storage similar to Git, and can easily query, clone and restore data at any point in time.
Combining LakeFS's data versioning capabilities and Databend's Git-like data management capabilities can ensure data integrity and provide efficient, highly available and consistent views for data analysis workflows.
If you would like to learn more, check out the resources listed below.
Highlights
Here are some noteworthy events, maybe you can find something of interest.
- Supported JSON operators:
?,?|and?&. - Supports permission management for STAGE and UDF.
- Support
flattentable functions. - Supports creating external tables using URL:
CREATE STAGE test_stage [URL =] 's3://load/files/'. - Read the document Example 2: Attaching Table in READ_ONLY Mode to learn how to use the read-only mode of ATTACH TABLE.
- Read the documentation Conversion Functions to learn about the type conversion functions and rules in Databend.
What's Up Next
We are always open to cutting-edge technologies and innovative ideas, and welcome you to join the community and inject vitality into Databend.
Support reading Delta table format
Databend currently supports mounting and reading data in Iceberg table format through Catalog, and the next target planned to support is Delta Lake.
Delta Lake is an open source storage format initiated by Databricks. It extends Parquet data files with a file-based transaction log, enabling it to handle ACID transactions and extensible metadata. Delta Lake is fully compatible with the Apache Spark API and is designed to be tightly integrated with structured streaming, allowing you to easily use a single copy of your data in batch and streaming operations and provide incremental processing at scale.
Issue #13429 | add read only support for Delta table
If you are interested in this topic, you can try to solve some of the problems or participate in discussions and PR reviews. Alternatively, you can click on https://link.databend.rs/im-feeling-lucky to pick a random question, good luck!
New Contributors
Meet new people in the community. Databend is a better place because of you.
Changelog
Check out the changelog for Databend's daily builds to stay up to date on the latest developments.
Address: https://github.com/datafuselabs/databend/releases
Contributors
A big thank you to the contributors for their great work this week.
Connect With Us
Databend is an open source, flexible, low-cost, new data warehouse based on object storage that can also perform real-time analysis. We look forward to your attention and exploring cloud native data warehouse solutions together to create a new generation of open source Data Cloud.
Alibaba Cloud suffered a serious failure, affecting all products (has been restored). The Russian operating system Aurora OS 5.0, a new UI, was unveiled on Tumblr. Many Internet companies urgently recruited Hongmeng programmers . .NET 8 is officially GA, the latest LTS version UNIX time About to enter the 1.7 billion era (already entered) Xiaomi officially announced that Xiaomi Vela is fully open source, and the underlying kernel is .NET 8 on NuttX Linux. The independent size is reduced by 50%. FFmpeg 6.1 "Heaviside" is released. Microsoft launches a new "Windows App"