Databend Open Source Weekly Issue 114

Databend is a modern cloud data warehouse. Designed for flexibility and efficiency to support your large-scale analysis needs. Free and open source. Experience the cloud service now: https://app.databend.cn .

What's On In Databend

Explore Databend's new developments this week and encounter Databend that is closer to your heart.

MERGE INTO

The MERGE INTO statement is used to insert, update, or delete rows in a target table using data from a specified source based on specified conditions and matching criteria.

MERGE INTO usually contains several MATCHEDor NOT MATCHEDclauses: for MATCHEDthe clause, the target table can be updated or deleted, and for NOT MATCHEDthe clause, data can be inserted.

-- Merge data into 'salaries' based on employee details from 'employees'
MERGE INTO salaries
USING (SELECT * FROM employees)
ON salaries.employee_id = employees.employee_id
WHEN MATCHED AND employees.department = 'HR' THEN
    UPDATE SET
        salaries.salary = salaries.salary + 1000.00
WHEN MATCHED THEN
    UPDATE SET
        salaries.salary = salaries.salary + 500.00
WHEN NOT MATCHED THEN
    INSERT (employee_id, salary)
    VALUES (employees.employee_id, 55000.00);

The MERGE INTO statement is currently experimental and can be executed SET enable_experimental_merge_into = 1;to enable this feature.

If you would like to learn more, check out the resources listed below.

Docs | MERGE

Code Corner

Let’s explore code snippets or projects in Databend and the surrounding ecosystem.

Python Binding and Local modes support configuring data paths

Databend's Local mode now supports using DATABEND_DATA_PATHenvironment variables to control the storage location of metadata and data files.

DATABEND_DATA_PATH=/tmp/data/ databend-query local -q "create table abc(a int); insert into abc values(3);"

Databend's Python Binding also follows DATABEND_DATA_PATH, but databendthis environment variable needs to be defined before use.

import os
os.environ["DATABEND_DATA_PATH"] = "/tmp/def/"

from databend import SessionContext

If you would like to learn more, check out the resources listed below.

PR #13089 | feat: support DATABEND_DATA_PATH in bendpy and databend-local

Highlights

Here are some noteworthy events, maybe you can find something of interest.

Improved Hash Join, performance increased by 10% in some scenarios.
Improved parallel execution of MERGE INTO.
Improve CI, use to quickinstallinstall relevant binary tools, and use nextestto perform unit testing.

What's Up Next

We are always open to cutting-edge technologies and innovative ideas, and welcome you to join the community and inject vitality into Databend.

Remove files when deleting Internal Stage

Internal Stage stages data files in the storage backend specified in the Databend Query configuration.

Considering that after deleting the Internal Stage, users will not be able to access the data files staged in it. Therefore, it is necessary to consider removing the staged files in the Internal Stage when deleting it.

Issue #12986 | remove files at the same time of drop internal stage

If you are interested in this topic, you can try to solve some of the problems or participate in discussions and PR reviews. Alternatively, you can click on https://link.databend.rs/im-feeling-lucky to pick a random question, good luck!

Changelog

Check out the changelog for Databend's daily builds to stay up to date on the latest developments.

Address: https://github.com/datafuselabs/databend/releases

Contributors

A big thank you to the contributors for their great work this week.

Connect With Us

Databend is an open source, flexible, low-cost, new data warehouse based on object storage that can also perform real-time analysis. We look forward to your attention and exploring cloud native data warehouse solutions together to create a new generation of open source Data Cloud.