Databend Open Source Weekly Issue 140

[Live broadcast preview] Will large models replace programmers? "

Databend is a modern cloud data warehouse. Designed for flexibility and efficiency to support your large-scale analysis needs. Free and open source. Experience the cloud service now: https://app.databend.cn .

What's On In Databend

Explore Databend's new developments this week and encounter Databend that is closer to your heart.

Support EXECUTE IMMEDIATE

Databend now supports the EXECUTE IMMEDIATE command, which allows users to execute complex SQL logic in a single statement, providing more flexible SQL processing capabilities.

Examples are as follows:

EXECUTE IMMEDIATE $$
BEGIN
    LET x := 1;
    LET y := x + 1;
    LET z RESULTSET := SELECT :y + 1;
    RETURN TABLE(z);
END;
$$;
----
┌───────┐
│ 2 + 1 │
│ UInt8 │
├───────┤
│     3 │
└───────┘

This feature enables Databend to better support complex data processing scenarios. and further advances our work in supporting stored procedures and SQL scripts. Interested partners are also welcome to participate in the development and iteration of related functions.

If you'd like to learn more, feel free to contact the Databend team or check out the resources listed below.

Code Corner

Let’s explore code snippets or projects in Databend and the surrounding ecosystem.

技术：Towards Efficient Distributed Group Aggregation

In large-scale data analysis (OLAP), group aggregation (Group Aggregation) is a widely used technology. Unlike regular SELECT queries, grouped aggregation divides the data set into groups and aggregates the group data, thereby changing the result cardinality.

Inspired by the DuckDB blog, Databend developed a new type of aggregate hash table to optimize the processing efficiency of grouped aggregation. In addition, in order to better adapt to distributed scenarios, Databend has also designed and implemented a series of optimizations for distribution.

Blog | Towards Efficient Distributed Group Aggregation

Highlights

Here are some noteworthy events, maybe you can find something of interest.

Supports ST_GEOMPOINTFROMGEOHASHand ST_MAKEPOLYGONgeographic information functions.
Support change table comments.
Support offsets for time travel.
Read the document Docs | Full-Text Index to learn about Databend's full-text indexing capabilities.

What's Up Next

We are always open to cutting-edge technologies and innovative ideas, and welcome you to join the community and inject vitality into Databend.

Add query summary to query log

A query summary is a summary of a query, and if two queries have the same summary, it means they share the same schema. For example, the following two query summaries are identical:

"SELECT id FROM t1 WHERE name = 'a';"
"SELECT id FROM t1 WHERE name = 'b';"

By examining the respective query summaries, you can determine which query mode is more time-consuming, helping to optimize database performance and resource utilization.

Issue #15164 | feat: add query digest to query_log

If you are interested in this topic, you can try to solve some of the problems or participate in discussions and PR reviews. Alternatively, you can click on https://link.databend.rs/im-feeling-lucky to pick a random question, good luck!

Changelog

Check out the changelog for Databend's daily builds to stay up to date on the latest developments.

Address: https://github.com/datafuselabs/databend/releases

Contributors

A big thank you to the contributors for their great work this week.

Connect With Us

Databend is an open source, flexible, low-cost, new data warehouse based on object storage that can also perform real-time analysis. We look forward to your attention and exploring cloud native data warehouse solutions together to create a new generation of open source Data Cloud.