Databend Open Source Weekly Issue 121

[Live broadcast preview] How far do we have to go to replace Oracle?

Databend is a modern cloud data warehouse. Designed for flexibility and efficiency to support your large-scale analysis needs. Free and open source. Experience the cloud service now: https://app.databend.cn .

What's On In Databend

Explore Databend's new developments this week and encounter Databend that is closer to your heart.

Support append stream

Databend now supports CREATE STREAMsyntax to create corresponding append streams for tables. The append stream only tracks insert operations, not update and delete operations.

The corresponding syntax is as follows:

CREATE STREAM [IF NOT EXISTS] [<database>.]<stream> 
  ON TABLE [<database>.]<table> 
    [<stream_point>] [COMMENT = '<string_literal>']

Here's a simple example:

databend> create table t(a int);
Query OK, 0 rows affected (0.07 sec)

databend> insert into t values(1);
Query OK, 1 row affected (0.06 sec)

databend> alter table t set options(change_tracking=true);
Query OK, 0 rows affected (0.06 sec)

databend> create stream s on table t;
Query OK, 0 rows affected (0.04 sec)

databend> insert into t values(2);
Query OK, 1 row affected (0.07 sec)

databend> select * from s;
+------+
| a    |
+------+
|    2 |
+------+
1 row in set (0.07 sec)
Read 2 rows, 59.00 B in 0.032 sec., 61.82 rows/sec., 1.78 KiB/sec.

If you'd like to learn more, feel free to contact the Databend team or check out the resources listed below.

PR #13602 | feat: append only stream

Code Corner

Let’s explore code snippets or projects in Databend and the surrounding ecosystem.

Switch from parquet2 to parquet-rs

As parquet2 faces a lack of bug fixes and long-term maintenance, Databend has now switched the default Parquet Reader to arrow-rs/parquet .

This change v1.2.223-nightlywill take effect from , and using arrow-rs/parquetwill help improve the stability of Parquet file reading and writing. Currently, Databend's Fuse engine still uses parquet2.

PR #13754 | feat: change default value of use_parquet2 to 0

Highlights

Here are some noteworthy events, maybe you can find something of interest.

Added current_timestamptime function.
New array_to_stringand array_flattenarray functions.
Added stream_statustable functions and improved fuse_encodingtable functions.
Added UDF authorization management support.

What's Up Next

We are always open to cutting-edge technologies and innovative ideas, and welcome you to join the community and inject vitality into Databend.

Utilize table engine to support Iceberg and Delta tables

Currently, Iceberg tables are only supported for use with Iceberg catalogs. If you can decouple the table from the catalog, you can try creating a table powered by the Iceberg or Delta engines using the following syntax:

create table <ident> from <location> engine=iceberg|delta [connection_name="my_connection", <other args>]

Issue #13787 | support iceberg and delta with table engine

If you are interested in this topic, you can try to solve some of the problems or participate in discussions and PR reviews. Alternatively, you can click on https://link.databend.rs/im-feeling-lucky to pick a random question, good luck!

Changelog

Check out the changelog for Databend's daily builds to stay up to date on the latest developments.

Address: https://github.com/datafuselabs/databend/releases

Contributors

A big thank you to the contributors for their great work this week.

Connect With Us

Databend is an open source, flexible, low-cost, new data warehouse based on object storage that can also perform real-time analysis. We look forward to your attention and exploring cloud native data warehouse solutions together to create a new generation of open source Data Cloud.