Dear community friends, Apache Doris version 2.0.2 has been officially released on October 6, 2023. This version has updated and optimized multiple functions to better meet the needs of users. 92 contributors have submitted functional optimization items and problem fixes for Apache Doris 2.0.2 version, which further improved the stability and performance of the system. Everyone is welcome to download and experience it.

GitHub download: https://github.com/apache/doris/releases/tag/2.0.2-rc05

Official download page: https://doris.apache.org/download/

Important updates

Remove the json "->" operator that conflicts with the lambda function syntax and use the function json_extract instead. #24679
metadata_failure_recoveryMoved parameters from # 24308
The null value in a common type is represented by \n. The null value in a complex type or nested type is consistent with the JSON type and is represented by null. # 24207
Optimize the binding strategy of the priority_network configuration item of the BE node. If the user configures the wrong priority_network, the startup will fail directly to avoid the user from mistakenly thinking that the configuration is correct. If the user does not configure priority_network, only the first IP is selected from the IPv4 list instead of selecting from all IPs to avoid the user's server not supporting IPv4. # 23795 # 23784
Supports canceling retrying import tasks and fixes the problem of failed loading cancellation. # 17730

Function optimization

01 Improved ease of use

In some scenarios, users need to add some custom libraries to the cluster, such as lzo.jar, orai18n.jar, etc. In past versions, these lib files were located in fe/lib or be/lib, but when upgrading the cluster, the lib library will be replaced by the new lib library, causing all customized lib libraries to be lost. In the new version, a new custom directory custom_lib is added for FE and BE , where users can place custom lib files. #23887
Supports user role-based permission access control and implements row-level fine-grained permission control strategies. #23022

02Improve query optimizer Nereids statistics collection

Disable File Cache when running Analysis tasks. Analysis tasks are background tasks and should not affect the user's local File Cache data. #23663
In past releases, columns with errors were ignored when viewing their statistics. In the new version, when the min or max value fails to deserialize, viewing the column's statistics will use N/A as the min or max value and still display the remaining statistics, including count, null_count, ndv, etc. #23703
Supports statistics collection of JDBC external tables. #23965
Skip statistics checking on __internal_schemaand on unknown columns. #24625information_schema

03 Multi-Catalog function optimization

Support Hadoop Viewfs; #24168
Optimize JDBC Catalog Checksum Replay and Range related issues; #22369
Optimized JDBC Catalog's Property check and error message prompts. #23868
Fixed the MaxCompute Catalog Decimal type parsing problem and the incorrect use of object storage address. #24242
Supports SQL Cache of Hive Metastore Catalog. #23391
Improved the metadata synchronization performance of Hive Metastore Catalog. #22869
Add metadata_name_ids to quickly obtain Catalogs, DB, Table, without Refresh Catalog when creating or deleting Catalog and Table, and add Profiling table to be compatible with MySQL. #22702

04Inverted index performance optimization

Increase the query cache of the bkd index. The cache can accelerate the query performance when the bkd index is hit. The effect is more obvious in high concurrency scenarios; #23952
Improve the query performance of inverted index on Count operator; #24678
Improved the efficiency of the Match operator when missing the index, and the performance increased by up to 60 times in test performance; #24751
Improved query performance of MATCH and MATCH_ALL on inverted index; #23871 #24389

05 Array function optimization

Optimized the problem that the Array function of the old version of the query optimizer cannot handle the Decimal type; #23630
Optimized array_union array function support for multiple parameters; #24327
Support processing array nested complex types through the explode function; #24455

Bug fixes

This version fixes some bugs that existed in the previous version, greatly improving the overall stability of the system. For the complete list of BugFix, please refer to the GitHub Commits record: https://github.com/apache/doris/issues/25011

Apache Doris version 2.0.2 is officially released!