Article directory
The content of this article comes from the OpenMLDB official website and has been deleted or modified.
About OpenMLDB
- Official website: https://www.openmldb.com
- github : https://github.com/4paradigm/OpenMLDB
- Official documentation: https://openmldb.ai/docs/zh/main/about/intro.html
- Official blog: https://openmldb.ai/blog/
- WeChat communication group website: http://qr61.cn/o9EEV6/qcGz53S
- Beginner's Guide: https://openmldb.ai/openmldb-Beginner's Guide-A must-read for developers to get started/
- Get started quickly: http://openmldb.ai/docs/zh/main/quickstart/openmldb_quickstart.html
OpenMLDB is an open source machine learning database.
Currently, it is mainly oriented towards artificial intelligence applications , providing an efficient feature platform with online and offline consistency . It has been deeply optimized for feature store requirements, achieving millisecond-level computing latency.
In addition, OpenMLDB itself also contains an efficient and fully functional time series database , which is used in finance, IoT, data annotation and other fields.
develop
OpenMLDB originated as commercial software from 4Paradigm, a leading artificial intelligence platform provider.
In 2021, its R&D team abstracted, enhanced, and community-friendly the core components of feature engineering in commercial products, and formed them into a systematic open source product to help more companies realize artificial intelligence transformation at low cost.
Before being open sourced, OpenMLDB has been deployed and launched in hundreds of scenarios as one of the commercial components of Fourth Paradigm.
Solve the problem of
traditional feature development: separation of offline development and online services, high cost and
inconsistency between online and offline. Possible reasons are:
-
Inconsistent tool capabilities
-
Poor understanding of needs communication
-
Two major AI applications: perception and decision-making
-
Hard real-time computing truly meets the needs of implementation decision-making – real-time data, real-time computing
-
Streaming computing designed for Big Data and BI
Architecture
The above figure shows the abstract architecture of OpenMLDB, which contains four important design components:
- Unified SQL programming language
- High-performance real-time SQL engine with millisecond latency
- Batch SQL engine based on the OpenMLDB Spark distribution
- Consistent execution plan generator that connects real-time and batch SQL engines to ensure online and offline consistency
Iori 2023-11-29 (Wednesday)