Date: 2020/8/4
The Kuiper team announced the release of Kuiper 0.9.0
Kuiper 0.9.0 can be downloaded from here .
EMQ X Kuiper is a lightweight IoT edge analysis and streaming open source software implemented by Golang. It can run on various resource-constrained edge devices. One of the main goals of Kuiper's design is to migrate real-time streaming computing frameworks (such as Apache Spark , Apache Storm and Apache Flink, etc.) running in the cloud to the edge. Kuiper reference to the above architecture and implementation of the cloud streaming program, the characteristics of the edge processing streaming data, based on the use of the preparation 源 (Source), SQL (业务逻辑处理), 目标 (Sink)rules engine to implement the edge processing streaming data ends.
Kuiper's application scenarios include: running in various edge use scenarios of the Internet of Things, such as real-time processing of production line data in the Industrial Internet of Things; real-time analysis of data from the car bus in the Internet of Vehicles; in the smart city scenario, Real-time analysis of data from various urban facilities. Through Kuiper's processing at the edge, system response speed can be improved, network bandwidth costs and storage costs can be saved, and system security can be improved.
Website: https://www.emqx.io/products/kuiper
Github repository: https://github.com/emqx/kuiper
Overview
Kuiper 0.9.0 is a major version that includes support for convection status, KubeEdge device model access, EdgeX array type support, and TDengine database storage.
Features and bug fixes
- State management function support . This feature allows Kuiper to implement a stateful flow:
- Support fault-tolerant processing. If an unexpected interruption occurs during stream processing, stream processing can be resumed after the rule is restarted;
- Support the realization of checkpointing (Checkpointing), which allows users to implement QoS settings during stream processing, including At-most-once(0), At-least-once(1) and Exactly-once(2);
- To consume data from a specified data offset (offset), users can extend the relevant interface to realize a data source that can consume data again, so that it can be offline or can be restored when there is an error in stream processing;
- Configurable state persistent storage. The system stores state in the file system by default, and also supports storing state data in third-party databases, such as Redis;
- Support the user to call the interface provided by Kuiper to realize the storage of custom state data when expanding the source, target and function;
- Provides access support for the KubeEdge data model , and a tool that automatically executes the distribution of configuration files through Kubernetes configmap. Users can use Kuiper to directly support analysis of device data from KubeEdge;
- Added TDengine plug-in , which can support saving analysis results to TDengine time series database;
- Translated and synchronized all Chinese documents;
- Optimized the execution process of FVT in Github Action, and deleted some unnecessary test procedures;
- Added support for RPM & APT installation package;
- Window
filterfilter supporting data , to implement the data filter, and then the window packet; this feature is important to count window: Results and filtered through a WHERE clause, and then the window will be different groups;
thank
- @chensheng0 provides a fix for Kubernetes configmap, which can be integrated with Baidu Baetyl framework
- @GZJ provides a fix to clean up the scene when Kuiper exits
- @smart33690 provides a fix for the Influxdb sink sample plug- in
Kuiper 2020 milestones
The Kuiper project will continue to develop rapidly in 2020, including improving more functions, integrating with more projects in the edge open source community, adding more continuous integration tests, and improving software quality. The main contents are as follows,
- EdgeX Hanoi version integration (Q3): Kuiper will support the newly added array data types in EdgeX; and support the management of Kuiper's flow and rules through EdgeX UI, making it more convenient for users to use Kuiper
- KubeEdge integration (Q3/Q4): By extending the Device Model, Kuiper is used to implement functions such as cleaning, caching and retransmission of bypass device data
- State management (Q3): Kuiper will provide built-in State support, and support functions such as fault-tolerant recovery. Kuiper will realize the persistence support required for long-term window processing through this function. In addition, it can also allow users to extend the plug-in process through API Call to realize the storage of custom state data
- Kuiper 1.0.0 (Q3/Q4) released: Version 1.0.0 will be released in Q3 or Q4 in 2020
You can click Kuiper 2020 milestones for more detailed information and the latest updates.
contact
If you have any questions about Kuiper, please feel free to contact us at [email protected].
Copyright statement: This article is EMQ original, please indicate the source for reprinting.
Original link: https://www.emqx.io/cn/news/release-notes-kuiper-0-9-0