CSharpFlink distributed real-time computing framework, adding Kafka data interface and loading multiple computing tasks

update content:

1. Multiple computing task instances (*.dll) can be added.
2. IChannelMesaageHandler interface can be added , so that the main node and the computing node can be reused.
3. Fix the problem that the SouceFunction starts the thread multiple times.
4. Increase the interface of Kafka data access to support real-time calculation of Kafka access data.

Note: Please download from the open source website.

system framework:

System test:

     CPU: 4-core I5-7400 2.7GHz, memory: 16G, random data point time window and calculation operator, main node CPU and memory usage: 15%-35%, 1500MB-2048MB, working node CPU and memory usage: 0.1%-2.5%, 18MB-30MB. This machine deploys 1 master node and 10 computing nodes. The master node generates a calculation task of 100,000 data points, and each data point generates a new data in 1 second, and calculates the maximum, minimum, average or sum of the time window. Operation effect, as shown below:

Guess you like

Origin www.oschina.net/news/121387/csharpflink-updated