1. Current status of real-time computing in the field of big data
With the overall development of the big data industry, enterprises have more and more demands for real-time computing. Especially when building a real-time data warehouse, many real-time data sources need to be connected, and the data warehouse is still layered. Real-time computing is required for each layer of data. At this time, many real-time computing programs need to be developed. SQL can realize real-time computing tasks, without writing a bunch of codes every time, improving work efficiency, and making it possible for ordinary developers who only know SQL to easily develop real-time computing tasks.
In order to solve this pain point, we developed a data processing bus platform, which can also be called a real-time data transfer platform.
2. What is a data processing bus
In order to make the processing of real-time data more efficient and simple, we have developed a one-stop real-time data development platform. You only need to select the data source, destination, and corresponding SQL calculation logic on the page, and you can easily realize the development of massive real-time data calculation tasks.
The main function of this platform is to support SQL to realize the development of real-time data computing tasks.
The goal we expect to achieve is to use SQL to solve more than 80% of real-time data computing needs through this platform.
3. Overview of the prototype of the data processing bus
Since the data processing bus involves the front-end and back-end, in the enterprise there are dedicated colleagues responsible for the development of the front-end code, and our big data department only needs to be responsible for the background function development, so the front-end page code is not involved in the course. Here, the prototype diagram is used to demonstrate the specific use process of the data processing bus to deepen everyone's understanding.
Note: The prototype diagram can only be demonstrated here, and cannot be sent out. I hope everyone understands.