Real-time big data statistical application using pipelinedb

Leveraging pipelinedb for real-time big data statistics applications

PipelineDB is built to continuously run SQL queries on streaming data. The output of these continuous queries is stored in regular tables and can be queried like any other table or view.
Summary and summary; perform calculations in sliding time windows; text search filtering; geospatial filtering, etc. By reducing the cardinality of its input streams, PipelineDB can significantly reduce the amount of information that needs to be retained to disk, as only the output of successive queries is stored. The original data will be discarded once a continuous query that needs to be read is read.

  Therefore, most of the data passed through PipelineDB can be considered dummy data. This idea of ​​data virtualization is at the heart of PipelineDB's concerns, allowing it to process large amounts of data very efficiently using relatively small hardware space.

  Raw data can be streamed directly into PipelineDB and refined and distilled in real-time through continuous queries that you have declared. This allows for periodic processing of granular data before fine-grained output is loaded into the database, as long as this processing can be defined by an SQL query.