Apache IoTDB has been used in many IoT systems. In order to let everyone understand how the time series data of the Industrial Internet of Things is collected, stored, inquired and analyzed, and visualized. We made an IoTDB showcase, which is a reduced version of IoTDB's functions in practical applications, showing several common scenarios for IoTDB to manage IoT time series data.
The text is 1978 words, and the expected reading time is 5 minutes.
Introduction to Showcase
Showcase diagram
This booth showed several common scenarios in which Apache IoTDB manages time series data of the Internet of Things.
In the industrial Internet era, more and more terminal devices collect monitoring data of themselves and their surroundings through sensors, and are time-tagged, forming the main body of industrial big data-time series data.
Display content:
(1) The light sensor in the mobile phone, the distance measuring sensor controlled by PLC, the gyroscope connected to the Raspberry Pi, etc., will send the collected time series data through the network and store it in the IoTDB of the device;
(2) The IoTDB on the device side can provide real-time query functions and visual display;
(3) The local data file of the device is sent back to the IoTDB on the server and loaded directly;
(4) The IoTDB on the server side provides long-term historical storage, and supports real-time query and complex data analysis, such as reading the data in IoTDB through Spark for calculation and analysis.
Sensor data collection
The source of IoT time series data is the sensors on various industrial equipment. This showcase has two sensors, a pull-wire ranging sensor and a gyroscope.
The first is a pull-wire distance sensor. This sensor can measure the distance the wire is pulled out in real time, forming a time series. The data of this sensor needs a Mitsubishi PLC that supports modbus protocol to collect. PLC (Programmable Logic Controller) is called the brain of industrial production system. PLC has two main functions: (1) Issue control commands, including switch and analog values (2) Collect data on sensors. The data collection is mainly shown here.
The second sensor is a gyroscope, which can measure the components of gravitational acceleration and angular velocity on three coordinate axes. A total of 6 time series. This sensor can be collected directly through the driver.
Device-side IoTDB
In the middle is a Raspberry Pi. The Raspberry Pi is a microcomputer with all the internal organs. Mainly used to simulate the field controller of the connected equipment. We deployed an IoTDB in this Raspberry Pi, and developed a program to collect data from the ranging sensor through PLC, and store the data in the IoTDB. This acquisition program uses Apache PLC4X. PLC4X provides a unified way to support various industrial protocols to interact with PLC. In addition, the gyroscope driver also runs in this Raspberry Pi. The data of the ranging sensor and gyroscope are stored in the IoTDB on the device side in real time.
Visualization
The screen in the upper left corner is the visualization software Grafana. We have developed the extension plug-in iotdb-grafana so that the time series data in IoTDB can be directly visualized in real time through Grafana. In this way, the monitoring personnel of the factory can monitor the operating status of the equipment in real time.
IoTDB-Grafana-Connector document:
https://iotdb.apache.org/#/Documents/progress/chap7/sec1
Data synchronization between IoTDB instances
There are many devices in the factory and IoTDB deployed by the field controller. We want to aggregate the data of all devices to the cloud. At this time, we don't want to write a copy of the data, so we developed the data synchronization function. The data file (TsFile) in one IoTDB instance can be periodically synchronized to another IoTDB instance. The Raspberry Pi on the right simulates a cloud server, and IoTDB is also deployed on it. The IoTDB on the device side periodically sends data files to the IoTDB on the server side.
File return function document:
https://iotdb.apache.org/#/Documents/progress/chap6/sec1
Big data analysis
IoTDB uses an integrated query and analysis architecture. A piece of data can not only support real-time query, but also support spark for big data analysis. We have developed 4 kinds of connectors.
MapReduce reads the data file directly (MapReduce-TsFile):
https://iotdb.apache.org/#/Documents/progress/chap7/sec2
Spark directly reads the data file (Spark-TsFile):
https://iotdb.apache.org/#/Documents/progress/chap7/sec3
Spark reads IoTDB directly (Spark-IoTDB-Connector):
https://iotdb.apache.org/#/Documents/progress/chap7/sec4
Hive reads the data file directly (Hive-TsFile):
https://iotdb.apache.org/#/Documents/progress/chap7/sec5
Internet data collection
In the Internet field, such as sensors in mobile phones, we have developed a simple APP to collect data and write it into IoTDB. Since IoTDB v0.9.0 supports dynamic registration of metadata when writing, client programming is also very convenient (0.9.0 will be released soon).
Physical map
to sum up
That's all for this showcase! It mainly includes data collection, storage, visualization, transmission, query, analysis and other functions of IoTDB in the industrial field. This showcase summarizes all the display functions we have done from Digital China in April 2018 to now.