I. Introduction
The preface article [1] of this series has already carried out a detailed description of the hardware layer and explained the process of equipment performance, development, filling and other links. This article will explain the related processes after the data is uploaded to the cloud.
As the project platform is under continuous construction, the current open source information is the second phase version, so the content of this article is explained based on the capabilities of this version. The architecture of this version further optimizes data authentication and database protection.
Home page effect of Phase II version
The home page effect
of the third-phase version under construction, while the third-phase version under construction has added functions such as log service, object storage, and small programs. Provides flexible log storage and high-speed retrieval capabilities, low-cost database backup + recovery + migration synchronization capabilities, mobile terminal interaction and other capabilities.
Second, the structure of the composition
The structure of the entire Phase II version is shown in the figure below, in which the part marked in light blue is the focus of the architecture changes relative to Phase 1.
Phase II architecture diagram
1. Deployment dimensions
Terminal: The equipment deployed at the monitoring point.
Cloud: A platform deployed on the cloud.
2. Functional dimension
Based on the basic components of the platform design, it is logically split, and the specific products selected are matched according to the design requirements of the architecture layer. The specific functions and selection instructions are as follows:
Monitoring node: consists of two parts : soft and hard. The firmware is developed based on Tencent's IoT terminal operating system TencentOS tiny, and the hardware devices compatible with Tencent Cloud are connected with Tencent Cloud IoT explorer at the protocol level.
Local gateway: This part is mainly based on communication protocol conversion, and does not parse specific data. It only needs to be configured according to the preceding instructions.
IoT access layer: Based on Tencent Cloud IoT explorer product, it provides related functions of decryption, access, and forwarding. After obtaining the data, a retweet action based on the http protocol was carried out and passed to the downstream API gateway.
Middleware layer: Based on the API gateway, it provides capabilities such as connection and load balancing. Based on the capabilities of authentication and flow control, it connects to cloud functions and provides reliable data input and output services.
Logic layer: Based on cloud functions, the ability to receive data into the database (data solidification), timed statistical summary (data analysis), and output application data (data output, support display) is realized.
Storage layer: Based on the cloud database TencentDB for MySQL, it stores monitoring data, summary data, and platform configuration.
Display layer: Based on Tencent Cloud Map, it provides map display, data display, and line chart display capabilities. And the configuration provides linkage processing, which improves the response speed and reduces the repeated transmission of resource files.
Three, module realization
The overall system is designed with data as the core, so this part is divided into two components:
Data structure: Introduce the core data structure and correlation mode of each link.
Module description: Introduce the realization of each functional module and the corresponding source code.
1. Data structure
(1) IoT explorer (IoT development platform) push data structure
Pushed in by the API gateway, the original data extracted by the cloud function in the environment variable event. The main parts are as follows:
The non-'body' part is related information of the API gateway;
'body' = IoT explorer raw data;
'body' = the original data reported by the terminal node, encoded by base64.
{ "body": "{\"payload\":\"eyJtZXRob2QiOiJyZXBvcnQiLCJjbGllbnRUb2tlbiI6IjIwMjAtMDgtMThUMTE6MTc6NDkuNDAzWiIsInBhcmFtcyI6eyJQTTFfQ0YxIjo1LCJQTTJkNV9DRjEiOjYsIlBNMTBfQ0YxIjo3LCJQTTEiOjUsIlBNMmQ1Ijo2LCJQTTEwIjo3LCJwYXJ0aWNsZXNfMGQzIjo3OTIsInBhcnRpY2xlc18wZDUiOjI1NCwicGFydGljbGVzXzEiOjE1LCJwYXJ0aWNsZXNfMmQ1IjozLCJwYXJ0aWNsZXNfNSI6MywicGFydGljbGVzXzEwIjowLCJ2ZXJzaW9uIjoxNTEsIkVycm9yIjowfX0=\", \"seq\":18639, \"timestamp\":1597749469,\"topic\":\"$thing/up/property/?????/?????\", \"devicename\":\"?????\", \"productid\":\"?????\"}", "headerParameters": {}, "headers": { "accept-encoding": "gzip", "content-length": "511", "content-type": "application/json", "endpoint-timeout": "15", "host": "service-???-???.gz.apigw.tencentcs.com", "user-agent": "Go-http-client/1.1", "x-anonymous-consumer": "true", "x-api-requestid": "???03a993936ae191f44651???", "x-b3-traceid": "???03a993936ae191f44651???", "x-qualifier": "$LATEST" }, "httpMethod": "POST", "path": "/??????", "pathParameters": {}, "queryString": {}, "queryStringParameters": {}, "requestContext": { "httpMethod": "POST", "identity": {}, "path": "/??????", "serviceId": "service-?????", "sourceIp": "?.?.?.?", "stage": "release" }}
'body' 经 base64 编码后：
{ "clientToken": "2020-08-18T11:17:49.403Z", "method": "report", "params": { "Error": 0, "PM1": 5, "PM10": 7, "PM10_CF1": 7, "PM1_CF1": 5, "PM2d5": 6, "PM2d5_CF1": 6, "particles_0d3": 792, "particles_0d5": 254, "particles_1": 15, "particles_10": 0, "particles_2d5": 3, "particles_5": 3, "version": 151 }}
（2）数据库表单结构
数据库详细格式，请参考github-sql：https://github.com/eckygao/ampservice_demo/blob/master/amservice.sql
各表用途：
config：系统配置表；
space：地点表。记录节点部署地点；
client：节点表。记录节点信息及最新关键信息，关联space表；
base_data：日志表。记录每次上报的信息，关联space、client表；
aggregate_data：汇总信息表。记录按小时、天等时段汇总的数据信息；
temp_base_data：临时日志表。上报设备名如不在节点表内，则存入此表，主要用于调测。
2. 模块说明
（1）数据入库
模块功能：接收IoT explorer推送数据，匹配并转换后，入数据库；
使用产品：API 网关、无服务器云函数；
关联源码：github-upload[2]；
部署方式：请参考相关文章 在线甲醛监测[3] “4.2.2 无服务器云函数/4.2.3 云API网关” 章节。
（2）终端在线更新
模块功能：按3分钟触发，更新终端信息，用于快速查询与展示；
使用产品：云函数；
关联源码：github-client_active_new；
部署方式：请参考相关文章 在线甲醛监测 “4.2.2 无服务器云函数/4.2.3 云API网关” 章节，但改为“定时触发”模式。
（3）数据统计
模块功能：按小时及天触发，统计监测数据，用于快速查询与展示；
使用产品：云函数；
关联源码：github-aggregate()；
部署方式：请参考相关文章 在线甲醛监测 “4.2.2 无服务器云函数/4.2.3 云API网关” 章节，但改为“定时触发”模式。
（4）数据接口-终端汇总信息表
模块功能：通过API网关提供数据查询能力，响应云图调用，提供终端汇总信息表；
使用产品：API 网关、云函数；
关联源码：github-client_info；
部署方式：请参考相关文章 在线甲醛监测 “4.2.2 无服务器云函数/4.2.3 云API网关” 章节。
（5）数据接口-终端单点信息表
模块功能：通过API网关提供数据查询能力，响应云图调用，提供终端单点信息表；
使用产品：API 网关、云函数；
关联源码：github-client_data；
部署方式：请参考相关文章 在线甲醛监测 “4.2.2 无服务器云函数/4.2.3 云API网关” 章节。
（6）数据展示
模块功能：提供WEB端展示能力，含地图绘点、列表展示、折线展示等；
使用产品：腾讯云图；
关联源码：无，当前为手工配置；
部署方式：请参考相关文章 在线甲醛监测 “4.2.4 腾讯云图” 章节，但数据源选择时，改为使用“API”。其中“首页”地图使用“数据接口-终端汇总信息表”接口、“单点数据页”使用“数据接口-终端单点信息表”接口。具体调用的接口URL，请对应API网关提供的服务链接。
四、成本分析
二期成本展示
上图表数为每节点每15秒上报一次的计算结果。
硬件部分由于厂商与采购量的差异，价格不同。此处以云服务成本计算，如果10000节点规模，每节点每分钟上报一次，单节点云服务年成本如下：
1. 二期架构单节点成本
(数 据库/年+云图/年)/10000+(API网关+云函数)/4=(468+48)/10000+(9+2.1)/4=3.335元。
2. 三期架构单节点成本
(消息队列/年+数据库/年\*2+云图/年)/10000+(云函数+日志服务+对象存储)/4=(3228+468*2+48)/10000+(2.1+2.24+0.02)/4=1.09元。
其中，二期无对象存储及日志服务模块，但当前的在建三期中已使用数月，故有实际数据可列入统计。
五、后续计划
预计完成三期建设，计划内容涉及：
Interactive optimization (done): Enable space table and display location.
Interaction construction (doing): increase the ability of small programs, provide mobile terminal query capabilities, and alarm push capabilities.
Disaster tolerance optimization 1 (done): Add object storage, provide low-cost data backup, recovery, and asynchronous migration synchronization capabilities.
Disaster Recovery Optimization 2 (todo): Increase the message queue CKafka to deal with the risk of data accumulation and loss under abnormal network and database conditions.
Disaster Recovery Optimization 3 (doing): Add log service for data reconciliation to deal with data loss or confusion that may occur in multi-step data processing.
Three architecture notice as shown below, light blue label out of two changes to the schema is relatively focus.
Three-phase structure
The partially completed effect of the third phase version is shown as follows:
WEB display
