This article is the founder of Tao Jianhui Tao think the data in the "State Power" in 2019 for the first time on 04 published articles, Tao Pan-thinking data networking hope in big data platforms in the power was play to their technical superiority, contribute.
Construction of the ubiquitous power of things, the State Grid Co., Ltd. to promote an important and critical part of the "three type two network" construction. Which, how to build a robust data platform, is to accelerate the premise and basis of things pan in the power construction.
The amount of data ubiquitous power of things will increase substantially
Always attached importance to the power industry data and information technology, from the 1980s, on the use of real-time database processing power grid and various data collection. But with the expansion of grid-scale, a substantial increase in the amount of data acquisition, real-time database and the traditional IT architecture it has been unable to meet the massive data processing requirements, so in recent years, the power industry began to use big data platform technology of the Internet industry, the most typical is to Kafka, Hadoop, HBase, Spark, Redis technologies integrate massive data processing. Such as smart meters with a production system, the tariff calculation, such programs are used.
Pan networking to promote the construction of electric power was needed to run a full range of network status, users of electricity such as real-time monitoring, early warning, analysis, data collection points and collection frequency will be a substantial increase in the amount of data will increase a hundredfold in the original basis.
With smart meters, for example, now users of smart meters a day to send a record. If instead of smart meters with the same business, 15 minutes to send a record to the cloud, the amount of data is increased at least 96-fold, the number of data insertion request corresponding increase more than 96-fold. The whole network of smart meters to 500 million units to statistics, the number of data pieces will one day generate up to 48 billion, the existing big data solutions and infrastructure will face a huge challenge, even though the increase in the number of servers through horizontal expansion to handle its operating costs will increase in magnitude.
From the point of view of the distribution network, even if the collection points and collection frequency is not significantly increased, but the D5000, CC2000 as the representative of mainstream products, subject to historical data processing capability, still only around real-time data acquisition, historical data to build applications section , topological analysis technology can not scale up in the time dimension.
电网数据采集及监控系统（SCADA）作为物联网的一部分，不但要看实时数据，还需要看历史数据，不单需要实时监控，更需要故障预警、趋势分析、运营指标的分析、效率分析等。通过快速存取、分析高频采集数据，将为电网的安全高效运行提供更精准的数据决策支撑。
另外一方面，泛在电力物联网与通用的物联网一样，不仅会存在云端的数据中心，也会存在边缘节点。这些边缘节点具备一定的计算和存储能力，能进行数据的预处理和缓存，大幅缓解数据中心平台的压力，而且能更好的保证边缘节点覆盖的区域有更好的数据实时响应能力，更好的支撑本地业务的实时智能化决策与执行。但是边缘计算与云计算需要通过紧密协同才能更好地满足各种需求场景的匹配，从而最大化边缘计算和云计算的应用价值。边云协同对打造泛在电力物联网大数据平台提出了新的要求。
采集点的增加和采集频次提高，能带来什么样的效益呢？以智能电表为例，如果将所有电表的数据采集频次提高到1次/15分钟，电网将实现对每个台区线损的实时监测，而不是现在的T-1模式，从而对异常线损实时处理。同时，对输电线路故障实时监测，再也无需用户上报，大大提升运维效率和服务质量。
以Hadoop体系为代表的互联网大数据解决方案，主要处理的是互联网领域的非结构化数据，比如爬虫数据、微博、微信数据等。但是，泛在电力物联网的数据与互联网数据有显著不同的特点，表现在几个方面：数据都是时序的，由传感器和设备不断产生，形成一个数据流；除视频、图像外，都是结构化的数据；数据是机器日志类型的，不会有删除或更新的动作；数据是有保留时长的，到期删除；数据流量是平稳可预测，知道测点数、采集频率，能较为准确估算流量大小；数据需要进行实时计算、分析；数据的分析、计算一般都是基于某一个时间段和地域进行；数据量巨大，一天产生上几百亿条记录。
Characterized in addition to the data are not the same, on the data processing, networking and ubiquitous power compared to typical Internet, there are different requirements. For example interpolation calculating section data and mathematical functions like a specific point in time. And the processing of these data are often directly linked to the acquisition and management of equipment, the need for a variety of categories based on ownership statistics collection devices, geographical and other attributes .
Construction of large data platform compatible
With the acceleration of the construction of the ubiquitous power of things, the existing Internet platform for Big Data technology will face enormous challenges, because power data size will increase several orders of magnitude, the amount of data analysis is also more real-time requirements are also higher . Therefore, further increase innovation in information technology, to build and improve accommodate large data platform ubiquitous power of things building needs.
This new generation of big data platform, have the following characteristics: full use of the ubiquitous data characteristics of power of things, do all kinds of technical optimization, greatly improved data insertion, query performance, reduce operating costs; must be able to data insertion processing in real time, a query request; level must be extended, with the increase in the amount of data, the server only needs to increase to expansion; support edge side of the cloud computing and cloud synergy; must be easy to maintain, reduce operation and maintenance personnel required; must be open, there is the popular industry standard SQL interfaces for application integration; must be integrated through a variety of machine learning Python, R, or other convenient interfaces, artificial intelligence algorithms.
At present, many domestic and foreign enterprises to see the rise of the Internet of Things, the traditional big data technology are facing new tests and challenges, and started to develop a new generation of big data platform. I believe that with the ubiquitous power of things continues to accelerate construction, will build a new generation of energy power big data platform to further tap the resources and the use of good data grid, grid operators to enhance efficiency and effectiveness, more reliable protection of the grid operator security, and provide new applications and services.
