Author: Zen and the Art of Computer Programming
1 Introduction
With the development and application of big data, Internet and Internet of Things technologies have also entered a new stage, undergoing changes from distributed systems to cloud computing and large-scale cluster technologies. In this process, the technical framework has undergone tremendous changes, and various business models have also evolved accordingly. This article will summarize the research and development path and countermeasures of Internet of Things technology in the era of big data.
2. Big data development background
(1) Historical review
Big Data Overview
Data is an objective physical phenomenon. More and more data are acquired and accumulated through various means, forming massive data sets. Data can be structured, semi-structured or unstructured data, static data or real-time dynamic data. At present, due to the continuous improvement of the technical capabilities of storing, processing and analyzing big data, it has become possible to process massive data with hundreds of millions of records, and has generated a lot of value, which has attracted widespread attention.
In the late 1990s, computer technology developed rapidly, and various simulators, game consoles, and cartoons produced by people on computers caused a sensation. During this period, IBM introduced the System/370, which allowed a personal computer to run advanced software in minutes. In the same year, the International Organization for Standardization ISO released ANSI C, the C language standard. Since then, new information technologies such as various databases, file systems, and network transmission protocols have emerged, providing infrastructure for the storage, processing, and analysis of big data.
At the beginning of the 21st century, with the development and popularization of the Internet, big data is facing explosive growth, and more importantly, the amount of data is rapidly expanding. In the past, centralized data centers stored and processed data, but now it has become a bottleneck. Many companies urgently need cloud computing services to help them get rid of centralized dependence and use the characteristics of distributed data storage to improve efficiency.
In 2008, Google launched MapReduce,