What is big data?
First ask a question: Is "big data" a specialized technology? Some people may think that big data is a specialized technology, but it is not. The three words "big data" are just a marketing language, behind which is the comprehensive application of a series of technologies such as hardware, database, operating system, and I-ladoop.
A brief history of big data
Big data can be said to be well-known now, but in fact, it is really retrospective, it was first proposed by SGI chief scientist John R.Masey at the USENIX conference in 1998. He first proposed this term in a paper he published called Big Data and the Next Wave of Infrastress to describe the phenomenon of data explosion. It is estimated that he might not have imagined that Big Data would be so popular in more than ten years.
If you trace the concept of big data, Alvin Toffler predicted in the book The Third Wave in 1980 that the arrival of the information age will bring about a data explosion, so scientists foresee it very early Here comes big data. Big data has a long history, but technology needs continuous accumulation to change from quantitative to qualitative.
For the industry, I have to mention the three technical papers on GFS, MapReduce and BigTable published by Google in 2003-2006. It is these three papers that laid the foundation for the development of big data. The father of Hadoop-Doug cuttingo is the reference paper, and later realized the current famous Hadoop, and the birth of Hadoop greatly promoted the vigorous development of big data technology.
Of course, it should be pointed out here that Hadoop is not equivalent to big data, and big data does not specifically refer to Hadoop. Big data is just a market language, representing a concept, a problem-solving idea, and a collection of a series of technologies. Hadoop is just one of the specific framework technologies for processing data.
State of Big Data
The 2016 technology maturity curve released by Gartner (see Figure 1.1) removes cloud computing, big data and related technologies for the first time. Gartner pointed out that these technologies are not unimportant, but are no longer "emerging." Although everyone's interest in big data is still unabated, the market has settled down, with a set of reasonable methods, and new technologies and practices are added. Existing programs. Therefore, big data has passed the peak period of technological expectations, and it is time to use big data to solve problems. In the future, the evolution of big data-related technologies will still show strong vitality for a long time, and the revenue of related markets will continue to increase.
As mentioned earlier, big data is a collection of ideas, problem-solving ideas, and a series of technologies. It has both similarities and differences with traditional Bl.
The same thing is to extract value from data to promote business success. The core of the difference is the development of distributed technology and the tremendous improvement of processing capabilities, making it possible to deal with things that were previously unthinkable. Therefore, the concept of data processing has also been expanded:
(1) Not limited to the traditional Bl sampling and modeling from the data, and then back to DW implementation, big data can directly find the law from the full amount of data, through the data sample Diversity compensates for the accuracy of the model.
(2) Not limited to the traditional Bl, simply find group commonality through summary and statistical analysis to output reports. Big data can directly portray individuals through enough data.
Although there are various differences, the boundaries between big data and Bl will be blurred in the future. The core driving goal of enterprises is to find business value from data assets, without caring about the methodology of construction and analysis.