Big Data technology first used in the Internet business, the Internet gives the characteristics of this emerging technology strengths in the processing of personal data. Today, the hot big data quickly "burn" into various industries, in the eve of the outbreak. And how to grasp the big data?
Technology needs to adapt to the trend of big data
Big Data processing The first is to acquire and record data; followed by the completion of data extraction, an important pre-processing or integration, aggregation and expression, such as cleaning and annotation and data (depending on the actual problem) work; the need for a complete analysis of the data again step, typically comprises data filtering, data such as summary, the data clustering or classification pretreated right into the final stage of analysis, at this stage, various algorithms and calculation tool is applied to the data, can be analyzed in order to see who wants or may be the result of interpretation.
Related to the huge amount of data, this set of process flow at various stages will challenge the traditional techniques. For example, network equipment mass, the mass of online users, uninterrupted network connection, in a large amount of time, the multi-format content data and status information, which via various client (web application or sensor, etc.) information from the data collection, along with thousands of access and operation request, the server system will apply pressure to the high concurrent manner.
In the analysis phase, in order to complete the purpose of data mining, often need to handle vast amounts of historical data and build complex mathematical statistics and analytical models (for example, affect the calculation of the temperature level of the winter down jacket sales of a particular thickness), and the for a large number of results association between the right to make efficient treatment, but also support the re-evaluation of the data update brings; and in the stage show, it should be hidden, such as data storage topology and data storage structures and other implementation details, data specification of exposure to business applications access interface that provides transparent support for complex data access requirements, which greatly reduces the difficulty of building business applications.
Complementary large cloud data
Traditional stand-alone mode is not only increasing the cost of treatment, and difficult to expand, and with increasing amount of data, increase the complexity of data processing, and the corresponding performance bottlenecks will be more and more extended. In this case, the cloud includes elastically stretchable and dynamic allocation, transparency virtualization and system resources to support multiple tenants billing according to the amount or support on-demand, and other basic elements of green energy just fit the new large data processing needs of technology; and cloud-computing model a typical representative of a new generation, as well as cloud computing platforms that support the underlying infrastructure of all upper layer application services, its high reliability, greater processing power and more large storage space, smooth migration, elastically stretchable, for transparency and a unified management and scheduling, and other characteristics of the user, is becoming an important development direction of the future of computing technology to solve big data problems.
Based on cloud computing platform built large data, large-scale distributed system capable of providing discrete polymeric communication, storage and processing capabilities, and provide a flexible, reliable and transparent form to the upper platforms and applications. It also provides for massive multi-format, multi-mode data across systems, cross-platform, cross-application means unified management and high availability, quick response mechanism system to support the goal of rapid change, the system environment and application configuration.
Cloud computing allows large data applications possible; no clouds appear computing, big data will still be castles in the air, the lack of foundation and floor possible. With cloud computing technology can improve the elasticity and flexibility of the overall system, reduce costs and manage risk, and improve application service availability and reliability; cloud computing not only to build an efficient, reliable system environment for large data processing, but also give full play the advantages of cloud computing platform, to find a more diversified export large data applications.
A correct understanding of cloud computing and big data
Value of Big Data has drawn attention to people's requirements for real-time data processing and effectiveness are also rising. Now for the big data applications have not limited BI (business intelligence) field, in all aspects of public services, scientific research, big data are also in play a huge influence, but also face much wider application. For example, the US National Oceanic and Atmospheric Administration attempts to exploit big data methods to assist in the study of climate, ecosystems, weather and commercial aspects of a Google Flu Trends is through the use of aggregated Google search data to estimate flu activity. Data has undoubtedly become an increasingly important resource for the information society.
Big Data significance lies not feature high capacity, diversity, but rather how we manage the data and analysis, and thus discover the value. If the lack of appropriate technical support in the analysis, the value of big data will be out of the question.
Traditional processing and analysis techniques in the face of these demands began to encounter bottlenecks, and the emergence of cloud computing, not only provides a large data mining tools to highlight the value of it for us, but also the use of big data have more possibilities .
Cloud computing includes two aspects; services and platforms, cloud computing is both a business model, but also count buckwheat mode. For example, the University of California, Berkeley, in a report on cloud computing, cloud computing refers both to think of applications as services over the Internet. Also refers to the hardware and software to provide these services in the data center.
On the current technology development, cloud computing resources in order to integrate data, including servers, storage, networks, applications, etc. as the center of virtualization technology as a means of using SOA architecture to provide users with safe, reliable and convenient various application data services; it has completed the process from component to system architecture level and then to pool resources to achieve different platforms (hardware, system and application) level of the iT system "universal" technology, breaking the barriers of physical devices to achieve centralized management, dynamic provisioning and on-demand purposes.
With the power of the "cloud" can be achieved unified management of big data for multi-format, multi-mode, and the efficient flow of real-time analysis, the value of big data mining, play big data real sense.
