1.1 Big Data era
First Wave: before and after 1980, the personal computer, the birth of the company: IBM
Second Wave: before and after 1995, the popularity of the Internet
Third Wave: things, cloud computing, large data
Lead to big data era
factor:
Technical Support:
1. Storage: storage capacity is increasing (personal data continues to increase, more and more corporate data)
2. Calculation: CPU processing capacity has increased significantly
3. Network: network bandwidth is increasing
Through three stages:
The first stage: stage operating systems
The second stage: UGC stage
Phase III: Phase Perception systems (large-scale popularization of the Internet of Things)
Big Data development process:
1.2 Big Data concepts and the impact
Characteristics: 4V
1. The amount of data:
Big Data Moore's Law: more than 50% growth per year
2. Diversity:
Structured data and unstructured data
3. Fast of:
The processing speed is very fast; many enterprise applications require second-level decision-making
4. Low density value:
Impact of big data:
1. The fourth paradigm
The first: Experimental
The second: Theory
Third: computing
Fourth: data
2. In terms of way of thinking, subvert the traditional way of thinking
1) rather than the whole sample sampling
2) rather than the precise efficiency (the first one basis)
3) rather than a causal correlation
1.3 Application of Big Data
TV drama Toupai: big data analysis software
Google predict flu: real-time control of citizens search for relevant information
1.4 The key technology of large data
Five levels: data collection, (core) data storage and management, data processing and analysis , data privacy and security
Two core: distributed storage, distributed processing
Distributed storage: Google behalf of the company
Big data calculation mode:
Different computing model requires the use of different products
1. batch computing :( real time is not good enough)
Representative products: MapReduce, Spark
2. Flow calculation:
Real-time processing, real-time response
3. FIG calculated:
Efficient processing of the data structure of FIG.
4. Analysis calculated query (Query Interactive):
Representatives: Hive, etc.
1.5 big data cloud computing, networking
cloud computing:
1. Cloud computing to solve the problem:
Two core: distributed storage, distributed processing
2. Typical features:
Virtualization, multi-user
3. The concept:
As a service to provide very low-cost IT resources to users over the network
4. Advantages:
Companies do not need self-built IT infrastructure, cloud resources can be rented
The three modes: public cloud, hybrid cloud, private cloud
Public cloud: for all users
Private cloud: for the enterprise
Hybrid cloud: part for himself, partly to outside
6. The three kinds of cloud services:
1.IaaS：基础设施即服务
将基础设施作为服务出租
2.PaaS：平台即服务
用提供的平台资源开发、部署软件
3.SaaS：软件即服务
典型案例：云财务软件
7.云计算关键技术：
1.多租户
2.虚拟化：虚拟机、虚拟专用网（VPN）
3.分布式存储
4.分布式计算
8.云计算数据中心
数据中心包含有大量刀片服务器
数据中心建设位置：
地质构造稳定、天气凉爽
慕课链接：https://www.icourse163.org/learn/XMU-1002335004?tid=1003965001#/learn/announce