Differences and connections between big data, machine learning, and artificial intelligence

What is the difference and connection between big data and machine learning or artificial intelligence?

• Big data does basic data storage and data statistical calculations . To discover or mine valuable information or data from a large amount of data , it is necessary to use machine learning algorithms to combine a large amount of data to build a machine learning model, and use the model to analyze real events. Make predictions .
  

What has changed in the era of big data?

• Changed the way humans think

  • The importance of data: the transformation from data resources to data assets ; intuitive phenomenon: some companies that buy and sell data have appeared.

4V characteristics of big data

• Big amount of data:
  • KB-----MB------GB----TB----PB----ZB-------1024-----2**10---- --index
  • Data shows exponential growth
  • Big data storage: HDFS distributed file system
  • Big data calculation: MR+Hive+spark+filnk
• various:
  • Structured data: Mysql, Oracle ===> standard SQL
  • Unstructured: video, audio, text ===> hdfs storage, mr-hive
  • Semi-structured: Json, Xml, Html====> hdfs storage, mr-hive
• high speed:
  • Data is growing fast
    • Data shows exponential growth
  • Fast data processing
    • Offline-batch processing-streaming computing (flink)
• Low value density:
  • The density of valuable data = valuable data/full data
  • Full data shows exponential growth
  • Valuable data grows slowly and has low value density
  • Need to use machine learning algorithms to solve :
    • How to mine valuable data or information?
      • It is necessary to use machine learning algorithms to combine a large amount of data to build a machine learning model, and use the model to make predictions about real events.

Several popular directions of artificial intelligence

• Image recognition: such as identifying a cat or a dog
• Driverless: Intelligent driverless cars such as Baidu's driverless cars
• Smart translation: Google's smart translation
• Speech recognition: Recognize where people are saying
• Medical diagnosis: Combining traditional doctor's method + medical data analysis
• Data Mining: Unearthing valuable data from massive data, the story of Walmart beer and diapers

Three waves of machine learning

• The three waves of machine learning can also be said to be the three waves of artificial intelligence , because machine learning is an inevitable product of the development of artificial intelligence (Artificial Intelligence) to a certain stage .

1956-AI-Development

• 1- Checkers: IBM defeated the human master-expert system
• 2-Chess: IBM Deep Blue-----------Statistical Model
• 3-Go: AlphaGo defeats Li Shishi-----big data genre, deep learning genre

Several basic concepts in the field of artificial intelligence

• Machine learning is a branch of artificial intelligence

  • Data mining branch
  • Pattern recognition branch

• Deep learning is a technology to implement machine learning

  • Deep learning is to solve the problem of machine learning in some aspects, such as image recognition, speech recognition, etc., and the ability of deep learning to extract a large number of features can be used for image recognition or speech recognition

• Artificial intelligence is smarter

  • Let the machine learn

Data, data analysis, data mining, machine learning

• Data: Observed value or measured value—1.7—1.8.—1.9
• Data analysis: the process from data to information, through data processing, processing, and analysis to achieve the purpose of information
• Information: credible data, for height-high school low
• Data mining: from information to valuable information, such as whether it is a programmer or a civil servant
• Machine learning (methods)-data mining (things)
• Machine learning (methods)-pattern recognition (things-face recognition)