1. Artificial Intelligence, Machine Learning and Deep Learning
What artificial intelligence needs to solve is natural language understanding, image recognition, speech recognition and other problems that humans can solve quickly through intuition but are difficult to solve through computers. How to make computers grasp common sense that is very intuitive to humans is a huge challenge for the development of artificial intelligence.
The reason why early artificial intelligence systems can be successfully applied to relatively specific environments (specific domains), such as IBM's Deep Blue; the establishment of knowledge bases (Ontology) is a big breakthrough, of which WordNet is the most famous, which further defines relationship between synonym sets. In addition to WordNet, there are also many people who organize the knowledge in Wikipedia into a knowledge base. Google's knowledge base is based on Wikipedia.
On the one hand, building a knowledge base requires a lot of manpower and material resources. On the other hand, the knowledge that can be clearly defined through the knowledge base is limited, and not all knowledge can be clearly defined in a fixed format that can be understood by computers. So how can computers gain new knowledge from historical experience just like humans do? This is the problem that machine learning needs to solve.
Definition of machine learning: A program can learn from experience if the effect P can also increase with the increase of experience E on the task T.
How to digitally represent real-world entities has always been a very important problem in computer science; that is, how to extract features from entities has a huge impact on the performance of many traditional machine learning algorithms. Using different representations of the same data can greatly affect the difficulty of solving the problem. Once data representation and feature extraction are solved, many AI tasks are 90% solved.
But feature extraction is not a simple thing. One of the core problems solved by deep learning is to automatically combine simple features into more complex features, and use these combined features to solve problems. Deep Learning Room A branch of machine learning that, in addition to learning associations between features and tasks, can automatically extract more complex features from simple ones.
In general, artificial intelligence, machine learning, and progression to study rooms are a few areas that are very relevant. The relationship between them is: artificial intelligence is a very broad class of problems, and an important means for the machine learning room to solve such problems. Deep learning is a branch of machine learning. In many artificial intelligence problems, deep learning methods break through the bottleneck of traditional machine learning methods and promote the development of artificial intelligence.
The history of deep learning
Early neural network models were similar to biomimetic machine learning, which attempted to mimic the learning mechanism of the brain, with the McCulloch-Pitts Neuron results and perceptron models greatly influencing modern machine learning, but they had limitations. Among them, the perceptual model can only solve the linearly separable problem, but cannot solve the XOR problem;
It was not until the late 1980s that the second wave of neural network research emerged with the introduction of distributed knowledge representation (distributed representation) and neural network backpropagation algorithms. The core idea of distributed knowledge representation is that knowledge and concepts in the real world should be expressed by neurons, and each neuron in the model should also participate in expressing multiple concepts. Distributed knowledge representation greatly strengthens the expressive ability of the model, allowing the neural network to move from the width direction to the depth direction, which lays the foundation for future deep learning. Deep neural networks can solve linear inseparable problems like the XOR problem very well.
In the late 1980s, researchers also made breakthroughs in reducing the computational complexity of training neural networks. Inverse algorithms, convolutional neural networks and recurrent neural networks are widely used. LSTM models (long short-term memory) can effectively model longer sequences, such as a sentence or a paragraph. Until today, LSTM is the most effective method to solve many natural language processing, machine translation, speech recognition, time series prediction and other problems.
3. The application of deep learning
Deep learning has performed very well in many machine learning fields, such as image recognition, speech recognition, audio processing, natural language processing, robotics, biological information processing, chemistry, computer games, search engines, online advertising, medical automatic diagnosis and It has applications in various fields such as finance. The following are some areas where deep learning is widely used.
1. Computer Vision
Computer vision is the first field of deep learning technology to achieve breakthrough achievements. With the deep learning algorithm AlexNet winning the image classification competition ILSVRC (ImageNet Large Scale Visual Recogntion Challenge) in 2012, deep learning began to receive widespread attention in academia.
In the object recognition problem, face recognition is a very widely used technology. Deep learning technology can solve this problem well by automatically learning more effective facial feature representation from massive data. On the face recognition dataset LFW (Labeled Faces in the Wild), the DeepID2 system based on the deep learning algorithm can achieve a recognition rate of 99.47%.
In the field of computer vision, optical character recognition (OCR) is also one of the earlier fields using deep learning. The so-called OCR is to use a computer program to convert the characters in the picture that the computer cannot understand, such as numbers, letters, Chinese characters and other symbols, into a text format that the computer can understand.
2. Voice recognition
The deep learning method reduces the error rate based on the traditional Gaussian mixture model (GMM) on the TIMIT dataset from 21.7% to 17.9% using the deep learning model. As the amount of data increases, the deep learning model is superior to the algorithm using the Gaussian mixture model in terms of the growth rate of the correct rate and the growth rate. The reason why deep learning can accomplish such a technological breakthrough is the most important The original is that it can automatically extract more complex and effective features from massive data. Instead of manually extracting features as in the Gaussian mixture model.
Speech recognition based on deep learning has been applied to various fields, such as the Siri system launched by Apple, the Google voice search launched by Google on the Android system, and the simultaneous interpretation system developed by Microsoft from English to Chinese.
3. Natural language processing
Deep learning has made achievements in language modeling (language modeling), machine translation, part-of-speech tagging (part-of-speech tagging), entity recognition (named entity recognition, NER), sentiment analysis (sentiment analysis), advertising recommendation and search ranking. outstanding achievements.
A very important technique for intelligent feature extraction using deep learning is word embedding. Because in the field of natural language processing, there are many words that express similar meanings, but the encoding of these two words may be very different in the computer, so the computer cannot well understand the semantics expressed by natural language. To solve this problem, researchers manually built a large number of corpora. Through these corpora, the relationship between words in natural language can be roughly described. Among the established corpora, WordNet, ConceptNet and FrameNet are the most influential ones. However, the establishment of the corpus requires a lot of manpower and material resources, and the expansion ability is limited. Word vectors provide a more flexible way to characterize the semantics of words.
Word vectors represent each word as a relatively low-dimensional vector (like 100 or 200). For words that are semantically similar, the distance of their corresponding word vectors in space should also be close.
4. Human-machine game
Such as AlphaGo defeating Lee Sedol