Table of contents
Relationship between deep learning, neural network, machine learning, artificial intelligence
Vernacular explanation of deep learning
Traditional Machine Learning VS Deep Learning
Pros and Cons of Deep Learning
4 Typical Deep Learning Algorithms
Convolutional Neural Network – CNN
Recurrent Neural Network – RNN
Generative Adversarial Networks – GANs
Deep Reinforcement Learning – RL
Relationship between deep learning, neural network, machine learning, artificial intelligence
deep learning, machine learning, artificial intelligence
simply put:
- Deep learning is a branch of machine learning (the most important branch)
- Machine learning is a branch of artificial intelligence
deep learning, neural network
The concept of deep learning originated from the research of artificial neural networks, but it is not completely equal to traditional neural networks.
However, in terms of name, many deep learning algorithms will contain the word "neural network", such as: convolutional neural network, recurrent neural network.
Therefore, deep learning can be said to be an upgrade based on traditional neural networks, which is approximately equal to neural networks.
Vernacular explanation of deep learning
It is assumed that the information to be processed by deep learning is "water flow", and the deep learning network that processes data is a huge water pipe network composed of pipes and valves (weight thresholds) . The inlets of the network are pipe openings and the outlets of the network are also pipe openings.
This network of water pipes has many layers, and each layer has many regulating valves that can control the direction and flow of water flow. According to the needs of different tasks, the number of layers of the water pipe network and the number of regulating valves in each layer can have different combinations. For complex tasks, the total number of regulating valves can be in the thousands or even more. In the water pipe network, each regulating valve on each floor is connected to all regulating valves on the next floor through water pipes to form a fully connected water flow system from front to back layer by layer.
So how can a computer use this vast network of water pipes to learn to read?
For example, when the computer sees a picture with the word "Tian", it simply puts all the numbers that make up the picture (in the computer, each color point of the picture is composed of "0" and "1") represented by numbers) all become streams of information, which are poured into the water pipe network from the inlet.
We pre-inserted a signboard at each outlet of the water pipe network, corresponding to each Chinese character we want the computer to recognize. At this time, because the input is the Chinese character "Tian", when the water flows through the entire water pipe network, the computer will run to the outlet of the pipe to see if the water flowing out of the pipe outlet marked with the word "Tian" is the most . If so, the pipe network meets the requirements. If this is not the case, just adjust each flow control valve in the water pipe network, so that the water that "flows out" from the "Tian" character outlet is at most.
Now, the computer has to be busy for a while, and there are so many valves to adjust! Fortunately, the speed of the computer is fast, and the violent calculation and the optimization of the algorithm can always give a solution quickly and adjust all the valves so that the flow at the outlet meets the requirements.
In the next step, when learning the word "Shen", we will use a similar method to turn each picture with the word "Shen" into a water flow composed of a large number of numbers, pour it into the water pipe network, and see if it is The outlet of the pipe with the word "Shen" is the most flowing. If not, we have to adjust all the valves. This time, we must not only ensure that the "Tian" character we have just learned is not affected, but also ensure that the new "Shen" character can be processed correctly.
Repeatedly, it is known that the water flow corresponding to all Chinese characters can flow through the entire water pipe network in the desired way. At this time, we say that this water pipe network is a well-trained deep learning model. When a large number of Chinese characters are processed by this pipeline network and all valves are adjusted in place, the entire water pipeline network can be used to recognize Chinese characters. At this time, we can "weld" all the adjusted valves and wait for the arrival of new water flow.
Similar to what is done during training, the unknown pictures will be converted by the computer into a stream of data, which will be poured into the trained water pipe network. At this time, the computer only needs to observe, which water outlet flows out the most, and which word is written in this picture.
Deep learning is roughly such an overall structure built with human mathematical knowledge and computer algorithms, combined with as much training data as possible and the large-scale computing power of computers to adjust internal parameters, so as to approach the problem goal as much as possible. The way experience is modeled.
Traditional Machine Learning VS Deep Learning
Similarities Between Traditional Machine Learning and Deep Learning
In terms of data preparation and preprocessing, the two are very similar.
They all may perform some operations on the data:
- data cleaning
- data label
- Normalized
- denoising
- Dimensionality reduction
The core difference between traditional machine learning and deep learning
The feature extraction of traditional machine learning mainly relies on manual labor. Manual feature extraction is simple and effective for specific simple tasks, but it is not universal.
The feature extraction of deep learning does not rely on manual work, but is automatically extracted by machines. This is why everyone says that the interpretability of deep learning is poor, because sometimes deep learning can perform well, but we don't know what its principle is.
Pros and Cons of Deep Learning
Advantage 1: strong learning ability
Judging from the results, the performance of deep learning is very good, and his learning ability is very strong.
Advantage 2: Wide coverage and good adaptability
The neural network of deep learning has many layers and a wide width. It can theoretically be mapped to any function, so it can solve very complex problems.
Advantage 3: data-driven, high upper limit
Deep learning is highly dependent on data, and the larger the amount of data, the better its performance. In some tasks such as image recognition, facial recognition, and NLP , it has even surpassed human performance. At the same time, he can further increase his upper limit by adjusting parameters.
Advantage 4: Good portability
Due to the excellent performance of deep learning, there are many frameworks that can be used, such as TensorFlow and Pytorch . These frameworks are compatible with many platforms.
Disadvantage 1: large amount of calculation, poor portability
Deep learning requires a lot of data and a lot of computing power, so the cost is very high. And many applications are not yet suitable for use on mobile devices. At present, many companies and teams are developing chips for portable devices. This issue will be resolved in the future.
Disadvantage 2: High hardware requirements
Deep learning requires high computing power, and ordinary CPUs can no longer meet the requirements of deep learning. The mainstream computing power uses GPU and TPU , so the requirements for hardware are very high, and the cost is also high.
Disadvantage 3: Complicated model design
The model design of deep learning is very complex, requiring a lot of manpower, material resources and time to develop new algorithms and models. Most people can only use off-the-shelf models.
Disadvantage 4: No "humanity", prone to prejudice
Because deep learning relies on data, and the interpretability is not high. In the case of imbalanced training data, problems such as gender discrimination and racial discrimination will appear.
4 Typical Deep Learning Algorithms
Convolutional Neural Network – CNN
Value of CNNs :
- Able to effectively reduce the dimensionality of a large amount of data into a small amount of data (without affecting the result)
- Ability to preserve the characteristics of pictures, similar to human visual principles
The basic principle of CNN:
- Convolutional layer – the main function is to preserve the characteristics of the image
- Pooling layer – the main function is to reduce the data dimension, which can effectively avoid overfitting
- Fully connected layer – output the results we want according to different tasks
Practical applications of CNNs:
- Image classification and retrieval
- Target location detection
- target segmentation
- face recognition
- bone identification
Recurrent Neural Network – RNN
RNN is an algorithm that can efficiently process sequence data . For example: article content, voice audio, stock price trend...
The reason why he can process sequence data is that the previous input in the sequence will also affect the subsequent output, which is equivalent to having a "memory function". But RNN has serious short-term memory problems, and long-term data has little effect (even if it is important information).
So variant algorithms such as LSTM and GRU appeared based on RNN . These variant algorithms mainly have several characteristics:
- Long-term information can be effectively retained
- Select important information to keep, unimportant information will choose "forget"
Several typical applications of RNN are as follows:
- text generation
- Speech Recognition
- machine translation
- generate image description
- video tag
Generative Adversarial Networks – GANs
Assuming that the law and order in a city is chaotic, there will be countless thieves in the city soon. Among these thieves, some may be master theft, and some may have no skills at all. If the city starts to improve its law and order, a "campaign" against crime is suddenly launched, and the police begin to resume patrolling in the city. Soon, a group of "not skilled" thieves will be caught. The reason why the unskilled thieves were caught was because the police were not skilled enough. After catching a group of low-end thieves, it is hard to say how the city's security level will become, but it is obvious that the city The average level of thieves here has been greatly improved .
The police began to continue training their crime-solving skills and began to catch the more and more cunning thieves. With the arrest of these professional habitual offenders, the police have also developed special skills. They can quickly spot suspicious persons from a group of people, so they go forward to interrogate and finally arrest the suspects; the life of thieves is also difficult. , because the level of the police has been greatly improved, if you still want to behave sneakily like before, you will be caught by the police soon. In order to avoid being arrested, the thieves try their best to appear less "suspicious", while the devil is one foot tall and the road is one foot high. The police are also constantly improving their own level, striving to distinguish thieves from innocent ordinary people. With this kind of "communication" and "communication" between the police and the thief, the thieves have become very cautious. They have extremely high stealing skills and behave exactly like ordinary people, while the police have all practiced "sighting eyes". ", once a suspicious person is found, it can be found immediately and controlled in time-in the end, we got the strongest thief and the strongest police at the same time.
Deep Reinforcement Learning – RL
The idea of the reinforcement learning algorithm is very simple. Taking games as an example, if a certain strategy can be adopted in the game to obtain higher scores , then this strategy should be further "strengthened" in order to continue to achieve better results. This strategy is very similar to various "performance rewards" in daily life. We often use this strategy to improve our game level.
In the Flappy bird game, we need simple click operations to control the bird, avoid various water pipes, and fly as far as possible, because the farther you fly, the higher the reward points.
This is a typical reinforcement learning scenario:
- Machines have a clear role as birdies - agents
- Need to control the bird to fly farther - the goal
- During the whole game, you need to avoid various water pipes - the environment
- The way to avoid the water pipe is to let the bird fly hard - action
- The farther you fly, the more points you get - Rewards
You will find that the biggest difference between reinforcement learning, supervised learning, and unsupervised learning is that it does not require a lot of "data feeding". Instead, you learn certain skills through your own constant attempts.
Learn more: " What is Reinforcement Learning in One Article? (Basic concepts + application scenarios + mainstream algorithms)》
Summarize
Deep learning belongs to the category of machine learning. Deep learning can be said to be an upgrade based on traditional neural networks, which is approximately equal to neural networks.
Both deep learning and traditional machine learning are similar in data preprocessing. The core difference lies in the feature extraction link. In deep learning, the feature extraction is done by the machine itself, and manual extraction is not required.
Advantages of deep learning:
- High learning capibility
- Wide coverage and good adaptability
- Data driven, high ceiling
- good portability
Disadvantages of deep learning:
- Large amount of calculation, poor portability
- high hardware requirements
- Complex model design
- No "humanity", prone to prejudice
4 typical algorithms for deep learning:
- Convolutional Neural Network – CNN
- Recurrent Neural Network – RNN
- Generative Adversarial Networks – GANs
- Deep Reinforcement Learning – RL
deep learning
The concept of ANN originated from the research of artificial neural network. A multi-layer perceptron with multiple hidden layers is a deep learning structure . Deep learning combines low-level features to form more abstract high-level representation attribute categories or features to discover distributed feature representations of data .
The concept of deep learning was proposed by Hinton et al. in 2006. Based on the deep belief network (DBN), a non-supervised greedy layer-by-layer training algorithm is proposed, which brings hope to solve the optimization problems related to the deep structure, and then a multi-layer autoencoder deep structure is proposed. In addition, the convolutional neural network proposed by Lecun et al. is the first real multi-layer structure learning algorithm, which uses the spatial relative relationship to reduce the number of parameters to improve training performance.
Deep learning is a method based on representation learning of data in machine learning.
Observations (such as an image) can be represented in a variety of ways, such as a vector of intensity values for each pixel, or more abstractly as a series of edges, regions of a specific shape, etc. Instead, it is easier to learn tasks from examples (e.g., face recognition or facial expression recognition) using certain representations. The advantage of deep learning is to use unsupervised or semi-supervised feature learning and hierarchical feature extraction efficient algorithms to replace manual feature acquisition.
Deep learning is a new field in machine learning research. Its motivation is to establish and simulate the neural network of human brain for analysis and learning. It imitates the mechanism of human brain to explain data, such as images, sounds and texts.
Like machine learning methods, deep machine learning methods can also be divided into supervised learning and unsupervised learning. The learning models established under different learning frameworks are very different. For example, Convolutional Neural Networks (CNNs) is a machine learning model under deep supervised learning, and Deep Belief Nets (DBNs) is a machine learning model under unsupervised learning. .
Deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations rather than task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised.
Deep learning architectures, such as deep neural networks, deep belief networks, and recurrent neural networks, have been applied in computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design, medical image analysis and other fields. Materials inspection and board game programs that produce results comparable to, and in some cases better than, human experts.
Deep learning models are vaguely inspired by information processing and communication patterns in biological nervous systems, but have various differences from the structural and functional properties of biological brains, especially human brains, making them incompatible with neuroscientific evidence.