1 Overview of Unsupervised Learning Models
In supervised training, the model can calculate the loss based on the difference between the predicted result and the label, and converge towards the direction with the smallest loss.
In unsupervised training, it is not possible to specify the convergence direction for the model weights through the sample labels, which requires the model to have the function of self-supervision.
1.1 Two typical neural network models
Two typical neural networks are the self-encoding neural network and the adversarial neural network:
① Self-encoding neural network: The data is used as a label to specify the direction of convergence.
② Adversarial neural network: Generally, two or more sub-models are used for training at the same time, and the relationship between multiple models is used to achieve the effect of mutual supervision.
2 Autoencoding Neural Networks
Autoencoder is a neural network whose goal is to reconstruct the input signal. A field of unsupervised learning that automatically learns features from unlabeled data.
2.1 Structure of Autoencoding Neural Network
Autoencoding consists of 3 neural network layers: input layer, implicit layer and output layer. Among them, the samples of the input layer will also act as the label of the output layer, that is, this neural network is a neural network that reproduces the input signal as much as possible. .
- The process from high-dimensional feature samples of the input layer to low-dimensional features is called encoding, and the network is called encoder;
- The process from the low-dimensional features of the hidden layer to the high-dimensional feature samples is called decoding, and the speed that implements this part of the function is called the decoder.
2.2 Computational process of autoencoding neural network
An autoencoder neural network is essentially a model whose output and input are equal. A simple self-encoding neural network structure can be represented by a 3-layer fully connected neural network.
2.2.1 Briefly explain the computation of autoencoding neural network
In the above figure, the input layer has the same dimension as the output layer, the middle layer is the output of the encoder, and the output layer can also be understood as the output of the decoder. The encoder is responsible for encoding and converting the encoded original data to the intermediate low-dimensional data, and the decoder is responsible for decoding the low-dimensional data back to the original input to implement the process of encryption and decryption.
During the training process, the MSE calculation is performed with the original input data and the reconstructed decoded data, and the calculation result is used as a loss value to guide the convergence direction of the model.
The self-encoding neural network requires that the output be as equal to the input as possible, and its hidden layer must satisfy a certain sparsity. The sparsity effect is achieved by reducing the number of neurons in the latter layer of the hidden layer than the previous layer. This is equivalent to compressing the input in the hidden layer and decompressing it in the output layer. During the whole process, information will be lost, but training can minimize the lost information and maximize the retention of its main features.
2.3 The role and significance of autoencoding neural network
The input data will undergo a series of feature transformations in the network model, and the output will be the same as the input. While this type of model is not meaningful for a single copy, it is valuable for the overall sample set. The distribution of the samples in the data set can be well learned, which can not only compress the data set to achieve the power of extracting the principal components of the data, but also fit with the characteristics of the data set to realize the function of generating simulated data.
The intermediate state after the transformation process can output a better feature description than the original data, which makes the self-encoding have strong feature learning ability, so the processing results of the intermediate state are often used to fit AI tasks.
2.3.1 Autoencoding and PCA Algorithm
In unsupervised learning, a common form is to train an encoder to encode the original dataset into a fixed-length vector that retains as much important information as possible from the original data. The auto-encoder formed by training can capture the most important factors representing the input data and find the main components that can represent the original information. (If the activation function in the autoencoder uses a linear function, it is a PCA model.)
2.3.2 Autoencoding and Deep Learning
The concept of encoder is widely used in deep learning models. For example, the backbone network model in target recognition and semantic segmentation can be understood as an encoder model. In classification tasks, the network structure before the output layer can be understood as an independent encoder model.
2.3.3 Types of Autoencoding Neural Networks
On top of the basic self-encoding, some better-performing self-encoding neural networks are derived, such as variational self-encoding neural networks, conditional variational self-encoding neural networks, and so on. Their input and output are no longer purely focused on a single sample, but autoencoder fitting for the distribution of the entire sample, which has better generalization ability.
3 Variational Autoencoder Neural Networks
The variational auto-encoding neural network learns the law of the sample, and the neural network not only has the function of reconstructing the sample, but also has the function of imitating the sample.
3.1 Decoding and encoding process of variational autoencoder neural network
The variational self-encoding neural network changes the distribution of samples during the encoding process (variation can be understood as changing the distribution), and the law of learning samples is the distribution of learning samples. Assuming that we know the distribution function of the sample, we can randomly take a sample from this function, and then perform the forward transmission of the network decoding layer to generate a new sample.
3.2 The Mystery of Variational Autoencoder Neural Networks
In order to obtain the distribution function of the sample, the training purpose of the model will be to generate the encoder into a Gaussian distribution data set by adding a constraint item, arbitrarily take the relevant data according to the Gaussian distribution mean and variance rules, and input the data into the decoder Restore to sample.
4 Conditional Variational Autoencoder Neural Networks
4.1 Problems with Variational Autoencoder Neural Networks
Although the variational autoencoder neural network can generate a sample, it can only output samples of the same category as the input picture. Exactly, we do not know which class the generated samples belong to.
4.2 The Role of Conditional Variational Autoencoder Neural Networks
The conditional variational autoencoder neural network is optimized on the basis of the variational autoencoder neural network, which allows the model to generate samples according to the specified categories.
4.3 Implementation of Conditional Variational Autoencoder Neural Networks
The conditional variational auto-encoding neural network only makes one change on the basis of the variational auto-encoding neural network: when training and testing, a label vector ((one-hot type) is added.
4.4 Principles of Conditional Variational Autoencoder Neural Networks
A condition is added to the variational auto-encoding neural network, and the label factor is added when the network learns the image distribution, so that the specified image can be generated according to the value of the label.