Convolutional neural network is to use the convolution kernel to extract the features of the input features, and then send the extracted features to the fully connected network for recognition and prediction.
The main modules of the convolutional neural network:
Convolutional (convolutional) -> batch standardization (BN (batch noemalization)) -> activation (Activation) -> pooling (Pooling) -> fully connected (FC)
feature extraction includes: convolution, batch normalization, activation and pooling Four-step
convolution is the feature extractor, which is CBAPD (convolutional, batch normalization, activation, pooling, dropout)
** Fully connected NN (neural network): **Each neuron has a connection relationship with every neuron in the adjacent layer before and after, the input is the feature, and the output is the predicted result.
Number of parameters:
In the neural network (due to the increase in the number of hidden layers, the network scale is too large) too many parameters to be optimized can easily cause the model to overfit.
In order to reduce the parameters to be trained, feature extraction is performed on the original image in practical applications, and then the extracted features are sent to the fully connected network.
Convolution can effectively extract image features. Generally, a square convolution kernel is used to slide on the input feature map according to the specified step size, and traverse a pixel in the input feature map. At each step, the convolution kernel will have an overlapping area with the input feature map, and the corresponding elements of the overlapping area will be multiplied, summed and biased to a pixel of the output feature.
Note : To make the number of channels of the convolution kernel consistent with the number of channels of the input feature map (because if you want the convolution kernel to match the corresponding points of the input feature map, you must make the depth of the convolution kernel consistent with the depth of the input feature map )
The depth of the input feature map (the number of channels) determines the depth of the current convolution layer
The number of convolution kernels in the current layer determines the depth of the output feature map of the current layer.
Receptive Field : The size of each pixel in each output feature map of the convolutional neural network, which is mapped on the original input image.
Since the convolution calculation will reduce the size of the input feature map, in order to ensure that the size of the input feature map remains unchanged, all 0 padding can be used.
Tensorflow describes all 0 padding with parameter padding='SAME' or padding='VALID'
Batch Normalization (Batch Normalization, BN)
standardization : make the data conform to the distribution
of 0 as the mean and 1 as the standard deviation. Batch standardization : for a small batch of data (
Batch ), the normalization process (usually between the convolution operation and the activation operation) is the i-th pixel in the output feature map of the k-th convolution kernel after batch normalization.
The BN layer is located after the convolutional layer and before the activation layer.
Convolutional (convolutional)----->Batch Normalization (BN)-------->Activation
Pooling (Pooling)-Pooling is used to reduce the amount of feature data Pooling
(including maximum pooling) of and mean pooling ), the maximum value of the texture images can be extracted pooled, the pooled mean background characteristics can be retained.
Abandonment : During neural network training, some neurons in the hidden layer are often temporarily discarded from the neural network according to a certain proportion. When used, all neurons are restored to the neural network.