1.1 Overview
The Fully Connected Layer is generally located at the end of the entire convolutional neural network , and is responsible for converting the two-dimensional feature map output by the convolution into a one-dimensional vector , thus realizing the end-to-end learning process (ie: input an image or A piece of speech, output a vector or message). Each node in the fully connected layer is connected to all nodes in the previous layer, so it is called a fully connected layer. Due to its fully connected characteristics, the parameters of the general fully connected layer are also the most.
1.2.1 Main functions
The main function of the fully connected layer is to integrate the features calculated by the previous layer (convolutional layer, pooling layer, etc.) into one value. Its advantage is to reduce the influence of feature positions on classification results and improve the robustness of the entire network. .
1.2.2 Personal understanding
5×3×3As shown in the figure above, a network generates a feature map before the fully connected layer. We only need to use 5 convolution kernels to perform convolution operations with the output of the activation function, and then add the values of the five outputs. The output value of a fully connected layer can be obtained. If the result is an N-dimensional vector, N×5a 3×3convolution kernel is required. Coupled with the weight corresponding to the summation operation, the number of parameters is very large, so the fully connected layer is generally only used after the network.
1.3 The difference between the fully connected layer and the convolutional layer
The convolutional neural network (CNN) consists of an input layer, a convolutional layer, an activation function, a pooling layer, and a fully connected layer, namely (INPUT-CONV-RELU-POOL-FC).
1.3.1 Convolution layer: use it for feature extraction.
For example, the input image is 32 * 32 * 3, 3 is its depth (ie R, G, B), and the convolutional layer is a 5 * 5 * 3filter (filter). Note here: the depth of the receptive field must be the same as the depth of the input image. By convolving a filter with the incoming image, we can get a 28*28*1feature map of a.
1.3.2 Fully connected layer
Each node of the fully connected layer is connected to all the nodes of the previous layer, which is used to integrate the features extracted earlier.
Due to its fully connected characteristics, the parameters of the general fully connected layer are also the most . The weight matrix of the fully connected layer is fixed , that is, the input of each feature map must be of a certain size (that is, the size that can be multiplied with the weight matrix), so the size of the input image of the network must be fixed at the beginning to ensure The size of the feature map sent to the fully connected layer matches the weight matrix of the fully connected layer.
The two places with the densest connections in the figure below are the fully connected layer. It is obvious that there are indeed many parameters in the fully connected layer.
1.4 Fully connected layer and forward calculation
In the forward calculation process, which is a linear weighted summation process , each output of the fully connected layer can be regarded as each node of the previous layer multiplied by a weight coefficient W, and finally added with a bias value b gets, ie. As shown in the first fully connected layer in the above figure, the input has 50 4 4 neuron nodes, and the output has 500 nodes, so a total of 50 4 4*500=400000 weight parameters W and 500 bias parameters b are required .
1.5 Fully connected layer and backpropagation
1. Deriving the output of the previous layer (that is, the input of the current layer)
2. Deriving the weight coefficient W
3. Deriving the bias coefficient b
The specific mathematical steps are omitted, and the specific calculation details can be clicked on this link
There is also a very vivid explanation: click here