CNN Notes + Backpropagation

CNN pooling layer

1. Both max and mean are extracting regional features, filtering out unnecessary information and retaining the main features, so better identification can be performed at the abstract level.

2. Pooling layer is also called subsampling layer. When the data passes through the convolution layer, the feature dimension may still be large, that is, there are still many feature parameters, so the problem of overfitting is prone to occur. Then, by pooling subsampling, the dimension of the feature can be greatly reduced and the risk of overfitting can be reduced.

2. The main benefit of the pooling layer is that it provides an invariance .
Invariance means that if the cat in the classification image is moved to the left or right, the cat can still be classified, that is, no matter where the final position is, it can be classified correctly. In object detection, sound recognition, we all want to have translation invariance.

-----------reason:

backpropagation

First, we randomly initialize a series of w, b, through forward propagation, layer by layer calculation, and finally calculate our predicted value. Then we will define a loss function to measure the difference between the true value and the predicted value. Then through backpropagation to find the most suitable w, b parameters to minimize the loss function. The w and b of each layer are updated by gradient descent.