This article comes from the public account "AI Great Principles"
Convolution and pooling layers are very common in neural networks.
Because of the habit, it is easy to be mistaken for essential.
So can convolution replace pooling?
Can pooling replace convolution?
To answer this question, one must delve into their nature to see if they are not available, and what would happen without them.
1. The essence of convolution
Convolution is an operation. The feature map is multiplied by the relative position of the convolution kernel and then added to obtain a number.
The role of convolution: feature extraction.
2. The essence of pooling
The pooling layer is sandwiched between consecutive convolutional layers to compress the amount of data and parameters and reduce overfitting.
The role of pooling: feature selection.
Or feature dimensionality reduction. An image contains a lot of information and many features, but some information is not very useful or repetitive for us to do image tasks. We can remove this kind of redundant information and extract the most important features. A major role of pooling operations.
3. Discussion on whether it can be replaced
Convolution is a process of feature extraction. Convolution can usually keep the size of the feature map unchanged, and the number of convolution kernels can be used to increase the number of channels.
The pooling layer can usually reduce the size (width, height) of the feature map, while the number of channels remains unchanged.
However this is the usual practice.
The essence of the convolution operation is the dot product, which is a mathematical operation of a matrix and another convolution kernel matrix.
The size of the feature map can be reduced through operations such as step size and padding to achieve the function of reducing the feature map of the pooling layer.
And can the pooling layer achieve the function of convolution?
The essential difference between the two is that the operation of the pooling layer is an internal operation of the feature map. The so-called step size and convolution kernel are empty concepts, a virtual concept, not real, and there is not really a convolution kernel. operate with it.
Therefore, convolution is playing with others, and pooling is playing with yourself.
There is also a fatal problem here.
This "other" is our model.
There are parameters in the model, and this parameter is the parameter of the convolution kernel.
The whole model cannot have no parameters, so it cannot have no convolution, so the pooling layer cannot completely replace the convolutional layer.
From another perspective, convolution is feature extraction, and pooling is feature selection.
Feature extraction is fundamental, and feature selection is optimization and icing on the cake.
Convolution is irreplaceable. If there is no pooling, it is just a problem of poor effect.
Therefore, convolution can replace pooling. This replacement means that it can still work instead of it, not that it can replace all its functions.
In fact, without pooling, it may be easy to overfit, because the original features extracted from the features are too complex, and the feature selection of pooling is the process of retaining important features and discarding other features.
Pooling cannot replace convolution because it cannot work without a convolution kernel.
If it is all pooling, the image can also be feature extracted to obtain a feature map (the maximum value is regarded as a feature in a certain sense), but only this map is obtained, which has nothing to do with the model.
4. Summary
Pooling layers cannot completely replace convolutional layers.
Convolutional layers can replace pooling layers.
——————
Talking about it is enough, and the principles of AI are nuanced
Scan the "AI Principles" below and select the "Follow" official account
—————————————————————
—————————————————————