1、Dropout
In order to prevent overfitting in the training phase, the neurons are removed immediately (throwing them completely randomly), as shown in the following figure:
2、DropConnect
There is no connection to apply dropout on the neurons, but to the weights and biases that connect these neurons (throw only at the connection, the neuron does not throw), as shown in the following figure:
3、Spatial Dropout
(Press the channel to throw immediately)
4、Cutout
Randomly select a point in the image as the center point to cover a fixed-size square (throw it randomly on the input layer according to the spatial block)
5. DropBlock (discard blocks in the neighborhood of the unit)
( Every feature map is randomly thrown according to the spatial block ) that is, an adjacent area in the feature map is put together and dropped. As shown below:
6, reference