0. Preface
The problem of overfitting is very common in machine learning. That is, the model performs well on the training set data, but has poor generalization ability on the test set and the new data set. This is called overfitting.
1. Reasons for overfitting
There are many reasons for overfitting. The common ones are as follows:
1. The training set data is too small, and it is easy to overfit.
2. The data distribution of the training set and the test set is inconsistent, which is easy to overlook. For example, some algorithms require the data set to conform to the Gaussian distribution, and the training set also meets the conditions, but the online data distribution has changed since it went online, and the effect will certainly not be very good.
3. The model itself is particularly complicated. For example, in a tree model, if the number of trees is too large and the depth is too large, it is easy to overfit.
2. Ways to solve the overfitting problem in deep learning
1. For the problem of too little training data, you can increase the training data.
2. Adding to the problem of too high model complexity can reduce the model complexity. For example, reducing the number of layers or reducing the number of neurons can reduce the size of the network.
3. Regularization, which is a common method to solve over-fitting.
4. Dropout, the neural network randomly drops the neurons in the neural network during each iteration. Whenever we discard different neurons, it is equivalent to retraining a new neural network.
5. Early stop, during the training process, if the training error continues to decrease, but the test error has begun to increase, you can stop training at this time.
6. Integrated learning, combining multiple models can reduce the risk of overfitting a few models.
7. BN, Batch Normalization. Between each layer of CNN, a regularization layer that adjusts the weight of the neuron to a standard normal distribution is added.