A term
1, under-fitting and over-fitting:
Underfitting: data model predictions and the actual data that much difference (Figure 1-1 left);
Overfitting: Results for the existing model predicts real actual data has a good accuracy, but the model can not be generalized to other new data (to the right in FIG. 1-1);
【Picture 1-1】
2, the training set, validation set, the test set:
Training sets: training data for gradient descent, so that the error is minimized;
Validation set: test the accuracy of the current model, we thereby adjusting the number of iterations, learning rate ...;
Test set: to test the accuracy of the final data set of;
3, regularization:
Restrictions objective function, in order to avoid over-fitting;
4, deviation, variance
Deviation: the predicted results and the actual error;
Variance: model data of different batches of the same type (e.g. training set, validation set), the degree of fluctuation of the output;
Second, our approach to the problem overview
1, for more training data - high variance (overfitting);
2, to reduce the feature values - higher variance (overfitting);
3, the feature value increases - high deviation (underfitting);
4, narrow items λ-- high positive deviation (underfitting) of the;
5, increasing the positive λ-- higher variance (overfitting) of the entry;
Third, the linear regression cost function regularization formula
Wherein, J (θ) is the cost function, x, y for the training data, the model parameters [theta], the end of the expression is a regularization term to refine each parameter.
The reason is the emergence of over-fitting model function is too complex, when coupled with a regularization term, the size can be well controlled parameters, simplified model.
When [lambda] greater, so that the parameter will be smaller, the model functions can be more simplified, easy to spread model, generalization.
Fourth, the logistic regression cost function regularization formula
