Q1 over-fitting problem
Training set good performance, poor performance of the test set. Quite robust. The following are two examples (a regression problem, a classification problem)
The first figure was significantly less fit, third figure over-fitting, fitting function complex, although with a very low cost function for the training set, but the ability to apply to the new sample is not high, drawing two is both balanced.
Solution:
(1) discard some can not help us correct prediction feature. Workers can use to choose which features to retain, or use some model selection algorithm to help (PCA);
(2) regularization. Retention feature is known, but reduce the size parameter.
Q2 cost function
Wherein λ is called the regularization parameter.
After possible for such model and the original model of the regularization process is as follows:
Not [theta] 0 regularization.
Q3 regularization of linear regression
For j = 1,2,3 ...... are:
As can be seen, it regularized gradient descent method of linear regression is that, every time the value of θ so that an additional reducing value update rule based on the original algorithm.
Calculating a global minimum formal method
Q4 regularized logistic regression model
Normalized with different normalization linear regression different logistic regression models that h (x) of
vocabulary
overfitting problem --- overfitting regularization --- regularization