Learning algorithm've learned so far: linear regression and logistic regression
First, overfitting
Examples: linear fit (Rate)
(1), 
: poor fitting, having a high deviation
(2), 
: fit well
(3), : 
over-fitting, with a high variance (usually too many variables, fitting out of the picture looks very consistent with the data set, and the cost function close to zero, but it can not be generalized to the new data set)
Second, to address over-fitting
(1) reduce the number of variables
(2), regularization
Keep all the characteristic variables, but reduce the size or magnitude of the parameters θ
Third, the regularization
Ideas:
Above diagram, for example, to add a penalty term cost function 
,
The cost function becomes
Coefficient before penalty term to 1000, in order to minimize a cost function, θ_3 and θ_4 close to 0, this function is assumed that 
the two almost zero (or corresponding to a quadratic function), thereby reducing the variable reaches without better fitting effect, this is the regularization of ideas.
Third, the regularization cost function
Many times, we do not know which variables are relatively low correlation, it can not be accurately reduced their values, so we consider the cost function to add additional regularization term to narrow all the parameters,
This usually does not require θ_0 regularization, which is prescriptive
If the regularization parameter is too large, the degree of punishment for parameter too large, so that the parameters are close to zero