In the process of fitting the data with machine learning algorithms, often start unsatisfactory results, such as AUC value is not high enough, then we need to tune the model, then what tuning direction? There is no tuning methodology to follow? The answer of course is yes, bias and variance of these two indicators will be able to play a role in guiding tuning.
Bias
Let's look at an example, suppose laboratory data collected weight and size of a mouse, we can build a model to predict the size of the mice's weight by entering mice, part of the data scatter plot below. Before training, we will split the data into two parts, red point of the training set, a green dot indicates that the test set:
Then we use the training data to fit the two models, the first model uses a linear algorithm, as follows:
Can be seen that a linear model is not a good depiction of real data, we generally use the MSE (Mean Squared Error) to quantify this fitting ability, i.e. the mean squared difference between the predicted and actual values.
Next we train a second more complex model, the curve of the model are as follows:
The second model is a perfect fit the training data, which uses this model to predict the training data, equal to the predicted value and the actual value obtained.
At this point, we then give the definition of bias is not difficult to understand the:
Bias measure of the difference between the training phase, the machine learning algorithms and real data.
As can be seen from the above example, the Model 2 Model far below the bias of a bias.
Variance
After completion of the training models, we also need to use the model evaluation test set, the figure is a model of evaluation results, we blue dotted line test results, the difference of actual and predicted values (MSE may also be used to measure ):
Similarly, two models of the evaluation results is as follows:
And Bias In contrast, a model predicting the effect is far better than the Model 2, which shows two models predictive ability is not stable, we try to give Variance is also a definition:
Variance represents the degree of deviation between the predicted effect, the greater the degree of variation between different test set, the greater the variance, the smaller the contrary.
Obviously the larger variance model II; and for the different test set, a model prediction accuracy is very close, we can say that a smaller model of variance.
Bias & Variance
The following diagram is taken from Scott Fortmann-Roe's a blog , it can be a good tune depicts our direction in machine learning, where the upper left corner is the ideal model, which is the ultimate goal, if it is not, you should move towards High Bias lower left corner + Low Variance efforts.
The figure above, the case has been referred to the upper right corner over-fitting (Overfit) , which represents the performance of your model in training is very good, but the performance in the testing process and very poor, Model 2 above is a typical the over-fitting case. There certainly have been fitted underfitting (Underfit) , it means that your model can not portray a good sample data. At the same time, overfitting general correspondence you use a more complex model, while the less fit and generally corresponds to a simple model. In many cases, we say that the model tuning, actually refers to:
To strike a balance between simple and complex model model .
How to do it? Here are some rules of thumb:
- How to deal with the larger problem of variance
- Reducing the number of features
- Use a simpler model
- Increase your training data set
- Use regularization
- Random factor is added, for example, by using bagging and boosting methods
- How to deal with the larger problem of bias
- Increasing the number of features
- The use of more complex models
- Remove regularization
reference:
Machine Learning Fundamentals: Bias and Variance
related articles:
Dichotomous assessment, from the confusion matrix Speaking of
understanding "cross-validation" (Cross Validation)
Reproduced in: https: //www.jianshu.com/p/e5c2af344327