Machine Learning - Regular + Regression and Classification Analysis

Machine Learning - Regular + Regression and Classification Analysis

This article is himself an early article about regularization and return supplements. At that time, when writing the article themselves also just started learning machine, a lot of knowledge is understanding, but lack a certain height from the summary. Combined with this point of view the original thing I read is not easy to chaos.

First be clear, is not a return of the patent, regression and classification can all be used. Use in the regression regularization is ridge regression (L2 regularization) and Lasso (L1 regularization), use is regularization our common loss function in the category (in fact, there are applications one aspect, know almost the a main mentioned, is configured to direct loss function, for example, on a regular L1 configured such that: img, L2 of a regular constructed such that it: img), e.g. sklearn parameter selection in the model has a corresponding.

As L1 and L2 regular trade-offs, you can see some of the other answers I mentioned above know almost answered:

Xu Jian Zi

L0 norm: the number of nonzero elements in the vector.

L1 norm (Lasso Regularization): vector absolute values ​​of each element.

L2 norm (Ridge Regression): vector square and the square root of each element.

L0 and L1 norm norm are able to achieve that the parameters sparse objects, but harder L0 norm optimal solution, optimal L1 norm is convex and easier to solve similar L0, it is widely used.

L2 norm primary role is to prevent over-fitting the model to improve the generalization ability of the model.

Wei and Jin

img

Blue line L1, L2 is a red line, obviously, L1 is more tolerant of extreme values ​​distribution. So if the loss of data entry using the L1 Norm, obviously, L1 Norm no L2 Norm less sensitive to outlier; if regularization loss term use L1, then the learned parameters tend sparse, use L2 Norm is no such tendency.

In practice, the data scientist Quaro experience, the practical application process, L1 nrom showed little better than when L2 norm, L2 norm preferably used is a better choice.

In fact, from one side, the difference between classification and regression to reflect on whether the threshold set threshold determination of return is classified. In fact, think about it, classification and regression problems in machine learning and deep learning almost half a dozen, just because they now do is target computer vision, so more emphasis on the classification model, in other areas, such as forecasting stock and so on like scene, regression is still a lot of difference between the two is reflected in the design of network structure, but if there are words in the context of supervised learning, the difference really is not too large, for example, this problem know almost said .

Guess you like

Origin www.cnblogs.com/jiading/p/12104854.html