Support vector machine regression (SVR) is an application model of support vector machine in regression problems. There are many variants of SVM regression models based on different loss functions . This article only describes the SVR model for insensitive loss function.
main idea
Find a separating hyperplane (hypersurface) that minimizes the expected risk.
-SVR
- loss function
- loss function, which is when the error is less than
, the error can be ignored. Conversely, the error is
. As shown in the figure:
based on
- The SVR of the loss function is called
-SVR.
The optimization problem is as follows:
-SVR
same
- Support vector machine classification, set another parameter
to adjust the number of support vectors.
The optimization problem is as follows:
It is set freely by the user, so directly set the
called
,
called
, the optimization function is equivalent to:
support vector
Intuitively, the support vector is the sample that plays a role in the calculation of the final w and b (
). Then according to
Insensitive function image, the insensitive area is like a "pipe". Sample correspondence within the pipeline
, is the non-support vector; the one located on the "tube wall" is the boundary support vector,
, is the boundary support vector; outside the "pipe" is the non-boundary support vector,
, is a non-boundary support vector (outliers are often selected from non-boundary support vectors during outlier detection);
Note: The picture is from the LIBSVM guidance document . Please correct me if I am wrong.