[Machine Learning]-Support Vector Machine

Welcome to reprint, please indicate the source : https://blog.csdn.net/qq_41709378/article/details/106241378
———————————————————————— ————————————————————————————

Support vector machines (SVM) is a two-class classification model. Its basic model is a linear classifier with the largest interval defined in the feature space. The largest interval makes it different from the perceptron; the support vector machine also includes kernel techniques, which makes it a substantial nonlinear classifier. The learning strategy of support vector machines is to maximize the interval, which can be formalized as a problem of solving convex quadratic programming (convex quadratic programming), which is also equivalent to the problem of minimizing the regularized hinge loss function. The learning algorithm of support vector machine is the optimal algorithm for solving convex quadratic programming.

method	Applicable issues	Model characteristics	Model type	Learning strategy	Learned loss function	Learning algorithm
Support Vector Machines	Two categories	Separation hyperplane, nuclear technique	Discriminant model	Minimize the loss of regular hinges and maximize the soft interval	Hinge loss	Sequential Minimal Optimization Algorithm (SMO)

1 What's the SVM?
First of all, a support vector machine is not a machine, but a machine learning algorithm.
　　1. Support Vector Machine, commonly known as support vector machine, is a supervised learning algorithm, which belongs to the category of classification.
　　2. In the application of data mining, it corresponds to and differs from the Clustering of unsupervised learning.
　　3. Widely used in Machine Learning (machine learning), Computer Vision (computer vision, just called cv) and Data Mining (data mining).

2 What is "Machine"?
Classification Machine is a classifier. There is nothing to say about this. It can also be understood as an algorithm. In the field of machine learning, the word "machine" is often used to indicate an algorithm.

3 What is "support vector"?
Popular understanding: support vector (support vector) means that certain points in the data set have a special location. For example, for the line x+y-2=0, the area above the line x+y-2>0 is all type A, and the area below x+y-2<0 is all type B. When we look for this line Generally speaking, it depends on the two types of data gathered together. The points of their respective most edge positions are the points closest to the dividing line, and the other points cannot play a role in determining the final position of the line, so I will call these points "support points" (meaning useful points), but in mathematics, there is no such saying. In mathematics, points can be called vectors. For example, a two-dimensional point (x, y) is two-dimensional. Vector, the three-dimensionality is the three-dimensional vector (x, y, z). So "support point" was changed to "support vector", which sounds more professional.
Unpopular understanding: These points on the maximum margin (maximum margin) are called "support vectors". What I want to add is why these points are called "support vectors" because of the expression of the final classification machine (classifier) The formula only contains the information of these "support vectors", and has nothing to do with other data points:

in this expression, only the coefficients of the support vector $a_i$It is not equal to 0, so it is very important to find the separating hyperplane at the end.
If you still don't understand it well, it doesn't matter. Look at the figure below. The "support vector" is the point circled with a red box in the figure.

4 What problems can support vector machines solve?
　　The most basic application is data classification, especially for non-linear inseparable data sets. Support vector machines can not only classify non-linear separable data sets, but also classify non-linear inseparable data sets (because in real scenes, sample data is often linearly inseparable).

Realistic scenario 1: Most of the sample data is linearly separable, there are no noise points, and hard-spaced support vector machines can be used for classification; but there are only a small amount of noise or peculiar points in the sample, and these noises or peculiarities are linearly separable => Use the soft interval method of the support vector machine to classify;
Realistic scenario 2: The sample data is completely linear and inseparable => Introduce the kernel function to convert the low-dimensional inseparable non-linear data set into the high-dimensional separable data set, and non Linear support vector machine for classification;

5 Next, analyze the support vector machine from four aspects.
[Machine learning] -Lagrangian multiplier method and KKT condition .
[Machine learning] -Support vector machine (1): Hard interval support vector machine .
[Machine learning]-Support vector machine (2): Soft interval support vector Machine .
[Machine Learning]-Support Vector Machine (3): Sequence Minimization Algorithm (SMO Algorithm) .
[Machine Learning]-Support Vector Machine (4): Kernel Function .
[Machine Learning]-Support Vector Machine (5): SVM program-Python .