Li Haha’s Pattern Recognition Notes [Part 4: Support Vector Machine]
Principles of Support Vector Machine
Support vector:
Support Vector is the sample point (vector point) closest to the classification decision boundary.
Through the support vector, the weight vector under the maximum classification interval can be found, so as to achieve a classification decision function with ideal price comparison.
Support Vector Machine: Support Vector Machine (SVM)
Guidelines for Minimizing Structural Risk
What is the optimal solution?
The weight vector w that makes the classification interval d obtain the maximum value.
Advantages of SVM:
- It does not require a large number of samples, and the final solution is only determined by a small number of support vectors.
- It has strong generalization ability.
Two risk descriptions:
- Empirical risk: the proportion of samples in the misclassified sample set
- Structural risk: the probability that a trained classifier will misclassify when facing unknown samples
If you blindly pursue the minimum experience risk, then over-fitting may occur. For example, when the boundary is a "hound-tooth interaction" shape, the result is not very good.
VC dimension:
Definition: the classification ability of a class of functions.
The VC dimension of the linear classifier is 3, and the VC dimension of the quadratic function classifier is at least 4.