Third, linear support vector machine
Linearly separable support vector machines can only linear separable training data set, but the reality of the data set is not completely pure, there may be some noise data in it, how to deal with this situation, linear support vector machine to be shipped born.
Online support vector machine which, for each sample point we set up a relaxation factor to relaxation of those singular points, for a certain singularity is unable to meet the objective function, that is, unable to reach greater than or equal to claim 1, in in this case, we will add this factor to the relaxation, he is greater than or equal to satisfy constraints 1. For each slack variable, representing an increase of complexity of the model, we have for each slack variable increase a cost of $ \ xi_ {i}, i = 1,2,3, ..., N $
1, according to the above definition, in fact, the original problem has been born
$argmin_{w,b,\xi}\frac{1}{2}|w|^{2}+C\sum_{i=1}^{N}\xi_{i}$
$s.t. y_{i}*(w*x_{i}+b) \ge 1-\xi_{i}$
$\xi_{i} \ge 0,i=1,2,3,...,N$
2, dual problem
By linearly separable SVM similar derivation method to first construct the Lagrangian function and derivative, then poured simplification, we can get the original problem of dual form
$argmin_{\alpha}\frac{1}{2}|w|^{2}-\sum_{i=1}^{N}\alpha_{i}$
$s.t. \sum_{i=1}^{N}\alpha_{i}*y_{i}=0$
$0 \le alpha_{i} \ge C,i=1,2,3,...,N$
How has the dual problem to restore the original problem, and restore method is linearly separable support vector machine is the same.
Fourth, non-linear SVM
The main technique is the use of a nuclear
Structure operation after the mapping between the two spaces in the scope of this core skill, rather than focus on ways of mapping
$ K (x, y) = \ phi (x) \ bullet \ phi (y) $
Where $ \ bullet $ is the meaning of the dot
Polynomial kernel functions common core, and Gaussian kernel (RBF)
reference
[1] statistical learning methods, Li Hang