Machine Learning - support vector machine (SVM) algorithm principle and the advantages and disadvantages

First, the principle of support vector machine (SVM) algorithm

　　SVM (Support Vector Machine, often referred to as SVM) is a method of supervised learning, and can be widely used statistical classification and regression analysis. It is a vector mapped to a higher dimensional space, the establishment of a maximum interval hyperplane in this space. In hyperplane parted data has two hyperplanes parallel to each other, the two spaced parallel hyperplanes hyperplane to maximize distance. Assumed that the greater the distance or gap between the parallel hyperplanes, the smaller the total error of the classification.

　　

 

　　For linearly separable support vector machine to solve the problem may actually be converted into a belt constraint optimization problem to solve:

　　　　Reasoning:

　　　　　　     

                                                                                  

                                                                                               

                                                                                              

　　　　　　　　　　　　　　　　　result:

　　　　                                                   

 

　　For linear SVM inseparable practically solve the problem can be transformed into a strip of soft-margin constraint optimization problem to solve:

　　　　                         

　　Algorithms advantages:　　

　　(1) using the kernel function can be mapped to a high-dimensional space

　　(2) the use of nuclear non-linear function of classification can be solved

　　(3) classification idea is simple, it is to maximize the sample spacing and decision-making plane

　　(4) better classification results

　　Algorithms disadvantages :

　　(1) SVM algorithm is difficult to implement large-scale training sample

　　(2) with a multi-classification SVM to solve difficult problems

　　(3) sensitive to missing data, and the choice of kernel function parameters sensitive　　

Second, code implementation

import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
from sklearn.preprocessing import StandardScaler
from sklearn.svm import LinearSVC

iris = datasets.load_iris()

X = iris.data
y = iris.target

X = X[y<2,:2]
y = y[y<2]

plt.scatter(X[y==0,0], X[y==0,1], color='red')
plt.scatter(X[y==1,0], X[y==1,1], color='blue')

standardScaler = StandardScaler()
standardScaler.fit(X)
X_standard = standardScaler.transform(X)

svc = LinearSVC(C=1e9)
svc.fit(X_standard, y)

def plot_svc_decision_boundary(model, axis):
    
    x0, x1 = np.meshgrid(
        np.linspace(axis[0], axis[1], int((axis[1]-axis[0])*100)).reshape(-1, 1),
        np.linspace(axis[2], axis[3], int((axis[3]-axis[2])*100)).reshape(-1, 1),
    )
    X_new = np.c_[x0.ravel(), x1.ravel()]

    y_predict = model.predict(X_new)
    zz = y_predict.reshape(x0.shape)

    from matplotlib.colors import ListedColormap
    custom_cmap = ListedColormap(['#EF9A9A','#FFF59D','#90CAF9'])
    
    plt.contourf(x0, x1, zz, linewidth=5, cmap=custom_cmap)
    
    w = model.coef_[0]
    b = model.intercept_[0]
    
    # w0*x0 + w1*x1 + b = 0
    # => x1 = -w0/w1 * x0 - b/w1
    plot_x = np.linspace(axis[0], axis[1], 200)
    up_y = -w[0]/w[1] * plot_x - b/w[1] + 1/w[1]
    down_y = -w[0]/w[1] * plot_x - b/w[1] - 1/w[1]
    
    up_index = (up_y >= axis[2]) & (up_y <= axis[3])
    down_index = (down_y >= axis[2]) & (down_y <= axis[3])
    plt.plot(plot_x[up_index], up_y[up_index], color='black')
    plt.plot(plot_x[down_index], down_y[down_index], color='black')
    
plot_svc_decision_boundary(svc, axis=[-3, 3, -3, 3])
plt.scatter(X_standard[y==0,0], X_standard[y==0,1])
plt.scatter(X_standard[y==1,0], X_standard[y==1,1])

输出结果：