SVM classification is to find a plane, so that the support vectors of the two classification sets or all the data (LSSVM) are the farthest from the classification plane;
SVR regression is to find a regression plane, so that all the data in a set have the closest distance to the plane.
SVR is the English abbreviation of support vector regression, which is an important application branch of support vector machine (SVM).
The traditional regression method considers the prediction correct if and only when the regression f(x) is exactly equal to y, as commonly used in linear regression ( f(x) − and )2to calculate its loss.
Support vector regression believes that as long as f(x) does not deviate too much from y, the prediction can be considered correct and no loss is calculated. Specifically, the threshold α is set., computes only | f(x)−y|>αThe loss of the data points, as shown in the figure below, we all believe that the model prediction is accurate for the data points in the shadow part, and only calculate the loss of the data points outside the shadow:
Data processing
preprocessing.scale() function:
scale() is used to scale the original sample, the range can be set by yourself, usually [0,1] or [-1,1].
The purpose of scaling is mainly
1) to prevent a feature from being too large or too small, thus playing an unbalanced role in training;
2) for calculation speed. Because in the kernel calculation, the inner product operation or the exp operation will be used, and the unbalanced data may cause calculation difficulties.
For the SVM algorithm, we first import the SVR module in sklearn.svm. SVR() is the method used by the SVM algorithm for regression (that is, the method used when the input label is a continuous value). The SVR mode is determined by the following statement (select several important parameters for testing. Randomly select one The stock starts a test of the relevant parameter selection).
svr = SVR(kernel='rbf', C=1e3, gamma=0.01)
kernel: The type of kernel function , commonly used are 'rbf', 'linear', 'poly', etc. As shown in Figure 4-1-2-1, it is found that the fitting effect of the function model is the best when the rbf parameter is used.
C: penalty factor
C represents how much you value outliers, the larger the C, the more important you are, the less you want to throw them away. When the value of C is large, the penalty for error classification increases, and when the value of C is small, the penalty for error classification decreases. When C is larger and approaches infinity, it means that the existence of classification errors is not allowed, and the smaller the margin, the easier it is to overfit; when C is close to 0, it means that we no longer care about whether the classification is correct, and only the larger the margin is required, Easy to underfit. It was found to be most suitable when 1e3 was used as shown.
gamma:
are the kernel coefficients of 'rbf', 'poly' and 'sigmoid' and the value of gamma must be greater than 0. With the increase of gamma, there is a situation where the classification effect is poor for the test set but good for the training classification, and it is easy to over-fit the generalization error. As shown in the figure, it is found that the accuracy is the highest when gamma=0.01.
The data we use this time is the salary corresponding to different promotion levels within the company
Let's take a look at how it is implemented in Python
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
dataset = pd.read_csv('Position_Salaries.csv')
X = dataset.iloc[:, 1:2].values
# 这里注意：1:2其实只有第一列，与1 的区别是这表示的是一个matrix矩阵，而非单一向量。
y = dataset.iloc[:, 2].valuesNext, process the data:
# Reshape your data either using array.reshape(-1, 1) if your data has a single feature
# array.reshape(1, -1) if it contains a single sample.
X = np.reshape(X, (-1, 1))
y = np.reshape(y, (-1, 1))
# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
sc_y = StandardScaler()
X = sc_X.fit_transform(X)
y = sc_y.fit_transform(y)
Next, get to the point and start SVR regression:
# Fitting SVR to the dataset
from sklearn.svm import SVR
regressor = SVR(kernel = 'rbf')
regressor.fit(X, y)
# Predicting a new result
y_pred = regressor.predict(sc_X.transform(np.array([[6.5]])))
# 转换回正常预测值
y_pred = sc_y.inverse_transform(y_pred)
# 图像中显示
plt.scatter(X, y, color = 'red')
plt.plot(X, regressor.predict(X), color = 'blue')
plt.title('Truth or Bluff (SVR)')
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()
# Visualising the SVR results (for higher resolution and smoother curve)
X_grid = np.arange(min(X), max(X), 0.01) # choice of 0.01 instead of 0.1 step because the data is feature scaled
X_grid = X_grid.reshape((len(X_grid), 1))
plt.scatter(X, y, color = 'red')
plt.plot(X_grid, regressor.predict(X_grid), color = 'blue')
plt.title('Truth or Bluff (SVR)')
plt.xlabel('Position level')
plt.ylabel('Salary')
plt.show()
