1 Visualization of training set ex6data2.mat
Plot the training set function plotData.m
Same as plotData.m in Machine Learning (20)
Training set visualization script GaussianSVM part code
%% Initialization
clear ; close all; clc
%% =============== Part 1: Loading and visualizing data================
fprintf('Loading and Visualizing Data ...\n')
% Loading from the file ex6data2.mat, you will find that there are X and y variable values in the environment:
load('ex6data2.mat');
% Plot the training set data
plotData(X, y);
fprintf('Program paused. Press enter to continue.\n');
pause;
Results of the
You can see that the data set here is divided into two categories, but the decision boundary of SVM is a curve. and clear boundaries
2 Gaussian kernel function of SVM gaussianKernel.m
function sim = gaussianKernel(x1, x2, sigma)
% gaussianKernel(x1, x2, sigma) returns a radial basis function kernel between x1 and x2,
% Input parameters: x1, x2 column vectors. Ensure that x1 and x2 are column vectors with consistent dimensions
% Return value: sim returns a Gaussian kernel between x1 and x2
x1 = x1(:); x2 = x2(:);
% You need to return the following variables correctly.
sim = 0;
sim = exp(-sum((x1-x2).^2)/(2*sigma^2));
end
3 Training function svmTrain.m
Same as svmTrain.m in Machine Learning (20)
4 Prediction function svmPredict.m
function pred = svmPredict(model, X)
% svmPredict(model, X) returns the prediction vector using the trained SVM model
% Input: model model obtained by svmTrain function
% X training matrix, the number of rows is the number of samples, and the number of columns is the number of input features
% Output: pred is a predicted column vector of {0,1} values with the number of rows being the number of samples and the number of columns being 1.
% Check if we got a column vector, if so then assume we only need to predict for a single example
if (size(X, 2) == 1)
% Examples should be in rows
X = X';
endif
% data set
m = size(X, 1); %number of samples
p = zeros(m, 1); % initialize the elements of the column vector to 0
pred = zeros(m, 1); % initialize the elements of the column vector to 0
if strcmp(func2str(model.kernelFunction), 'linearKernel')
% If using linear kernel function, we can directly use weights and biases
p = X * model.w + model.b;
elseif strfind(func2str(model.kernelFunction), 'gaussianKernel')
% Vectorized RBF kernel
% This is equivalent to computing the kernel on each pair of examples
X1 = sum(X.^2, 2);
X2 = sum(model.X.^2, 2)';
K = bsxfun(@plus, X1, bsxfun(@plus, X2, - 2 * X * model.X'));
K = model.kernelFunction(1, 0) .^ K;
K = bsxfun(@times, model.y', K);
K = bsxfun(@times, model.alphas', K);
p = sum(K, 2);
else
% Other nonlinear kernels
for i = 1:m
prediction = 0;
for j = 1:size(model.X, 1)
prediction = prediction + ...
model.alphas(j) * model.y(j) * ...
model.kernelFunction(X(i,:)', model.X(j,:)');
endfor
p(i) = prediction + model.b;
endfor
endif
% Convert prediction to 0 / 1
pred(p >= 0) = 1;
pred(p < 0) = 0;
end
5 Visualize SVM decision boundary function visualizeBoundary.m
function visualizeBoundary(X, y, model, varargin)
% visualizeBoundary draws the nonlinear decision boundary learned by the support vector machine
% Input: X training matrix, the number of rows is the number of samples, and the number of columns is the number of input features
% y The output feature vector of the training set is a column vector containing 1 and 0. The number of rows is the number of samples and the number of columns is 1.
% model The model obtained by the svmTrain function
% varargin
plotData(X, y) % Plot training data
% Make classification predictions for a value grid
x1plot = linspace(min(X(:,1)), max(X(:,1)), 100)';
x2plot = linspace(min(X(:,2)), max(X(:,2)), 100)';
[X1, X2] = meshgrid(x1plot, x2plot);
vals = zeros(size(X1));
for i = 1:size(X1, 2)
this_X = [X1(:, i), X2(:, i)];
vals(:, i) = svmPredict(model, this_X);
endfor
% Draw SVM decision boundary
hold on
contour(X1, X2, vals, [0.5 0.5], 'b');
hold off;
end
6 Nonlinear SVM script GaussianSVM.m
%% Initialization
clear ; close all; clc
%% =============== Part 1: Loading and visualizing data================
fprintf('Loading and Visualizing Data ...\n')
% Loading from the file ex6data2.mat, you will find that there are X and y variable values in the environment:
load('ex6data2.mat');
% Plot the training set data
plotData(X, y);
fprintf('Program paused. Press enter to continue.\n');
pause;
%% ==================== Part 2: Linear SVM training ====================
% After implementing the kernel, we can now use it to train the SVM classifier.
% Loading from the file ex6data2.mat, you will find that there are X and y variable values in the environment:
fprintf('\nTraining SVM with RBF Kernel (this may take 1 to 2 minutes) ...\n');
% Load from ex6data2:
% You will have X, y in your environment
load('ex6data2.mat');
% SVM parameters
C = 1; sigma = 0.1;
% Here we set the tolerance and max_passes lower,
% so that the code runs faster. In practice, however, you will want to run the training until convergence.
model= svmTrain(X, y, C, @(x1, x2) gaussianKernel(x1, x2, sigma));
visualizeBoundary(X, y, model);
fprintf('Program paused. Press enter to continue.\n');
pause;
7 Results of executing the script GaussianSVM.m
The picture on the left cannot use a linear decision boundary to divide positive and negative sample data.
The picture on the right is the decision boundary trained with Gaussian kernel SVM. This decision boundary can correctly separate most positive and negative samples and fit the data well.