该程序是根据二层神经网络实现的手写数字0-9的识别。
一、手写数字0-9
训练集为5000幅像素为20*20的手写数字0-9,如图所示为100个样本展示图像:
二、神经网络结构
二层的神经网络结构如下图所示:
输入层神经元401个(含1个偏执单元),隐藏层神经元26个(含偏执单元),输出层神经元10个
系数矩阵的阶数为25401,的阶数为1026。
三、数学推导
数学推导与《简单的二层BP神经网络-实现逻辑与门(Matlab和Python)》一文中的推导过程类似,在此不再赘述。
文章链接:简单的二层BP神经网络-实现逻辑与门(Matlab和Python)
四、程序(Matlab)
clear all;clc
%数据初始化
num_labels = 10;
% Load Training Data
load('data.mat');
m = size(X, 1);
% Randomly select 100 data points to display
sel = randperm(size(X, 1));
sel = sel(1:100);
figure
displayData(X(sel, :));
%Loading Parameters
load('weights.mat');
alpha = 3; %learning_rate
number_iters = 5000; %number_of_training_iterations
%迭代
for i=1:number_iters
%Forward Propagation
a1 = [ones(m, 1) X]; %5000x401
z2 = a1 * Theta1'; %5000x25 Theta1 25*401
a2 = sigmoid(z2); %5000x25
a2 = [ones(m, 1) a2]; %5000x26
z3 = a2 * Theta2'; %5000x10 Theta2 10*26
a3 = sigmoid(z3); %5000x10
h = a3; %5000x10
u = eye(num_labels);
y1 = u(y,:);
%Back Propagation
delta3 = a3 - y1; % 5000 * 10
delta2 = delta3 * Theta2; % 5000 * 26
delta2 = delta2(:,2:end); % 5000 * 25
delta2 = delta2 .* sigmoidGradient(z2); % 5000 * 25
Delta1 = zeros(size(Theta1)); % 25 * 401
Delta2 = zeros(size(Theta2)); % 10 * 26
Delta1 = Delta1 + delta2' * a1; % 25 * 401 5000×25' * 5000x401
Delta2 = Delta2 + delta3' * a2; % 10 * 26 5000×10' * 5000x26
Theta2_grad = 1/m * Delta2;
Theta1_grad = 1/m * Delta1;
Theta1 = Theta1 - alpha * Theta1_grad;
Theta2 = Theta2 - alpha * Theta2_grad;
J(i) = 1/m*(sum(sum(-y1 .* log(h) - (1 - y1) .* log(1 - h))));
end
%绘制代价函数曲线
figure
plot(J);
xlabel('number of iterations')
ylabel('Costfunction in the output layer');
%预测
pred = predict(Theta1, Theta2, X);
fprintf('\nTraining Set Accuracy: %f\n', mean(double(pred == y)) * 100);
输出结果:
Training Set Accuracy: 100.000000
代价函数图像:
注:相关内容参照吴恩达机器学习课程和编程作业修改而来,如侵可删。