BP Neural Network Realizes Handwritten Digit Recognition
The complete code is at the end
1. Import the package
import numpy as np
from sklearn.datasets import load_digits
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report,confusion_matrix
import matplotlib.pyplot as plt
The pixel value can be obtained from digits.data.shape. The output here indicates that there are 1797 pictures, and each picture is 64 pixels, so there are 64 inputs. The tag value is stored in digits.target, the value is 0~9.
2. Import data
digits = load_digits()
# 数据 图片的灰度信息存在digits.data中
X = digits.data
# 标签
y = digits.target
3. Define the neural network
64-100-10, that is, 64 neurons in the input layer, 100 hidden layers, and 10 output layers. Why is there no bias in the input layer x 0 = 1 x_0 = 1x0=1. Personal understanding is that adding a piece of data to each picture has little effect on the training result, and it is not very convenient to add a pixel to the picture?
# 定义神经网络64-100-10,64像素即64通道输入，隐藏层稍多于输入层，输出层为0-9，所以是10个
# 输入层到隐藏层v,隐藏层到输出层w
V = np.random.random((64,100)) * 2 - 1
W = np.random.random((100,10)) * 2 - 1
4. Data segmentation Use
part of the data in digits for testing and part for training. Use the
function train_test_split()
# 数据切分 train_test_split(X,y)可以把输入数据和标签数据切分为训练集和测试集
# default: 1/4测试集，3/4训练集
X_train,X_test,y_train,y_test = train_test_split(X,y)
5. Label binarization (one hot encoding) The
neural network output is only 0, 1, which cannot directly represent 0, 1, 2, 3, 4... Therefore, classification problems must be binarized
0→100000000
3→001000000
9→ 000000001
# 标签二值化
# 0->100000000
# 3->001000000
# 9->000000001
label_train = LabelBinarizer().fit_transform(y_train)
6. Define the activation function
Use the sidmoid function as the activation function
# 激活函数
def sigmoid(x):
return 1/(1+np.exp(-x))
def dsigmoid(x):
s = 1/(1+np.exp(-x))
return s*(1-s)
7. Training model
Give the training function a training set, randomly extract a piece of data from the training set as the input x of this training, and then update the weights according to the BP algorithm.
The accuracy rate will be calculated every 1000 times of training. The calculation method is: take the test set as input, call the prediction function, and return the neuron output L 2 L_2 under the current number of training times.L2, L 2 L_2 L2It is a 10-dimensional vector with values between 0 and 1. The largest value is considered to be 1, and its position is filtered out and compared with the label value of the test set, and the average value is obtained to obtain the accuracy.
x = np.atleast_2d(x) here is actually converting x into a one-dimensional matrix, because x itself is a list, matrix operations cannot be performed, it needs to be a one-dimensional matrix to do matrix multiplication
# 训练模型
def train(X,y,Steps = 10000,lr = 0.12):
global W,V
for n in range(Steps):
# 随机选取一个数据
i = np.random.randint(X.shape[0])
x = X[i]
# 把数据变为2维数据，一维数据不能矩阵乘法
# ?????????
x = np.atleast_2d(x)
# BP算法公式
L1 = sigmoid(np.dot(x,V))
L2 = sigmoid(np.dot(L1,W))
# 学习信号
L2_delta = (y[i]-L2)*dsigmoid(np.dot(L1,W))
L1_delta = np.dot(L2_delta,W.T)*dsigmoid(np.dot(x,V))
# 更新权值
W += lr*np.dot(L1.T,L2_delta)
V += lr*np.dot(x.T,L1_delta)
#每训练1000次预测一次准确率
if n%1000 == 0:
output = predict(X_test)
# predictions：最大值所在位置（0-9）->预测的标签值
predictions = np.argmax(output,axis = 1)
acc = np.mean(np.equal(predictions,y_test))
print("steps:",n,"accuracy:",acc)
def predict(x):
L1 = sigmoid(np.dot(x,V))
L2 = sigmoid(np.dot(L1,W))
return L2
8. Train 20000 times
train(X_train,label_train,20000)
Complete code and results
import numpy as np
from sklearn.datasets import load_digits
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report,confusion_matrix
import matplotlib.pyplot as plt
digits = load_digits()
# 数据 图片的灰度信息存在digits.data中
X = digits.data
# 标签
y = digits.target
# 定义神经网络64-100-10,64像素即64通道输入，隐藏层稍多于输入层，输出层为0-9，所以是10个
# 输入层到隐藏层v,隐藏层到输出层w
V = np.random.random((64,100)) * 2 - 1
W = np.random.random((100,10)) * 2 - 1
# 数据切分 train_test_split(X,y)可以把输入数据和标签数据切分为训练集和测试集
# default: 1/4测试集，3/4训练集
X_train,X_test,y_train,y_test = train_test_split(X,y)
# 标签二值化
# 0->100000000
# 3->001000000
# 9->000000001
label_train = LabelBinarizer().fit_transform(y_train)
# 激活函数
def sigmoid(x):
return 1/(1+np.exp(-x))
def dsigmoid(x):
s = 1/(1+np.exp(-x))
return s*(1-s)
# 训练模型
def train(X,y,Steps = 10000,lr = 0.12):
global W,V
for n in range(Steps):
# 随机选取一个数据
i = np.random.randint(X.shape[0])
x = X[i]
# 把数据变为2维数据，一维数据不能矩阵乘法
# ?????????
x = np.atleast_2d(x)
# BP算法公式
L1 = sigmoid(np.dot(x,V))
L2 = sigmoid(np.dot(L1,W))
# 学习信号
L2_delta = (y[i]-L2)*dsigmoid(np.dot(L1,W))
L1_delta = np.dot(L2_delta,W.T)*dsigmoid(np.dot(x,V))
# 更新权值
W += lr*np.dot(L1.T,L2_delta)
V += lr*np.dot(x.T,L1_delta)
#每训练1000次预测一次准确率
if n%1000 == 0:
output = predict(X_test)
# predictions：最大值所在位置（0-9）->预测的标签值
predictions = np.argmax(output,axis = 1)
acc = np.mean(np.equal(predictions,y_test))
print("steps:",n,"accuracy:",acc)
def predict(x):
L1 = sigmoid(np.dot(x,V))
L2 = sigmoid(np.dot(L1,W))
return L2
train(X_train,label_train,20000)
result
steps: 0 accuracy: 0.06888888888888889
steps: 1000 accuracy: 0.6066666666666667
steps: 2000 accuracy: 0.7733333333333333
steps: 3000 accuracy: 0.8022222222222222
steps: 4000 accuracy: 0.8222222222222222
steps: 5000 accuracy: 0.8488888888888889
steps: 6000 accuracy: 0.84
steps: 7000 accuracy: 0.86
steps: 8000 accuracy: 0.8422222222222222
steps: 9000 accuracy: 0.86
steps: 10000 accuracy: 0.8666666666666667
steps: 11000 accuracy: 0.8533333333333334
steps: 12000 accuracy: 0.8666666666666667
steps: 13000 accuracy: 0.8644444444444445
steps: 14000 accuracy: 0.8644444444444445
steps: 15000 accuracy: 0.8688888888888889
steps: 16000 accuracy: 0.9044444444444445
steps: 17000 accuracy: 0.94
steps: 18000 accuracy: 0.9444444444444444
steps: 19000 accuracy: 0.94