1 Introduction

This time we use Recurrent Neural Networks (RNN, Recurrent Neural Networks) for classification, using the MNIST data set, mainly using the SimpleRNN layer.

2. Use Keras to build RNN recurrent neural network

2.1. Import necessary modules

import numpy as np
from keras.datasets import mnist    #手写体数据集模块
from keras.utils import np_utils
from keras.models import Sequential   #构建网络必需模块  
from keras.layers import SimpleRNN, Activation, Dense    #RNN、激活函数、全连接层模块
from keras.optimizers import Adam   #优化器模块
np.random.seed(42)   #随机数种子

2.2. Hyperparameter setting

The image resolution in MNIST is 28 × 28. In order to use RNN, we understand the image as serialized data. Each line serves as an input unit, so the input data size is INPUT_SIZE = 28; first, the first line is input, then the second line, the third line, the fourth line, ..., the 28th line input, this is a picture, that is a Sequence, so step TIME_STEPS = 28.

TIME_STEPS = 28     #可理解为每张图片的行数
INPUT_SIZE = 28     #可理解为每张图片的列数
BATCH_SIZE = 50     #批量大小
BATCH_INDEX = 0   
OUTPUT_SIZE = 10    #输出维度大小
CELL_SIZE = 50      #经过RNN后的输出大小
LR = 0.001

2.3. Data preprocessing

The training data should be normalized. Because the original data is an 8-bit grayscale image, it needs to be divided by 255.

(X_train, y_train),(X_test, y_test) = mnist.load_data()     #拆分训练集与测试集

X_train = X_train.reshape(-1,28,28)/255    #满足输入RNN为(-1,28,28)
X_test = X_test.reshape(-1,28,28)/255
y_train = np_utils.to_categorical(y_train,num_classes=10)    #将类别向量转换为二进制（只有0和1）的矩阵类型表示
y_test = np_utils.to_categorical(y_test,num_classes=10)

2.4. Build a model

First add the RNN layer, the input is training data, and the output data size is defined by CELL_SIZE.

Then add the output layer and select softmax as the excitation function

model = Sequential()
model.add(SimpleRNN(
    batch_input_shape = (None,TIME_STEPS,INPUT_SIZE),
    output_dim = CELL_SIZE,
    unroll = True
))

model.add(Dense(OUTPUT_SIZE))
model.add(Activation('softmax'))

2.5. Activate the model

After setting the optimization method, you can start training after the loss function and metrics method. Each training does not take all the data, just take BATCH_SIZE sequences, or BATCH_SIZE pictures, which can greatly reduce the calculation time and improve training efficiency.

adam = Adam(LR)
model.compile(optimizer=adam,
              loss = 'categorical_crossentropy',
              metrics=['accuracy'])

2.6. Training + testing

for step in range(10001):
  X_batch = X_train[BATCH_INDEX:BATCH_INDEX+BATCH_SIZE,:,:]
  y_batch = y_train[BATCH_INDEX:BATCH_INDEX+BATCH_SIZE,:]
  cost = model.train_on_batch(X_batch,y_batch)
  BATCH_INDEX += BATCH_SIZE
  BATCH_INDEX = 0 if BATCH_INDEX >= X_train.shape[0] else BATCH_INDEX
  if step % 500 == 0:
    cost, accuracy = model.evaluate(X_test, y_test, batch_size=y_test.shape[0],verbose=False)
    print('test cost:',cost,'test accuracy:',accuracy)