Convolutional Neural Networks for tf & MNIST

The construction process is also to load the data first, then build the grid model, and finally train and evaluate the model

Download Data

• 1) Define the input data and preprocess the data. Here, we first read the data MNIST, and get the image and label matrix of the training set, and the
  image and label matrix of the test set, respectively

from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
mnist = input_data.read_data_sets('./MNIST_data', one_hot=True)

• 2) Next, the input data needs to be processed, and the shapes of the above trX and teX are changed to [-1, 28, 28, 1], -1 means
  the , and 28×28 is the long sum of the pictures The number of pixels wide, 1 is the number of channels, because the MNIST
  image is black and white, so the channel is 1, if it is an RGB color image, the channel is 3

trX = trX.reshape(-1,28,28,1) # 28*28*1 input img
teX = teX.reshape(-1,28,28,1) # 28*28*1 input img
X = tf.placeholder("float",[None,28,28,1])
Y = tf.placeholder("float",[None,10])

Build the model

• Initialize the weights and define the lattice network structure. Here, we are going to build a convolutional neural network with 3 convolutional layers and 3 pooling layers, followed by 1 fully connected layer and 1 output layer. First define the function to initialize the weights

def init_weights(shape):
    return tf.Variable(tf.random_normal(shape,stddev=0.01))

• The weight grid is initialized as follows, and we set the size of the convolution kernel to be 3×3:

w = init_weights([3,3,1,32]) # patch 大小为 3 × 3，输入维度为1,输出维度为32
w2 = init_weights([3,3,32,64]) # patch 大小为 3 × 3 输入维度为32,输出维度为64
w3 = init_weights([3,3,64,128]) # patch 大小为 3 × 3，输入维度为 64,输出维度为128
w4 = init_weights([128*4*4,625]) # 全链接层，输入维度为128×4×4，是上一层的输入数据由三维转换成一维,输出维度为 625
w_o = init_weights([625,10]) # 输出层，输入维度为625,输出维度为10,代表10类（labels）

• Then, define a model function:

# 神经网络模型构建函数，传入参数
# X：输入数据
# w: 每一层的权重
# p_keep_conv,p_keep_hidden:droupt 要保留神经元的比例
def model(X,w,w2,w3,w4,w_o,p_keep_conv,p_keep_hidden):
    # 第一组卷积层及池化层，最后 droupt 一些神经元
    l1a = tf.nn.relu(tf.nn.conv2d(X,w,strides= [1,1,1,1],padding = 'SAME'))
    # l1a shape = (?,28,28,32)
    l1 = tf.nn.max_pool(l1a,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
    # l1 shape = (?,14,14,32)
    l1 = tf.nn.dropout(l1,p_keep_conv)

    #第二组卷积层及池化层，最后 droupt 一些神经元
    l2a = tf.nn.relu(tf.nn.conv2d(l1,w2,strides=[1,1,1,1],padding='SAME'))
    # L2a shape = (?,14,14,64)
    l2 = tf.nn.max_pool(l2a,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
    # 12 shaep = (?,7,7,64)
    l2 = tf.nn.dropout(l2,p_keep_conv)

    #第三组卷积及池化层，最后 droupt 一些神经元
    l3a = tf.nn.relu(tf.nn.conv2d(l2,w3,strides=[1,1,1,1],padding='SAME'))
    # l3a shape = (?,7,7,128)
    l3 = tf.nn.max_pool(l3a,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
    # l3 shape = (?,4,4,128)
    l3 = tf.reshape(l3,[-1,w4.get_shape().as_list()[0]]) # reshape to (?,2048)
    l3 = tf.nn.dropout(l3,p_keep_conv)

    #全链接层，最后 droupt 一些神经元
    l4 = tf.nn.relu(tf.matmul(l3,w4))
    l4 = tf.nn.dropout(l4,p_keep_hidden)

    #输出层
    pyx = tf.matmul(l4,w_o)
    return pyx # 返回预测值

• We define a dropout placeholder—keep_conv, which represents how many neurons in a layer are kept. Generate the lattice network model and get the predicted value, as follows:

p_keep_conv = tf.placeholder("float")
p_keep_hidden = tf.placeholder("float")
py_x = model(X,w,w2,w3,w4,w_o,p_keep_conv,p_keep_hidden) # 得到预测值

• Next, define the loss function, here we use tf.nn.softmax_cross_entropy_with_logits to compare the difference between the predicted value and the true value, and perform mean processing; define the training operation (train_op), using the optimizer tf.train.RMSPropOptimizer that implements the RMSProp algorithm , the learning rate is 0.001, and the decay value is 0.9 to minimize the loss

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=py_x,labels=Y))
train_op = tf.train.RMSPropOptimizer(0.001,0.9).minimize(cost)
predict_op = tf.argmax(py_x,1)

Train the model and evaluate the model

• First define the batch size at training time and the batch size at evaluation time

batch_size = 64
test_size = 256

#Launch the graph in a session
with tf.Session() as sess:
    tf.global_variables_initializer().run()
    for i in range(100):
        training_batch = zip(range(0,len(trX),batch_size),
                             range(batch_size,len(trX)+1,batch_size))
        for start,end in training_batch:
            sess.run(train_op,feed_dict={X:trX[start:end],Y:trY[start:end],
                                        p_keep_conv:0.8,p_keep_hidden:0.5})
        test_indices = np.arange(len(teX)) # Get A Test Batch
        np.random.shuffle(test_indices)
        test_indices = test_indices[0:test_size]
        print(i,np.mean(np.argmax(teY[test_indices],axis=1) ==
                       sess.run(predict_op,feed_dict={X:teX[test_indices],
                                                     p_keep_conv:1.0,
                                                     p_keep_hidden:1.0})))