TensorFlow入门篇(三)：MNIST数据集简易分类

版权声明：本文为博主原创文章，博主欢迎各位转载。 https://blog.csdn.net/tuwenqi2013/article/details/84558804

数据集：http://yann.lecun.com/exdb/mnist/

环境：Python 3.7

          TensorFlow 1.12

          numpy 1.15.4

问题：1、增加隐藏层后，效果为何变差？

           2、二次代价函数替换成交叉熵代价函数，初期效果提升明显？

代码：

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data


#载入数据集
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)

#每个批次的大小
batch_size = 100
#计算一共有多少个批次
n_batch = mnist.train.num_examples // batch_size

#定义两个placeholder
x = tf.placeholder(tf.float32,[None,784])
y = tf.placeholder(tf.float32,[None,10])

#创建一个简单的神经网络
W = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))
prediction = tf.nn.softmax(tf.matmul(x,W)+b)

#二次代价函数
loss = tf.reduce_mean(tf.square(y-prediction))
# 交叉熵代价函数
# loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=prediction))    
#使用梯度下降法
train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)

#初始化变量
init = tf.global_variables_initializer()

#结果存放在一个布尔型列表中
correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置
#求准确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))

with tf.Session() as sess:
    sess.run(init)
    for epoch in range(51):
        for batch in range(n_batch):
            batch_xs,batch_ys =  mnist.train.next_batch(batch_size)
            sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys})
        
        acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels})
        print("Iter " + str(epoch) + ",Testing Accuracy " + str(acc))

结果：

iter0, test accuracy: 0.7689
iter1, test accuracy: 0.8456
iter2, test accuracy: 0.867
iter3, test accuracy: 0.8761
iter4, test accuracy: 0.8819
iter5, test accuracy: 0.8848
iter6, test accuracy: 0.8888
iter7, test accuracy: 0.8927
iter8, test accuracy: 0.8955
iter9, test accuracy: 0.8971
iter10, test accuracy: 0.8988
iter11, test accuracy: 0.9004
iter12, test accuracy: 0.9012
iter13, test accuracy: 0.9028
iter14, test accuracy: 0.9038
iter15, test accuracy: 0.9046
iter16, test accuracy: 0.9055
iter17, test accuracy: 0.9057
iter18, test accuracy: 0.9061
iter19, test accuracy: 0.9072
iter20, test accuracy: 0.9075
iter21, test accuracy: 0.9082
iter22, test accuracy: 0.9095
iter23, test accuracy: 0.9095
iter24, test accuracy: 0.9101
iter25, test accuracy: 0.9098
iter26, test accuracy: 0.9108
iter27, test accuracy: 0.9116
iter28, test accuracy: 0.912
iter29, test accuracy: 0.9128
iter30, test accuracy: 0.9127
iter31, test accuracy: 0.9131
iter32, test accuracy: 0.9135
iter33, test accuracy: 0.9136
iter34, test accuracy: 0.9141
iter35, test accuracy: 0.9146
iter36, test accuracy: 0.9144
iter37, test accuracy: 0.9155
iter38, test accuracy: 0.9153
iter39, test accuracy: 0.9159
iter40, test accuracy: 0.9159
iter41, test accuracy: 0.9157
iter42, test accuracy: 0.916
iter43, test accuracy: 0.9162
iter44, test accuracy: 0.9173
iter45, test accuracy: 0.9172
iter46, test accuracy: 0.9176
iter47, test accuracy: 0.918
iter48, test accuracy: 0.9176
iter49, test accuracy: 0.918
iter50, test accuracy: 0.9179
 

第一次提升性能：采用Dropout  (随机激活) 只激活部分神经元

                  作用：防止过拟合

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

# 载入数据集
mnist = input_data.read_data_sets("../data/MNIST", one_hot=True)
# 每个批次的大小
batch_size = 100
# 计算一共有多少个批次
n_batch = mnist.train.num_examples // batch_size

# 定义两个placeholder
x = tf.placeholder(tf.float32, [None, 784])
y = tf.placeholder(tf.float32, [None, 10])
keep_prob = tf.placeholder(tf.float32)

# 创建一个简单的神经网络
W1 = tf.Variable(tf.truncated_normal([784, 1000], stddev=0.1))
b1 = tf.Variable(tf.zeros([1000]) + 0.1)
L1 = tf.nn.tanh(tf.matmul(x, W1) + b1)
L1_drop = tf.nn.dropout(L1, keep_prob)

W2 = tf.Variable(tf.truncated_normal([1000, 1000], stddev=0.1))
b2 = tf.Variable(tf.zeros([1000]) + 0.1)
L2 = tf.nn.tanh(tf.matmul(L1_drop, W2) + b2)
L2_drop = tf.nn.dropout(L2, keep_prob)

W3 = tf.Variable(tf.truncated_normal([1000, 1000], stddev=0.1))
b3 = tf.Variable(tf.zeros([1000]) + 0.1)
L3 = tf.nn.tanh(tf.matmul(L2_drop, W3) + b3)
L3_drop = tf.nn.dropout(L3, keep_prob)

W4 = tf.Variable(tf.truncated_normal([1000, 10], stddev=0.1))
b4 = tf.Variable(tf.zeros([10]) + 0.1)
prediction = tf.nn.softmax(tf.matmul(L3_drop, W4) + b4)

# 二次代价函数
# loss = tf.reduce_mean(tf.square(y-prediction))
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=prediction))
# 使用梯度下降法
train_step = tf.train.GradientDescentOptimizer(0.2).minimize(loss)

# 初始化变量
init = tf.global_variables_initializer()

# 结果存放在一个布尔型列表中
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(prediction, 1))  # argmax返回一维张量中最大的值所在的位置
# 求准确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

with tf.Session() as sess:
    sess.run(init)
    for epoch in range(51):
        for batch in range(n_batch):
            batch_xs, batch_ys = mnist.train.next_batch(batch_size)
            sess.run(train_step, feed_dict={x: batch_xs, y: batch_ys, keep_prob: 0.7})  # 只训练70%的神经元

        test_acc = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0})
        train_acc = sess.run(accuracy, feed_dict={x: mnist.train.images, y: mnist.train.labels, keep_prob: 1.0})
        print("Iter " + str(epoch) + ",Testing Accuracy " + str(test_acc) + ",Training Accuracy " + str(train_acc))

结果：

Iter 45,Testing Accuracy 0.9738,Training Accuracy 0.984
Iter 46,Testing Accuracy 0.9751,Training Accuracy 0.98425454
Iter 47,Testing Accuracy 0.9754,Training Accuracy 0.9846
Iter 48,Testing Accuracy 0.9755,Training Accuracy 0.9849091
Iter 49,Testing Accuracy 0.9748,Training Accuracy 0.9848
Iter 50,Testing Accuracy 0.9765,Training Accuracy 0.9856

进一步改进措施：1、使用reLU激活函数，加快收敛速度

                             2、使用逐渐减小的学习率，减小收敛过程的震荡