本次将使用四隐藏层神经元进行模型构建，每层神经元个数为 256、128、64、32，其中我们将首次引入 dropout 层来防止多层神经元带来的过拟合问题，dropout 层是通过随机放弃部分神经元参与模型训练，让每次神经元之间的连接不同，关于具体的 drop out层将在博客中详细讲解
采用的激活函数仍是 softmax
$output = f(z) = f(\sum_{i=1}^nx_i*w_i+b)$

导入必要模块

import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
import math
import matplotlib.pyplot as plt
import time
import os


mnist = input_data.read_data_sets('./mnist_dataset/', one_hot=True)

Extracting ./mnist_dataset/train-images-idx3-ubyte.gz
Extracting ./mnist_dataset/train-labels-idx1-ubyte.gz
Extracting ./mnist_dataset/t10k-images-idx3-ubyte.gz
Extracting ./mnist_dataset/t10k-labels-idx1-ubyte.gz

构建输入层

x = tf.placeholder(tf.float32, [None, 784], name='X')
y = tf.placeholder(tf.float32, [None, 10], name='Y')

第 1 ~ 4 隐藏层

注意，多层神经网络中，生成权值不适用完全随机数，使用截断的方法 tf.truncated_normal() 生成随机数，否则容易造成 Loss = Nan 情况

H1_NN = 256

W1 = tf.Variable(tf.truncated_normal([784, H1_NN], stddev=0.1))
b1 = tf.Variable(tf.zeros([H1_NN]))

Y1 = tf.nn.relu(tf.matmul(x, W1) + b1)
Y1 = tf.nn.dropout(Y1, rate=0.2)

H2_NN = 128

W2 = tf.Variable(tf.truncated_normal([H1_NN, H2_NN], stddev=0.1))
b2 = tf.Variable(tf.zeros([H2_NN]))

Y2 = tf.nn.relu(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, rate=0.2)

H3_NN = 64

W3 = tf.Variable(tf.truncated_normal([H2_NN, H3_NN], stddev=0.1))
b3 = tf.Variable(tf.zeros([H3_NN]))

Y3 = tf.nn.relu(tf.matmul(Y2, W3) + b3)
Y3 = tf.nn.dropout(Y3, rate=0.2)

H4_NN = 32

W4 = tf.Variable(tf.truncated_normal([H3_NN, H4_NN], stddev=0.1))
b4 = tf.Variable(tf.zeros([H4_NN]))

Y4 = tf.nn.relu(tf.matmul(Y3, W4) + b4)
Y4 = tf.nn.dropout(Y4, rate=0.2)

构建输出层

W_out = tf.Variable(tf.truncated_normal([H4_NN, 10], stddev=0.1))
b_out = tf.Variable(tf.zeros([10]))

forward = tf.matmul(Y4, W_out) + b_out
pred = tf.nn.softmax(forward)

定义损失函数

原始定义方式：
loss_function = tf.reduce_mean(-tf.reduce_sum(y * tf.log(pred), reduction_indices=1))
为了避免 $\lim_{x\to0}log(x)$ 造成值趋于无穷大，使得最终的 loss 值超出认可范围，将使用 TensorFlow 1.x 自带的 Softmax 交叉熵损失函数定义方法

loss_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=forward, labels=y))

设置训练参数

epochs = 150
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01

选择优化器

optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)

定义准确率

correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

开始训练

startTime = time.time()

sess = tf.Session()

train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []

sess.run(tf.global_variables_initializer())
for epoch in range(epochs):
    if epoch > 99:
        learning_rate = 0.001
    for batch in range(total_batch):
        xs, ys = mnist.train.next_batch(batch_size)
        sess.run(optimizer, feed_dict={x:xs, y:ys})
    train_loss, train_acc = sess.run([loss_function, accuracy],
                         feed_dict={x:mnist.train.images, y:mnist.train.labels})
    val_loss, val_acc = sess.run([loss_function, accuracy],
                         feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
    train_loss_list.append(train_loss)
    train_acc_list.append(train_acc)
    val_loss_list.append(val_loss)
    val_acc_list.append(val_acc)
    if (epoch + 1) % display_step == 0:
        print('Epoch: %3d' % (epoch + 1), 
              'train_loss=%6f' % train_loss, 
              'train_accuracy=%4f' % train_acc,
              'val_loss=%6f' % val_loss, 
              'val_accuracy=%4f' % val_acc,
              'learning_rate=%.4f' % learning_rate)

duration = time.time() - startTime
print('INFO:Train Finished!\nTaking for %d seconds!' % duration)

部分训练如下
Epoch: 120 train_loss=0.065860 train_accuracy=0.981527 val_loss=0.132282 val_accuracy=0.965000 learning_rate=0.0010
Epoch: 121 train_loss=0.064610 train_accuracy=0.982327 val_loss=0.125659 val_accuracy=0.966600 learning_rate=0.0010
Epoch: 122 train_loss=0.068193 train_accuracy=0.980200 val_loss=0.128001 val_accuracy=0.964800 learning_rate=0.0010
Epoch: 123 train_loss=0.062760 train_accuracy=0.982182 val_loss=0.132810 val_accuracy=0.969000 learning_rate=0.0010
Epoch: 124 train_loss=0.065039 train_accuracy=0.981309 val_loss=0.132855 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 125 train_loss=0.063242 train_accuracy=0.981782 val_loss=0.127155 val_accuracy=0.971200 learning_rate=0.0010
Epoch: 126 train_loss=0.064617 train_accuracy=0.981655 val_loss=0.130808 val_accuracy=0.968800 learning_rate=0.0010
Epoch: 127 train_loss=0.062583 train_accuracy=0.982545 val_loss=0.124542 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 128 train_loss=0.061034 train_accuracy=0.982873 val_loss=0.114629 val_accuracy=0.971200 learning_rate=0.0010
Epoch: 129 train_loss=0.062726 train_accuracy=0.982455 val_loss=0.124787 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 130 train_loss=0.059900 train_accuracy=0.982818 val_loss=0.129608 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 131 train_loss=0.059676 train_accuracy=0.982400 val_loss=0.128246 val_accuracy=0.970400 learning_rate=0.0010
Epoch: 132 train_loss=0.060196 train_accuracy=0.982491 val_loss=0.117495 val_accuracy=0.970600 learning_rate=0.0010
Epoch: 133 train_loss=0.059927 train_accuracy=0.982473 val_loss=0.125890 val_accuracy=0.971400 learning_rate=0.0010
Epoch: 134 train_loss=0.061151 train_accuracy=0.982855 val_loss=0.127866 val_accuracy=0.969600 learning_rate=0.0010
Epoch: 135 train_loss=0.059084 train_accuracy=0.982327 val_loss=0.136047 val_accuracy=0.965600 learning_rate=0.0010
Epoch: 136 train_loss=0.059557 train_accuracy=0.983018 val_loss=0.126029 val_accuracy=0.968600 learning_rate=0.0010
Epoch: 137 train_loss=0.054957 train_accuracy=0.984564 val_loss=0.119716 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 138 train_loss=0.058328 train_accuracy=0.983000 val_loss=0.126019 val_accuracy=0.969600 learning_rate=0.0010
Epoch: 139 train_loss=0.054724 train_accuracy=0.984727 val_loss=0.140184 val_accuracy=0.967400 learning_rate=0.0010
Epoch: 140 train_loss=0.057109 train_accuracy=0.984309 val_loss=0.117697 val_accuracy=0.966800 learning_rate=0.0010
Epoch: 141 train_loss=0.054068 train_accuracy=0.984127 val_loss=0.130676 val_accuracy=0.970800 learning_rate=0.0010
Epoch: 142 train_loss=0.055419 train_accuracy=0.984455 val_loss=0.131557 val_accuracy=0.968200 learning_rate=0.0010
Epoch: 143 train_loss=0.053236 train_accuracy=0.984491 val_loss=0.120054 val_accuracy=0.971600 learning_rate=0.0010
Epoch: 144 train_loss=0.055174 train_accuracy=0.984273 val_loss=0.128828 val_accuracy=0.969400 learning_rate=0.0010
Epoch: 145 train_loss=0.051997 train_accuracy=0.985164 val_loss=0.128098 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 146 train_loss=0.052802 train_accuracy=0.984127 val_loss=0.133567 val_accuracy=0.969400 learning_rate=0.0010
Epoch: 147 train_loss=0.052215 train_accuracy=0.984691 val_loss=0.131717 val_accuracy=0.970400 learning_rate=0.0010
Epoch: 148 train_loss=0.050656 train_accuracy=0.985600 val_loss=0.138663 val_accuracy=0.968600 learning_rate=0.0010
Epoch: 149 train_loss=0.051594 train_accuracy=0.985455 val_loss=0.130331 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 150 train_loss=0.049943 train_accuracy=0.985691 val_loss=0.125346 val_accuracy=0.970800 learning_rate=0.0010
INFO:Train Finished!
Taking for 254 seconds!

test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)

Test dataset accuracy: 0.969800

损失与准确率可视化

fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_loss_list, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(val_loss_list, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_acc_list, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(val_acc_list, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()

sess.close()

定义全连接函数简化网络定义代码并保存模型

def fcn_layer(inputs,
              input_dim,
              output_dim,
              activation=None):
    W = tf.Variable(tf.truncated_normal([input_dim, output_dim], stddev=0.1))
    b = tf.Variable(tf.zeros([output_dim]))
    XWb = tf.matmul(inputs, W) + b
    if activation is None:
        outputs = XWb
    else:
        outputs = activation(XWb)
        outputs = tf.nn.dropout(outputs, rate=0.2)
    
    return outputs

x = tf.placeholder(tf.float32, [None, 784], name='X')

h1 = fcn_layer(x, 784, 256, tf.nn.relu)
h2 = fcn_layer(h1, 256, 128, tf.nn.relu)
h3 = fcn_layer(h2, 128, 64, tf.nn.relu)
h4 = fcn_layer(h3, 64, 32, tf.nn.relu)

forward = fcn_layer(h4, 32, 10, None)
pred = tf.nn.softmax(forward)

epochs = 201
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01
save_step = 5

ckpt_dir = './mnist_ckpt'
if not os.path.exists(ckpt_dir):
    os.makedirs(ckpt_dir)

loss_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=forward, labels=y))

optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)

correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

startTime = time.time()

saver = tf.train.Saver()
sess = tf.Session()

train_loss_list = []
train_acc_list = []
val_loss_list = []
val_acc_list = []

sess.run(tf.global_variables_initializer())
for epoch in range(epochs):
    if epoch > 99:
        learning_rate = 0.001
    for batch in range(total_batch):
        xs, ys = mnist.train.next_batch(batch_size)
        sess.run(optimizer, feed_dict={x:xs, y:ys})
    train_loss, train_acc = sess.run([loss_function, accuracy],
                         feed_dict={x:mnist.train.images, y:mnist.train.labels})
    val_loss, val_acc = sess.run([loss_function, accuracy],
                         feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
    train_loss_list.append(train_loss)
    train_acc_list.append(train_acc)
    val_loss_list.append(val_loss)
    val_acc_list.append(val_acc)
    if (epoch + 1) % display_step == 0:
        print('Epoch: %3d' % (epoch + 1), 
              'train_loss=%6f' % train_loss, 
              'train_accuracy=%4f' % train_acc,
              'val_loss=%6f' % val_loss, 
              'val_accuracy=%4f' % val_acc,
              'learning_rate=%.4f' % learning_rate)
    if (epoch + 1) % save_step == 0 or (epoch + 1) == epochs:
        saver.save(sess, os.path.join(ckpt_dir, 'mnist_model_{0:03d}.ckpt'.format(epoch + 1)))
        print('INFO:mnist_model_{0:03d}.ckpt saved'.format(epoch + 1))

duration = time.time() - startTime
print('INFO:Model saved\n', 'INFO:Train Finished!\nTaking for %d seconds!' % duration)

部分训练过程如下
Epoch: 180 train_loss=0.044205 train_accuracy=0.987273 val_loss=0.141515 val_accuracy=0.969000 learning_rate=0.0010
INFO:mnist_model_180.ckpt saved
Epoch: 181 train_loss=0.042262 train_accuracy=0.987473 val_loss=0.129795 val_accuracy=0.972400 learning_rate=0.0010
Epoch: 182 train_loss=0.040721 train_accuracy=0.988127 val_loss=0.141154 val_accuracy=0.969200 learning_rate=0.0010
Epoch: 183 train_loss=0.041534 train_accuracy=0.987800 val_loss=0.135395 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 184 train_loss=0.041826 train_accuracy=0.987873 val_loss=0.133711 val_accuracy=0.971600 learning_rate=0.0010
Epoch: 185 train_loss=0.042363 train_accuracy=0.988218 val_loss=0.137348 val_accuracy=0.970800 learning_rate=0.0010
INFO:mnist_model_185.ckpt saved
Epoch: 186 train_loss=0.041273 train_accuracy=0.988145 val_loss=0.139761 val_accuracy=0.968400 learning_rate=0.0010
Epoch: 187 train_loss=0.041105 train_accuracy=0.987873 val_loss=0.131214 val_accuracy=0.972200 learning_rate=0.0010
Epoch: 188 train_loss=0.038462 train_accuracy=0.988927 val_loss=0.116528 val_accuracy=0.972000 learning_rate=0.0010
Epoch: 189 train_loss=0.040760 train_accuracy=0.988255 val_loss=0.136059 val_accuracy=0.969800 learning_rate=0.0010
Epoch: 190 train_loss=0.039955 train_accuracy=0.988545 val_loss=0.130248 val_accuracy=0.970600 learning_rate=0.0010
INFO:mnist_model_190.ckpt saved
Epoch: 191 train_loss=0.040176 train_accuracy=0.988982 val_loss=0.136247 val_accuracy=0.971000 learning_rate=0.0010
Epoch: 192 train_loss=0.036810 train_accuracy=0.989509 val_loss=0.139191 val_accuracy=0.970000 learning_rate=0.0010
Epoch: 193 train_loss=0.039340 train_accuracy=0.988782 val_loss=0.139221 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 194 train_loss=0.038176 train_accuracy=0.988709 val_loss=0.119204 val_accuracy=0.972800 learning_rate=0.0010
Epoch: 195 train_loss=0.039080 train_accuracy=0.988709 val_loss=0.138383 val_accuracy=0.968600 learning_rate=0.0010
INFO:mnist_model_195.ckpt saved
Epoch: 196 train_loss=0.039176 train_accuracy=0.988945 val_loss=0.127481 val_accuracy=0.972000 learning_rate=0.0010
Epoch: 197 train_loss=0.038120 train_accuracy=0.988545 val_loss=0.128279 val_accuracy=0.973200 learning_rate=0.0010
Epoch: 198 train_loss=0.035479 train_accuracy=0.989873 val_loss=0.138994 val_accuracy=0.967800 learning_rate=0.0010
Epoch: 199 train_loss=0.037645 train_accuracy=0.989036 val_loss=0.134254 val_accuracy=0.967200 learning_rate=0.0010
Epoch: 200 train_loss=0.035901 train_accuracy=0.989291 val_loss=0.133777 val_accuracy=0.968600 learning_rate=0.0010
INFO:mnist_model_200.ckpt saved
Epoch: 201 train_loss=0.036870 train_accuracy=0.989473 val_loss=0.128872 val_accuracy=0.969800 learning_rate=0.0010
INFO:mnist_model_201.ckpt saved
INFO:Model saved
 INFO:Train Finished!
Taking for 335 seconds!

test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)

Test dataset accuracy: 0.970000

fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_loss_list, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(val_loss_list, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_acc_list, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(val_acc_list, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()

sess.close()

恢复模型进行预测

模型仅仅保存着训练出的权重参数，并未定义网络结构，需要定义网络结构，读取模型，在这里由于前面已运行网络结构，这里就不再重新定义

saver = tf.train.Saver()

sess = tf.Session()
sess.run(tf.global_variables_initializer())

ckpt_dir = './mnist_ckpt'
ckpt = tf.train.get_checkpoint_state(ckpt_dir)

if ckpt and ckpt.model_checkpoint_path:
    saver.restore(sess, ckpt.model_checkpoint_path)
    print('Restore model from ' + ckpt.model_checkpoint_path)

INFO:tensorflow:Restoring parameters from ./mnist_ckpt\mnist_model_201.ckpt
Restore model from ./mnist_ckpt\mnist_model_201.ckpt

准确率不同缘由

因为只保存了模型权重，并未保存网络结构，网络含有 dropout 层，每次读入的权重一样，但网络结构会因为 dropout 层不同，去掉 dropout 层，准确率就会跟前面测试集准确率一样，或者将网络结构一并保存

test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)

Test dataset accuracy: 0.969300

随机可视化显示预测值与标签值

def plot_apply_images(num):
    if num < 1:
        print('INFO:The number of input pictures must be greater than zero!')
    else:
        choose_list = []
        for i in range(num):
            choose_n = np.random.randint(len(mnist.test.images))
            choose_list.append(choose_n)
        fig = plt.gcf()
        fig.set_size_inches(18, 5 * math.ceil(num / 3))
        prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
        for i in range(num):
            ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
            plt_img = mnist.test.images[choose_list[i]].reshape(28, 28)
            ax_img.imshow(plt_img, cmap='binary')
            ax_img.set_title('Original label:' \
                             + str(np.argmax(mnist.test.labels[choose_list[i]])) \
                             + ' Predict label:' \
                             + str(prediction_result[choose_list[i]]),
                             fontsize=10)
            ax_img.set_xticks([])
            ax_img.set_yticks([])
        plt.show()

plot_apply_images(9)

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-88gThrAW-1587876488808)(output_41_0.png)]

扫描二维码关注公众号，回复： 11542418 查看本文章

随机从预测错误中图像

prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
predict_bool = (prediction_result == np.argmax(mnist.test.labels, 1))
predict_bool_list = list(predict_bool)
err_list = []
for i in range(len(predict_bool_list)):
    if not predict_bool_list[i]:
        err_list.append(i)
print('There are %d images that are misrecongnized' % len(err_list))

There are 313 images that are misrecongnized

def plot_wrong_images(num):
    prediction_result = sess.run(tf.argmax(pred, 1), feed_dict={x:mnist.test.images})
    predict_bool = (prediction_result == np.argmax(mnist.test.labels, 1))
    predict_bool_list = list(predict_bool)
    err_list = []
    for i in range(len(predict_bool_list)):
        if not predict_bool_list[i]:
            err_list.append(i)
    print('There are %d images that are misrecongnized' % len(err_list))
    if num < 1:
        print('INFO:The number of input pictures must be greater than zero!')
    else:
        choose_list = []
        for i in range(num):
            choose_n = np.random.randint(len(err_list))
            choose_list.append(err_list[choose_n])
        fig = plt.gcf()
        fig.set_size_inches(18, 5 * math.ceil(num / 3))
        for i in range(num):
            ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
            plt_img = mnist.test.images[choose_list[i]].reshape(28, 28)
            ax_img.imshow(plt_img, cmap='binary')
            ax_img.set_title('Original label:' \
                             + str(np.argmax(mnist.test.labels[choose_list[i]])) \
                             + ' Predict label:' \
                             + str(prediction_result[choose_list[i]]),
                             fontsize=10)
            ax_img.set_xticks([])
            ax_img.set_yticks([])
        plt.show()

plot_wrong_images(9)

There are 282 images that are misrecongnized

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-3J1aLjWQ-1587876488810)(output_45_1.png)]

mnist 全连接神经网络 TensorFlow 2.x 实现

TensorFlow 低阶 API

导入必要的包

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt


tf.__version__

'2.0.0'

多层神经网络

数据预处理

mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

total_num = len(train_images)
valid_split = 0.2
train_num = int(total_num * (1 - valid_split))

train_x = train_images[:train_num]
train_y = train_labels[:train_num]

valid_x = train_images[train_num:]
valid_y = train_labels[train_num:]

test_x = test_images
test_y = test_labels

train_x = train_x.reshape(-1, 784)
valid_x = valid_x.reshape(-1, 784)
test_x = test_x.reshape(-1, 784)

train_x = tf.cast(train_x / 255.0, tf.float32)
valid_x = tf.cast(valid_x / 255.0, tf.float32)
test_x = tf.cast(test_x / 255.0, tf.float32)

train_y = tf.one_hot(train_y, depth=10)
valid_y = tf.one_hot(valid_y, depth=10)
test_y = tf.one_hot(test_y, depth=10)

创建变量

Input_dim = 784
H1_NN = 64
W1 = tf.Variable(tf.random.normal([Input_dim, H1_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B1 = tf.Variable(tf.zeros([H1_NN]), dtype=tf.float32)

H2_NN = 32
W2 = tf.Variable(tf.random.normal([H1_NN, H2_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B2 = tf.Variable(tf.zeros([H2_NN]), dtype=tf.float32)

H3_NN = 10
W3 = tf.Variable(tf.random.normal([H2_NN, H3_NN], mean=0.0, stddev=1.0, dtype=tf.float32))
B3 = tf.Variable(tf.zeros([10]), dtype=tf.float32)

建立待优化列表

W = [W1, W2, W3]
B = [B1, B2, B3]

模型建立

def model(x, w, b):
    x = tf.matmul(x, w[0] + b[0])
    x = tf.nn.relu(x)
    x = tf.matmul(x, w[1] + b[1])
    x = tf.nn.relu(x)
    x = tf.matmul(x, w[2] + b[2])
    pred = tf.nn.softmax(x)
    
    return pred

定义损失函数

def loss(x, y, w, b):
    pred = model(x, w, b)
    loss_ = tf.keras.losses.categorical_crossentropy(y_true=y, y_pred=pred)
    return tf.reduce_mean(loss_)

定义梯度下降

def grad(x, y, w, b):
    var_list = w + b
    with tf.GradientTape() as tape:
        loss_ = loss(x, y, w, b)
    return tape.gradient(loss_, var_list)

设置训练参数

epochs = 400
batch_size = 50
learning_rate = 0.001

定义优化器

optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)

定义准确率

def accuracy(x, y, w, b):
    pred = model(x, w, b)
    correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
    return tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

total_step = int(train_num / batch_size)

loss_list_train = []
loss_list_valid = []
acc_list_train = []
acc_list_valid = []

for epoch in range(epochs):
    if epoch > 10:   
        learning_rate = 0.00001 
    for step in range(total_step):
        xs = train_x[step * batch_size:(step + 1) * batch_size]
        ys = train_y[step * batch_size:(step + 1) * batch_size]
        
        grads = grad(xs, ys, W, B)
        optimizer.apply_gradients(zip(grads, W + B))
    loss_train = loss(train_x, train_y, W, B).numpy()
    loss_valid = loss(valid_x, valid_y, W, B).numpy()
    acc_train = accuracy(train_x, train_y, W, B).numpy()
    acc_valid = accuracy(valid_x, valid_y, W, B).numpy()
    loss_list_train.append(loss_train)
    loss_list_valid.append(loss_valid)
    acc_list_train.append(acc_train)
    acc_list_valid.append(acc_valid)
    print('Epoch: {:03}'.format(epoch + 1),
          'train_loss= %.6f' % loss_train,
          'train_acc=%.6f' % acc_train,
          'val_loss= %.6f' % loss_valid,
          'val_acc=%.6f' % acc_valid,
          'learning_rate=%.6f' % learning_rate)
print('INFO:Train Finished!')

部分训练过程如下
Epoch: 380 train_loss= 0.163936 train_acc=0.989479 val_loss= 0.480753 val_acc=0.968667 learning_rate=0.000010
Epoch: 381 train_loss= 0.222664 train_acc=0.984979 val_loss= 0.571479 val_acc=0.962583 learning_rate=0.000010
Epoch: 382 train_loss= 0.193336 train_acc=0.987521 val_loss= 0.521553 val_acc=0.966417 learning_rate=0.000010
Epoch: 383 train_loss= 0.157278 train_acc=0.989875 val_loss= 0.493450 val_acc=0.967917 learning_rate=0.000010
Epoch: 384 train_loss= 0.149343 train_acc=0.990354 val_loss= 0.482244 val_acc=0.968583 learning_rate=0.000010
Epoch: 385 train_loss= 0.170983 train_acc=0.988792 val_loss= 0.516505 val_acc=0.966500 learning_rate=0.000010
Epoch: 386 train_loss= 0.179502 train_acc=0.988333 val_loss= 0.516758 val_acc=0.966417 learning_rate=0.000010
Epoch: 387 train_loss= 0.177936 train_acc=0.988396 val_loss= 0.536245 val_acc=0.965250 learning_rate=0.000010
Epoch: 388 train_loss= 0.156727 train_acc=0.989937 val_loss= 0.486693 val_acc=0.968417 learning_rate=0.000010
Epoch: 389 train_loss= 0.214938 train_acc=0.986083 val_loss= 0.537136 val_acc=0.965833 learning_rate=0.000010
Epoch: 390 train_loss= 0.149115 train_acc=0.990417 val_loss= 0.452357 val_acc=0.970583 learning_rate=0.000010
Epoch: 391 train_loss= 0.176035 train_acc=0.988583 val_loss= 0.553299 val_acc=0.963833 learning_rate=0.000010
Epoch: 392 train_loss= 0.173085 train_acc=0.988937 val_loss= 0.467770 val_acc=0.969833 learning_rate=0.000010
Epoch: 393 train_loss= 0.162568 train_acc=0.989604 val_loss= 0.455658 val_acc=0.970250 learning_rate=0.000010
Epoch: 394 train_loss= 0.212621 train_acc=0.986042 val_loss= 0.525348 val_acc=0.965917 learning_rate=0.000010
Epoch: 395 train_loss= 0.179633 train_acc=0.988521 val_loss= 0.499386 val_acc=0.967500 learning_rate=0.000010
Epoch: 396 train_loss= 0.157325 train_acc=0.989937 val_loss= 0.524120 val_acc=0.966417 learning_rate=0.000010
Epoch: 397 train_loss= 0.235230 train_acc=0.984333 val_loss= 0.562348 val_acc=0.963833 learning_rate=0.000010
Epoch: 398 train_loss= 0.154102 train_acc=0.990062 val_loss= 0.499408 val_acc=0.967917 learning_rate=0.000010
Epoch: 399 train_loss= 0.178979 train_acc=0.988333 val_loss= 0.535948 val_acc=0.965333 learning_rate=0.000010
Epoch: 400 train_loss= 0.153352 train_acc=0.990021 val_loss= 0.471192 val_acc=0.969750 learning_rate=0.000010
INFO:Train Finished!

损失率与准确率可视化

fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(loss_list_train, color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(loss_list_valid, color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(acc_list_train, color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(acc_list_valid, color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()

在这里插入图片描述

模型评估

acc_test = accuracy(test_x, test_y, W, B).numpy()
print('Test Accuracy:', acc_test)

Test Accuracy: 0.9694

Keras 序列模型建模

基于 onehot 编码

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import math


tf.__version__

'2.0.0'

mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

train_images = train_images / 255.0
test_images = test_images / 255.0

train_labels_onehot = tf.one_hot(train_labels, depth=10).numpy()
test_labels_onehot = tf.one_hot(test_labels, depth=10).numpy()

模型建立

model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))

model.add(tf.keras.layers.Dense(units=256,
                                kernel_initializer='normal',
                                activation='relu'))

model.add(tf.keras.layers.Dense(units=128,
                                kernel_initializer='normal',
                                activation='relu'))

model.add(tf.keras.layers.Dense(units=64,
                                kernel_initializer='normal',
                                activation='relu'))

model.add(tf.keras.layers.Dense(units=32,
                                kernel_initializer='normal',
                                activation='relu'))

model.add(tf.keras.layers.Dense(10, activation='softmax'))

模型摘要

model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten (Flatten)            (None, 784)               0         
_________________________________________________________________
dense (Dense)                (None, 256)               200960    
_________________________________________________________________
dense_1 (Dense)              (None, 128)               32896     
_________________________________________________________________
dense_2 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_3 (Dense)              (None, 32)                2080      
_________________________________________________________________
dense_4 (Dense)              (None, 10)                330       
=================================================================
Total params: 244,522
Trainable params: 244,522
Non-trainable params: 0
_________________________________________________________________

参数设置及训练

model.compile(optimizer='adam',
              loss='categorical_crossentropy',
              metrics=['accuracy'])

train_epochs = 10
batch_size = 40

train_history = model.fit(train_images, train_labels_onehot,
                          validation_split=0.2,
                          epochs=train_epochs,
                          batch_size=batch_size,
                          verbose=1)

Train on 48000 samples, validate on 12000 samples
Epoch 1/10
48000/48000 [==============================] - 145s 3ms/sample - loss: 0.3239 - accuracy: 0.9032 - val_loss: 0.1386 - val_accuracy: 0.9586
Epoch 2/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.1158 - accuracy: 0.9654 - val_loss: 0.1302 - val_accuracy: 0.9592
Epoch 3/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0807 - accuracy: 0.9758 - val_loss: 0.1019 - val_accuracy: 0.9699
Epoch 4/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0575 - accuracy: 0.9824 - val_loss: 0.0994 - val_accuracy: 0.9714
Epoch 5/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.0468 - accuracy: 0.9853 - val_loss: 0.1204 - val_accuracy: 0.9672
Epoch 6/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0386 - accuracy: 0.9883 - val_loss: 0.1029 - val_accuracy: 0.9723
Epoch 7/10
48000/48000 [==============================] - 143s 3ms/sample - loss: 0.0313 - accuracy: 0.9898 - val_loss: 0.1074 - val_accuracy: 0.9737
Epoch 8/10
48000/48000 [==============================] - 141s 3ms/sample - loss: 0.0298 - accuracy: 0.9906 - val_loss: 0.1065 - val_accuracy: 0.9740
Epoch 9/10
48000/48000 [==============================] - 142s 3ms/sample - loss: 0.0248 - accuracy: 0.9920 - val_loss: 0.0963 - val_accuracy: 0.9778
Epoch 10/10
48000/48000 [==============================] - 143s 3ms/sample - loss: 0.0221 - accuracy: 0.9929 - val_loss: 0.1000 - val_accuracy: 0.9753

train_history.history 保存了训练中，四种基本数据 loss，accuracy，val_loss，val_accuracy 的值，并以字典形式存储

train_history.history

{'loss': [0.32386956638656555,
  0.11580678951577283,
  0.08069420200908402,
  0.057474158356781116,
  0.046842153635103995,
  0.03864645366907401,
  0.03133942916045271,
  0.02975376448806249,
  0.02482963896606937,
  0.02210053606606228],
 'accuracy': [0.90316665,
  0.9653958,
  0.97577083,
  0.982375,
  0.9853333,
  0.98833334,
  0.9897708,
  0.99058336,
  0.992,
  0.99291664],
 'val_loss': [0.1386103215805876,
  0.13024117694740805,
  0.10186999143411715,
  0.09937536970265985,
  0.1203663934887542,
  0.10294981063261123,
  0.10740806087201539,
  0.10651071226072721,
  0.09627585080283704,
  0.10002141902339645],
 'val_accuracy': [0.95858335,
  0.95925,
  0.96991664,
  0.97141665,
  0.96716666,
  0.9723333,
  0.97375,
  0.974,
  0.97783333,
  0.97533333]}

fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_history.history['loss'], color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(train_history.history['val_loss'], color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_history.history['accuracy'], color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(train_history.history['val_accuracy'], color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fkRHwXQ8-1587877486141)(output_21_0.png)]

模型评估

返回值为一个列表，分别赋值给 test_loss，test_acc

test_loss, test_acc = model.evaluate(test_images, test_labels_onehot, verbose=2)
print('test_loss:', test_loss,
      '\ntest_acc:', test_acc,
      '\nmetrics_names:', model.metrics_names)

10000/1 - 25s - loss: 0.0496 - accuracy: 0.9759
test_loss: 0.09910532588202041 
test_acc: 0.9759 
metrics_names: ['loss', 'accuracy']

模型预测

def plot_apply_images(num):
    if num < 1:
        print('INFO:The number of input pictures must be greater than zero!')
    else:
        test_pred = model.predict_classes(test_images)
        choose_list = []
        for i in range(num):
            choose_n = np.random.randint(len(test_images))
            choose_list.append(choose_n)
        fig = plt.gcf()
        fig.set_size_inches(18, 5 * math.ceil(num / 3))

        for i in range(num):
            ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
            plt_img = test_images[choose_list[i]]
            ax_img.imshow(plt_img, cmap='binary')
            ax_img.set_title('Original label:' \
                             + str(test_labels[choose_list[i]]) \
                             + ' Predict label:' \
                             + str(test_pred[choose_list[i]]),
                             fontsize=10)
            ax_img.set_xticks([])
            ax_img.set_yticks([])
        plt.show()

plot_apply_images(9)

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ULYu6Vj5-1587877486143)(output_26_0.png)]

注意

以下两个 cell 等价，predict_classes 能直接预测整数分类值，predict返回 onehot 预测值

test_pred = model.predict(test_images)
print('test_pred.shape', test_pred.shape,
      '\ntest_pred[0]', np.argmax(test_pred[0]))

test_pred = model.predict_classes(test_images)
print('test_pred.shape', test_pred.shape,
      '\ntest_pred[0]', test_pred[0],
      '\ntest_labels[0]', test_labels[0])

test_pred.shape (10000, 10) 
test_pred[0] 7
test_pred.shape (10000,) 
test_pred[0] 7 
test_labels[0] 7

基于整数型编码

mnist = tf.keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

train_images = train_images / 255.0
test_images = test_images / 255.0

model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(units=64,
                                kernel_initializer='normal',
                                activation='relu'),
    tf.keras.layers.Dense(units=32,
                                kernel_initializer='normal',
                                activation='relu'),
    tf.keras.layers.Dense(10, activation='softmax')
])

model.summary()

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_1 (Flatten)          (None, 784)               0         
_________________________________________________________________
dense_5 (Dense)              (None, 64)                50240     
_________________________________________________________________
dense_6 (Dense)              (None, 32)                2080      
_________________________________________________________________
dense_7 (Dense)              (None, 10)                330       
=================================================================
Total params: 52,650
Trainable params: 52,650
Non-trainable params: 0
_________________________________________________________________

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

train_epochs = 10
batch_size = 40

train_history = model.fit(train_images, train_labels,
                          validation_split=0.2,
                          epochs=train_epochs,
                          batch_size=batch_size,
                          verbose=1)

Train on 48000 samples, validate on 12000 samples
Epoch 1/10
48000/48000 [==============================] - 5s 99us/sample - loss: 0.3811 - accuracy: 0.8907 - val_loss: 0.1879 - val_accuracy: 0.9455
Epoch 2/10
48000/48000 [==============================] - 4s 88us/sample - loss: 0.1703 - accuracy: 0.9496 - val_loss: 0.1588 - val_accuracy: 0.9517
Epoch 3/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.1252 - accuracy: 0.9621 - val_loss: 0.1207 - val_accuracy: 0.9653
Epoch 4/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.1004 - accuracy: 0.9698 - val_loss: 0.1209 - val_accuracy: 0.9644
Epoch 5/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0806 - accuracy: 0.9752 - val_loss: 0.1053 - val_accuracy: 0.9693
Epoch 6/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0673 - accuracy: 0.9789 - val_loss: 0.1070 - val_accuracy: 0.9668
Epoch 7/10
48000/48000 [==============================] - 4s 87us/sample - loss: 0.0572 - accuracy: 0.9825 - val_loss: 0.1037 - val_accuracy: 0.9701
Epoch 8/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0481 - accuracy: 0.9851 - val_loss: 0.1038 - val_accuracy: 0.9701
Epoch 9/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0440 - accuracy: 0.9861 - val_loss: 0.1091 - val_accuracy: 0.9694
Epoch 10/10
48000/48000 [==============================] - 4s 85us/sample - loss: 0.0351 - accuracy: 0.9884 - val_loss: 0.1005 - val_accuracy: 0.9722

训练可视化

fig = plt.gcf()
fig.set_size_inches(10, 5)
ax1 = fig.add_subplot(111)
ax1.set_title('Train and Validation Picture')
ax1.set_ylabel('Loss value')
line1, = ax1.plot(train_history.history['loss'], color=(0.5, 0.5, 1.0), label='Loss train')
line2, = ax1.plot(train_history.history['val_loss'], color=(0.5, 1.0, 0.5), label='Loss valid')
ax2 = ax1.twinx()
ax2.set_ylabel('Accuracy value')
line3, = ax2.plot(train_history.history['accuracy'], color=(0.5, 0.5, 0.5), label='Accuracy train')
line4, = ax2.plot(train_history.history['val_accuracy'], color=(1, 0, 0), label='Accuracy valid')
plt.legend(handles=(line1, line2, line3, line4), loc='best')
plt.show()

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dDVVigzW-1587877486144)(output_38_0.png)]

模型预测

def plot_apply_images(num):
    if num < 1:
        print('INFO:The number of input pictures must be greater than zero!')
    else:
        test_pred = model.predict_classes(test_images)
        choose_list = []
        for i in range(num):
            choose_n = np.random.randint(len(test_images))
            choose_list.append(choose_n)
        fig = plt.gcf()
        fig.set_size_inches(18, 5 * math.ceil(num / 3))

        for i in range(num):
            ax_img = plt.subplot(math.ceil(num / 3), 3, i + 1)
            plt_img = test_images[choose_list[i]]
            ax_img.imshow(plt_img, cmap='binary')
            ax_img.set_title('Original label:' \
                             + str(test_labels[choose_list[i]]) \
                             + ' Predict label:' \
                             + str(test_pred[choose_list[i]]),
                             fontsize=10)
            ax_img.set_xticks([])
            ax_img.set_yticks([])
        plt.show()

plot_apply_images(9)

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-e4IDXpdk-1587877486147)(output_41_0.png)]

mnist TensorFlow 1.x Tensorboard 应用

以 mnist 手写数字识别为例

为了简单方便，我们以单个神经元为例进行

import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
import math
import matplotlib.pyplot as plt
import os


mnist = input_data.read_data_sets('./mnist_dataset/', one_hot=True)

Extracting ./mnist_dataset/train-images-idx3-ubyte.gz
Extracting ./mnist_dataset/train-labels-idx1-ubyte.gz
Extracting ./mnist_dataset/t10k-images-idx3-ubyte.gz
Extracting ./mnist_dataset/t10k-labels-idx1-ubyte.gz

模型建立与训练

x = tf.placeholder(tf.float32, [None, 784], name='X')
y = tf.placeholder(tf.float32, [None, 10], name='Y')
image_shape_input = tf.reshape(x, [-1, 28, 28, 1])
tf.summary.image('input', image_shape_input, 9)

W = tf.Variable(tf.random_normal([784, 10]), name='W')
b = tf.Variable(tf.zeros([10]), name='b')

forward = tf.matmul(x, W) + b
pred = tf.nn.softmax(forward)

epochs = 100
batch_size = 100
total_batch = int(mnist.train.num_examples / batch_size)
display_step = 1
learning_rate = 0.01
save_step = 5

ckpt_dir = './mnist_Tensorboard_tutorial_ckpt'
if not os.path.exists(ckpt_dir):
    os.makedirs(ckpt_dir)

loss_function = tf.reduce_mean(-tf.reduce_sum(y * tf.log(pred), reduction_indices=1))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss_function)

correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

tf.summary.histogram('forward', forward)
tf.summary.scalar('loss', loss_function)
tf.summary.scalar('accuracy', accuracy)

saver = tf.train.Saver()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
merged_summary_op = tf.summary.merge_all()
writer = tf.summary.FileWriter('./Tensorboard_tutorial_log', sess.graph)
for epoch in range(epochs):
    for batch in range(total_batch):
        xs, ys = mnist.train.next_batch(batch_size)
        summary_str = sess.run(merged_summary_op, feed_dict={x:xs, y:ys})
        writer.add_summary(summary_str, epoch)
        sess.run(optimizer, feed_dict={x:xs, y:ys})
    val_loss, val_acc = sess.run([loss_function, accuracy],
                         feed_dict={x:mnist.validation.images, y:mnist.validation.labels})
    if (epoch + 1) % display_step == 0:
        print('Epoch:{:03d}'.format(epoch + 1), 
              'val_loss=%6f' % val_loss, 
              'val_accuracy=%4f' % val_acc,
              'learning_rate=%.4f' % learning_rate)
    if (epoch + 1) % save_step == 0 or (epoch + 1) == epochs:
        saver.save(sess, os.path.join(ckpt_dir, 'mnist_model_{0:03d}.ckpt'.format(epoch + 1)))
        print('INFO:mnist_model_{0:03d}.ckpt saved'.format(epoch + 1))
print('INFO:Train Finished!')

部分训练如下
Epoch:080 val_loss=0.552384 val_accuracy=0.880600 learning_rate=0.0100
INFO:mnist_model_080.ckpt saved
Epoch:081 val_loss=0.550444 val_accuracy=0.881000 learning_rate=0.0100
Epoch:082 val_loss=0.548490 val_accuracy=0.881600 learning_rate=0.0100
Epoch:083 val_loss=0.546378 val_accuracy=0.882200 learning_rate=0.0100
Epoch:084 val_loss=0.544601 val_accuracy=0.882400 learning_rate=0.0100
Epoch:085 val_loss=0.542755 val_accuracy=0.882800 learning_rate=0.0100
INFO:mnist_model_085.ckpt saved
Epoch:086 val_loss=0.540885 val_accuracy=0.883800 learning_rate=0.0100
Epoch:087 val_loss=0.539172 val_accuracy=0.882800 learning_rate=0.0100
Epoch:088 val_loss=0.537330 val_accuracy=0.883600 learning_rate=0.0100
Epoch:089 val_loss=0.535444 val_accuracy=0.884400 learning_rate=0.0100
Epoch:090 val_loss=0.533857 val_accuracy=0.884600 learning_rate=0.0100
INFO:mnist_model_090.ckpt saved
Epoch:091 val_loss=0.532277 val_accuracy=0.883800 learning_rate=0.0100
Epoch:092 val_loss=0.530567 val_accuracy=0.885800 learning_rate=0.0100
Epoch:093 val_loss=0.528905 val_accuracy=0.885400 learning_rate=0.0100
Epoch:094 val_loss=0.527362 val_accuracy=0.885600 learning_rate=0.0100
Epoch:095 val_loss=0.525661 val_accuracy=0.884000 learning_rate=0.0100
INFO:mnist_model_095.ckpt saved
Epoch:096 val_loss=0.524464 val_accuracy=0.886400 learning_rate=0.0100
Epoch:097 val_loss=0.522694 val_accuracy=0.886400 learning_rate=0.0100
Epoch:098 val_loss=0.521264 val_accuracy=0.887600 learning_rate=0.0100
Epoch:099 val_loss=0.519479 val_accuracy=0.886800 learning_rate=0.0100
Epoch:100 val_loss=0.518293 val_accuracy=0.887600 learning_rate=0.0100
INFO:mnist_model_100.ckpt saved
INFO:Train Finished!

test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)

Test dataset accuracy: 0.877300

模型读取恢复

saver = tf.train.Saver()

sess = tf.Session()
sess.run(tf.global_variables_initializer())

ckpt_dir = './mnist_Tensorboard_tutorial_ckpt'
ckpt = tf.train.get_checkpoint_state(ckpt_dir)

if ckpt and ckpt.model_checkpoint_path:
    saver.restore(sess, ckpt.model_checkpoint_path)
    print('Restore model from ' + ckpt.model_checkpoint_path)

INFO:tensorflow:Restoring parameters from ./mnist_Tensorboard_tutorial_ckpt\mnist_model_100.ckpt
Restore model from ./mnist_Tensorboard_tutorial_ckpt\mnist_model_100.ckpt

test_acc = sess.run(accuracy, feed_dict={x:mnist.test.images, y:mnist.test.labels})
print('Test dataset accuracy: %f' % test_acc)

Test dataset accuracy: 0.877300

如何打开 Tensorboard 并查看日志文件查看 Tensorflow 笔记 Ⅰ——TensorFlow 编程基础

完结

TensorFlow1.x 与 TensorFlow2.x 在读入数据的方式不同，这篇仅涉及单个神经元的搭建，下篇将使用多层神经网络进行搭建，整个文件已经上传在该页面，土豪任性下载，我将在整个系列完成后，上传至我的 GitHub

Tensorflow 笔记 Ⅴ——mnist全连接神经网络

文章目录

特别说明

mnist 全连接神经网络 TensorFlow 1.x 实现

全连接神经网络建模实现

使用多层神经元进行模型构建

导入必要模块

构建输入层

第 1 ~ 4 隐藏层

构建输出层

定义损失函数

设置训练参数

选择优化器

定义准确率

开始训练

损失与准确率可视化

定义全连接函数简化网络定义代码并保存模型

恢复模型进行预测

准确率不同缘由

随机可视化显示预测值与标签值

随机从预测错误中图像

mnist 全连接神经网络 TensorFlow 2.x 实现

TensorFlow 低阶 API

导入必要的包

多层神经网络

数据预处理

创建变量

建立待优化列表

模型建立

定义损失函数

定义梯度下降

设置训练参数

定义优化器

定义准确率

损失率与准确率可视化

模型评估

Keras 序列模型建模

基于 onehot 编码

模型建立

模型摘要

参数设置及训练

模型评估

模型预测

注意

基于整数型编码

训练可视化

模型预测

mnist TensorFlow 1.x Tensorboard 应用

以 mnist 手写数字识别为例

模型建立与训练

模型读取恢复

完结

猜你喜欢