Article directory
TensorBoardVisualization
import tensorflow as tf
# 定义命名空间
with tf.name_scope('input'):
# fetch：就是同时运行多个op的意思
# 定义名称，会在tensorboard中代替显示
input1 = tf.constant(3.0,name='A')
input2 = tf.constant(4.0,name='B')
input3 = tf.constant(5.0,name='C')
with tf.name_scope('op'):
#加法
add = tf.add(input2,input3)
#乘法
mul = tf.multiply(input1,add)
with tf.Session() as ss:
#默认在当前py目录下的logs文件夹，没有会自己创建
result = ss.run([mul,add])
wirter = tf.summary.FileWriter('logs/demo/',ss.graph)
print(result)
[27.0, 9.0]
This code mainly demonstrates how to use TensorFlow and TensorBoard to create and visualize calculation graphs.
TensorFlow is an open source software library for numerical calculations based on data flow graphs. It has fast calculation speed and flexible construction methods, and is widely used in fields such as machine learning and deep learning. TensorBoard is a visualization tool provided by TensorFlow, which can help developers better understand, debug and optimize the calculation graph in TensorFlow.
In this code, three constants , and are first tf.constantcreated through methods , assigned values 3.0, 4.0 and 5.0 respectively, and an alias is given to these constants, namely "A", "B" and "C", so In the follow-up TensorBoard, we can clearly see the relationship between them.input1input2input3
Next, the addition and multiplication operations are defined using the tf.addand method, where addition uses the sum and multiplication uses the sum of the result of the addition. Two namespaces and are also defined here , representing the process of input and operation respectively.tf.multiplyinput2input3input1inputop
Then, with tf.Session() as ss:create a session using ss.runthe method to run the computation graph and save the result in result.
Finally, use tf.summary.FileWriterthe method to write the calculation graph to logs/demo/the directory for viewing in TensorBoard. After running python 文件名.py, enter in the command line to tensorboard --logdir=logs/demostart the TensorBoard service, open the browser, and enter http://localhost:6006/to access the visual interface of TensorBoard.
In the TensorBoard interface, you can view the visual structure of the calculation graph, the values of constants, the operation process and other information, helping developers better understand, debug and optimize TensorFlow's calculation graph.
TensorBoard case
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import argparse
import sys
import os
import tensorflow as tf
import warnings
warnings.filterwarnings("ignore")
from tensorflow.examples.tutorials.mnist import input_data
max_steps = 200 # 最大迭代次数 默认1000
learning_rate = 0.001 # 学习率
dropout = 0.9 # dropout时随机保留神经元的比例
data_dir = os.path.join('data', 'mnist')# 样本数据存储的路径
if not os.path.exists('log'):
os.mkdir('log')
log_dir = 'log' # 输出日志保存的路径
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)
sess = tf.InteractiveSession()
with tf.name_scope('input'):
x = tf.placeholder(tf.float32, [None, 784], name='x-input')
y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')
#使用tf.summary.image保存图像信息，在tensorboard上还原出输入的特征数据对应的图片
with tf.name_scope('input_reshape'):
image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])
tf.summary.image('input', image_shaped_input, 10)
def weight_variable(shape):
"""Create a weight variable with appropriate initialization."""
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
"""Create a bias variable with appropriate initialization."""
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
def variable_summaries(var):
"""Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
with tf.name_scope('summaries'):
# 计算参数的均值，并使用tf.summary.scaler记录
mean = tf.reduce_mean(var)
tf.summary.scalar('mean', mean)
# 计算参数的标准差
with tf.name_scope('stddev'):
stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
# 使用tf.summary.scaler记录记录下标准差，最大值，最小值
tf.summary.scalar('stddev', stddev)
tf.summary.scalar('max', tf.reduce_max(var))
tf.summary.scalar('min', tf.reduce_min(var))
# 用直方图记录参数的分布
tf.summary.histogram('histogram', var)
"""
构建神经网络层
创建第一层隐藏层
创建一个构建隐藏层的方法,输入的参数有：
input_tensor：特征数据
input_dim：输入数据的维度大小
output_dim：输出数据的维度大小(=隐层神经元个数）
layer_name：命名空间
act=tf.nn.relu：激活函数（默认是relu)
"""
def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):
"""Reusable code for making a simple neural net layer.
It does a matrix multiply, bias add, and then uses relu to nonlinearize.
It also sets up name scoping so that the resultant graph is easy to read,
and adds a number of summary ops.
"""
# 设置命名空间
with tf.name_scope(layer_name):
# 调用之前的方法初始化权重w，并且调用参数信息的记录方法，记录w的信息
with tf.name_scope('weights'):
weights = weight_variable([input_dim, output_dim]) #神经元数量
variable_summaries(weights)
# 调用之前的方法初始化权重b，并且调用参数信息的记录方法，记录b的信息
with tf.name_scope('biases'):
biases = bias_variable([output_dim])
variable_summaries(biases)
# 执行wx+b的线性计算，并且用直方图记录下来
with tf.name_scope('linear_compute'):
preactivate = tf.matmul(input_tensor, weights) + biases
tf.summary.histogram('linear', preactivate)
# 将线性输出经过激励函数，并将输出也用直方图记录下来
activations = act(preactivate, name='activation')
tf.summary.histogram('activations', activations)
# 返回激励层的最终输出
return activations
hidden1 = nn_layer(x, 784, 500, 'layer1')
"""
创建一个dropout层，,随机关闭掉hidden1的一些神经元，并记录keep_prob
"""
with tf.name_scope('dropout'):
keep_prob = tf.placeholder(tf.float32)
tf.summary.scalar('dropout_keep_probability', keep_prob)
dropped = tf.nn.dropout(hidden1, keep_prob)
"""
创建一个输出层，输入的维度是上一层的输出:500,输出的维度是分类的类别种类：10，
激活函数设置为全等映射identity.（暂且先别使用softmax,会放在之后的损失函数中一起计算）
"""
y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)
"""
创建损失函数
使用tf.nn.softmax_cross_entropy_with_logits来计算softmax并计算交叉熵损失,并且求均值作为最终的损失值。
"""
with tf.name_scope('loss'):
# 计算交叉熵损失（每个样本都会有一个损失）
diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)
with tf.name_scope('total'):
# 计算所有样本交叉熵损失的均值
cross_entropy = tf.reduce_mean(diff)
tf.summary.scalar('loss', cross_entropy)
"""
训练，并计算准确率
使用AdamOptimizer优化器训练模型，最小化交叉熵损失
计算准确率,并用tf.summary.scalar记录准确率
"""
with tf.name_scope('train'):
train_step = tf.train.AdamOptimizer(learning_rate).minimize(
cross_entropy)
with tf.name_scope('accuracy'):
with tf.name_scope('correct_prediction'):
# 分别将预测和真实的标签中取出最大值的索引，弱相同则返回1(true),不同则返回0(false)
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
with tf.name_scope('accuracy'):
# 求均值即为准确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
tf.summary.scalar('accuracy', accuracy)
# summaries合并
merged = tf.summary.merge_all()
# 写到指定的磁盘路径中
#删除src路径下所有文件
def delete_file_folder(src):
'''delete files and folders'''
if os.path.isfile(src):
try:
os.remove(src)
except:
pass
elif os.path.isdir(src):
for item in os.listdir(src):
itemsrc=os.path.join(src,item)
delete_file_folder(itemsrc)
try:
os.rmdir(src)
except:
pass
#删除之前生成的log
if os.path.exists(log_dir + '/train'):
delete_file_folder(log_dir + '/train')
if os.path.exists(log_dir + '/test'):
delete_file_folder(log_dir + '/test')
train_writer = tf.summary.FileWriter(log_dir + '/train', sess.graph)
test_writer = tf.summary.FileWriter(log_dir + '/test')
# 运行初始化所有变量
tf.global_variables_initializer().run()
#现在我们要获取之后要喂入的数据
def feed_dict(train):
"""Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""
if train:
xs, ys = mnist.train.next_batch(100)
k = dropout
else:
xs, ys = mnist.test.images, mnist.test.labels
k = 1.0
return {
x: xs, y_: ys, keep_prob: k}
"""
开始训练模型。 每隔10步，就进行一次merge, 并打印一次测试数据集的准确率，
然后将测试数据集的各种summary信息写进日志中。 每隔100步，记录原信息
其他每一步时都记录下训练集的summary信息并写到日志中。
"""
for i in range(max_steps):
if i % 10 == 0: # 记录测试集的summary与accuracy
summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))
test_writer.add_summary(summary, i)
print('Accuracy at step %s: %s' % (i, acc))
else: # 记录训练集的summary
if i % 100 == 99: # Record execution stats
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
summary, _ = sess.run([merged, train_step],
feed_dict=feed_dict(True),
options=run_options,
run_metadata=run_metadata)
train_writer.add_run_metadata(run_metadata, 'step%03d' % i)
train_writer.add_summary(summary, i)
print('Adding run metadata for', i)
else: # Record a summary
summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))
train_writer.add_summary(summary, i)
train_writer.close()
test_writer.close()
WARNING:tensorflow:From <ipython-input-3-27b4be5f38e0>:25: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Please write your own downloading logic.
WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data/train-images-idx3-ubyte.gz
WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.data to implement this functionality.
Extracting MNIST_data/train-labels-idx1-ubyte.gz
WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use tf.one_hot on tensors.
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
WARNING:tensorflow:From /home/nlp/anaconda3/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.
Instructions for updating:
Please use alternatives such as official/mnist/dataset.py from tensorflow/models.
WARNING:tensorflow:From <ipython-input-3-27b4be5f38e0>:109: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From <ipython-input-3-27b4be5f38e0>:123: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.
See `tf.nn.softmax_cross_entropy_with_logits_v2`.
Accuracy at step 0: 0.0639
Accuracy at step 10: 0.7139
Accuracy at step 20: 0.8271
Accuracy at step 30: 0.8647
Accuracy at step 40: 0.8818
Accuracy at step 50: 0.8932
Accuracy at step 60: 0.8984
Accuracy at step 70: 0.8986
Accuracy at step 80: 0.9062
Accuracy at step 90: 0.9128
Adding run metadata for 99
Accuracy at step 100: 0.9134
Accuracy at step 110: 0.9212
Accuracy at step 120: 0.9156
Accuracy at step 130: 0.9226
Accuracy at step 140: 0.9251
Accuracy at step 150: 0.9238
Accuracy at step 160: 0.9259
Accuracy at step 170: 0.9265
Accuracy at step 180: 0.9291
Accuracy at step 190: 0.932
Adding run metadata for 199
This code mainly demonstrates how to create and visualize a Convolutional Neural Network (CNN) using Tensorflow and TensorBoard.
CNN is a deep learning structure and a type of neural network that can be applied to image recognition, speech recognition and other fields. In this code, we will use CNN to complete the MNIST handwritten digit recognition task. The input is a 28×28 pixel handwritten digit image, and the output is the probability of one of the numbers 0-9.
tf.placeholderFirst, two placeholder variables x and y_ are created through the method, representing the input and output of the network respectively. In the processing of input data, in order to visualize the input data (28×28 pixels), the tf.summary.imageimage information is recorded and reshapethe input feature data is reconstructed using the method to ensure that the input image is 28×28×1 in size. , and tf.summary.imagerecord it with.
Secondly, in terms of building the neural network, we created two hidden layers and an output layer. Among them, each hidden layer includes a linear calculation layer and a ReLU activation function layer, and the tf.summary.histogramrelevant parameters of each layer are recorded using methods so that changes in each layer can be viewed in TensorBoard.
Then, we added dropoutlayers after the first hidden layer and randomly turned off a certain proportion of neurons to avoid overfitting. In the output layer, tf.nn.softmax cross_entropy_with_logitsthe cross-entropy loss is calculated using the method and tf.summary.scalarthe loss information is recorded using the method. We tf.train.AdamOptimizertrain the model using , tf.reduce_mean(tf.cast(correct_prediction, tf.float32))calculate the accuracy using , and tf.summary.scalarrecord the accuracy information using .
Finally, we defined mergedthe variables, brought together all the information that needed to be recorded, tf.summary.merge_all()merged them all through the method, and finally tf.summary.FileWriterwrote all the information to the log file through the method. During the training process, the accuracy and related information of the test set are recorded every 10 steps and recorded in the log; the original information of the training set is recorded every 100 steps and recorded in the log; other training steps are recorded The summary of the set is written to the log. Finally, the log file is passed train_writer.close()and test_writer.close()closed.
Throughout the code, the usage of namespaces is standardized, and the recording method of each parameter is clear and clear, allowing us to clearly understand the parameter changes, loss changes, accuracy changes, etc. of each layer in TensorBoard. Therefore, TensorBoard can well help developers debug, analyze and optimize models.