Basic steps of training based on tensorflow model: construct graph + execution graph

1 The basic steps of training the model

1.1 Structure diagram

The process of constructing a diagram is generally as follows:

• Step 1: Read data: tf.data
• Step 2: Create placeholders for input and label tags: tf.placeholder()
• Step 3: Create parameter variables, etc.: tf.get_variable()
• Step 4: inference
• Step 5: Construct loss function
• Step 6: create optimizer optimizer

eg:
opt = tf.train.GradientDescentOptimizer(learning_rate=0.001)
optimizer = opt.minimize(loss)

1.2 Training model

• Step 1: Initialize variables
• Step 2: Run the optimizer and use feed_dict to pass the input value and label into the model
• Step 3: Use FileWriter to write log files

writer = tf.summary.FileWriter('./graphs/result', sess.graph)

• Step 4: Use TensorBoard to see the results

tensorboard --logdir='./graphs'

2 Tensorflow read data and optimizer function

2.1 tf.data

The advantage of Placeholder is that it puts the data processing outside of tensorflow, which is convenient to use python, but the disadvantage is that it uses a single thread to process data, which usually reduces the execution process of the entire program.
The following are some functions for reading data, which can be read according to the format of the actual file:

• tf.data.Dataset.from_tensor_slices((features, labels))
• tf.data.TextLineDataset(filenames)
• tf.data.FixedLengthRecordDataset(filenames)
• tf.data.TFRecordDataset(filenames)

Is tf.data really more efficient?
Indeed, it takes time to use placeholder: 9.05s, and tf.data under the same conditions, the processing time is: 6.12s.

2.2 Optimizer

How does tensorflow know which variables need to be updated?
Session will update all the trainable variables that loss depends on during the training process.

All variable parameters are trainable by default, unless we specify that they do not need to be updated during the training process:

tf.Variable(initial_value=None, trainable=True, ...)

Let's take a look at some optimizers in tensorflow:

• tf.train.GradientDecscentOptimizer
• tf.train.AdagradOptimizer
• tf.train.MomentumOptimizer
• tf.train.AdamOptimizer
• tf.train.FtrlOptimizer
• tf.train.RMSPropOptimizer
• …