Beginner deep learning (1) Summary of the basic development steps of TensorFlow

Through the blog: Beginner Deep Learning (1) TensorFlow basic development steps prepare data and build models (forward + reverse) and Beginner Deep Learning (1) TensorFlow basic development steps iterative training and
introduction to the model .

The basic steps of tensorflow development can now be summarized as follows:

• Define tensorflow input node
• Define the "learning parameter" variable
• Define "operation"
• Optimization function
• Initialize all variables
• Iteratively update the parameters to the optimal solution
• Test model
• Use model

1. The method of defining tensorflow input nodes

There are several ways to define input nodes in tensorflow:

• Defined by placeholders: usually use this method
• Defined by dictionary type: generally used in the case of more input,
• Direct definition: rarely used,

(1) Use placeholders to define instances of input nodes

In the blog: Beginner Deep Learning (1) TensorFlow basic development steps prepare data and build a model (forward + reverse) is to define the input nodes through placeholders:

# X,Y为占位符
X = tf.placeholder("float") # 代表x的输入
Y = tf.placeholder("float") # 代表对应的真实值y

Complete code:

在这里插入代码片

(2) Through the dictionary type to define the instance of the input node

The way to define input nodes by dictionary type is similar to that of placeholders, but they are piled up together.

#占位符
inputdict = {
    
    
    'x':tf.placeholder("float"),
    'y':tf.placeholder("float")
}

(3) Examples of directly defining input nodes

Direct definition is to put the defined python variables directly into the OP node to participate in the input operation, and put the variables of the simulated data directly into the model for training.

2. Define the variables of "learning parameters"

The definition of "learning parameter" is very similar to the definition of input, which is divided into two parts : direct definition and dictionary type definition .
Since there are many parameters in the deep neural network example, the second type is generally used more.

(1) Directly define "learning parameters"

# 模型参数
W = tf.Variable(tf.random_normal([1]), name="weight") # w初始化为[-1,1]的随机数，形状为一维的数字
b = tf.Variable(tf.zeros([1]), name="bias") # b的初始化是0，形状为一维的数字
# 通过tf.Variable()函数对参数直接定义

(2) Define "learning parameters" by dictionary type

# 模型参数
paradict = {
    
    
    'W':tf.Variable(tf.random_normal([1])) # w初始化为[-1,1]的随机数，形状为一维的数字
    'b':tf.Variable(tf.zeros([1])) # b的初始化是0，形状为一维的数字
}

3. Define "Operation"

The process of defining "operation" is the core process of establishing the model , which directly determines the fitting effect of the model .

(1) Define the forward propagation model

# 创建正向模型
# X,Y为占位符
X = tf.placeholder("float") # 代表x的输入
Y = tf.placeholder("float") # 代表对应的真实值y

# 模型参数
W = tf.Variable(tf.random_normal([1]), name="weight") # w初始化为[-1,1]的随机数，形状为一维的数字
b = tf.Variable(tf.zeros([1]), name="bias") # b的初始化是0，形状为一维的数字

# 前向结构
z = tf.multiply(X, W)+ b # x*w+b

As shown above (blog: beginner deep learning (1) TensorFlow basic development steps: preparing data and building a model (forward + reverse) ) The network structure used is very simple, with only one neuron.
Later, you will learn more and more complex networks such as: multi-layer neural network, convolutional neural network, recurrent neural network, etc. They are all network structures composed of neurons in different combinations.

(2) Define the loss function

The loss function is mainly used to calculate the error between the "output value" and the "true value", which is used in conjunction with back propagation.
In order to find the minimum value (corresponding to the optimal solution) in backpropagation, the function must be derivable.

4. Optimize function, optimize target

With the forward structure and loss function, the learning parameters are optimized through the optimization function , and this process is also completed in back propagation.

PS：

Back-propagation process: It is to pass the error in the opposite direction along the forward-propagation structure .

5. Initialize all variables

There is only one line of code, but it is a very critical link.
After the session is created, the first thing is to initialize it.
For example, in the blog: Beginner Deep Learning (1) Iterative training and use model of the basic development steps of TensorFlow :

# 初始化变量
init = tf.global_variables_initializer() # global_variables_initializer（）函数初始化所有变量

note:

Use the tf.global_variables_initializer() function to initialize all variables, which must be completed after all variables and OP are defined . Only in this way can the defined content be valid , otherwise, the variables and OPs defined after initialization can not be calculated using the run in the session.

6. Iteratively update the parameters to the optimal solution

In the process of iterative training, it is necessary to establish a session to complete. It usually uses the with syntax , which can be closed after the session ends.

For example, in the blog: Beginner Deep Learning (1) Iterative training and use model of the basic development steps of TensorFlow :

with tf.Session() as sess:
    sess.run(init) # 通过sess.进行网络节点的运算
    plotdata = {
    
    "batchsize":[],"loss":[]} # 存放批次值和损失值

Run the nodes in the model through run in the session. The same is true in the iterative training process, except that the OP of the optimized operation is placed in the run , and the number of cycles needs to be added to the outer layer .

For example, in the blog: Beginner Deep Learning (1) Iterative training and use model of the basic development steps of TensorFlow :

    # 向模型输入数据
    for epoch in range(training_epochs):
        for (x, y) in zip(train_X, train_Y):
            sess.run(optimizer, feed_dict={
    
    X: x, Y: y}) # 通过feed机制将真实数据灌到占位符（第二步创建模型时）对应的位置

In the actual use process, a concept called MINIBATCH will be introduced for iterative training, that is, each time a certain amount of data is taken and placed in the network for training.

7. Test model

The test model is not the core part of the neural network, that is, the accuracy or error rate of the model is calculated to describe the quality of the model.

For example, in the blog: Beginner Deep Learning (1) Iterative training and use model of the basic development steps of TensorFlow :

    print ("cost=", sess.run(cost, feed_dict={
    
    X: train_X, Y: train_Y}), "W=", sess.run(W), "b=", sess.run(b))

8. Use the model

The use model is similar to the test model, that is, the loss value node can be replaced with an output node.

For example, in the blog: Beginner Deep Learning (1) Iterative training and use model of the basic development steps of TensorFlow :

print ("x=0.2，z=", sess.run(z, feed_dict={
    
    X: 0.2}))