TensorFlow2.0 Tutorial - using low-level api training (non tf.keras)

TensorFlow2.0 Tutorial - using low-level api training (non tf.keras)

tensorflow2.0 recommended tf.keras such a high level api to build a network, but still maintained a low-level api tensorflow flexible structure of the network, this section we will introduce how to construct a neural network by using these low-level api and training.

Original Address: https://doit-space.blog.csdn.net/article/details/95040964

The most complete Tensorflow 2.0 Getting Started tutorial continuously updated: https://blog.csdn.net/qq_31456593/article/details/88606284

See the complete tensorflow2.0 tutorial code https://github.com/czy36mengfei/tensorflow2_tutorials_chinese (welcome star)

This tutorial focuses on learning by individuals reproduce notes tensorflow2.0 official tutorial from finishing, Chinese to explain, easy to enjoy reading tutorials Chinese friends, the official tutorial: https://www.tensorflow.org

一、Variables

TensorFlow tensor is stateless immutable object. When we have to change tensor, you can use the characteristics of python, the calculated value assigned to the variable python.

x = tf.ones([6,6])
x = x + 3 # x+3后得到了一个新的张量，并把这个张量赋给x
print(x)

tf.Tensor(
[[4. 4. 4. 4. 4. 4.]
 [4. 4. 4. 4. 4. 4.]
 [4. 4. 4. 4. 4. 4.]
 [4. 4. 4. 4. 4. 4.]
 [4. 4. 4. 4. 4. 4.]
 [4. 4. 4. 4. 4. 4.]], shape=(6, 6), dtype=float32)

However, the intermediate state machine learning needs to be changed (loss becomes smaller toward each parameter change in direction, but also built so TensorFlow state operation, which is Variables, which may represent the model parameters, and convenient and efficient.

Variables is an object value exists, when it is used, it is implicitly is read from the memory, and when there is such tf.assign_sub, tf.scatter_update when such operation, the new value is stored to the original object.

v = tf.Variable(2)
v.assign(6)
print(v)
v.assign_add(tf.square(3))
print(v)

<tf.Variable 'Variable:0' shape=() dtype=int32, numpy=6>
<tf.Variable 'Variable:0' shape=() dtype=int32, numpy=15>

Note: When the gradient will automatically track calculation variables (not Watch), a variable representing the embedded, TensorFlow uses sparse update the default, which can improve storage efficiency and calculated.

Second, Example: linear model

Use Tensor, Variable and GradientTape these simple, if you can build a simple model. Proceed as follows:

• Definition Model
• Defined loss function
• Data acquisition training
• Model training, using the adjustment variable optimizer

In the following we will construct a simple linear model: f (x) = W + b, it has two variables W and B, and we will use W = 3.0, b = 2.0 to structure data for learning.

1. Definition Model

We simple model is defined as a class, which encapsulates and calculated variables

class Model(object):
    def __init__(self):
        # 初始化变量
        self.W = tf.Variable(5.0)
        self.b = tf.Variable(0.0)
    
    def __call__(self, x):
        return self.W * x + self.b
# 测试
model = Model()
print(model(2))

tf.Tensor(10.0, shape=(), dtype=float32)

2. Defined loss function

Matching degree model output with a desired output loss measured as a function of a given input. As used herein, standard loss L2.

def loss(predicted_y, true_y):
    return tf.reduce_mean(tf.square(predicted_y - true_y))

3. Get the training data

Generating data with noise

TRUE_W = 3.0
TRUE_b = 2.0
num = 1000

# 随机输入
inputs = tf.random.normal(shape=[num])
# 随机噪音
noise = tf.random.normal(shape=[num])

# 构造数据
outputs = TRUE_W * inputs + TRUE_b + noise

Before our training model, so that we can see where the model now live. We will draw the prediction model in red, the training data plotted in blue.

import matplotlib.pyplot as plt

plt.scatter(inputs, outputs, c='b')
plt.scatter(inputs, model(inputs), c='r')
plt.show()

# 当前loss
print('Init Loss:')
print(loss(model(inputs), outputs))

Init Loss:
tf.Tensor(8.763554, shape=(), dtype=float32)

4. Define training cycle

We now have a model and training data. We are ready to start training, that is to update the model variables (W and b) using training data for use gradient descent to minimize losses. Achieve many variants of gradient descent scheme in the tf.train.Optimizer. It is strongly recommended that you use these implementations, but in the spirit of building from the first principle, in this particular case, we will achieve the basic optimizer themselves.

def train(model, inputs, outputs, learning_rate):
    # 记录loss计算过程
    with tf.GradientTape() as t:
        current_loss = loss(model(inputs), outputs)
        # 对W，b求导
        dW, db = t.gradient(current_loss, [model.W, model.b])
        # 减去梯度×学习率
        model.W.assign_sub(dW*learning_rate)
        model.b.assign_sub(db*learning_rate)
        

We repeated training, and observe the changes of W and b

model= Model()

# 收集W，b画图
Ws, bs = [], []
for epoch in range(10):
    Ws.append(model.W.numpy())
    bs.append(model.b.numpy())
    # 计算loss
    current_loss = loss(model(inputs), outputs)
    train(model, inputs, outputs, learning_rate=0.1)
    print('Epoch %2d: W=%1.2f b=%1.2f, loss=%2.5f' %
        (epoch, Ws[-1], bs[-1], current_loss))
# 画图
# Let's plot it all
epochs = range(10)
plt.plot(epochs, Ws, 'r',
         epochs, bs, 'b')
plt.plot([TRUE_W] * len(epochs), 'r--',
         [TRUE_b] * len(epochs), 'b--')
plt.legend(['W', 'b', 'true W', 'true_b'])
plt.show()

Epoch  0: W=5.00 b=0.00, loss=8.76355
Epoch  1: W=4.61 b=0.40, loss=5.97410
Epoch  2: W=4.30 b=0.72, loss=4.18118
Epoch  3: W=4.05 b=0.98, loss=3.02875
Epoch  4: W=3.85 b=1.18, loss=2.28800
Epoch  5: W=3.69 b=1.35, loss=1.81184
Epoch  6: W=3.56 b=1.48, loss=1.50577
Epoch  7: W=3.46 b=1.58, loss=1.30901
Epoch  8: W=3.38 b=1.67, loss=1.18253
Epoch  9: W=3.31 b=1.73, loss=1.10123