Univariate Linear Regression of TensorFlow Notes

---

foreword

Record the process of completing univariate linear regression using TensorFlow1.x and TensorFlow2.x.

---

1. Dataset generation

Generate a y=2x+1 dataset with standard normal distribution noise

import numpy as np
import matplotlib.pyplot as plt
#数据集生成
x_data = np.linspace(-1, 1, 100)
#引入随机数噪声
np.random.seed(0)
y_data = 2 * x_data + 1 + 0.5 * np.random.randn(x_data.shape[0])
plt.scatter(x_data, y_data)

Two, TensorFlow1.x

1. Define the model

import tensorflow.compat.v1 as tf
tf.disable_eager_execution()
#创建变量
x = tf.placeholder('float', name='x')
y = tf.placeholder('float', name='y')
w = tf.Variable(0.0, name='w')
b = tf.Variable(0.0, name='b')
#构建模型
def model(x, w, b):
    return w * x + b
#预测节点
pred = model(x, w, b)

2. Training model

#训练参数:训练轮数,学习率
train_epoch = 10
learning_rate = 0.01
#显示损失函数
step = 0
display_step = 5
loss_list = []
loss_function = tf.reduce_mean(tf.square(y - pred))
#定义优化器,给定学习率,以最小化损失函数为目的
optimizer = tf.train.GradientDescentOptimizer(learning_rate)\
    .minimize(loss_function)
#变量初始化
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
#迭代训练
for epoch in range(train_epoch):
    for xi, yi in zip(x_data, y_data):#按第一维度拼接
        #权重更新
        _, loss = sess.run([optimizer, loss_function], feed_dict={
    
    x:xi, y:yi})
        #显示训练过程
        step = step + 1
        if step%display_step == 0:
            loss_list.append(loss)
    print('w=',sess.run(w),'b=',sess.run(b))
plt.plot(loss_list)

Weight update tends to be stable

Stochastic gradient descent, each time a sample is used for training, the loss function as a whole gradually decreases, fluctuating due to the influence of a single data point

3. Model prediction

x_test = np.linspace(1, 10, 100)
y_test = 2 * x_test + 1
y_pred = sess.run(pred, feed_dict={
    
    x:x_test})
plt.scatter(x_test, y_test, c='b')
plt.scatter(x_test, y_pred, c='g')
sess.close()

3. TensorFlow2.x

1. Define the model

import tensorflow as tf
#创建变量
w = tf.Variable(np.random.randn(), tf.float32)
b = tf.Variable(np.random.randn(), tf.float32)
#建立模型
def model(x, w, b):
    return w * x + b

2. Training model

#定义超参数
train_epoch = 50
learning_rate = 0.1
#显示损失函数
step = 0
display_step = 1
loss_list = []
def loss_function(x, y, w, b):
    pred = w * x + b
    loss = tf.reduce_mean(tf.square(y - pred))
    return loss
#迭代训练
for epoch in range(train_epoch):
    #自动求导
    with tf.GradientTape() as tape:
        loss = loss_function(x_data, y_data, w, b)
        grad_w, grad_b = tape.gradient(loss, [w, b])
    #显示训练过程
    step = step + 1
    if step % display_step == 0:
        print('w=', w.numpy(), 'b=', b.numpy())
        loss_list.append(loss.numpy())
    #权重更新
    w.assign_sub(grad_w * learning_rate)
    b.assign_sub(grad_b * learning_rate)
plt.plot(loss_list)

Batch gradient descent, using the entire data set for training each time, is less affected by a single data point, due to the reduced frequency of weight update, it is necessary to appropriately increase the learning rate and the number of training rounds

3. Model prediction

x_test = np.linspace(1, 10, 100)
y_test = 2 * x_test + 1
y_pred = w * x_test + b
plt.scatter(x_test, y_test, c='b')
plt.scatter(x_test, y_pred, c='g')

---

Summarize

TensorFlow1.x uses gradient descent optimizer for weight update, while TensorFlow2.x uses derivation tool for weight update. When switching from 1.x to 2.x, you need to exit the console to restart imperative programming. Stochastic gradient descent uses one sample each time for weight update, which has a small amount of calculation and is greatly affected by a single data point; batch gradient descent uses all samples for weight update each time, is less affected by a single data point, and requires the number of iterations Increase, the amount of calculation is large; stochastic gradient descent uses some samples to update the weight each time, between the two, when the sample size and the difference between samples are large, it can be selected.