Concise implementation of d2l linear regression

Simple implementation of linear regression

• Previous section Tensors: data storage, linear algebra; automatic differentiation: calculating gradients
• Open source framework that automates repetitive tasks in gradient-based learning algorithms
• Data iterators, loss functions, optimizers, neural network layers

• Using a deep learning framework to concisely implement a linear regression model to generate a data set

1. Generate data set

import numpy as np
import torch
from torch.utils import data
from d2l import torch as d2l

true_w = torch.tensor([2, -3.4])
true_b = 4.2
features, labels = d2l.synthetic_data(true_w, true_b, 1000)

2. Call the existing API of the framework to read data

def load_array(data_arrays,batch_size,is_train=True):
    dataset = data.TensorDataset(*data_arrays)
    return data.DataLoader(dataset,batch_size,shuffle=is_train)#shuffle是否要随机打乱顺序
batch_size=10
data_iter=load_array((features,labels),batch_size)
next(iter(data_iter))#使用next从迭代器中获取第一项

3. Use the framework’s predefined layers

• Standard deep learning model, using the framework’s predefined layers
• Focus on which layers are used to construct the model, not on the implementation details of the layers
• The Sequential class connects multiple layers together (standard pipeline)
  • When given input data, the Sequential instance passes the data into the first layer,
  • Then use the output of the first layer as the input of the second layer, and so on.

# nn 神经网络缩写
from torch import nn
# 线性回归 全连接层 （指定输入特征形状，指定输出特征形状）
# Sequential 容器
net=nn.Sequential(nn.Linear(2,1))

4. Initialize model parameters

• Weights and biases in linear regression models

# net 就是一个layer网络中的第一个图层， weight访问到w，normal使用正态分布替换掉data的值
net[0].weight.data.normal_(0, 0.01)
# bias偏差
net[0].bias.data.fill_(0)

5. Mean square error

• MSELoss class square norm (L2 norm)
• The default returns the average loss of all samples

loss=nnLMSELoss()

6. Instantiate SGD instance (optimization algorithm)

• Mini-batch stochastic gradient descent, only need to set the lr value

# 通过net.parameters()从模型中获得
trainer = torch.optim.SGD(net.parameters(), lr=0.03)

7. Training

• No need to assign parameters individually; no need to define a loss function; no need to manually implement mini-batch stochastic gradient descent
• Complex model, with all basic components
   
• every iteration
  • Completely traverse the data set once
  • (Get a mini-batch input and corresponding labels from it)
    • Call net(x) to generate predictions and calculate loss 1 (forward propagation)
    • Compute gradients via backpropagation
    • Update model parameters by calling the optimizer
  • Calculate the loss after each iteration and print it to monitor the training process

num_epochs = 3
for epoch in range(num_epochs):
    for X, y in data_iter:
        l = loss(net(X) ,y)
        trainer.zero_grad()
        l.backward()
        trainer.step()
    l = loss(net(features), labels)
    print(f'epoch {
      
      epoch + 1}, loss {
      
      l:f}')

• Compare the true parameters of the generated data set with the model parameters obtained by training on the limited data set
• • Access the desired layer from net and read the weights and biases of that layer

w = net[0].weight.data
print('w的估计误差：', true_w - w.reshape(true_w.shape))
b = net[0].bias.data
print('b的估计误差：', true_b - b)