In the previous sections, simple linear models were involved. This chapter will use the tools provided by pytorch to implement linear regression. We will introduce things such as Module, constructing Loss function, constructing stochastic gradient descent optimizer, etc. These things will be provided by pytorch, so the main role of this chapter is how to use the tools provided by pytorch to help us realize the construction of linear models more conveniently .
First of all, let’s review what we said earlier. The first step is to determine the Linear Model, and then define the loss function for optimization. When optimizing in pytorch, sgd (stochastic gradient descent) is used, and the core step of using sgd is to ask gradient) of each loss function with respect to the weight . This is the step of optimization. work to be done. Next, we began to use the tools provided by pytorch to implement the linear model. Although the following code may be more troublesome, it has better scalability, in other words, it can be used to do a lot of things.
- Prepare dataset
- Build a model with class (inherited from nn.Moudle) to calculate y_hat
- Construct loss function and optimizer (using pytorch API)
- Write the training cycle (forward, backward, update; feedforward calculates loss, feedback calculates gradient, stochastic gradient descent updates weights)
The sample dataset we prepared here is very simple, as shown below, both x and y are 3×1 matrices.
x_data = torch.Tensor([[1.0], [2.0], [3.0]])
y_data = torch.Tensor([[2.0], [4.0], [6.0]])When we were training in the front, we had to manually seek derivatives. In pytorch, we no longer manually seek derivatives. Our focus should be shifted to the construction of the calculation graph . As long as the calculation graph can be successfully constructed, the gradient and the like will be automatically calculated, and then we can optimize it. A simple computational graph is shown in Figure 1.
After x is input, y_hat is obtained through a linear unit operation, and then enters the loss function to obtain the loss value. After calculating the loss value, you can call backward on the loss, and then backpropagate the entire calculation graph. The following is the code part of the model construction:
class LinearModel (torch.nn.Module) : # 必须继承自 torch.nn.Module
def __init__(self) : # 所有的模型至少都要实现 init 和 forward
super(LinearModel, self).__init__() # 这行不用管他为什么，直接写上就对了
self.linear = torch.nn.Linear(1, 1) # 构造对象，linear 中已经包含了 ω 和 b，两个参数分别为输入输出的 size
# 用 module 构造出来的对象，会自动帮助实现 backward 的过程
def forward (self, X) : # forward 是前馈的过程中所要进行的计算
y_pred = self.linear (x)
return y_pred
model = LinearModel()
# 优化器
criterion = torch.nn.MSELoss(size_average = False)
optimizer = torch.optim.SGD(model.parameters()，lr=0.01)The following is the training process: This set of processes is carried out in the framework, and the principle is exactly the same as the manual training mentioned earlier. Obviously, using a framework can greatly reduce the amount of code.
for epoch in range(100) :
y_pred = model(x_data) # 先在前馈中算y_hat
loss = criterion(y_pred, y_data) # 计算损失
print(epoch, loss)
optimizer.zero_grad () # 一定注意--梯度清零--
loss.backward() # 反向传播
optimizer.step () # 进行更新
# 训练情况的输出
print('W = '，model.linear.weight.item())
print('b = '，model.linear.bias.item())
# Test Model
x_test = torch.Tensor([[4.0]])
y_test = model(x_test)
print('y_pred ='，y_test.data)
The above is a complete routine of model training.
The complete reproducible code is as follows:
import torch
x_data = torch.Tensor([[1.0], [2.0], [3.0]])
y_data = torch.Tensor([[2.0], [4.0], [6.0]])
class LinearModel(torch.nn.Module): # 必须继承自 torch.nn.Module
def __init__(self): # 所有的模型至少都要实现 init 和 forward
super(LinearModel, self).__init__() # 这行不用管他为什么，直接写上就对了
self.linear = torch.nn.Linear(1, 1) # 构造对象，linear 中已经包含了 ω 和 b，两个参数分别为输入输出的 size
# 用 module 构造出来的对象，会自动帮助实现 backward 的过程
def forward(self, X): # forward 是前馈的过程中所要进行的计算
y_pred = self.linear(x)
return y_pred
model = LinearModel()
# 优化器
criterion = torch.nn.MSELoss(size_average = False)
optimizer = torch.optim.SGD(model.parameters(), lr = 0.01)
for epoch in range(100):
y_pred = model(x_data) # 先在前馈中算y_hat
loss = criterion(y_pred, y_data) # 计算损失
print(epoch, loss)
optimizer.zero_grad() # 一定注意--梯度清零--
loss.backward() # 反向传播
optimizer.step() # 进行更新
# 训练情况的输出
print('W = ', model.linear.weight.item())
print('b = ', model.linear.bias.item())
# Test Model
x_test = torch.Tensor([[4.0]])
y_test = model(x_test)
print('y_pred =', y_test.data)