4- depth hands-on learning simple linear regression pytorch achieve

• Introduced before introducing the same package or module
• Generating a data set
• Read data pytorch
• Definition Model
• Model initialization
• Defined loss function
• Custom optimization algorithm
• Trainer
• summary

To achieve the same functions in this section use pytorch module, generate a more compact code

Introduced before introducing the same package or module

%matplotlib inline
import torch
from IPython import display
from matplotlib import pyplot as plt
import numpy as np
import random

Generating a data set

num_inputs =2   ## 特征数量
num_examples=1000   # 样本量
true_w=[2,-3.4]  # 真实的权重系数
true_b=4.2  # 真实的偏置量
features = torch.randn(num_examples,num_inputs,dtype=torch.float32)   # 生成随机的特征
labels = true_w[0]*features[:,0]+true_w[1]*features[:,1]+true_b  # 生成随机的标签
labels += torch.tensor(np.random.normal(0,0.01,size=labels.size()),dtype=torch.float32)  #在标签上加上随机噪声项

Read data pytorch

Providing pytotch data packet to read the data. Since data is often used as a variable name, we introduced the data module data Data instead. In each iteration, we will random reads small batch comprising 10 samples

import torch.utils.data as Data
batch_size=10
dataset= Data.TensorDataset(features,labels)
data_iter = Data.DataLoader(dataset,batch_size,shuffle=True)
for X,y in data_iter:
    print(X,y)
    break

tensor([[ 1.9769,  1.5576],
        [-0.4743,  0.8653],
        [ 0.2641,  1.9682],
        [-2.3385,  0.3753],
        [ 0.3972, -0.6515],
        [ 1.1317, -0.2586],
        [ 1.6896,  1.0102],
        [-0.6803,  0.7734],
        [-0.3525, -0.7764],
        [ 0.3199,  0.9397]]) tensor([ 2.8727,  0.3198, -1.9711, -1.7576,  7.2187,  7.3517,  4.1360,  0.2171,
         6.1193,  1.6434])

Definition Model

• From the realization of a zero-based, we need to define the model parameters and use them step by step description of how the model is calculated. When the model results is complicated, the steps become more complicated. In fact pytorch provides a number of predefined layers, which makes me, just need to focus on what layer to use to construct the model. Here pytorch more concise definition of linear regression.

• First import torch.nn module, in fact, nn is the acronym of neural network. The module defines a number of layers of the neural network, used before autograd, and nn is the use autograd to define the model.
• Nn is the core data structure Module1, it is an abstract concept, both represent a layer in the neural network, the neural network may be expressed comprise many layers.

• In actual use, through inheritance torch.Module will compose their own network / layers. A nn.Module instance should contain and have a forward propagation (Forward) method returns the output.

from torch import nn

class LinearNet(nn.Module):
    def __init__(self,n_features):
        super(LinearNet,self).__init__()
        self.linear = nn.Linear(n_features,1)   # 这里的1是指out_features
    def forward(self,x):
        y = self.linear(x)
        return y
net = LinearNet(num_inputs)  # 输入特维度为2，输出结果维度为1
print(net)

LinearNet(
  (linear): Linear(in_features=2, out_features=1, bias=True)
)

In fact, we can also use nn.Sequential be more convenient to build the network, is a Sequential ordered container, the network layer are sequentially added to the calculation in accordance with FIG. Sequential order of incoming.

# 写法1
net= nn.Sequential(nn.Linear(num_inputs,1)
                  # 此处还可以传入其他的层
                  )
# 写法2
net = nn.Sequential()
net.add_module('linear',nn.Linear(num_inputs,1))
# net.add_module .....


# 写法3

from collections import OrderedDict

net = nn.Sequential(
OrderedDict([
    ('linear',nn.Linear(num_inputs,1))
    # 其他的层
    
])
)

print(net)
print(net[0])

Sequential(
  (linear): Linear(in_features=2, out_features=1, bias=True)
)
Linear(in_features=2, out_features=1, bias=True)

 for param in net.parameters():
        print(param)

Parameter containing:
tensor([[-0.4229, -0.0282]], requires_grad=True)
Parameter containing:
tensor([0.0852], requires_grad=True)

As a single neural network, the input layer neurons and output layer of the linear regression of each input completed fully connected, so a linear regression output layer is called fully connected layer

Model initialization

Before using the net, we need to initialize the model parameters, such as linear regression model weights and biases. Providing a variety of parameters pytorch initialization method init model. Init here is an abbreviated form of initializer. We init.normal_ the weight parameter for each element is initialized to a random sample mean 0 and standard deviation of the normal distribution of 0.01. Deviation is initialized to 0

from torch.nn import init
init.normal_(net[0].weight,mean=0,std=0.01)
init.constant_(net[0].bias,val=0)
# 该写法与后面的三种写法才可以使用，如果使用一开始的写法，net[0].weight 因改为net.linear.weight bias亦然。
#因为net[0]这样的写法只有当net是ModuleList或者Sequential实例时才可以。
# 

Parameter containing:
tensor([0.], requires_grad=True)

Defined loss function

# pytorch在nn模块中提供了各种损失函数，这些损失函数可以看做是一种特殊的层，pytorch也将这些损失函数实现为nn.Module的子类。这里我们使用
# pytorch提供的均方误差损失作为模型的损失函数
loss = nn.MSELoss()

Custom optimization algorithm

Similarly, we also do not need to own small quantities stochastic gradient descent algorithm, torch.optim module provides a number of commonly used optimization algorithm, SGD, Adam, RMSPorp and so on. Let's create an instance for the optimization optimizer net of all parameters, and specify the rate of learning in small quantities stochastic gradient decreased 0.03 (SGD) for the optimization algorithm.

import torch.optim as optim
optimizer = optim.SGD(net.parameters(),lr=0.03)
print(optimizer)

SGD (
Parameter Group 0
    dampening: 0
    lr: 0.03
    momentum: 0
    nesterov: False
    weight_decay: 0
)

We can set different rates for different learning sub-networks, which are often used in finetune time.

optimizer = optim.SGD(
[
    {'params':net.subset1.parameters()},  # lr 默认用最外层的学习率
    {'params':net.subset2.parameters(),'lr':0.01}
],
    lr= 0.03
)

Trainer

When using pytorch training model, we have to iterate the model parameters by calling optim examples of step functions. By definition of small quantities of stochastic gradient descent, we specify the size of the batch in step functions
so that the bulk of the sample are averaged gradient

num_epochs =3
for epoch in range(1,num_epochs+1):
    for X,y in data_iter:
        output = net(X)
        l = loss(output,y.view(-1,1))
        optimizer.zero_grad()  # 梯度清零，等价于net.zero_grad()
        l.backward()
        optimizer.step()
    print('epoch %d loss: %f'%(epoch,l.item()))
        

epoch 1 loss: 0.000286
epoch 2 loss: 0.000199
epoch 3 loss: 0.000072

Below we compare the model parameters and learned the true model parameters. We need to get from the net layers, and to access its weight (weight) and bias (bias). Learned parameters and actual parameters is very close

dense=net[0]
print(true_w,dense.weight)
print(true_b,dense.bias)

[2, -3.4] Parameter containing:
tensor([[ 2.0010, -3.3996]], requires_grad=True)
4.2 Parameter containing:
tensor([4.2005], requires_grad=True)

summary

• Use pytorch can achieve a more compact model
• torch.utils.data module provides data about the processing tool, torch.nn module defines a number of layers of the neural network, torch.nn.init initialization module defines various methods, torch.optim module provides initialization parameters of the model Various methods