Deep Learning Practicing Project (2)-----Using PyTorch for Linear Regression

Preface

Deep learning is not as difficult as imagined, and even simpler than some traditional machine learning. The mathematical knowledge used does not need to be particularly advanced. This article will use PyTorch to implement the classic model of linear regression.

1. Linear regression theory

Linear regression is a statistical analysis method that uses regression analysis in mathematical statistics to determine the quantitative relationship between two or more variables. It is widely used. In regression analysis, only one independent variable and one dependent variable are included, and the relationship between the two can be approximated by a straight line. This kind of regression analysis is called univariate linear regression analysis. If the regression analysis includes two or more independent variables, and the relationship between the dependent variable and the independent variable is linear, it is called multiple linear regression analysis.
Simply put: linear regression has a mapping f for input x and output y, y=f(x), and the form of f is aX+b . Among them, a and b are two adjustable parameters. The purpose of our training is to train the two parameters a and b.

Two, PyTorch code implementation

Let's use the code of pyTorch to do a detailed explanation

First, we need to import related packages

# 注意，这里我们使用了一个新库叫 seaborn 如果报错找不到包的话请使用pip install seaborn 来进行安装
import torch
from torch.nn import Linear, Module, MSELoss
from torch.optim import SGD
import numpy as np
import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
import seaborn as sns
torch.__version__

Let's define a linear function, here we use $Y = 5 x + 7$ , where 5 and 7 are the parameters a and b mentioned above, let’s use matplot to visualize this function first


x = np.linspace(0,20,500)
y = 5*x + 7
plt.plot(x,y)

Insert picture description here

Let’s generate some random points as our training data

x = np.random.rand(256)
noise = np.random.randn(256) / 4
y = x * 5 + 7 + noise
df = pd.DataFrame()
df['x'] = x
df['y'] = y

Show the data we generated on the graph

sns.lmplot(x='x', y='y', data=df)

Insert picture description here

We randomly generated some points. Next, we will use PyTorch to build a linear model to fit them. This is the so-called training process. Since there is only one layer of linear model, we will use it directly.

model=Linear(1, 1)

Among them, the parameter (1, 1) represents the number of input and output features (feature) is 1. The expression of the Linear model is $\cdot x+b$ , where $w$ stands for weight, $b$ stands for bias

Loss function we use the mean square loss function: MSELoss

criterion = MSELoss()

Optimizer We choose the most common optimization method SGD, which is to calculate the gradient of the mini-batch in each iteration, and then update the parameters, the learning rate is 0.01

optim = SGD(model.parameters(), lr = 0.01)

Train 3000 times

epochs = 3000

Prepare training data: The shape of x_train, y_train is (256, 1), which means that the mini-batch size is 256, and the feature is 1. astype('float32') is to be directly converted to torch.float in the next step.

x_train = x.reshape(-1, 1).astype('float32')
y_train = y.reshape(-1, 1).astype('float32')

Start training

for i in range(epochs):
    # 整理输入和输出的数据，这里输入和输出一定要是torch的Tensor类型
    inputs = torch.from_numpy(x_train)
    labels = torch.from_numpy(y_train)
    #使用模型进行预测
    outputs = model(inputs)
    #梯度置0，否则会累加
    optim.zero_grad()
    # 计算损失
    loss = criterion(outputs, labels)
    # 反向传播
    loss.backward()
    # 使用优化器默认方法优化
    optim.step()
    if (i%100 == 0):
        #每 100次打印一下损失函数，看看效果
        print('epoch {}, loss {:1.4f}'.format(i,loss.data.item()))

The training is complete, let's see what the results of the training are. Use model.parameters() to extract model parameters. $w$ ， $b$ is the model parameter we need to train. We expect the data $w = 5$ ， $b = 7$ can do a comparison

[w, b] = model.parameters()
print (w.item(),b.item())

Visualize our model again and take a look at the data we trained. If you don’t like seaborn, you can use matplot directly


predicted = model.forward(torch.from_numpy(x_train)).data.numpy()
plt.plot(x_train, y_train, 'go', label = 'data', alpha = 0.3)
plt.plot(x_train, predicted, label = 'predicted', alpha = 1)
plt.legend()
plt.show()

Insert picture description here

references

https://github.com/zergtant/pytorch-handbook/blob/master/chapter2/2.2-deep-learning-basic-mathematics.ipynb