pytorch learn ter simple regression

Return to solve the problem using neural networks

Machine learning problems are divided into two categories:

• Regression
• Classification

For the output value is continuous, called regression.
For output only a finite number of discrete values, called classification.

Today to see the video tutorial, a neural network to solve a simple regression.
Directly on the code

import torch
from torch.autograd import Variable
import torch.nn.functional as F
import matplotlib.pyplot as plt

#fake data
x = torch.unsqueeze(torch.linspace(-1,1,100),dim=1)#这句话是把x本来是个一维的数据进行升维 
y = x.pow(2) + 0.2*torch.rand(x.size())           #把 x^2乘2的数据加上一个随机噪声。 rand应该是生成0-1之间的一个均匀分布    

x,y = Variable(x),Variable(y) #老版本的包装代码。自动微分变量

#plt.scatter( x.data.numpy(),y.data.numpy() )#打印散点图
#plt.show()  

class Net(torch.nn.Module):  #继承torch.nn.Moudle这个模块，可以调用这个模块的接口
    def __init__(self, n_features,n_hidden,n_output ):     #搭建计算图的一些变量的初始化
        super(Net,self).__init__()  #将self（即自身）转换成Net类对应的父类，调用父类的__init__（）来初始化
        self.hidden = torch.nn.Linear( n_features,n_hidden )
        self.predict = torch.nn.Linear( n_hidden,n_output )
        
              
    def forward(self,x):      #前向传播，搭建计算图的过程
        x = F.relu(self.hidden(x))
        x = self.predict(x)
        return x

net = Net(1,10,1)  #初始时的参数只是多少，貌似没有赋值？或者说默认是全零？
print(net)

plt.ion() #matplotlib变成一个实时打印的状态
plt.show()

optimizer = torch.optim.SGD( net.parameters(),lr = 0.5 )   #随机梯度下降算法
loss_func = torch.nn.MSELoss();                            #代价函数,应该是二次代价函数。

for t in range(100):
    prediction = net(x)
    loss = loss_func( prediction, y )
    optimizer.zero_grad()                                #清空上一部的梯度值，否则应该会累加上去。
    loss.backward()                                       #反向传播就散梯度值
    optimizer.step()                                      #用计算得到的梯度值，来更新神经网络里面的各个参数
    

    
    if t % 5 == 0:
        plt.cla()                                         #清楚上幅图的散点
        plt.scatter( x.data.numpy(),y.data.numpy() )      #原始数据的散点图
        plt.plot(x.data.numpy(),prediction.data.numpy(),'r-',lw=5)
        plt.text( 0.5, 0,'LOSS=%.4f'% loss.data.numpy(),fontdict={'size':20,'color': 'red'} )
        plt.pause(0.1)

plt.ioff()
plt.show()
        

Code Analysis

Dummy data structure

x = torch.unsqueeze(torch.linspace(-1,1,100),dim=1)#这句话是把x本来是个一维的数据进行升维 
y = x.pow(2) + 0.2*torch.rand(x.size())

torch.linspace function, apparently from -1 to 1, 100 aliquots intermediate point, and then forming an array,
sense unsqueeze function in that one-dimensional array operates liter dimensional,
i.e., from [], is changed to [ []].
Y is equal to said value x ^ 2, while the value obtained by adding a random fluctuation, thus obtaining a set of dummy data.

Class defines a neural network

class Net(torch.nn.Module):  #继承torch.nn.Moudle这个模块，可以调用这个模块的接口
    def __init__(self, n_features,n_hidden,n_output ):     #搭建计算图的一些变量的初始化
        super(Net,self).__init__()  #将self（即自身）转换成Net类对应的父类，调用父类的#__init__（）来初始化
        self.hidden = torch.nn.Linear( n_features,n_hidden )
        self.predict = torch.nn.Linear( n_hidden,n_output )
        
              
    def forward(self,x):      #前向传播，搭建计算图的过程
        x = F.relu(self.hidden(x))
        x = self.predict(x)
        return x

This class is inherited from torch.nn.Module's
first look at the constructor
Super (Net, Self). The init ()
This is a call to the parent class constructor to initialize

self.hidden = torch.nn.Linear( n_features,n_hidden )
self.predict = torch.nn.Linear( n_hidden,n_output )

nn.Linear generate a linear model, wx + b a computational model of this form, n_features number is entered,
n_hidden is the number of hidden layers.

If n_features is 2, n_hidden 3, a calculation model should correspond as follows:

self.hidden, self.predict this member should be a definition similar operation on such a linear model of FIG.

However __init __ () function, there are no operation process does not create a computing node in FIG. Create a real
function DAG node is the forward (x) function,

    def forward(self,x):      #前向传播，搭建计算图的过程,输入值为x， 经过下面的运算得到一个返回值就是y。
        x = F.relu(self.hidden(x))
        x = self.predict(x)
        return x

When the current spread of the input variables x,
self.hidden (x) is called calculation model just created, set up an algorithm node, while the dependence of X is also identified F.relu activation function, such incentives I heard a very popular function neural network learning in depth in recent years, slowly replaces the sigmoid function of

the chart is a comparison of several activation functions. relu most simple function, a value less than 0 are all 0, 0 is greater than x, the simplest but most popular. interesting.

= self.predict the X-(the X-)
return the X-
After another layer of mapping linear model, this is not coupled with a relu function. Return x, x is the predicted value is the y-
. Where will call forward function builds calculation drawing it?
This sentence should be a
prediction = net (x), this sentence should call forward.

Optimization objective function

net = Net(1,10,1)  #初始化神经网络模型，输入层个数为1，隐藏层为10，输出个数为1.初始时的参数值是多少，貌似没有赋值？或者说默认是全零？
print(net)

optimizer = torch.optim.SGD( net.parameters(),lr = 0.5 )   #随机梯度下降算法，注意传入了net.parameters()，说明，optimizer已经关联上了神#经网络的参数
loss_func = torch.nn.MSELoss();                            #代价函数,应该是二次代价函数。

new = Net (1,10,1)
here is to create a number of input layer 1, the hidden layer neuron number is 10, the number of a neural network output layer 1. This invokes __init__ function is actually calculated figure is not generated, but the neural network to take out approximately

optimizer = torch.optim.SGD( net.parameters(),lr = 0.5 )
设置一个一个优化算法。这里选择的是随机下降算法，之前学过，有点忘了，大致的含义就是：实际梯度下降应该每次迭代都应该用所有的数据来计算一次，但是呢，那个计算量太大了，或者说一次没办法知道所有的数据。所以改为每次都用一个数据来进行反向传播，这种方法就叫做随机梯度下降，这种方法几年前学过原理，有点忘了，以后再复习，还得注意到optimizer已经和神经网络的参数net.parameter()挂钩了，以后它做的所有操作，都会影响到net.parameter.当然主要是zeros_grad()以及step(),前者用来每次将参数的梯度值进行清零操作，后者用每次计算的梯度值来调整参数值。

loss_func = torch.nn.MSELoss( );
代价函数一般都用这个，(y’-y)^2。这个值越小越好，我们就是要通过迭代，找到一组合适的参数，使得这个代价函数最小。

反向传播，调整参数

   for t in range(100):
        prediction = net(x)
        loss = loss_func( prediction, y )
        optimizer.zero_grad()                                #清空上一步的参数梯度值，否则应该会累加上去。
        loss.backward()                                      #反向传播，计算梯度值
        optimizer.step()      								 #调整参数值

循环100次呢，代表是要反向传播100次，来迭代得寻找最优的那组神经网络参数。
prediction = net(x），这显然是个典型的神经网络的前向传播过程，那么它肯定会调用net的forward函数，构建计算图节点
loss = loss_func（prediction，y)又构建了新的一个计算代价函数的节点。optimizer.zero_grad()是把优化器中记录的各个参数的梯度值置零，这步操作是在每次反向传播的之前进行的。之前看一个教程是每个梯度如果不清除的话，会叠加上去，而不是覆盖，可能这是需要这步操作的原因。

FIG whole calculation model as follows:

optimizer.step () is the gradient value of the parameter of each calculation, a parameter update.

Each update parameters, parameter values will make towards making the smallest LOSS direction update function (of course, a single point of view, perhaps necessarily, but the overall trend is definitely), set the number of iterations of 100 times, 100 times back-propagation, each times will use the gradient value of the parameter to update the look parameter values. Finally fitting parameter values, predicted values and sample values obtained before the spread is very similar to the
first time

last