python diary: with Pytorch build a simple neural network

A recent study pytorch framework, by far the most to share a basic set up with pytorch neural network and method of training. It was the first to write this article to share, I hope to help beginners pytorch friends!

A task

At first said that we are building a network to complete the task of learning: Let's Academy of Neurology network exclusive OR operation, XOR operation is commonly known as "the same as taking 0, 1 different take." Then we need to say simply, that is, we need to build such a neural network, let us input and output 0:00 (1,1), output 1 input (1,0) (0 same take, take a different ), and so on.

Second, the realization of ideas

Because we need need to have two inputs and one output, so we need to set two input nodes and output layer is provided at an output node of the input layer. Because the problem is relatively simple, so we just need to set the hidden layer of 10 nodes can achieve good results, the activation function of the hidden layer we use ReLU function, we use the output layer Sigmoid function, so that the output remains at 0-1 in a range, if the output is greater than 0.5, to allow the output to 1, less than 0.5, so that the output is 0.

Third, the implementation process

When setting up the network, we use a simple method to quickly build, as building blocks of this method allows you to quickly and efficiently build up a neural network, specifically to achieve the following:

The first step: the introduction of the necessary libraries

code show as below:

import torch
import torch.nn as nn
import numpy as np

Of course, be introduced by pytorch torch package, in order to write the code and facilitate torch bag nn nn is replaced, this package is the abbreviation nn neural network, designed to take a neural network packet. Numpy introduced in order to create a matrix as input.

Step 2: Create an input set

code show as below:

# Construction input set 
X = np.mat ( ' 0 0; ' 
           ' 0. 1; ' 
           ' . 1 0; ' 
           ' . 1. 1 ' )
x = torch.tensor(x).float()
y = np.mat ( 1; 0; 0; ' ' 1 ' )
           
           
           
y = torch.tensor(y).float()

I personally prefer constructing a matrix with np.mat this way, I feel the wording is relatively simple, of course, you can also use other methods. But the matrix must have finished building this step torch.tensor (x) .float (), you have to put input into tensor created variables.

What is tensor it? You can simply understand that he is a kind of variable used in the pytorch, pytorch you want to use this framework must first convert your variable to a tensor variables. And we will ask you this neural network input and output must be a float floating-point, referring to the tensor variables float, while the input you use np.mat create is int type, will be converted into a tensor itensor automatically converted to the int type, so to add .float later () is converted to float.

Thus we have constructed a complete input and output (matrix are x and y matrix), x is a matrix of four rows and two columns, each row is a his input, a two input values, where we put all the input cases They are listed out. Y is an output of a matrix of four rows, each row is an output, the input of each line corresponding to the matrix x.

The third step: to build the network

code show as below:

# Build the network 
Mynet = nn.Sequential (
    nn.Linear(2, 10),
    nn.ReLU ();
    nn.Linear(10, 1),
    nn.Sigmoid()
)
print (Mynet)

We use nn package Sequential build the network, this function is that you can let us take a thing like building blocks neural network.

nn.Linear (2,10) means input layer structures, which represents the number of input nodes, the number 10 represents the output node. Linear is linear in English, which is the meaning of this layer does not include any other activation function, you enter what he will give you the output of the Han. nn.ReLU () This represents the activation of a function layer, you enter just threw ReLU to function. Then again a Linear, finally thrown to the Sigmoid function. 2,10,1 to represent the number of three layers, simple.

Step four: Set Optimizer

code show as below:

# Set the optimizer 
optimzer = torch.optim.SGD (myNet.parameters (), LR = 0.05 )
loss_func = nn.MSELoss()

The understanding is that this step, you need to have an optimized method to train your network, so this step we set the optimization method to be employed.

torch.optim.SGD means that the use of SGD training method, you only need it and you learn the parameters of the network will be able to pass in, respectively myNet.paramets and lr. loss_func phrase set the cost function, because our problem is relatively simple, so the use of MSE, which is the mean square error cost function.

Fifth step: training network

code show as below:

for epoch in range(5000):
    out = myNet(x)
    Loss = loss_func (OUT, Y)   # calculating an error 
    optimzer.zero_grad ()   # clear gradient 
    loss.backward ()
    optimzer.step()

I am here to set up a circulation of 5,000 (may not need so many times), so that the training action iteration 5000. Each output directly myNet (x), to enter your network get thrown into the output out (is so simple and crude!), Then the cost function and your standard output y seek error. Clear gradient that step in order to remove the last of the gradient obtained each time re-iteration, you'll remember this step on the line, beginners do not understand too. loss.backward () of course is to make back-propagation, then optimzer.step () is to let the optimizer we just set to work.

Step Six: Testing

code show as below:

print(myNet(x).data)

Output:

tensor([[0.9424],
        [0.0406],
        [0.0400],
        [0.9590]])

We can see the results very close to the results we expect, of course, you can also change the test data, the results will be similar. Simple explanation of why we here at the end of code with a .data, because our tensor variable actually consists of two parts, in part, tensor data, the other is automatic derivation of tensor parameters, we add meaning .data tensor data output is taken in, and if not it will output the following:

tensor([[0.9492],
        [0.0502],
        [0.0757],
        [0.9351]], grad_fn=<SigmoidBackward>)

Share on here today, my first post to a blog, written in inappropriate places also hope you criticism, do not know if I have you, to help beginners pytorch complete code is as follows:

import torch
import torch.nn as nn
import numpy as np

# Construction input set 
X = np.mat ( ' 0 0; ' 
           ' 0. 1; ' 
           ' . 1 0; ' 
           ' . 1. 1 ' )
x = torch.tensor(x).float()
y = np.mat ( 1; 0; 0; ' ' 1 ' )
           
           
           
y = torch.tensor(y).float()

# Build the network 
Mynet = nn.Sequential (
    nn.Linear(2, 10),
    nn.ReLU ();
    nn.Linear(10, 1),
    nn.Sigmoid()
)
print (Mynet)

# Set the optimizer 
optimzer = torch.optim.SGD (myNet.parameters (), LR = 0.05 )
loss_func = nn.MSELoss()

for epoch in range(5000):
    out = myNet(x)
    Loss = loss_func (OUT, Y)   # calculating an error 
    optimzer.zero_grad ()   # clear gradient 
    loss.backward ()
    optimzer.step()


print(myNet(x).data)