Super simple to build a network from scratch learning pytorch Lesson 1

1 task

At first said that we are building a network to complete the task of learning:
let us learn neural network exclusive OR operation, XOR operation is commonly known as "the same as taking 0, 1 different take." Then we need to say simply, that is, we need to build such a neural network, let us input and output 0:00 (1,1), output 1 input (1,0) (0 same take, take a different ), and so on.

2 realization of ideas

Because we need need to have two inputs and one output, so we need to set two input nodes and output layer is provided at an output node of the input layer. Because the problem is relatively simple, so we just need to set the hidden layer of 10 nodes can achieve good results, the activation function of the hidden layer we use ReLU function, we use the output layer Sigmoid function, so that the output remains at 0-1 in a range, if the output is greater than 0.5, to allow the output to 1, less than 0.5, so that the output is 0.

3 implementation process

Quickly build simple method we use.

3.1 introduce the necessary library

import torch
import torch.nn as nn
import numpy as np

Of course, be introduced by pytorch torch package, in order to write the code and facilitate torch bag nn nn is replaced, this package is the abbreviation nn neural network, designed to take a neural network packet. Numpy introduced in order to create a matrix as input.

3.2 Creating training set

# 构建输入集
x = np.mat('0 0;'
           '0 1;'
           '1 0;'
           '1 1')
x = torch.tensor(x).float()
y = np.mat('1;'
           '0;'
           '0;'
           '1')
y = torch.tensor(y).float()

I personally prefer constructing a matrix with np.mat this way, I feel the wording is relatively simple, of course, you can also use other methods. But the matrix must have finished building this step torch.tensor (x) .float (), you have to put input into tensor created variables.

What is tensor it? You can simply understand that he is a kind of variable used in the pytorch, pytorch you want to use this framework must first convert your variable to a tensor variables. And we will ask you this neural network input and output must be a float floating-point, referring to the tensor variables float, while the input you use np.mat create is int type, will be converted into a tensor itensor automatically converted to the int type, so to add .float later () is converted to float.

Thus we have constructed a complete input and output (matrix are x and y matrix), x is a matrix of four rows and two columns, each row is a his input, a two input values, where we put all the input cases They are listed out. Y is an output of a matrix of four rows, each row is an output, the input of each line corresponding to the matrix x.

3.3 to build the network

myNet = nn.Sequential( 
    nn.Linear(2,10),
    nn.ReLU(),
    nn.Linear(10,1),
    nn.Sigmoid()
    )
print(myNet)

Output:

We use nn package Sequential build the network, this function is that you can let us take a thing like building blocks neural network.

nn.Linear (2,10) means input layer structures, which represents the number of input nodes, the number 10 represents the output node. Linear is linear in English, which is the meaning of this layer does not include any other activation function, you enter what he will give you the output of the Han. nn.ReLU () This represents the activation of a function layer, you enter just threw ReLU to function. Then again a Linear, finally thrown to the Sigmoid function. 2,10,1 to represent the number of three layers, simple.

Set Optimizer 3.4

optimzer = torch.optim.SGD(myNet.parameters(),lr=0.05)
loss_func = nn.MSELoss()

The understanding is that this step, you need to have an optimized method to train your network, so this step we set the optimization method to be employed.

torch.optim.SGD means that the use of SGD training method, you only need it and you learn the parameters of the network will be able to pass in, respectively myNet.paramets and lr. loss_func phrase set the cost function, because our problem is relatively simple, so the use of MSE, which is the mean square error cost function.

3.5 Training Network

for epoch in range(5000):
    out = myNet(x)
    loss = loss_func(out,y)
    optimzer.zero_grad()
    loss.backward()
    optimzer.step()

I am here to set up a circulation of 5,000 (may not need so many times), so that the training action iteration 5000. Each output directly myNet (x), to enter your network get thrown into the output out (is so simple and crude!), Then the cost function and your standard output y seek error. Clear gradient that step in order to remove the last of the gradient obtained each time re-iteration, you'll remember this step on the line, beginners do not understand too. loss.backward () of course is to make back-propagation, then optimzer.step () is to let the optimizer we just set to work.

3.6 Test

print(myNet(x).data)

operation result:

You can see the results very close to the results we expect, of course, you can also change the test data, the results will be similar. Simple explanation of why we here at the end of code with a .data, because our tensor variable actually consists of two parts, in part, tensor data, the other is automatic derivation of tensor parameters, we add meaning .data tensor data output is taken in, and if not it will output the following:

Bloggers also beginners, until like a decision tree model, but began to thoroughly understand the inspirational today one of deep learning framework PyTorch. Such bigwigs hear when loaded to force at least understand.