Pytorch automatically solves the gradient

To understand Pytorch's solution to gradients, we first need to understand the concept of the calculation graph in Pytorch. In the calculation graph, each Variable represents a node. Each node can represent a neuron. We can only put variables into nodes. To solve the gradient of the variables in the nodes, suppose we have a matrix:

1., 2., 3.
4., 5., 6.

We first initialize this matrix (two-dimensional tensor) in Pytorch and put it into the nodes in the calculation graph. In Pytorch, a node is represented by Variable, so we can write the following code:

Import Torch
 from torch.autograd Import Variable 

# we're creating a two-dimensional tensor (ie matrix) rather than the standard amount, and therefore error 
# in Pytorch which allows only scalar scalar or scalar solving partial to vector (or tensor) Derivative 
x = Variable (torch.Tensor ([[1., 2., 3.], [4., 5., 6.]]), requires_grad = True)

Among the nodes, there are parameters of requires_grad, the system default is False, that is, the gradient of the variables cannot be solved, but we need the gradient of the variables inside, so we need to name the entire parameter True.

Finally, we write expressions about x in other variables:

y=2*x*x+2

j=y.mean()

In this way, the value of j is obtained, which is a scalar, because the mean value of mean expressed in Pytorch can only solve the partial derivative of scalar to scalar, or scalar to tensor, otherwise it will report an error.

Now that our computational graph model is built, there is only one node in the entire model, the other representations are equivalent to the weights in the neurons, and J represents the loss function. We call the backward propagation function backward () in Pytorch, for x To solve the gradient of j, the code is as follows:

 

j.backward()

 

Then the gradient is solved, we print out the value of the gradient of x with respect to j, and the following parameter x.grad represents the magnitude of the gradient of x solved out:

print ( " The gradient of x is: " , x.grad)

Output:

The gradient of x is: 
tensor ([[0.6667, 1.3333, 2.0000 ], 
        [ 2.6667, 3.3333, 4.0000]]

Understand!