Getting started with Pytorch

Recommended information:

• "Eat Pytorch in 20 Days" : https://jackiexiao.github.io/eat_pytorch_in_20_days/ (the sample code is very comprehensive)
• "Pytorch in a simple way" : https://datawhalechina.github.io/thorough-pytorch/index.html (the descriptive language is richer and helps to understand the concept)

1. Basic grammar of Pytorch

1.1 Know Pytorch

Pytorch is a Numpy-based scientific computing package that provides its users with two major functions.
As a substitute for Numpy, it provides users with the ability to use the powerful functions of the GPU.
As a deep learning platform, it provides users with the largest flexibility and speed.

(1) Basic element operations of Pytorch

• Tensor tensor: The concept of tensor is similar to the ndarray data structure in Numpy, the biggest difference is that Tensor can take advantage of the acceleration function of GPU.

• When we use Pytorch, the general procedure is to first reference torch, as follows:

from __future__ import print_function
import torch

• Create an uninitialized matrix:

x = torch.empty(5, 3)
print(x)

output result:

tensor([[2.4835e+27, 2.5428e+30, 1.0877e-19],
        [1.5163e+23, 2.2012e+12, 3.7899e+22],
        [5.2480e+05, 1.0175e+31, 9.7056e+24],
        [1.6283e+32, 3.7913e+22, 3.9653e+28],
        [1.0876e-19, 6.2027e+26, 2.3685e+21]])

• Create an initialized matrix:

x = torch.rand(5, 3)
print(x)

output result:

tensor([[0.1368, 0.8070, 0.4567],
        [0.4369, 0.8278, 0.5552],
        [0.6848, 0.4473, 0.1031],
        [0.5308, 0.9194, 0.2761],
        [0.0484, 0.9941, 0.2227]])

Contrast with and without initialized matrices: When declaring an uninitialized matrix, it does not itself contain any exact values. When creating an uninitialized matrix, whatever values ​​are in the memory allocated to the matrix are assigned to the matrix, Essentially meaningless data.

• Create an all-zero matrix and specify the data element type as long

x = torch.zeros(5, 3, dtype=torch.long)
print(x)

output result:

tensor([[0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]])

• Create tensors directly from data

x = torch.tensor([2.5, 3.5])
print(x)

output result:

tensor([2.5000, 3.3000])

• Create a new tensor of the same size from an existing tensor

# 利用news_methods方法得到一个张量
x = x.new_ones(5, 3, dtype=torch.double)
print(x)

# 利用randn_like方法得到相同张量尺寸的一个新张量, 并且采用随机初始化来对其赋值
y = torch.randn_like(x, dtype=torch.float)
print(y)

output result:

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]], dtype=torch.float64)

tensor([[-0.1497, -0.5832, -0.3805],
        [ 0.9001,  2.0637,  1.3299],
        [-0.8813, -0.6579, -0.9135],
        [-0.1374,  0.1000, -0.9343],
        [-1.1278, -0.9140, -1.5910]])

• get the dimensions of the tensor

print(x.size())

output result:

torch.Size([5, 3])

Note: torch.SizeThe function essentially returns a tuple, so it supports all tuple operations.

(2) Basic operations of Pytorch

• Addition operation:

y = torch.rand(5, 3)
print(x + y)

output result:

tensor([[ 1.6978, -1.6979,  0.3093],
        [ 0.4953,  0.3954,  0.0595],
        [-0.9540,  0.3353,  0.1251],
        [ 0.6883,  0.9775,  1.1764],
        [ 2.6784,  0.1209,  1.5542]])

• The second addition method:

print(torch.add(x, y))

output result:

tensor([[ 1.6978, -1.6979,  0.3093],
        [ 0.4953,  0.3954,  0.0595],
        [-0.9540,  0.3353,  0.1251],
        [ 0.6883,  0.9775,  1.1764],
        [ 2.6784,  0.1209,  1.5542]])

• The third addition method:

# 提前设定一个空的张量
result = torch.empty(5, 3)
# 将空的张量作为加法的结果存储张量
torch.add(x, y, out=result)
print(result)

output result:

tensor([[ 1.6978, -1.6979,  0.3093],
        [ 0.4953,  0.3954,  0.0595],
        [-0.9540,  0.3353,  0.1251],
        [ 0.6883,  0.9775,  1.1764],
        [ 2.6784,  0.1209,  1.5542]])

• The fourth addition method: in-place (in-place replacement)

y.add_(x)
print(y)

output result:

tensor([[ 1.6978, -1.6979,  0.3093],
        [ 0.4953,  0.3954,  0.0595],
        [-0.9540,  0.3353,  0.1251],
        [ 0.6883,  0.9775,  1.1764],
        [ 2.6784,  0.1209,  1.5542]])

Note: All in-place operation functions have an underscore suffix.
For example, x.copy_(y), x.add_(y), will directly change the value of x.

• Operate on tensors in a Numpy-like manner:

print(x[:, 1])

output result:

tensor([-2.0902, -0.4489, -0.1441,  0.8035, -0.8341])

• Change the shape of a tensor: torch.view()

x = torch.randn(4, 4)
# tensor.view()操作需要保证数据元素的总数量不变
y = x.view(16)
# -1代表自动匹配个数
z = x.view(-1, 8)
print(x.size(), y.size(), z.size())

output result:

torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])

• If there is only one element in the tensor, you can use .item() to get the value out as a python number

x = torch.randn(1)
print(x)
print(x.item())

output result:

tensor([-0.3531])
-0.3530771732330322

(3) About the mutual conversion between Torch Tensor and Numpy array

• Torch Tensor and Numpy array share the underlying memory space , so changing the value of one will also change the other.

a = torch.ones(5)
print(a)

output result:

tensor([1., 1., 1., 1., 1.])

• Convert Torch Tensor to Numpy array

b = a.numpy()
print(b)

output result:

[1. 1. 1. 1. 1.]

• Adding one of them changes the other:

a.add_(1)
print(a)
print(b)

output result:

tensor([2., 2., 2., 2., 2.])
[2. 2. 2. 2. 2.]

• Convert Numpy array to Torch Tensor:

import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)

output result:

[2. 2. 2. 2. 2.]
tensor([2., 2., 2., 2., 2.], dtype=torch.float64)

Note: All Tensors on CPU, except CharTensor, can be converted to Numpy array and vice versa.

• About Cuda Tensor: Tensors can .to()be moved to any device with methods.

# 如果服务器上已经安装了GPU和CUDA
if torch.cuda.is_available():
    # 定义一个设备对象, 这里指定成CUDA, 即使用GPU
    device = torch.device("cuda")
    # 直接在GPU上创建一个Tensor
    y = torch.ones_like(x, device=device)
    # 将在CPU上面的x张量移动到GPU上面
    x = x.to(device)
    # x和y都在GPU上面, 才能支持加法运算
    z = x + y
    # 此处的张量z在GPU上面
    print(z)
    # 也可以将z转移到CPU上面, 并同时指定张量元素的数据类型
    print(z.to("cpu", torch.double))

output result:

tensor([0.6469], device='cuda:0')
tensor([0.6469], dtype=torch.float64)

1.2 autograd in Pytorch

In the entire Pytorch framework, all neural networks are essentially an autograd package (automatic derivation toolkit). The
autograd package provides an automatic differentiation function for all operations on Tensors.

(1) About torch.Tensor

• torch.Tensor is the core class in the entire package. If the property .requires_grad is set to True, it will track all operations defined on this class. When the code needs to be backpropagated, just call .backward() directly Automatically calculate all gradients. All gradients on this Tensor will be accumulated into attribute .grad.
• If you want to terminate the backtracking of a Tensor in the calculation graph, you only need to execute .detach() to remove the Tensor from the calculation graph, and the Tensor will not be calculated in future backtracking calculations.
• In addition to .detach(), if you want to terminate the backtracking of the calculation graph, that is, no longer perform the process of direction propagation to obtain derivatives, you can also use the code block method with torch.no_grad():, which is very suitable for model When making predictions, because the prediction stage no longer needs to calculate the gradient.

(2) About torch.Function:

• The Function class is a core class that is as important as the Tensor class. It builds a complete class together with Tensor. Each Tensor has a .grad_fn attribute, which represents which specific Function was referenced to create the Tensor.
• If a Tensor is user-defined, its corresponding grad_fn is None.

(3) Operations on Tensor

x1 = torch.ones(3, 3)
print(x1)

x = torch.ones(2, 2, requires_grad=True)
print(x)

output result:

tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])

tensor([[1., 1.],
        [1., 1.]], requires_grad=True)

• Perform an addition operation on a Tensor with requires_grad=True

y = x + 2
print(y)

output result:

tensor([[3., 3.],
        [3., 3.]], grad_fn=<AddBackward0>)

• Print Tensor's grad_fn attribute:

print(x.grad_fn)
print(y.grad_fn)

output result:

None
<AddBackward0 object at 0x10db11208>

• Perform more complex operations on Tensor:

z = y * y * 3
out = z.mean()
print(z, out)

output result:

tensor([[27., 27.],
        [27., 27.]], grad_fn=<MulBackward0>) tensor(27., grad_fn=<MeanBackward0>)

• About the method .requires_grad_(): This method can change the value of the Tensor attribute .requires_grad in place. If it is not actively set, it defaults to False.

a = torch.randn(2, 2)
a = ((a * 3) / (a - 1))
print(a.requires_grad)
a.requires_grad_(True)
print(a.requires_grad)
b = (a * a).sum()
print(b.grad_fn)

output result:

False
True
<SumBackward0 object at 0x7f191afd6be0>

(4) About gradient Gradients

• In Pytorch, backpropagation is implemented by .backward().

out.backward()
print(x.grad)

output result:

tensor([[4.5000, 4.5000],
        [4.5000, 4.5000]])

• About the property setting of automatic derivation: You can perform automatic derivation by setting .requires_grad=True, or you can stop automatic derivation by restricting the code block.

print(x.requires_grad)
print((x ** 2).requires_grad)

with torch.no_grad():
    print((x ** 2).requires_grad)

output result:

True
True
False

• A new Tensor can be obtained by .detach(), with the same content but without automatic differentiation.

print(x.requires_grad)
y = x.detach()
print(y.requires_grad)
print(x.eq(y).all())

output result:

True
False
tensor(True)

2. Initial application of Pytorch

2.1 Building a neural network using Pytorch

• About torch.nn:

  • Use Pytorch to build neural networks, the main tools are in the torch.nn package.
  • nn relies on autograd to define the model and automatically derive it.

• A typical process for building a neural network:

  • Define a neural network with learnable parameters
  • Iterate over the training dataset
  • Process the input data to flow through the neural network
  • Calculate the loss value
  • Backpropagating the gradient of the network parameters
  • Update the weight of the network with certain rules

---

(1) Define the network

• We first define a neural network implemented by Pytorch:

# 导入若干工具包
import torch
import torch.nn as nn
import torch.nn.functional as F


# 定义一个简单的网络类
class Net(nn.Module):

    def __init__(self):
        super(Net, self).__init__()
        # 定义第一层卷积神经网络, 输入通道维度=1, 输出通道维度=6, 卷积核大小3*3
        self.conv1 = nn.Conv2d(1, 6, 3)
        # 定义第二层卷积神经网络, 输入通道维度=6, 输出通道维度=16, 卷积核大小3*3
        self.conv2 = nn.Conv2d(6, 16, 3)
        # 定义三层全连接网络
        self.fc1 = nn.Linear(16 * 6 * 6, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        # 在(2, 2)的池化窗口下执行最大池化操作
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x
	# 将向量转换为一维向量（不考虑batch_size那一个维度），即flatten
    def num_flat_features(self, x):
        # 计算size, 除了第0个维度上的batch_size
        size = x.size()[1:]
        num_features = 1
        for s in size:
            num_features *= s
        return num_features


net = Net()
print(net)

output result:

Net(
  (conv1): Conv2d(1, 6, kernel_size=(3, 3), stride=(1, 1))
  (conv2): Conv2d(6, 16, kernel_size=(3, 3), stride=(1, 1))
  (fc1): Linear(in_features=576, out_features=120, bias=True)
  (fc2): Linear(in_features=120, out_features=84, bias=True)
  (fc3): Linear(in_features=84, out_features=10, bias=True)
)

• All trainable parameters in the model can be obtained through net.parameters().

params = list(net.parameters())
print(len(params))
print(params[0].size())

output result:

10
torch.Size([6, 1, 3, 3])

• Suppose the input size of the image is 32*32:

input = torch.randn(1, 1, 32, 32) # 4个维度依次为：batch_size维度、通道、长、宽
out = net(input)
print(out)

output result:

tensor([[ 0.1242,  0.1194, -0.0584, -0.1140,  0.0661,  0.0191, -0.0966,  0.0480,
          0.0775, -0.0451]], grad_fn=<AddmmBackward>)

• Once the output tensor is available, the gradient zeroing and backpropagation operations can be performed.

net.zero_grad()
out.backward()

Note : The neural network built by torch.nn only supports the input of mini-batches, and does not support the input of a single sample, that is, it must contain a batch_sizedimension

For example : nn.Conv2d requires a 4D Tensor with a shape of (nSamples, nChannels, Height, Width). If your input only has a single sample form, you need to execute input.unsqueeze(0) to actively expand the 3D Tensor into a 4D Tensor.

(2) Loss function

• The input of the loss function is an input pair: (output, target), and then calculates a value to evaluate the gap between output and target.

• There are several different loss functions available in torch.nn. For example, nn.MSELoss evaluates the gap between the input and the target value by calculating the mean square error loss.

---

• An example of calculating the loss using nn.MSELoss:

output = net(input)
target = torch.randn(10)

# 改变target的形状为二维张量, 为了和output匹配
target = target.view(1, -1)
criterion = nn.MSELoss()

loss = criterion(output, target)
print(loss)

output result:

tensor(1.1562, grad_fn=<MseLossBackward>)

• About the chain of direction propagation: If we track the direction of loss backpropagation and use the .grad_fn property to print, we can see a complete calculation graph as follows:

input -> conv2d -> relu -> maxpool2d -> conv2d -> relu -> maxpool2d
      -> view -> linear -> relu -> linear -> relu -> linear
      -> MSELoss
      -> loss

• When loss.backward() is called, the entire calculation graph will automatically derive the loss, and all Tensors with the property requires_grad=True will participate in the gradient derivation operation, and the gradient will be accumulated to the .grad attribute in the Tensors.

print(loss.grad_fn)  # MSELoss
print(loss.grad_fn.next_functions[0][0])  # Linear
print(loss.grad_fn.next_functions[0][0].next_functions[0][0])  # ReLU

output result:

<MseLossBackward object at 0x7fdba3216da0>
<AddmmBackward object at 0x7fdba3216f28>
<AccumulateGrad object at 0x7fdba3216f28>

(3) Backpropagation (backpropagation)

• Performing backpropagation in Pytorch is very simple, all operations are loss.backward().
• Before performing backpropagation, the gradient must be cleared to zero, otherwise the gradient will be accumulated between different batches of data.

---

• A small example of performing backpropagation:

# Pytorch中执行梯度清零的代码
net.zero_grad()

print('conv1.bias.grad before backward')
print(net.conv1.bias.grad)

# Pytorch中执行反向传播的代码
loss.backward()

print('conv1.bias.grad after backward')
print(net.conv1.bias.grad)

output result:

conv1.bias.grad before backward
tensor([0., 0., 0., 0., 0., 0.])
conv1.bias.grad after backward
tensor([-0.0002,  0.0045,  0.0017, -0.0099,  0.0092, -0.0044])

(4) Update network parameters

• The simplest algorithm for updating parameters is SGD (Stochastic Gradient Descent).
• The specific algorithm formula expression is: weight = weight - learning_rate * gradient

---

• First implement SGD with traditional Python code as follows:

learning_rate = 0.01
for f in net.parameters():
    f.data.sub_(f.grad.data * learning_rate)

• Then use the standard code officially recommended by Pytorch as follows:

# 首先导入优化器的包, optim中包含若干常用的优化算法, 比如SGD, Adam等
import torch.optim as optim

# 通过optim创建优化器对象
optimizer = optim.SGD(net.parameters(), lr=0.01)

# 将优化器执行梯度清零的操作
optimizer.zero_grad()

output = net(input)
loss = criterion(output, target)

# 对损失值执行反向传播的操作
loss.backward()
# 参数的更新通过一行标准代码来执行
optimizer.step()

2.2 Building a classifier using Pytorch

(1) Classifier task and data introduction

• Construct a neural network classifier that can classify different images, discriminate the input images and complete the classification.
• In this case, the CIFAR10 dataset is used as the original image data.

• CIFAR10 dataset introduction: The size of each picture in the dataset is 3 * 32 * 32, representing 3 color channels

• The CIFAR10 dataset has a total of 10 different categories, namely "airplane", "automobile", "bird", "cat", "deer", "dog", "frog", "horse", "ship", " truck".

A sample of the CIFAR10 dataset is shown in the figure below:

Steps to train the classifier
1: Download the CIFAR10 dataset using torchvision
2: Define the convolutional neural network
3: Define the loss function
4: Train the model on the training set
5: Test the model on the test set

(2) Use torchvision to download the CIFAR10 dataset

Import the torchvision package to assist in downloading datasets

import torch
import torchvision
import torchvision.transforms as transforms

Download the data set and adjust the picture, because the output of the torchvision data set is in PILImage format, and the data domain is in [0, 1]. We convert it to the tensor format of the standard data domain [-1, 1].

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')

output result:

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz
Extracting ./data/cifar-10-python.tar.gz to ./data
Files already downloaded and verified

Note: If you are running the above code under the Windows system, and the error message "BrokenPipeError" appears, you can try to set num_workers in torch.utils.data.DataLoader() to 0.

Images showing several training sets

# 导入画图包和numpy
import matplotlib.pyplot as plt
import numpy as np

# 构建展示图片的函数
def imshow(img):
    img = img / 2 + 0.5
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()


# 从数据迭代器中读取一张图片
dataiter = iter(trainloader)
images, labels = dataiter.next()

# 展示图片
imshow(torchvision.utils.make_grid(images))
# 打印标签label
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

Output picture result:

Output label result:

bird truck   cat   cat

(3) Define convolutional neural network

The class here is constructed like the class in section 2.1, the only difference is that 3-channel 3-channel is used here

import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x


net = Net()

(4) Define the loss function

A cross-entropy loss function and a stochastic gradient descent optimizer are used.

import torch.optim as optim

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

(5) Train the model on the training set

The optimization algorithm based on gradient descent requires many rounds of iterative training.

for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # data中包含输入图像张量inputs, 标签张量labels
        inputs, labels = data

        # 首先将优化器梯度归零
        optimizer.zero_grad()

        # 输入图像张量进网络, 得到输出张量outputs
        outputs = net(inputs)

        # 利用网络的输出outputs和标签labels计算损失值
        loss = criterion(outputs, labels)

        # 反向传播+参数更新, 是标准代码的标准流程
        loss.backward()
        optimizer.step()

        # 打印轮次和损失值
        running_loss += loss.item()
        if (i + 1) % 2000 == 0:
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')

output result:

[1,  2000] loss: 2.227
[1,  4000] loss: 1.884
[1,  6000] loss: 1.672
[1,  8000] loss: 1.582
[1, 10000] loss: 1.526
[1, 12000] loss: 1.474
[2,  2000] loss: 1.407
[2,  4000] loss: 1.384
[2,  6000] loss: 1.362
[2,  8000] loss: 1.341
[2, 10000] loss: 1.331
[2, 12000] loss: 1.291
Finished Training

Save the model:

# 首先设定模型的保存路径
PATH = './cifar_net.pth'
# 保存模型的状态字典
torch.save(net.state_dict(), PATH)

(6) Test the model on the test set

• The first step, display several pictures in the test set

dataiter = iter(testloader)
images, labels = dataiter.next()

# 打印原始图片
imshow(torchvision.utils.make_grid(images))
# 打印真实的标签
print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))

Output picture result:

Output label result:

GroundTruth:    cat  ship  ship plane

• The second step, load the model and predict the test picture

# 首先实例化模型的类对象
net = Net()
# 加载训练阶段保存好的模型的状态字典
net.load_state_dict(torch.load(PATH))

# 利用模型对图片进行预测
outputs = net(images)

# 共有10个类别, 采用模型计算出的概率最大的作为预测的类别
_, predicted = torch.max(outputs, 1)

# 打印预测标签的结果
print('Predicted: ', ' '.join('%5s' % classes[predicted[j]] for j in range(4)))

output result:

Predicted:    cat  ship  ship plane

• Next, look at the performance on all test sets

correct = 0
total = 0
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print('Accuracy of the network on the 10000 test images: %d %%' % (
    100 * correct / total))

output result:

Accuracy of the network on the 10000 test images: 53 %

• Analysis results: For a data set with 10 categories, the accuracy of random guessing is 10%, and the model reaches 53%, indicating that the model has learned the real thing.

• In order to take a more detailed look at which categories the model performs better and which categories perform worse, we calculate the accuracy by category.

class_correct = list(0. for i in range(10))
class_total = list(0. for i in range(10))
with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        _, predicted = torch.max(outputs, 1)
        c = (predicted == labels).squeeze()
        for i in range(4):
            label = labels[i]
            class_correct[label] += c[i].item()
            class_total[label] += 1


for i in range(10):
    print('Accuracy of %5s : %2d %%' % (
        classes[i], 100 * class_correct[i] / class_total[i]))

output result:

Accuracy of plane : 62 %
Accuracy of   car : 62 %
Accuracy of  bird : 45 %
Accuracy of   cat : 36 %
Accuracy of  deer : 52 %
Accuracy of   dog : 25 %
Accuracy of  frog : 69 %
Accuracy of horse : 60 %
Accuracy of  ship : 70 %
Accuracy of truck : 48 %

2.3 Training the model on the GPU

In order to really take advantage of the excellent properties of Tensor in Pytorch and accelerate the training of the model, we can transfer the training process to the GPU.

First the device is defined, if CUDA is available it is defined as GPU, otherwise it is defined as CPU.

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

print(device)

output result:

cuda:0

When training the model, you only need to transfer the model to the GPU, and at the same time transfer the input pictures and label pages to the GPU.

# 将模型转移到GPU上
net.to(device)

# 将输入的图片张量和标签张量转移到GPU上
inputs, labels = data[0].to(device), data[1].to(device)