[Introduction to Artificial Intelligence] Code running takes time to calculate, using GPU to train the network, converting List elements of Tensor into Tensor, model.train() and model.eval()

[Introduction to Artificial Intelligence] Code running takes time to calculate, using GPU to train the network, converting List elements of Tensor into Tensor, model.train() and model.eval()

---

1. Calculation of code running time

import time
import math

def time_duration(start):
    s = time.time() - start
    m = math.floor(s/60)
    s -= m * 60
    return '%d分%d秒' %(m,s)

start = time.time()

print("hello world")

print('本轮训练时长：',time_duration(start))

2. Use GPU to train the network

• It is easy to use PyTorch to call the GPU to train the network. You only need to send the model and data to the GPU.
• Code example:

# 1.创建模型
model = Net()
# 2.定义device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# 3.将模型加载到GPU(所定义的device) 
model.to(device)
# 4.将输入和输出加载到GPU
for data in train_dataloader:
	inputs, target = data
	inputs, target = inputs.to(device), target.to(device)

• In actual practice, training a certain network for 160 rounds took 20 minutes and 29 seconds using the CPU and 3 minutes and 56 seconds using the GPU. The effect is still very obvious.

3. The List whose elements are Tensor is converted into Tensor

• Converting an ordinary List into a Tensor is as simple as the following.

a = [1, 2, 3]
b = torch.tensor(a)

• But for a List containing tensor, an error will be reported if you use the above method. You can torch.stackconvert it as follows.

a = torch.rand((2, 3))
a_list = [a for _ in range(3)]
A = torch.stack(a_list)

4. model.train() and model.eval()

• PyTorch provides two ways to switch between training and evaluation (inference) modes: model.train() and model.eval().
• The general usage is: write model.trian() before training starts, and model.eval() during testing.
• Add model.train() above the training program 启用 batch normalization 和 dropout to ensure that the BN layer can use the mean and variance of each batch of data, and Dropout can randomly ignore some network connections to train and update parameters.
• Add model.eval() above the inference program 不启用 Batch Normalization 和 Dropoutto ensure that the mean and variance of the BN layer remain unchanged, and Dropout fails, that is, all network connections are utilized and neurons are not randomly discarded.