First of all, attach the link first, the small soil push pytorch at station b explains, PyTorch deep learning quick start tutorial (absolutely easy to understand!) [small mound]_哔哩哔哩_bilibili
This video is very friendly to Xiaobai, and the learning time is not too long (a little faster, about a day), combined with the official pytorch documentation. PyTorch documentation — PyTorch 1.12 documentation
This article is just to record the personal learning process. Where there are mistakes, please advise, and welcome to exchange and discuss.
How to download pytorch, including downloading the GPU version (a very troublesome process, each version number must correspond well, I have solved the problem after stepping on the pit) Here is an article that explains it in detail: torch.cuda.is_available() returns false—— solved Method_Nefu_lyh's Blog-CSDN Blog
When everything is ready, we start to build a very simple neural network of our own. The neural network has a fixed template, and the content of too many principles will not be described. There are many courses on the Internet including Wu Enda, Li Hongyi, Li Mu and so on. Create a CNN class here, initialize it, and then define forward. This simple method implements an addition operation.
import torch
from torch import nn
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
def forward(self, input):
output = input + 1
return output
cnn = CNN()
x = torch.tensor(1.0)
output = cnn(x)
print(output)The program running result is: tensor(2.). A simple neural network is built in this way. Let's see how to build a convolutional neural network, that is, CNN.
We know that a convolutional neural network generally includes a convolutional layer (Convolutional layer), a linear layer (ReLU), a pooling layer (Pooling layer), a fully connected layer (Fully-Connected layer)
The corresponding calculation relationship between layers has been shown in the link at the beginning of the article very clearly and in detail, including what parameters are included in each layer, and the effect of each parameter will affect everything!
For those who don’t like to watch videos, here is an article that includes the calculation process: CNN implementation process (Convolutional Neural Networks)_National Service Strongest Diaochan’s Blog-CSDN Blog_cnn Implementation
To sum up, I think writing a complete training process should clarify the thinking. First , the dataset should be loaded, including training and testing sets. Then build the convolutional neural network model we want, and then define the loss, including the parameters of the optimizer, learning_rate and so on. Finally , save our trained model for validation. Then start our whole training process.
Above code:
import torch
import torchvision.datasets
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
dataset = torchvision.datasets.CIFAR10("../dataset", train=False, transform=torchvision.transforms.ToTensor(), download=True)
dataloader = DataLoader(dataset, batch_size = 64)
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = Conv2d(3, 32, 5, padding=2)
self.maxpool1 = MaxPool2d(2)
self.conv2 = Conv2d(32, 32, 5, padding=2)
self.maxpool2 = MaxPool2d(2)
self.conv3 = Conv2d(32, 64, 5, padding=2)
self.maxpool3 = MaxPool2d(2)
self.flatten = Flatten()
self.linear1 = Linear(1024, 64)
self.linear2 = Linear(64, 10)
def forward(self, x):
x = self.conv1(x)
x = self.maxpool1(x)
x = self.conv2(x)
x = self.maxpool2(x)
x = self.conv3(x)
x = self.maxpool3(x)
x = self.flatten(x)
x = self.linear1(x)
x = self.linear2(x)
return x
loss = nn.CrossEntropyLoss()
cnn = CNN()
for data in dataloader:
imgs, targets = data
outputs = cnn(imgs)
result_loss = loss(outputs, targets)
result_loss.backward()
print(result_loss)We will see that when we define this convolutional neural network, many layers are created, and the code looks very cumbersome. We can use Sequential to optimize. Also, the above program does not include some settings of the optimizer. Let's do it below. optimization.
import torch
import torchvision.datasets
from torch import nn
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from model import *
import time
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
train_data = torchvision.datasets.CIFAR10("../dataset", train=True, transform=torchvision.transforms.ToTensor(),
download=True)
test_data = torchvision.datasets.CIFAR10("../dataset", train=False, transform=torchvision.transforms.ToTensor(),
download=True)
train_data_size = len(train_data)
test_data_size = len(test_data)
print("训练数据集的长度为:{}".format(train_data_size))
print("测试数据集的长度为:{}".format(test_data_size))
train_dataloader = DataLoader(train_data, batch_size=64)
test_dataloader = DataLoader(test_data, batch_size=64)
cnn = CNN()
cnn = cnn.to(device)
loss_fn = nn.CrossEntropyLoss()
loss_fn = loss_fn.to(device)
learning_rate=0.01 # 1e-1
optimizer = torch.optim.SGD(cnn.parameters(), lr=learning_rate)
total_train_step = 0
total_test_step = 0
epoch = 10
writer = SummaryWriter("../train_logs")
start_time = time.time()
for i in range(epoch):
print("---------------第{}轮训练开始:-------------------".format(i+1))
cnn.train()
for data in train_dataloader:
imgs, targets = data
imgs = imgs.to(device)
targets = targets.to(device)
outputs = cnn(imgs)
loss = loss_fn(outputs, targets)
# 优化
optimizer.zero_grad()
loss.backward()
optimizer.step()
total_train_step = total_train_step + 1
if total_train_step % 100 == 0:
end_time = time.time()
print(end_time - start_time)
print("训练次数:{}, Loss:{}".format(total_train_step, loss.item()))
writer.add_scalar("train_loss", loss.item(), total_train_step)
# 测试步骤开始
cnn.eval()
total_test_loss = 0
total_accuracy = 0
with torch.no_grad():
for data in test_dataloader:
imgs, targets = data
imgs = imgs.to(device)
targets = targets.to(device)
outputs = cnn(imgs)
loss = loss_fn(outputs, targets)
total_test_loss = total_test_loss + loss
accuracy = (outputs.argmax(1) == targets).sum()
total_accuracy = total_accuracy + accuracy
print("整体测试集上的Loss:{}".format(total_test_loss))
print("整体测试集上的正确率:{}".format(total_accuracy/test_data_size))
writer.add_scalar("test_loss", total_test_loss, total_test_step)
writer.add_scalar("test_accuracy", total_accuracy/test_data_size, total_test_step)
total_test_step = total_test_step + 1
torch.save(cnn, "cnn_{}.pth".format(i+1))
print("模型已保存")
writer.close()
A complete training process is shown in the figure above. GPU is used for training. It should be noted that we have model.eval() and with torch.no_grad() during testing.
model.eval() is to switch to the test program. At this time, the k and b parameters of the model will not be updated, and the dropout layer and batchnorm layer will be notified to switch between train and val. In the train mode, the dropout layer will set the probability of retaining the activation unit according to the set parameter p (retention probability=p, such as keep_prob=0.8), and the batchnorm layer will continue to calculate the mean and var of the data and update them. In val mode, the dropout layer will pass all activation units, and the batchnorm layer will stop calculating and updating mean and var, and directly use the mean and var values that have been learned in the training phase.
with torch.no_grad() In the process of model training, because the performance of the model needs to be monitored, after several training runs , a performance verification on the verification setepoch is required . Generally speaking, in the verification or testing phase, because it is enough to run a forward propagation (forward), there is no need to save the gradient of the variable.
Next, let's verify the trained model. This is a picture of a cat I found on the Internet to see if the model can hit. First, we need to do some processing on the found pictures, and then load the model with the best training results in our previous training process.
This is the label of our data set, and the output of the program is as follows:
Our model correctly classified this image! ! [3] is cat. Similarly, I judged the other two photos, but unfortunately I didn't guess right. The correct rate of this training model is only a little over 60%, and it seems that the correct rate is not that high. On the one hand, it may be that the data set is small, and the picture is 32x32, which is blurry. On the other hand, the number of training sessions may be less. Later, I will explore how to optimize the model to improve the accuracy rate.
Welcome to exchange and discuss! !