learn better from others,
be the better one.
—— "Weika Zhixiang"
The length of this article is 1739 words , and it is expected to read for 5 minutes
foreword
As a newcomer to pyTorch, you need to do demos to practice by yourself, so this entry is a series, starting from pyTorch training, saving the model, and then using C++ OpenCV DNN for reasoning, and then transplanting to Andorid to directly realize handwritten mathematics Identification can be regarded as a small project actual combat of the whole process. Today is the first article, writing the training of the simplest fully connected Minist dataset pyTorch.
achieve effect
Code
Micro card Zhixiang
Fully connected network model
import torch
import torch.nn.functional as F
##Minist的图像为1X28X28的
class LinearNet(torch.nn.Module):
def __init__(self):
super(LinearNet, self).__init__()
##784是图像为 1X28X28，通道X宽度X高度，然后总的输入就是784
self.l1 = torch.nn.Linear(784, 512)
self.l2 = torch.nn.Linear(512, 256)
self.l3 = torch.nn.Linear(256, 128)
self.l4 = torch.nn.Linear(128, 64)
self.l5 = torch.nn.Linear(64, 10)
##定义损失函数
self.criterion = torch.nn.CrossEntropyLoss()
def forward(self, x):
##将输入的图像矩阵改为N行784列
x = x.view(-1, 784)
x = F.relu(self.l1(x))
x = F.relu(self.l2(x))
x = F.relu(self.l3(x))
x = F.relu(self.l4(x))
##最后一层激活在损失函数中加入了，这里直接输出，不要加上rule了
return self.l5(x)The fully connected network model is also very simple, as can be seen from the Init initialization function, it is a five-layer Linear, and the input value of each layer is the output value of the previous layer, because the image in the Minist dataset is 1*28* 28, so the input of the first layer is 1X28X28 = 784. Here, in the creation of the LinearNet class, the loss function has been defined and can be called directly outside. The whole class is also very simple.
training model
The file of the training model is the focus of this article, because this file is used for training when other network models are used later. We load different training models according to the name of the set model. So I created a ministmodel.py file to write
01
Import related files and basic parameters
As can be seen from the figure above, here we imported the LinearNet model created above, and then set a few parameters, mainly to set which model to use for this training. This is to change the training model every time, and train this The code can be reused, no need for Ctrl + C and Ctrl + V.
02
Load the Minist dataset
The mean and standard deviation in transfrom at the top, because many online have already been calculated in advance, so directly input here, call the training set and test set, and realize it through torchvision. After setting the directory, if it does not exist in the current directory, it will be automatically download.
03
load model
A switch function is added to process the currently loaded model. If you add a new model, just return the new model directly according to the input. Because I am using python 3.9 version, there is no switch method to achieve it, only I wrote if else by myself, and there is a match case syntax after python 3.10 .
The loaded model in the blue box directly selects the corresponding model according to the train_name defined above, and the setting of the optimizer is also through the learning rate and momentum defined above, and then the training can start.
04
Train and save the model
training function
test function
The test function should pay attention to the toppredicted variable we defined at the beginning. It was said at the time that if the prediction rate is higher than the current one, we will update the prediction rate and save the current model. Use global to declare toppredicted, which is used to modify inside the function and outside the function Declared global variables, otherwise an error will be reported.
saved model file
start training
The last thing is to start training. A total of 10 rounds of training have been set up. When the training is completed, the total training time will be printed out. The complete code is as follows:
import torch
import time
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import torch.optim as optim
from NetLinear import LinearNet
batch_size = 64
##设置本次要训练用的模型
train_name = 'LinearNet'
print("train_name:" + train_name)
##设置模型保存名称
savemodel_name = train_name + ".pt"
print("savemodel_name:" + savemodel_name)
##设置初始预测率,用于判断高于当前预测率的保存模型
toppredicted = 0.0
##设置学习率
learnrate = 0.01
##设置动量值，如果上一次的momentnum与本次梯度方向是相同的，梯度下降幅度会拉大，起到加速迭代的作用
momentnum = 0.5
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(mean=(0.1307,), std=(0.3081,))
]) ##Normalize 里面两个值0.1307是均值mean， 0.3081是标准差std，计算好的直接用了
##训练数据集位置，如果不存在直接下载
train_dataset = datasets.MNIST(
root = '../datasets/mnist',
train = True,
download = True,
transform = transform
)
##读取训练数据集
train_dataloader = DataLoader(
dataset= train_dataset,
shuffle=True,
batch_size=batch_size
)
##测试数据集位置，如果不存在直接下载
test_dataset = datasets.MNIST(
root= '../datasets/mnist',
train= False,
download=True,
transform= transform
)
##读取测试数据集
test_dataloader = DataLoader(
dataset= test_dataset,
shuffle= True,
batch_size=batch_size
)
##设置选择训练模型,因为python用的是3.9,用不了match case语法
def switch(train_name):
if train_name == 'LinearNet':
return LinearNet()
##定义训练模型
class Net(torch.nn.Module):
def __init__(self, train_name):
super(Net, self).__init__()
self.model = switch(train_name= train_name)
self.criterion = self.model.criterion
def forward(self, x):
x = self.model(x)
return x
model = Net(train_name)
##加入判断是CPU训练还是GPU训练
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model.to(device)
##优化器
optimizer = optim.SGD(model.parameters(), lr= learnrate, momentum= momentnum)
##训练函数
def train(epoch):
running_loss = 0.0
for batch_idx, data in enumerate(train_dataloader, 0):
inputs, target = data
##加入CPU和GPU选择
inputs, target = inputs.to(device), target.to(device)
optimizer.zero_grad()
#前馈，反向传播，更新
outputs = model(inputs)
loss = model.criterion(outputs, target)
loss.backward()
optimizer.step()
running_loss += loss.item()
##计算每300次打印一次学习效果
if batch_idx % 300 == 299:
print('[%d, %5d] loss: %.3f' % (epoch + 1, batch_idx + 1, running_loss / 300))
running_loss = 0.0
def test():
correct = 0
total = 0
##with这里标记是不再计算梯度
with torch.no_grad():
for data in test_dataloader:
inputs, labels = data
##加入CPU和GPU选择
inputs, labels = inputs.to(device), labels.to(device)
outputs = model(inputs)
##预测返回的是两列，第一列是下标就是0-9的值，第二列为预测值，下面的dim=1就是找维度1（第二列）最大值输出
_, predicted = torch.max(outputs.data, dim=1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
currentpredicted = (100 * correct / total)
##用global声明toppredicted,用于在函数内部修改在函数外部声明的全局变量，否则报错
global toppredicted
##当预测率大于原来的保存模型
if currentpredicted > toppredicted:
toppredicted = currentpredicted
torch.save(model.state_dict(), savemodel_name)
print(savemodel_name+" saved, currentpredicted:%d %%" % currentpredicted)
print('Accuracy on test set: %d %%' % currentpredicted)
##开始训练
timestart = time.time()
for epoch in range(10):
train(epoch)
test()
timeend = time.time() - timestart
print("use time: {:.0f}m {:.0f}s".format(timeend // 60, timeend % 60))over
Wonderful review of the past
Android Kotlin makes signature whiteboard and saves pictures
Super simple pyTorch training->onnx model->C++ OpenCV DNN reasoning (with source code address)
Kotlin uses Select expression in coroutine to select fastest result