WDK study notes_Use of Pytorch framework

Use of Pytorch framework

Summary

In terms of engineering: I learned the use of the Pytorch framework. In terms of data processing, I used Dataset to read data from the file and numbered it. I used Dataloader to package the data in batches, so that I could use small batch gradient descent to optimize the model during training. In terms of visualization, Tensorboard was used to realize the visualization of building the model and the visualization of training data; on the model, a CNN-DNN model was built to classify handwritten digit recognition, and the accuracy rate was achieved without any parameter adjustment and model improvement. About 80%.
In terms of academics: This week, I read in detail the description of fluid motion from Euler's point of view and Lagrangian's point of view, and made some derivations, and partly learned the mathematical derivation of how SVM maps low-latitude data to high-latitude data for classification.

---

1. Data processing

In Pytorch, reading data mainly involves two classes—Dataset and Dataloader:

1. Dataset is often used to read data in files, read each data and its corresponding Label (label).
2. Dataloader packs the data read by Dataset and sends them to the neural network in batches.

1.1 Overloading of Dataset

Dataset official documentation
Create a class Mydataset, inherit the parent class Dataset, rewrite the initialization function, __getitem__ function and __len__ function. The effect of the following code is to read the image data of ants and bees in the upper directory file dataset/val/. The role of the function:

1. os.path.join: Input the file address, connect the address and output, such as: os.path.join(“dataset”, “val”) = dataset/val
2. os.listdir: Input a file address and output a list of file names in that address.
3. Image.open: Input the address of the image and output the image.

import os.path

from PIL import Image
from torch.utils.data import Dataset

class Mydataset(Dataset):
    def __init__(self, root_dir, label_dir):
        self.label_dir = label_dir
        self.root_label_dir = os.path.join(root_dir, label_dir)
        self.imgs_dir_list = os.listdir(self.root_label_dir)

    def __getitem__(self, idex):#输入图片索引，返回图片及其label
        img_path = self.imgs_dir_list[idex]
        img = Image.open(os.path.join(self.root_label_dir, img_path))
        img_label = self.label_dir
        return img, img_label

    def __len__(self):#返回图片的长度
        return len(self.imgs_dir_list)

root_dir = "../dataset/val"
ants_dir = "ants"
bees_dir = "bees"
ants_dataset = Mydataset(root_dir, ants_dir)
bees_dataset = Mydataset(root_dir, bees_dir)
img, label = ants_dataset.__getitem__(2)
img.show()
print(label)

1.2 DataLoader

DataLoader official document
Function: Pack data in batches

import torch
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

trans_tensor = torchvision.transforms.ToTensor() #实例化图片转tensor的类
train_data = torchvision.datasets.MNIST("./data_minst", train=True, transform=trans_tensor, download=True)
test_data = torchvision.datasets.MNIST("./data_minst", train=False, transform=trans_tensor, download=True)
writer =SummaryWriter("./logs")

batch = 50
train_data_batch = DataLoader(train_data, batch_size=batch)
test_data_batch = DataLoader(test_data, batch_size=batch)

i = 0
for data in train_data_batch:
    img, label = data
    print(img.shape)
    img = torch.reshape(img, (batch, 1, 28, 28))
    writer.add_images("手写数字", img, i)
    i += 1

writer.close()

2. Visualization

Tensorboard Official Documentation

2.1 Use of Tensorboard

2.1.1 Tensorboard draws according to the data

from torch.utils.tensorboard import SummaryWriter#从tensorboard中导入类SummaryWriter
writer = SummaryWriter("./logs")#类的实例化
for i in range(100):
    writer.add_scalar("y=x^2", i**2, i)#添加数据
writer.close()

2.1.2 Display pictures on Tensorboard

1. writer.add_images: The input tensor format is (n, m, h, w), where n represents the number of images, m represents the number of channels, and h and w represent image pixels.

import torch
from PIL import Image
from torchvision import transforms
from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter("./logs")
img = Image.open("./image/0013035.jpg")
tensor_trans = transforms.ToTensor()
img = tensor_trans(img)
print(img.shape)
img = torch.reshape(img,(1,3,512,768))
writer.add_images("image", img)
writer.close()

2.1.3 Tensorboard display model network structure

1. writer.add_graph(): Input model class and model input.

import torch
from torch import nn
from torch.utils.tensorboard import SummaryWriter


class CnnModel(nn.Module):
    def __init__(self):
        super(CnnModel, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=3, kernel_size=3, stride=1, padding=0)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(3, 3, 3, 1, 0)
        self.pool2 = nn.MaxPool2d(2)
        self.flaten = nn.Flatten()
        self.linear = nn.Linear(75, 10)
        self.sofmax = nn.Softmax(1)

    def forward(self, x):
        x = self.conv1(x)
        x = self.pool1(x)
        x = self.conv2(x)
        x = self.pool2(x)
        x = self.flaten(x)
        x = self.linear(x)
        x = self.sofmax(x)
        return x

if __name__ == '__main__':      #测试model是否正确
    x = torch.ones(1, 1, 28, 28)
    model = CnnModel()
    output = model.forward(x)
    writer = SummaryWriter("./logs_model")
    writer.add_graph(model, x)
    writer.close()

3. Model practice

torch.nn official document

3.1 Handwritten digit recognition

Read the handwritten data set from torchvision, use torchvision.transforms.ToTensor to convert the image into a tensor, use DataLoader to pack the data set in batches, inherit the model class from nn.Module, and use nn.Conv2d to build the convolutional layer. nn.MaxPool2dl builds the pooling layer, nn.Linear builds the linear layer, nn.Flatten() flattens the data, and nn.Sofemax() serves as the activation function. The built model is visualized with tensorboard as shown below:

3.1.1 Model program

import torch
from torch import nn

class CnnModel(nn.Module):
    def __init__(self):
        super(CnnModel, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=3, kernel_size=3, stride=1, padding=0)
        self.pool1 = nn.MaxPool2d(2)
        self.conv2 = nn.Conv2d(3, 3, 3, 1, 0)
        self.pool2 = nn.MaxPool2d(2)
        self.flaten = nn.Flatten()
        self.linear = nn.Linear(75, 10)
        self.sofmax = nn.Softmax(1)

    def forward(self, x):
        x = self.conv1(x)
        x = self.pool1(x)
        x = self.conv2(x)
        x = self.pool2(x)
        x = self.flaten(x)
        x = self.linear(x)
        x = self.sofmax(x)
        return x

if __name__ == '__main__':      #测试model是否正确
    x = torch.ones(1, 1, 28, 28)
    model = CnnModel()
    output = model.forward(x)
    print(output.shape)
    print(output)

3.1.2 Main function program

Read the data set and convert the image data into a tensor. Every 50 pictures are regarded as a batch and thrown into the model for training. Cross-entropy is used as the loss function to record the loss value of each batch of the model and each epoch data set. All loss values, after training, use torch.save to keep all the parameters of the trained model.

import torch
import torchvision
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter
from Model import *		#导入Model

#读取图片并转换为tensor型数据
trans_tensor = torchvision.transforms.ToTensor() #实例化图片转tensor的类
train_data = torchvision.datasets.MNIST("./data_minst", train=True, transform=trans_tensor, download=True)
test_data = torchvision.datasets.MNIST("./data_minst", train=False, transform=trans_tensor, download=True)

writer =SummaryWriter("./logs")

#将数据分批量打包
batch = 50
train_data_batch = DataLoader(train_data, batch_size=batch)
test_data_batch = DataLoader(test_data, batch_size=batch)

#模型实例化
MinModel = CnnModel()

#优化器实例化
learingrate = 1e-2
optimizer = torch.optim.SGD(MinModel.parameters(), lr=learingrate)

#损失函数
loss_f = nn.CrossEntropyLoss()

epoach = 20
train_size = 1


for i in range(epoach):
    total_loss = 0
    print("——————————第{}轮训练开始——————————————".format(i+1))
    for data in train_data_batch:
        img, label = data   #取出img和label
        y = MinModel(img)   #得到model输出
        loss = loss_f(y, label)    #计算损失值
        writer.add_scalar("Loss值随训练次数的变化", loss, train_size)
        total_loss += loss  #每一轮的累计损失值
        optimizer.zero_grad()   #每一次训练将梯度清0
        loss.backward()     #自动求导
        optimizer.step()    #用优化器自动更新参数

        acc_train = ((y.argmax(1)==label).sum())/batch
        writer.add_scalar("准确率随训练次数的变化", acc_train, train_size)

        if(train_size%100==0): #每训练100次输出此batch的损失值
            print("第{}次训练的loss值为{}".format(train_size, loss))
        train_size += 1

    print("********第{}轮训练全部的loss值是：{}***********".format(i+1, total_loss))

torch.save(MinModel, "./Model_save/model.pth") #保持model

writer.close()

Summarize

Realized the use of pytorch framework to build a network from 0, practiced with a data set of handwritten numbers, and mastered the visualization of model and training data with tensorboard. In terms of papers, a more systematic understanding of Euler and Lagrange's views on fluid motion.