Master Liu Er - Convolutional Neural Network

Solve multi-classification problems based on MNIST dataset + use CNN based on MNIST

Using Softmax and CrossEntroyLoss to solve multi-classification problems (MNIST dataset)

Multi-classification problem: In fact, the distribution of random events is solved, and the solution should be based on two requirements:

Probability ≥ 0 for each class (exponential over all outputs)

The sum of the probabilities of each category = 1 (normalize the results)

Based on the above analysis, softmax is introduced to realize the basic requirements of multi-classification problems.

Loss Loss: Just use the cross-entropy loss in pytorch. The one-hot code generated by the category (label) and the softmax are calculated after the logarithm. These calculations are combined together to form the cross-entropy loss, so cross-entropy is used During the loss, the last layer of the neural network cannot be activated because it is included in the cross-entropy loss.

Therefore, solving multi-classification problems can be divided into four steps, namely:

prepare data

design model

Define loss and optimizer

Do training and testing

Before preparing data, import the packages we need in torch, including transforms, datasets, DataLoader, etc.

Prepare the data. The MNIST dataset is a dataset that comes with Pytorch. Setting download to True will automatically download the dataset.

After the download is successful, you can extract the picture and use cv2 to save the picture.

define model

As shown in the figure below, the left picture is a picture in MNIST, and the right picture corresponds to its form represented by a matrix. A single digital picture in the MNIST dataset is a grayscale image with a size of 28*28=784, and the value of each pixel is The value is {0,255}, and the value of each pixel needs to be mapped to [0,1].

In this example, the original image is to be converted into a tensor, so each batch of input data to the neural network will be (N, 1, 28, 28), each with N samples, and each sample is 1-dimensional ( channel) 28*28 image.

Since the fully connected neural network requires the input of a matrix, the first thing to do is to convert the third-order tensor of 1*28*28 into a first-order vector (that is, splicing multiple rows into one row), and use the view to change the shape of the tensor. , 784 refers to the number of pixels, -1 can automatically calculate N, and then the corresponding model.

In the code, first initialize the linear layer, define the linear model in the initialization method, a total of five layers, and finally become 10.

然后在forward里面使用relu对每一层进行激活，最后一层不需要激活，因为直接是线性层的输出接到softmax层，最后网络定义层model。

关于损失和优化

损失使用pytorch自带的交叉熵损失，经过softmax求对数然后和one-hot做相应的乘法计算出损失，优化器使用带有冲量值的梯度下降，优化训练过程。

训练和测试

将一轮循环封装成一个函数train，使用enumerate将训练样本提取出来，inputs相当于x，target相当于Y，然后将优化器清零；接下来就是前馈（计算输出和损失）、反馈和优化；最后累计损失，每训练300次进行一次迭代，输出其损失。

在测试过程中不需要反向传播，只需要计算正向传播和分类正确的数量即可。在测试过程中不需要计算梯度，此时使用with torch.no_grad()，correct是指计算正确的是多少，然后从test_loader里面提取数据做预测，使用torch.max()计算出每一行的最大值和下标（预测结果），然后求总数total（size(0)取得是样本形状的第一个值，如(N,1)就是N），接下来将预测出来的值和真实值作比较，相等为真，否则为假。最后正确数除以总数就是正确率。

使用CNN跑MNIST数据集

CNN更多是针对具有某种特征的图像的处理，如下图所示，他可以分为两个步骤，分别是特征提取和图像分类，在特征提取过程中包含两个步骤，分别是卷积和下采样。在卷积层，可以通过不断的改变卷积核矩阵的值来关注不同的细节，提取不同的特征；也就是说，在初始化卷积核的矩阵值(即权重参数)后，我们通过梯度下降不断降低loss来获得最好的权重参数，整个过程都是自动调整的。在池化层（也称之为下采样）可以对数据进行降维避免过拟合，如下图就是基本的一个使用CNN进行图像处理的一个过程。