1. Experiment name
Pytorch implements convolutional neural network
2. Experimental requirements
Implementation of Convolutional Neural Networks with Python's Pytorch module. The network structure is an input layer, two convolutional layers, a fully connected layer, and an output layer.
3. Purpose of the experiment
- Familiar with and master the pytorch framework
- Master the fundamentals of convolutional neural networks
- Master the structure of convolutional neural network
- Master the code flow of convolutional neural network
4. Experimental process
This experiment is completely operated and run in itc's virtual environment. In order to highlight the key points, the operation process will not be repeated, but the programming process will be described in detail.
The first is the input picture, after a layer of convolutional layer, and then use the pooling method to process the convolutional information, and then go through the same processing again, and pass the obtained second processed information into the fully connected neural layer, Finally, a classifier is connected to perform classification prediction.
Detailed process:
- Convert the image to a tensor and normalize it.
- Download the training and test sets.
- DataLoader for building datasets and test sets.
- Define the network structure.
- Convert the model to device and display its structure.
- Defines the cross-entropy loss function.
- Transfer images and labels to the device.
- Clear the gradient of the model.
- Run the model forward.
- Calculate the loss for this round.
- Compute the accuracy for this round.
- Perform backpropagation to find the gradient of the model parameters.
- Use iterator to update model weights.
- Visualize the results.
- Repeat the above 7-14 process until the specified number of iterations is reached.
- Computes the average accuracy over the total tests.
- Computes the average loss over the total tests.
5. Experimental results
6. Experiment summary
Compared with the fully connected neural network in the previous experiment, the convolutional neural network has added convolutional layers and pooling layers. At the same time, it is no longer fully connected, but local connected, and a strategy of weight sharing is adopted. Among them, local connection and weight sharing reduce the scale of parameters, which greatly reduces the complexity of training and reduces overfitting. The downsampling of the pooling layer further reduces the scale of output parameters and improves the generalization ability of the model. .
It is not difficult to get started with convolutional neural networks, but it still takes time and effort to learn deeply in this field. When learning convolutional neural networks, you need to know more about several network structures, from shallow to deep, thinking about why you do this, the benefits and significance of doing so. More practice and more hands-on, not only allows us to have a deeper understanding and mastery of the process of convolutional neural networks, but also allows us to lay a solid foundation and have more foundation and strength to study in depth.
7. Source code
import torch
from torch import nn
import torchvision
from tqdm import tqdm
torch.cuda.is_available()
BATCH_SIZE = 100
EPOCHS = 10
learning_rate = 1e-4
keep_prob_rate = 0.7
device = "cuda:0" if torch.cuda.is_available() else "cpu"
transform = torchvision.transforms.Compose(
[torchvision.transforms.ToTensor(),
torchvision.transforms.Normalize(mean = [0.5], std = [0.5])]m
)
path = './data/'
trainData = torchvision.datasets.MNIST(
path, train = True, transform = transform, download= True
)
testData = torchvision.datasets.MNIST(
path, train = False, transform = transform
)
trainDataLoader = torch.utils.data.DataLoader(
dataset = trainData, batch_size = BATCH_SIZE,shuffle = True
)
testDataLoader = torch.utils.data.DataLoader(
dataset = testData, batch_size = BATCH_SIZE
)
class Net(torch.nn.Module):
def __init__(self):
super(Net,self).__init__()
self.model = torch.nn.Sequential(
torch.nn.Conv2d(
in_channels=1,out_channels=32,
kernel_size=7,padding=3,stride=1
),
torch.nn.ReLU(),
torch.nn.MaxPool2d(
kernel_size=2,stride=2
),
torch.nn.Conv2d(
in_channels=32,out_channels=64,
kernel_size=5,stride=1,padding=2
),
torch.nn.ReLU(),
torch.nn.MaxPool2d(
kernel_size=2,stride=2
),
torch.nn.Flatten(),
torch.nn.Linear(in_features=7*7*64, out_features= 1024),
torch.nn.ReLU(),
torch.nn.Dropout(1-keep_prob_rate),
torch.nn.Linear(in_features=1024,out_features=10),
torch.nn.Softmax(dim=1)
)
def forward(self, input):
output = self.model(input)
return output
net = Net()
print(net.to(device))
lossF = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(params=net.parameters(),lr=learning_rate)
history = {'Test Loss':[], 'Test Accuracy':[]}
for epoch in range(1,EPOCHS+1):
processBar = tqdm(trainDataLoader,unit='step')
net.train(True)
for step,(trainImgs, labels) in enumerate(processBar):
trainImgs = trainImgs.to(device)
labels = labels.to(device)
net.zero_grad()
outputs = net(trainImgs)
loss = lossF(outputs,labels)
predictions = torch.argmax(outputs,dim=1)
accuracy = torch.sum(predictions==labels)/labels.shape[0]
loss.backward()
optimizer.step()
processBar.set_description(
"[%d/%d] Loss: %.4f, Acc: %.4f" %(
epoch,EPOCHS,loss.item(),accuracy.item()
)
)
if step == len(processBar)-1:
correct, totalLoss = 0,0
net.train(False)
for testImgs,labels in testDataLoader:
testImgs = testImgs.to(device)
labels = labels.to(device)
outputs = net(testImgs)
loss = lossF(outputs,labels)
predicions = torch.argmax(outputs,dim=1)
totalLoss+=loss
correct+=torch.sum(predicions==labels)
testAccuracy = correct/(BATCH_SIZE*len(testDataLoader))
testLoss = totalLoss/len(testDataLoader)
history['Test Loss'].append(testLoss.item())
history['Test Accuracy'].append(testAccuracy.item())
processBar.set_description(
"[%d/%d] Loss: %.4f, Acc: %.4f, Test Loss: %.4f, Test Acc: %.4f" % (
epoch,EPOCHS,loss.item(),accuracy.item(),
testLoss.item(),testAccuracy.item()
)
)
processBar.close()