Hello everyone, I am Weixue AI. Today, I will introduce to you Deep Learning Skills Application 22 - the skill of building a universal data generation class, which is suitable for debugging and training of CNN, RNN, and GNN models. This article will realize the writing of a universal data generation class, and use the PyTorch framework to train CNN, RNN and GNN models.
Contents:
1. Background introduction
2. Dependent library introduction
3. Universal data generator introduction
4. CNN, RNN, GNN model building
5. Data generation and model training
6. Training results and summary
1. Background introduction
In the process of artificial intelligence model training, we need to conduct some experiments, tests or debugging, we may need a data set with a specific shape and quantity to verify our algorithm or model. By building a universal data generator, we can flexibly generate datasets of various shapes and sizes without manually crafting and preparing real datasets.
Second, the data generator can also be used to explore the nature and characteristics of the dataset. By generating data sets with specific distributions, characteristics, or regularities, we can gain a deeper understanding of the relationship between data, the interaction between features, and the structure of data, etc. This is very helpful for data preprocessing, feature engineering, and model selection.
In addition, data generators can also be used to implement data augmentation techniques. Data augmentation refers to generating new training samples by performing a series of transformations or perturbations on the original data to increase the diversity and generalization ability of the training data. By building a universal data generator, we can define various data augmentation methods and dynamically generate augmented samples during training to improve the robustness and reliability of the model.
2. Introduction to dependent libraries
First, we need to introduce the following dependent libraries:
- torch: PyTorch framework
- torch.optim: torch.optim is a module in the PyTorch framework for optimizing the parameters of the model. It provides various optimization algorithms such as Stochastic Gradient Descent (SGD), Adam, Adagrad, etc. By choosing an appropriate optimization algorithm and tuning parameters, the model can be better converged and achieve better performance during training.
- torch.utils.data: torch.utils.data is a module in the PyTorch framework, a tool class for processing datasets. It provides some commonly used data processing operations, such as data loading, batch processing, data iteration, and data transformation. By using torch.utils.data, it is convenient to load the dataset into the model for training, and it can flexibly handle data in different formats.
- numpy: numpy is a Python library primarily used for numerical and scientific computing. It provides a multidimensional array object (ndarray) and a series of functions for manipulating arrays. Numpy can efficiently perform numerical operations and supports broadcasting and vectorization operations, so it is widely used in scientific computing, data analysis, and machine learning. In PyTorch, numpy can be seamlessly converted with torch.Tensor to facilitate data processing and conversion.
3. Introduction to universal data generator
First, we need to define a dataset class called UniversalDataset, which is used to generate data and labels with a specific shape and quantity.
In the class initialization method __init__, we pass in three parameters: data_shape represents the shape of the data (a tuple), target_shape represents the shape of the label (a tuple), and num_samples represents the number of samples in the dataset. Generate data through these three parameters.
Next, we implemented the __len__ method, which returns the number of samples in the dataset, num_samples.
Then define the __getitem__ method, which returns the data and labels corresponding to the index in the dataset according to the index idx. In this method, we first create an all-zeros tensor data of the same shape as data_shape, and an all-zeros tensor target of the same shape as target_shape.
Then, we calculated the dimensions of the data and labels, namely data_dims and target_dims, respectively.
This article uses the torch.linspace function to generate an equally spaced data range data_range with a length of data_dim_size between 0 and 1, and reshape it into a tensor of data_shape_expanded shape through the reshape method. Then, we add this shaped data range to the data tensor data.
We do a similar operation on labels, generating a regular label tensor target.
Finally, we return the data tensor data and label tensor target as the sample corresponding to this index.
Through this class, we can generate a data set with a specified shape and quantity as needed, and the data and labels are regular, which is convenient for subsequent training and evaluation.
import torch
from torch import nn
from torch.utils.data import DataLoader, Dataset
class UniversalDataset(Dataset):
def __init__(self, data_shape, target_shape, num_samples):
self.data_shape = data_shape
self.target_shape = target_shape
self.num_samples = num_samples
def __len__(self):
return self.num_samples
def __getitem__(self, idx):
# 生成数据和标签
data = torch.zeros(self.data_shape)
target = torch.zeros(self.target_shape)
# 计算数据和标签的维度
data_dims = len(self.data_shape)
target_dims = len(self.target_shape)
# 生成有规律的数据和标签
for dim in range(data_dims):
data_dim_size = self.data_shape[dim]
data_range = torch.linspace(0, 1, data_dim_size)
data_shape_expanded = [1] * data_dims
data_shape_expanded[dim] = data_dim_size
data += data_range.reshape(data_shape_expanded)
for dim in range(target_dims):
target_dim_size = self.target_shape[dim]
target_range = torch.linspace(0, 1, target_dim_size)
target_shape_expanded = [1] * target_dims
target_shape_expanded[dim] = target_dim_size
target += target_range.reshape(target_shape_expanded)
return data, target
4. CNN, RNN, GNN model building
class CNNModel(nn.Module):
def __init__(self, input_shape):
super(CNNModel, self).__init__()
self.conv1 = nn.Conv2d(input_shape[0], 16, kernel_size=3, stride=1, padding=1)
self.fc = nn.Linear(16 * (input_shape[1] // 2) * (input_shape[2] // 2), 10)
def forward(self, x):
x = self.conv1(x)
x = nn.functional.relu(x)
x = nn.functional.max_pool2d(x, 2)
x = x.view(x.size(0), -1)
x = self.fc(x)
return x
class RNNModel(nn.Module):
def __init__(self, input_shape):
super(RNNModel, self).__init__()
self.rnn = nn.RNN(input_shape[1], 64, batch_first=True)
self.fc = nn.Linear(64, 10)
def forward(self, x):
_, h_n = self.rnn(x)
x = self.fc(h_n.squeeze(0))
return x
class GNNModel(nn.Module):
def __init__(self, input_shape):
super(GNNModel, self).__init__()
self.fc1 = nn.Linear(input_shape[1], 32)
self.fc2 = nn.Linear(32, 10)
def forward(self, x):
x = torch.mean(x, dim=1)
x = self.fc1(x)
x = nn.functional.relu(x)
x = self.fc2(x)
return x
5. Data generation and model training
# 定义训练函数
def train(model, dataloader, criterion, optimizer):
running_loss = 0.0
correct = 0
total = 0
for inputs, labels in dataloader:
optimizer.zero_grad()
outputs = model(inputs)
loss = criterion(outputs, labels)
_, predicted = torch.max(outputs.data, dim=1)
total += labels.size(0)
correct += (predicted == labels.argmax(dim=1)).sum().item()
loss.backward()
optimizer.step()
running_loss += loss.item()
epoch_loss = running_loss / len(dataloader)
epoch_acc = correct / total
return epoch_loss, epoch_acc
# 设置参数
data_shape_cnn = (3, 32, 32) # (channels, height, width)
target_shape = (10,)
num_samples = 1000
batch_size = 32
learning_rate = 0.001
num_epochs = 10
# 创建数据集和数据加载器
dataset = UniversalDataset(data_shape_cnn, target_shape, num_samples)
dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
# 创建CNN模型、优化器和损失函数
cnn_model = CNNModel(data_shape_cnn)
cnn_optimizer = torch.optim.Adam(cnn_model.parameters(), lr=learning_rate)
cnn_criterion = nn.CrossEntropyLoss()
print('CNN模型训练:')
# 训练CNN模型
for epoch in range(num_epochs):
cnn_loss, cnn_acc = train(cnn_model, dataloader, cnn_criterion, cnn_optimizer)
print(f'CNN - Epoch {
epoch+1}/{
num_epochs}, Loss: {
cnn_loss:.4f}, Accuracy: {
cnn_acc:.4f}')
# 重新创建数据集和数据加载器
data_shape_rnn = (20, 32) # (sequence_length, input_size, hidden_size)
dataset = UniversalDataset(data_shape_rnn, target_shape, num_samples)
dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
# 创建RNN模型、优化器和损失函数
rnn_model = RNNModel(data_shape_rnn)
rnn_optimizer = torch.optim.Adam(rnn_model.parameters(), lr=learning_rate)
rnn_criterion = nn.CrossEntropyLoss()
print('RNN模型训练:')
# 训练RNN模型
for epoch in range(num_epochs):
rnn_loss, rnn_acc = train(rnn_model, dataloader, rnn_criterion, rnn_optimizer)
print(f'RNN - Epoch {
epoch+1}/{
num_epochs}, Loss: {
rnn_loss:.4f}, Accuracy: {
rnn_acc:.4f}')
# 重新创建数据集和数据加载器
data_shape_gnn = (10, 100) # (num_nodes, node_features)
dataset = UniversalDataset(data_shape_gnn, target_shape, num_samples)
dataloader = DataLoader(dataset, batch_size=batch_size, shuffle=True)
# 创建GNN模型、优化器和损失函数
gnn_model = GNNModel(data_shape_gnn)
gnn_optimizer = torch.optim.Adam(gnn_model.parameters(), lr=learning_rate)
gnn_criterion = nn.CrossEntropyLoss()
print('GNN模型训练:')
# 训练GNN模型
for epoch in range(num_epochs):
gnn_loss, gnn_acc = train(gnn_model, dataloader, gnn_criterion, gnn_optimizer)
print(f'GNN - Epoch {
epoch+1}/{
num_epochs}, Loss: {
gnn_loss:.4f}, Accuracy: {
gnn_acc:.4f}')
6. Training Results and Summary
operation result:
CNN模型训练:
CNN - Epoch 1/10, Loss: 10.4031, Accuracy: 0.5840
CNN - Epoch 2/10, Loss: 10.2561, Accuracy: 1.0000
CNN - Epoch 3/10, Loss: 10.2503, Accuracy: 1.0000
CNN - Epoch 4/10, Loss: 10.2496, Accuracy: 1.0000
CNN - Epoch 5/10, Loss: 10.2495, Accuracy: 1.0000
CNN - Epoch 6/10, Loss: 10.2494, Accuracy: 1.0000
CNN - Epoch 7/10, Loss: 10.2493, Accuracy: 1.0000
CNN - Epoch 8/10, Loss: 10.2493, Accuracy: 1.0000
CNN - Epoch 9/10, Loss: 10.2493, Accuracy: 1.0000
CNN - Epoch 10/10, Loss: 10.2493, Accuracy: 1.0000
RNN模型训练:
RNN - Epoch 1/10, Loss: 10.3851, Accuracy: 0.9680
RNN - Epoch 2/10, Loss: 10.2606, Accuracy: 1.0000
RNN - Epoch 3/10, Loss: 10.2551, Accuracy: 1.0000
RNN - Epoch 4/10, Loss: 10.2531, Accuracy: 1.0000
RNN - Epoch 5/10, Loss: 10.2520, Accuracy: 1.0000
RNN - Epoch 6/10, Loss: 10.2513, Accuracy: 1.0000
RNN - Epoch 7/10, Loss: 10.2509, Accuracy: 1.0000
RNN - Epoch 8/10, Loss: 10.2506, Accuracy: 1.0000
RNN - Epoch 9/10, Loss: 10.2504, Accuracy: 1.0000
RNN - Epoch 10/10, Loss: 10.2502, Accuracy: 1.0000
GNN模型训练:
GNN - Epoch 1/10, Loss: 10.9591, Accuracy: 0.0400
GNN - Epoch 2/10, Loss: 10.3914, Accuracy: 1.0000
GNN - Epoch 3/10, Loss: 10.2818, Accuracy: 1.0000
GNN - Epoch 4/10, Loss: 10.2635, Accuracy: 1.0000
GNN - Epoch 5/10, Loss: 10.2569, Accuracy: 1.0000
GNN - Epoch 6/10, Loss: 10.2539, Accuracy: 1.0000
GNN - Epoch 7/10, Loss: 10.2524, Accuracy: 1.0000
GNN - Epoch 8/10, Loss: 10.2515, Accuracy: 1.0000
GNN - Epoch 9/10, Loss: 10.2509, Accuracy: 1.0000
GNN - Epoch 10/10, Loss: 10.2505, Accuracy: 1.0000
This article mainly introduces how to create a universal data generation class that can generate data of different shapes according to the input shape parameters. Then, input the generated data and labels into CNN, RNN and GNN models for training, and print out the loss value and accuracy rate. In the future, we can make more modifications and extensions according to specific tasks that may be required in actual applications.