The connection and difference between nn.Sequential and nn.ModuleList()

nn.Sequentialand nn.ModuleList()are PyTorchtwo different ways of managing submodules in a neural network model.

nn.Sequentialis a container class for building sequential models. It allows adding a series of submodules in a given order and concatenating them together to form a sequential network structure. nn.SequentialIt can simplify the definition of the model and the writing of the forward propagation, especially for those simple network structures without complex control processes. By adding submodules nn.Sequentialto , these submodules are automatically connected together in the order they were added to form an overall model. When the method nn.Sequentialof forward, the input data will go through each sub-module in the order they were added, enabling the forward propagation of the entire model.

The example uses nn.Sequentialto build a simple model:

import torch
import torch.nn as nn

model = nn.Sequential(
    nn.Linear(10, 20),
    nn.ReLU(),
    nn.Linear(20, 10)
)

input_tensor = torch.randn(32, 10)
output_tensor = model(input_tensor)

In this example, we nn.Sequentialdefine . The sequential model consists of three submodules: a linear layer, an ReLUactivation function, and another linear layer. When we call the model's forwardmethod , the input data input_tensorwill go through each submodule in the order they were added, and the output data will be generated output_tensor.

---

In contrast, nn.ModuleList()is a Pythonlist container for storing and managing any number of submodules. nn.SequentialUnlike with , nn.ModuleList()the submodules are not linked automatically, but stored as a list. Therefore, when using to nn.ModuleList()define a model, we need to define the connection relationship between sub-modules by ourselves. This makes it nn.ModuleList()more flexible and suitable for network structures that have complex control processes or require custom connection methods.

The example uses nn.ModuleList()to build a simple model:

import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()

        self.module_list = nn.ModuleList([
            nn.Linear(10, 20),
            nn.ReLU(),
            nn.Linear(20, 10)
        ])

    def forward(self, x):
        for module in self.module_list:
            x = module(x)
        return x

model = MyModel()
input_tensor = torch.randn(32, 10)
output_tensor = model(input_tensor)

In this example, we define a custom model class MyModelthat uses nn.ModuleList()to store three submodules: a linear layer, an ReLUactivation function, and another linear layer. In the model's forwardmethod , we iterate through module_listthe submodules in , xpassing , and get the final output.

Thus, the difference between andnn.Sequential is the ability to autowire submodules. Automatically connect submodules in the order they were added, suitable for simple sequential models. However, you need to manually define the connection mode between submodules, which is suitable for models with complex control processes or custom connections.nn.ModuleList()nn.Sequentialnn.ModuleList()

Additionally, nn.Sequentiala cleaner syntax is provided for defining models, since it is possible to create models directly by passing in a list of submodules. nn.ModuleList()Instead, you need to explicitly define and initialize submodules in the model class.

nn.Sequentialand nn.ModuleList()are nn.Moduleboth subclasses of , so they can both be registered and managed as attributes of the model.