[FIXED] What is difference between nn.Module and nn.Sequential

Issue

I am just learning to use PyTorch as a beginner. If anyone is familiar with PyTorch, would you tell me the difference between nn.Module and nn.Sequential?

My questions are

1. What is the advantage to use nn.Module instead of nn.Sequential?

2. Which is regularly utilised to build the model?

3. How we should select nn.Module or nn.Sequential?

Solution

TLDR; answering your questions

1. What is the advantage to use nn.Module instead of nn.Sequential?

While nn.Module is the base class to implement PyTorch models, nn.Sequential is a quick way to define a sequential neural network structures inside or outside an existing nn.Module.

2. Which is regularly utilized to build the model?

Both are widely used.

3. How we should select nn.Module or nn.Sequential?

All neural networks are implemented with nn.Module. If the layers are sequentially used (self.layer3(self.layer2(self.layer1(x))), you can leverage nn.Sequential to not have to define the forward function of the model.

---

I should start by mentioning that nn.Module is the base class for all neural network modules in PyTorch. As such nn.Sequential is actually a direct subclass of nn.Module, you can look for yourself on this line.

When creating a new neural network, you would usually go about creating a new class and inheriting from nn.Module, and defining two methods: __init__ (the initializer, where you define your layers) and forward (the inference code of your module, where you use your layers). That's all you need, since PyTorch will handle backward pass with Autograd. Here is an example of a module:

class NN(nn.Module):
    def __init__(self):
        super().__init__()
        
        self.fc1 = nn.Linear(10, 4)
        self.fc2 = nn.Linear(4, 2)

    def forward(self, x)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        return x

If the model you are defining is sequential, i.e. the layers are called sequentially on the input, one by one. Then, you can simply use a nn.Sequential. As I explained earlier, nn.Sequential is a special kind of nn.Module made for this particular widespread type of neural network. The equivalent here is:

class NN(nn.Sequential):
    def __init__(self):
        super().__init__(
           nn.Linear(10, 4),
           nn.ReLU(),
           nn.Linear(4, 2),
           nn.ReLU())

Or a simpler way of putting it is:

NN = Sequential(
   nn.Linear(10, 4),
   nn.ReLU(),
   nn.Linear(4, 2),
   nn.Linear())

The objective of nn.Sequential is to quickly implement sequential modules such that you are not required to write the forward definition, it being implicitly known because the layers are sequentially called on the outputs.

In a more complicated module though, you might need to use multiple sequential submodules. For instance, take a CNN classifier, you could define a nn.Sequential for the CNN part, then define another nn.Sequential for the fully connected classifier section of the model.

Answered By - Ivan