[FIXED] Dimensions Not Matching In PyTorch Linear Layer

Issue

Following the training lesson in PyTorch on this page: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py

It's basically their 'Hello World!' version of an image classifier.

What I'm trying to do is manually code the training steps in the network to make sure I understand each one, but I am currently getting a dimension mismatch in one of my linear layers, which has me stumped. Especially since (AFAIK) I'm recreating the steps in the tutorial exactly.

Anyways........

MY NETWORK:

class net(nn.Module):
def __init__(self):
    super(net, self).__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool  = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1   = nn.Linear(16*5*5, 120)
    self.fc2   = nn.Linear(120, 84)
    self.fc2   = nn.Linear(84, 10)

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = self.pool(F.relu(self.conv2(x)))
    x = x.view(-1, 16 * 5 * 5)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)

    return x

net = net()

I believe this is exactly as they have it on their own page.

I'm trying to calculate the following step without a loop:

for epoch in range(2):  # loop over the dataset multiple times

  running_loss = 0.0
  for i, data in enumerate(trainloader, 0):
    # get the inputs; data is a list of [inputs, labels]
    inputs, labels = data

    # zero the parameter gradients
    optimizer.zero_grad()

    # forward + backward + optimize
    outputs = net(inputs)
    loss = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

    # print statistics
    running_loss += loss.item()
    if i % 2000 == 1999:    # print every 2000 mini-batches
        print('[%d, %5d] loss: %.3f' %
              (epoch + 1, i + 1, running_loss / 2000))
        running_loss = 0.0

  print('Finished Training')

What I'm doing is this:

data = enumerate(trainloader)
inputs, labels = next(data)[1]
outputs = net(inputs)

And the last line gives me the following traceback:

RuntimeError                              Traceback (most recent call last)
<ipython-input-285-d4be5abf5bb1> in <module>
----> 1 outputs = net(inputs)

~\Anaconda\lib\site-packages\torch\nn\modules\module.py in __call__(self, 
*input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

<ipython-input-282-a6eca2e3e9db> in forward(self, x)
    14         x = x.view(-1, 16 * 5 * 5)
    15         x = F.relu(self.fc1(x))
---> 16         x = F.relu(self.fc2(x))
    17         x = self.fc3(x)

Which closes out with:

RuntimeError: size mismatch, m1: [4 x 120], m2: [84 x 10] at 
c:\a\w\1\s\tmp_conda_3.7_110206\conda\conda- 
bld\pytorch_1550401474361\work\aten\src\th\generic/THTensorMath.cpp:940

I know this means my dimension values don't match, and I suspect it has to do with the line x = x.view(-1, 16 * 5 * 5) where I go from the convolutional to linear layer, but I have two confusions:

• As far as I can tell my network matches exactly what's on the PyTorch page
• My error occurs on the second linear layer, not the first, and columns of the preceeding layer match the rows of the current one, so I find it confusing why this error is happening.

Solution

Actually, there is no self.fc3(x) in __init__() as you have mentioned in forward() function. Try running you code by changing

self.fc2 = nn.Linear(84, 10) in __init__() function to self.fc3 = nn.Linear(84, 10).

Above mistake is the reason why you are getting the error. As you are initializing self.fc2 twice in the above code, see below lines:

self.fc2   = nn.Linear(120, 84)
self.fc2   = nn.Linear(84, 10)

Here, first value of self.fc2 is overriden by later value. So, finally it is initialized with a Linear layer with input channels 84 and output channels 10. Later on, in the forward function you are passing the output channels of x = F.relu(self.fc1(x)), i.e., 120 as an input channels to x = F.relu(self.fc2(x)), which has been changed to 84 because of the above explained reasons, you are getting the error.

Apart from this, I don't think if something wrong with your code.

Answered By - Anubhav Singh