[FIXED] probleme of neural network :mat1 and mat2 shapes cannot be multiplied

Issue

I implemented a simple neuron network like this:

import torch
from torch import nn

class Simple_NN(nn.Module):
  '''
    Multilayer Perceptron.
  '''
  def __init__(self, input_dim):
    super().__init__()
    self.input = input_dim
    #self.out = out_dim
    self.layer = nn.Linear(self.input, 1, bias=False)
      
  def getweights(self):
    return self.layer.weight
  def normalize(self):
    self.layer.weight.data /= self.layer.weight.data.sum()
    return self.layer.weight

  def forward(self, x, dim = 0):
    sort = torch.sort(x, dim, descending = True)[0]
    #top = torch.topk(x, 4, dim)
    sort = self.layer(sort)
    return sort

when I run this piece of code:

outputs = torch.tensor([[1.9, 0.4, 1.3, 0.8, 0.2, 0.0],[1.7, 1.4, 0.3, 1.8, 1.2, 1.1]])
model = Simple_NN(input_dim = outputs.shape[0])
model.getweights()
model.normalize()

I get the following result:

Parameter containing:
tensor([[0.9772, 0.0228]], requires_grad=True)

but, when I run this line:

model(outputs, dim=0)

I get this error:

<ipython-input-1-dd06de9bb6ad> in forward(self, x, dim)
     20     sort = torch.sort(x, dim, descending = True)[0]
     21     #top = torch.topk(x, 4, dim)
---> 22     sort = self.layer(sort)
     23     return sort

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x6 and 2x1)

How can I solve this problem?

Solution

As you didn't provide more details, here's 2 possible ways to solve this:

1. If the batch_size=2, the input_dim should be 6, not 2:

   model = Simple_NN(input_dim = outputs.shape[1])  # change [0] to [1]
   

2. If the batch_size=6, then outputs needs to be transposed:

   model(outputs.t(), dim=0)  # add .t()
   

I think the correct solution to your case is the first one, but both of them work. It depends on what you actually want.

Answered By - Berriel