[FIXED] Dimensionality problem with PyTorch Conv layers

Issue

I'm trying to train a neural network in PyTorch with some input signals. The layers are conv1d. The shape of my input is [100, 10], meaning 100 signals of a length of 10.

But when I execute the training, I have this error: Given groups=1, weight of size [100, 10, 1], expected input[1, 1, 10] to have 10 channels, but got 1 channels instead

config = [10, 100, 100, 100, 100, 100, 100, 100] 
batch_size = 1
epochs = 10
learning_rate = 0.001
kernel_size = 1
    
class NeuralNet(nn.Module):
            
        def __init__(self, config, kernel_size=1):
             super().__init__()
                
             self.config = config
                
             self.layers = nn.ModuleList([nn.Sequential(
                    nn.Conv1d(self.config[i], self.config[i + 1], kernel_size = kernel_size),
                    nn.ReLU()) 
                    for i in range(len(self.config)-1)])
         
             self.last_layer = nn.Linear(self.config[-1], 3)
             self.layers.append(nn.Flatten())
             self.layers.append(self.last_layer)
        
        
        
      def forward(self, x):
           
            for i, l in enumerate(self.layers):
                x = l(x)
            return x
    
    
def loader(train_data, batch_size):
     
  inps = torch.tensor(train_data[0])
  tgts = torch.tensor(train_data[1])
        
  inps = torch.unsqueeze(inps, 1)
  dataset = TensorDataset(inps, tgts)
  train_dataloader = DataLoader(dataset, batch_size = batch_size)
        
  return train_dataloader

At first, my code was without the unsqueez(inps) line and I had the exact same error, but then I added this line thinking that I must have an input of size (num_examples, num_channels, lenght_of_signal) but it didn't resolve the problem at all.

Thank you in advance for your answers

Solution

nn.Conv1d expects input with shape of form (batch_size, num_of_channels, seq_length). It's parameters allow to directly set number of ouput channels (out_channels) and change length of output using, for example, stride. For conv1d layer to work correctly it should know number of input channels (in_channels), which is not the case on first convolution: input.shape == (batch_size, 1, 10), therefore num_of_channels = 1, while convolution in self.layers[0] expects this value to be equal 10 (because in_channels set by self.config[0] and self.config[0] == 10). Hence to fix this append one more value to config:

config = [10, 100, 100, 100, 100, 100, 100, 100]  # as in snippet above
config = [1] + config

At this point convs should be working fine, but there is another obstacle in self.layers -- linear layer at the end. So if kernel_size of 1 was used, then after final convolution batch will have shape (batch_size, 100, 10), and after flatten (batch_size, 100 * 10), while last_layer expects input of shape (batch_size, 100). So, if length of sequence after final conv layer is known (which is certainly the case if you're using kernel_size of 1 with default stride of 1 and default padding of 0 -- length stays same), last_layer should be defined as:

self.last_layer = nn.Linear(final_length * self.config[-1], 3)

and in snippet above final_length can be set to 10 (since conditions in previous brackets satisfied). To catch idea of how shapes in conv1d transformed take look at simple example in gif below (here batch_size is equal to 1):

Answered By - draw