[FIXED] PyTorch LSTM: RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 1219 and 440 in dimension 1

Issue

I have a basic PyTorch LSTM:

import torch.nn as nn
import torch.nn.functional as F

class BaselineLSTM(nn.Module):
    def __init__(self):
        super(BaselineLSTM, self).__init__()

        self.lstm = nn.LSTM(input_size=13, hidden_size=13)
        
    def forward(self, x):
        x = self.lstm(x)

        return x

For my data, I have:

train_set = CorruptedAudioDataset(corrupted_path, train_set=True)
train_loader = torch.utils.data.DataLoader(train_set, batch_size=128, shuffle=True, **kwargs)

My CorruptedAudioDataset has:

    def __getitem__(self, index):
        corrupted_sound_file = SoundFile(self.file_paths[index])
        corrupted_samplerate = corrupted_sound_file.samplerate
        corrupted_signal_audio_array = corrupted_sound_file.read()

        clean_path = self.file_paths[index].split('/')
        # print(self.file_paths[index], clean_path)
        clean_sound_file = SoundFile(self.file_paths[index])
        clean_samplerate = clean_sound_file.samplerate
        clean_signal_audio_array = clean_sound_file.read()


        corrupted_mfcc = mfcc(corrupted_signal_audio_array, samplerate=corrupted_samplerate)
        clean_mfcc = mfcc(clean_signal_audio_array, samplerate=clean_samplerate)

  
        print('return', corrupted_mfcc.shape, clean_mfcc.shape)
        return corrupted_mfcc, clean_mfcc

My training loop looks like:

    model = BaselineLSTM()
    for epoch in range(300):
        for inputs, outputs in train_loader:
            print('inputs', inputs)

And that's the line that I get the error on:

  File "train_lstm_baseline.py", line 47, in train
    for inputs, outputs in train_loader:
...
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 1219 and 440 in dimension 1 at ../aten/src/TH/generic/THTensor.cpp:612

Solution

This exception is thrown basically because you're loading batches with different shapes. As they're stored in the same tensor, all samples must have the same shape. In this case, you have an input in dimension 0 with 1219 and 440, which is not possible. For example, you have something like:

torch.Size([1, 1219])
torch.Size([1, 440])
torch.Size([1, 550])
...

You must have:

torch.Size([1, n])
torch.Size([1, n])
torch.Size([1, n])
...

The easiest way to solve this problem is setting batch_size=1. However, it may delay your code.

The best way is setting the data to the same shape. In this case, you need to assess your problem to check if it's possible.

Answered By - André Pacheco