[FIXED] How to train an RNN with examples of different lengths in Keras?

Issue

I am learning about RNNs and I'm using TensorFlow/Keras. I understand the basics of vanilla RNN and LSTM layers, but I'm having trouble understanding how to fit my model to the data.

My dataset consists of several time series examples of different sizes say for example:

x1 = [1, 2, 3, 4]
x2 = [3, 7]
x3 = [5, 6, 8, 9, 10, 11]

In the keras documentation, it says the input to an RNN layer must have shape (batch_size, timesteps, input_dim), and so I think, based on the above lists, I would need to create an input array of shape (3, None, 1).

How do I arrange my data so that my model can fit it?

Solution

The best way of handling this is to use masking: https://www.tensorflow.org/guide/keras/masking_and_padding The basic idea is to pad your time sequences, as suggested by Marco Cerliani in his comment, but then to mask off the padding when processing the sequences. That way you're only training your network on the actual data values.

Answered By - Dennis Sosnoski