[FIXED] Keras model.predict causing a ValueError

Issue

I have a Keras LSTM model that I trained that is supposed to predict the next in sequence:

from tensorflow.keras.layers import LSTM, Dropout, Input, Dense, Flatten
from tensorflow.keras.models import Sequential
import tensorflow.keras.optimizers as o

model = Sequential(
    [
        Input(shape= (500,5)), #500 arrays like this one -> [0,0,0,0,0]
        LSTM(500, return_sequences=False),
    
        Dense(972, activation="softmax"), #the 972 unique words in the vocab
    ]
)

optimizer = o.Adam(learning_rate=0.01)
model.compile(loss="categorical_crossentropy", optimizer=optimizer)
model.fit(corpuswithids, np.asarray(ydata), batch_size=200, epochs=20)

Here is my predict function so far:

def predict(text):
  #text = "this is a test"
  text = text.split(" ")
  ids = []
  for i in text:
     ids.append(texids[np.where(textfull == i)][0]) #Converts text to its id which is 
     #in this format [0,0,0,0,0]
  ids = np.asarray(ids)
  print(ids)
  #prints [[ 95.   0.   0.   5.   5.] 
  #[883.   0.   0.   4.   3.]
  #[ 44.   0.   0.   2.  88.]
  #[ 36.   0.   0.   3. 255.]]

  print(ids.shape)
  #prints (4, 5)
  model.predict(x = ids)
  return ids

This causes the following error:

    ValueError: Input 0 of layer sequential_13 is incompatible with the layer: expected 
    ndim=3, found ndim=2. Full shape received: (None, None)

Do I need to change or pad the length of the ids so that is it 500 long to match the train data? Thanks for any help!

Solution

Problem is with LSTM input shape, LSTM expects input of A 3D tensor with shape [batch, timesteps, feature].

I could reproduce the issue

import tensorflow as tf
input = tf.keras.layers.Input(500,5)
lstm = tf.keras.layers.LSTM(500, return_sequences=False)
output = lstm(input)
print(output.shape)

Output

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-15-0a52ed439ffd> in <module>()
      2 input = tf.keras.layers.Input(500,5)
      3 lstm = tf.keras.layers.LSTM(500, return_sequences=False)
----> 4 output = lstm(input)
      5 print(output.shape)

6 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    216                          'expected ndim=' + str(spec.ndim) + ', found ndim=' +
    217                          str(ndim) + '. Full shape received: ' +
--> 218                          str(tuple(shape)))
    219     if spec.max_ndim is not None:
    220       ndim = x.shape.rank

ValueError: Input 0 of layer lstm_13 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (5, 500)

Working Sample code

import tensorflow as tf
input = tf.keras.layers.Input(500,5)
inputs = tf.expand_dims(input, axis=0)
print(inputs)
lstm = tf.keras.layers.LSTM(4)
output = lstm(inputs)
print(output.shape)

Output

KerasTensor(type_spec=TensorSpec(shape=(1, 5, 500), dtype=tf.float32, name=None), name='tf.expand_dims_2/ExpandDims:0', description="created by layer 'tf.expand_dims_2'")
(1, 4)

Answered By - TFer