[FIXED] Keras model.predict does not work with 2 inputs

Issue

I'm completely new to Keras and AI. I have Keras 2.9 with Python 3.8.10 under Ubuntu 20.04. I have a model trained using 2 X inputs and an Y, and technically the training runs. Now I wanted to predict the Y using 2 inputs, but it fails. The training is done using this code fragment (I think only input and output format is interesting here):

def generate(aBatchSize:int=32, aRepeatParameter:int=2, aPort:int=12345):
  dim = (512, 512)
  paraShape = (aRepeatParameter * 2,)

  def generator():
    while True:
    # fill variables
      yield ((xParameter, xImage), y)

  dataset = tensorflow.data.Dataset.from_generator(generator,
    output_signature=(
      (tensorflow.TensorSpec(shape=paraShape, dtype=tensorflow.float32),
      tensorflow.TensorSpec(shape=dim, dtype=tensorflow.float32)),
      tensorflow.TensorSpec(shape=(1), dtype=tensorflow.float32)
            ))
  dataset = dataset.batch(aBatchSize)
  return dataset

repeatParameter = 2
batchSize = 16
model.fit(landscapeGenerator.generate(batchSize, repeatParameter, port), validation_data=landscapeGenerator.generate(batchSize, repeatParameter, port),
  epochs=50, steps_per_epoch=math.ceil(sampleSize / batchSize), validation_steps=validationSize/batchSize )

Printing the model input and output from training code yields this:

model.input [<KerasTensor: shape=(None, 4) dtype=float32 (created by layer 'input_1')>, <KerasTensor: shape=(None, 512, 512, 1) dtype=float32 (created by layer 'input_2')>]
model.output KerasTensor(type_spec=TensorSpec(shape=(None, 1), dtype=tf.float32, name=None), name='dense_4/BiasAdd:0', description="created by layer 'dense_4'")

This is the failing inference code:

image = numpy.multiply(imageio.imread(filename), 1.0 / 255.0)
model = tensorflow.keras.models.load_model(modelDir)

repeatParameter = 2
paraShape = (repeatParameter * 2,)
parameter = numpy.empty(paraShape, dtype=float)
# fill parameters
tempDiff = 5.0 * model.predict((parameter, image))

It writes, because does not understand that the model has 2 inputs of different size:

ValueError: Data cardinality is ambiguous:
  x sizes: 4, 512
Make sure all arrays contain the same number of samples.

I also wanted to make the prediction using a generator, because for that I know how to provide shape info, but no success:

def generate(aParameter, aParaShape, aImage):
  dim = (512, 512)
  def generator():
    while True:
      yield (aParameter, aImage)

  dataset = tensorflow.data.Dataset.from_generator(generator,
    output_signature=(
      tensorflow.TensorSpec(shape=paraShape, dtype=tensorflow.float32),
      tensorflow.TensorSpec(shape=dim, dtype=tensorflow.float32)
    ))
  dataset = dataset.batch(1)
  return dataset

image = numpy.multiply(imageio.imread(filename), 1.0 / 255.0)
model = tensorflow.keras.models.load_model(modelDir)
repeatParameter = 2
paraShape = (repeatParameter * 2,)
parameter = numpy.empty(paraShape, dtype=float)
# fill parameters
tempDiff = 5.0 * model.predict(generate(parameter, paraShape, image), batch_size=1, steps=1)

This one complains: ValueError: Layer "model_2" expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, 4) dtype=float32>]

EDIT: current model generation

def createMlp(aRepeatParameter:int):
  vectorSize = aRepeatParameter * 2
  inputs = Input(shape=(vectorSize,))
  x = inputs
# do not process now, raw data are better  x = Dense(vectorSize, activation="relu")(x)
  return Model(inputs, x)

def createCnn():
  filters=(256, 64, 16)
  inputShape = (512, 512, 1)
  chanDim = -1
  inputs = Input(shape=inputShape)
  x = inputs
  for (i, f) in enumerate(filters):
    x = Conv2D(f, (3, 3), padding="same")(x)
    x = Activation("relu")(x)
    x = BatchNormalization(axis=chanDim)(x)
    x = MaxPooling2D(pool_size=(2, 2))(x)
  x = Flatten()(x)
  x = Dense(512, activation='relu')(x)
  x = Dense(16, activation='relu')(x)
  x = BatchNormalization(axis=chanDim)(x)
  x = Dropout(0.5)(x)
  x = Dense(4)(x)
  x = Activation("relu")(x)
  return Model(inputs, x)


repeatParameter:int = 2
mlp = createMlp(repeatParameter)
cnn = createCnn()

combinedInput = Concatenate(axis=1)([mlp.output, cnn.output])
x = Dense(4, activation="relu")(combinedInput)
x = Dense(1, activation="linear")(x)
model = Model(inputs=[mlp.input, cnn.input], outputs=x)

Solution

It turned out I needed reshaping my inputs, and even that had a typo in it. The working solution is:

def generate(aParameter, aParaShape, aImage):
  dim = (512, 512)
  def generator():
    while True:
      yield (aParameter, aImage)

  dataset = tensorflow.data.Dataset.from_generator(generator,
    output_signature=(
      tensorflow.TensorSpec(shape=paraShape, dtype=tensorflow.float32),
      tensorflow.TensorSpec(shape=dim, dtype=tensorflow.float32)
    ))
  dataset = dataset.batch(1)
  return dataset

image = numpy.multiply(imageio.imread(filename), 1.0 / 255.0)
model = tensorflow.keras.models.load_model(modelDir)
repeatParameter = 2
paraShape = (repeatParameter * 2,)

parameter = numpy.empty(paraShape, dtype=float)
# fill it

parameter = numpy.reshape(parameter, (1, 4))
image = numpy.reshape(image, (1, 512, 512))
tempDiff = 5.0 * model.predict([parameter, image], batch_size=1, steps=1)

Answered By - Balázs Bámer