[FIXED] Tensorflow 2, Serve prediction issue in cpp

Issue

I'm trying to use my final saved_model.pb in cpp to send a text as input and receive predicted class id and score. But I get this error:

W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at strided_slice_op.cc:108 : Invalid argument: slice index 0 of dimension 0 out of bounds.

My model is trained using python. And now I'm using it in TensorFlow 2.5.

More info using saved_model_cli:

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['classification']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: Placeholder:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 74)
        name: head/Tile:0
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 74)
        name: head/predictions/probabilities:0
  Method name is: tensorflow/serving/classify

signature_def['predict']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['content'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: Placeholder:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['all_class_ids'] tensor_info:
        dtype: DT_INT32
        shape: (-1, 74)
        name: head/predictions/Tile:0
    outputs['all_classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 74)
        name: head/predictions/Tile_1:0
    outputs['class_ids'] tensor_info:
        dtype: DT_INT64
        shape: (-1, 1)
        name: head/predictions/ExpandDims:0
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 1)
        name: head/predictions/hash_table_Lookup/LookupTableFindV2:0
    outputs['logits'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 74)
        name: add:0
    outputs['probabilities'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 74)
        name: head/predictions/probabilities:0
  Method name is: tensorflow/serving/predict

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['inputs'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: Placeholder:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['classes'] tensor_info:
        dtype: DT_STRING
        shape: (-1, 74)
        name: head/Tile:0
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 74)
        name: head/predictions/probabilities:0
  Method name is: tensorflow/serving/classify

My source code:

// run the model
std::string test(const char *modelPath, const char *text) {
    tensorflow::SavedModelBundle model;
    tensorflow::Status status = tensorflow::LoadSavedModel(
            tensorflow::SessionOptions(),
            tensorflow::RunOptions(),
            modelPath,
            {"serve"},
            &model);

    TF_CHECK_OK(status);

    // Provide input data.
    tensorflow::Tensor tensor(tensorflow::DT_STRING, tensorflow::TensorShape());
    tensor.scalar<tensorflow::tstring>()() = tensorflow::tstring(text);

    // Link the data with some tags so tensorflow know where to put those data entries.
    std::vector<std::pair<std::string, tensorflow::Tensor>> feedInputs = {{"Placeholder:0", tensor}};
    std::vector<std::string> fetches = {"head/Tile:0", "head/predictions/probabilities:0"};

    // We need to store the results somewhere.
    std::vector<tensorflow::Tensor> outputs;

    // Let's run the model...
    status = model.GetSession()->Run(feedInputs, fetches, {}, &outputs);
    TF_CHECK_OK(status);

    for (const auto& output : outputs) {
        // TODO::
    }

    return "";
}

I've tried changing input and output names in cpp, but it seems the correct names are the ones used in this source code. I'm not very familiar with ternsorflow cpp api. Thanks.

Solution

The problem is in

tensorflow::Tensor tensor(tensorflow::DT_STRING, tensorflow::TensorShape());

It should be

tensorflow::Tensor tensor(tensorflow::DT_STRING, tensorflow::TensorShape({1}));

The error tells you Invalid argument: slice index 0 of dimension 0 out of bounds. Looks like tensor did not have extra dimension (1-D) for 1 string.

Answered By - Symmetrical