[FIXED] Tensorflow lite micro neural network layers building

Issue

I try to run some ML on my ESP32, and I want to use Tensorflow lite micro. But I don't really understand, how they build up the layers. So here is the example how to train the person detection model: Person detection model training

It is clear, but at the end they say:

MobileNet v1 is a stack of 14 of these depthwise separable convolution layers with an average pool, then a fully-connected layer followed by a softmax at the end.

If I check the sample code, where they build up the tf lite micro model, it only has 3 lines:

 static tflite::MicroMutableOpResolver<3> micro_op_resolver;
  micro_op_resolver.AddAveragePool2D();
  micro_op_resolver.AddConv2D();
  micro_op_resolver.AddDepthwiseConv2D();

There is the Average pool, and the depthwise layer, but where the Conv2D layer comes from? And only 1 depthwise layer is presented, but in the documentation, there is 14 depthwise layers in the model.

So the question is, is there any relation between training model, and the model I should build in tensoflow lite micro? If there is, how can I determine how to build up. And that is the question if there is no relation, in what way I need to build up the model.

Solution

They don't explicitly build the model, they rely on a model file that contains the architecture (source):

  model = tflite::GetModel(g_person_detect_model_data);

Where g_person_detect_model_data.cc is generated from a tflite model (containing the architecture) with the following command (See Converting into a c source file in the Readme) :

# Install xxd if it is not available
!apt-get -qq install xxd
# Save the file as a C source file
!xxd -i vww_96_grayscale_quantized.tflite > person_detect_model_data.cc

So the code you shared doesn't build the model. What you see is that for performances reasons, they explicitly add the ops needed by the model instead of relying on the more complex tflite::AllOpsResolver. It is indicated by this comment above the code you shared :

  // Pull in only the operation implementations we need.
  // This relies on a complete list of all the ops needed by this graph.
  // An easier approach is to just use the AllOpsResolver, but this will
  // incur some penalty in code space for op implementations that are not
  // needed by this graph.
  //
  // tflite::AllOpsResolver resolver;
  // NOLINTNEXTLINE(runtime-global-variables)
  static tflite::MicroMutableOpResolver<5> micro_op_resolver;
  micro_op_resolver.AddAveragePool2D();
  micro_op_resolver.AddConv2D();
  micro_op_resolver.AddDepthwiseConv2D();
  micro_op_resolver.AddReshape();
  micro_op_resolver.AddSoftmax();

Answered By - Lescurel