[FIXED] Pytorch -> [Onnx -> tensorflow] -> tflite generates lots of redundant conv2d operators

Issue

I am converting efficientnet from onnx to tensorflow for further conversion to tflite. The conversion from onnx to tensorflow yields strange results

Onnx has 1 conv2d operator

And after converting to tensorflow and later tflite model I get this stack of convs(All of them couldn't fit in the picture, there are 32 of the in total, as the number of kernels in onnx)

I suspect the issue is with onnx-tf conversion. It seems like the conv2d output are split into 32 separate outputs, then they are processed individually on separate conv2d operator and then concatenated together.

There are 2 connected issues on their github.

First has a fix for depthwise convs but it can't be applied in my case

https://github.com/onnx/onnx-tensorflow/issues/473

https://github.com/onnx/onnx-tensorflow/issues/754

Looking at the source of https://github.com/onnx/onnx-tensorflow/blob/master/onnx_tf/handlers/backend/conv_mixin.py They do indeed split the output produce separate convolutions and later concatenate them.

Can this split to multiple convolutions be avoided?

Solution

After some additional digging I've found the following

• My convs were depthwise(conv2d is depthwise in pytorch and onnx if it has groups parameter > 1)
• This bunch of convs is an inefficient way of doing a depthwise conv. To do it efficiently we need to use tf.depthwiseconv

To fix this in onnx-tf v1.7.0 you should apply a patch to onnx-tf source code posted here https://github.com/onnx/onnx-tensorflow/issues/473#issuecomment-533522917

In current master branch there is an attempt to fix the issue by detecting depthwise convs but it currently contains a bug. To fix the bug in master branch you can apply a fix I posted here https://github.com/onnx/onnx-tensorflow/issues/754#issuecomment-801775203 I've used the master branch and the fix above resulting in a small graph with depthwise conv

I've also created a fork with the fix above, so you can do

pip install git+https://github.com/Vozf/onnx-tensorflow

to apply it instead of patching on your own

It seems like the issue should be fixed in the next major release(probably 1.8.0)

Also consider using https://github.com/nerox8664/onnx2keras as it supports depthwise convolutions and moreover supports full nhwc conversion which allows removing all the transpose ops.

Answered By - vozman