[FIXED] How to use yolov5 tflite export with OpenCV

Issue

I have exported a tflite file from Yolov5 and I got the output data using the code below:

import numpy as np
import tensorflow as tf
from PIL import Image
import os

img = Image.open(os.path.join('dataset', 'images','val','IMG_6099.JPG'))
img = img.resize((256,256),Image.ANTIALIAS)
numpydata = np.asarray(img)
interpreter = tf.lite.Interpreter(model_path="yolov5s-fp16.tflite")
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = np.array(img,dtype=np.float32)
input_data = tf.expand_dims(input_data, 0)
interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])

print out output_data:

[[[1.6754180e-02 3.2771632e-02 8.4546164e-02 ... 2.2025524e-05
   3.0189141e-05 6.1972853e-05]
  [1.5505254e-02 3.5847023e-02 9.6953809e-02 ... 1.9333076e-05
   1.5587271e-05 3.6931968e-05]
  [1.6107641e-02 3.6390714e-02 8.2990780e-02 ... 1.6197217e-05
   1.4623029e-05 3.6216315e-05]
  ...
  [8.6931992e-01 8.8494051e-01 2.4040593e-01 ... 3.1457843e-05
   2.4052188e-05 2.2471884e-05]
  [8.6244017e-01 9.0521729e-01 4.4481179e-01 ... 5.1936011e-05
   3.9207229e-05 3.5609013e-05]
  [8.6841702e-01 9.0255147e-01 7.0057535e-01 ... 1.0812500e-04
   1.0073676e-04 7.7818921e-05]]]

What are these numbers? and more important how can I show the results on the image? I also see this post already.

and here is my code trying to capture objects in real time:

cap = cv2.VideoCapture(0)
ret, frame = cap.read()
print(ret)
frame = cv2.resize(frame, (256 , 256))
    

for i in range(len(scores)):
    if ((scores[i] > 0.1) and (scores[i] <= 1.0)):
        H = frame.shape[0]
        W = frame.shape[1]
        xmin = int(max(1,(xyxy[0][i] * W)))
        ymin = int(max(1,(xyxy[1][i] * H)))
        xmax = int(min(H,(xyxy[2][i] * W)))
        ymax = int(min(W,(xyxy[3][i] * H)))

        # cv2.rectangle(frame, (xmin,ymin), (xmax,ymax), (10, 255, 0), 2)
plt.imshow(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))

Solution

The post you attached is clear said: [x ,y ,w ,h , conf, class0, class1, ...] total 85 columns and many objects (rows) detected. col 0-3 is boxes, col 4 is conf, and the other 80 is the class

You need filter the rows with the conf. Otherwise you will get a lot false detected object.

Also [x ,y ,w ,h] is not the real scale of your input image.

To obtain the real boxes, you may need do some processing like e.g rescale, xywh to xyxy, NMS ( non max suppression) etc..

You can check the detect.py and utils/general.py in Yolov5 source code.

After you get the real boxes, draw box on image.

Below doc show you how to draw box on image using open cv https://docs.opencv.org/4.x/dc/da5/tutorial_py_drawing_functions.html

If you google, there are a lot of example teach you how to draw using open cv

Answered By - Marcus Wong