[FIXED] How to save information about the result of instance segmentation by YOLACT?

Issue

Is there any way to save the detected categories, their number, MASK area, etc. to a TXT file or CSV file when performing instance segmentation using YOLACT?

I’m using YOLACT (https://github.com/dbolya/yolact) to challenge instance segmentation. I was able to use eval.py to do an instance segmentation of my own data and save that image or video. However, what I really need is the class names and their numbers detected and classified by YOLACT's AI, and the area of ​​MASK. If we can output this information to a txt file or csv file, we can use YOLACT even more advanced.

If I can achieve that by adding an option in eval.py or modifying the code, please teach me.

Thank you.

Solution

You already have that information from the eval.py.

This line in the eval.py gives you the information.

# line 160
classes, scores, boxes = [x[:args.top_k].cpu().numpy() for x in t[:3]]
masks = t[3][:args.top_k]

areas = []

for mask in masks:
    # Use OpenCV's findContours() to detect and obtain the contour of the variable mask
    # Use OpenCV's contourArea() to calculate the area
    areas.append(area)

Getting the no. of detections:

# line 162
num_dets_to_consider = min(args.top_k, classes.shape[0])
    for j in range(num_dets_to_consider):
        if scores[j] < args.score_threshold:
            num_dets_to_consider = j
            break

To save the classes and scores as a csv file:

import pandas as pd

df = pd.DataFrame(columns=['class', 'score'])
c = 0
for i in reversed(range(num_dets_to_consider)):
    classname = cfg.dataset.class_names[classes[j]]
    score = scores[i]
    df.loc[c] = [classname, score]
    c += 1
df.to_csv('info.csv', index=None)

EDIT1: You can return the values (classes, scores, areas) from the prep_display() and then retrieve them in evalimage(). Note that evalimage() calls prep_display(). You can do something like this:

classes, scores, areas = prep_display()
# Here you can include the pandas code. Note that it stores the information only for one image. You can use a loop and then store the information of all the images.

EDIT2:

# This is the default line in eval.py at line ~600
# This is found at evalimage()
img_numpy = prep_display(preds, frame, None, None, undo_transform=False)

# Change the above line to this:
classes, scores, areas, img_numpy = prep_display(preds, frame, None, None, undo_transform=False)

# Also, remember to return these 4 values from the prep_display(). 
# Otherwise, it will lead to an error. 
# img_numpy is the default one that will be returned from prep_display(). 
# You're now simply returning 4 values instead of 1.

# NOTE: I have included img_numpy because I found that prep_display() is being used in many places. 
# You need to modify the returns wherever prep_display() is being called in the eval.py. I think it's being called 3 times from different regions in eval.py.

Answered By - planet_pluto