[FIXED] Having trouble when implementing CNN+LSTM model that i made into a video

Issue

I'm trying to test my model on video for prediction.

I want to make prediction using my cnn(alexnet)+lstm model on a video that I have, but when it runs, nothing from the video appear.

Here is my code:

vid = cv2.VideoCapture("Data Fix/Data16_133.mp4")

while(vid.isOpened()):
    ret, frame = vid.read()
    vid.set(3, 480)
    vid.set(4, 240)
    start = time.time()
    if ret == True:
        total_frame += 1
        draw = frame.copy()
        draw = cv2.cvtColor(draw, cv2.COLOR_BGR2RGB)
        scale_percent = 20 # percent of original size
        width = 224
        height = width
        dim = (width, height)
        frame_set = cv2.resize(draw, dim, interpolation = cv2.INTER_AREA)
        frame_set=np.arange(10*width*height*3).reshape(10,width, height, 3)
        frame_set.reshape(10, width, height, 3).shape
        frame_set = np.expand_dims(frame_set, axis=0)
        result=model.predict_on_batch(frame_set)

        cv2.imshow('Result', result)
        print(result)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break            

vid.release()
cv2.destroyAllWindows()

when I print the result, it keep printing this value over and over without showing nothing from the cv2.imshow

[[0.0602112  0.01403825 0.3782384  0.5362205  0.01129159]]

does anyone have a clue about this? any answer would be grateful

currently I'm trying this tutorial to make the model and how I put the dataset too are same, the different are I didn't use the MobileNet transfer learning, I modified it using AlexNet model.

Solution

The problem is that your results are not an image that you can display using OpenCV. Your results are output from your model which according to the shared notebook is a classification model and represents the class probabilities. I assume you are trying to predict some class corresponding to the video. If you want to see the frame then you have to use it like this:

cv2.imshow('frame', frame) # to see the frame
# below to see the draw
cv2.imshow('draw', draw)

Edit: If you want to show the predicted class on the image then do the following

# Get the predicted class from the result using argmax
pred_class = np.argmax(result)
# Here I assume that the index is the desired class like most cases

# Now we will write the class label on the image
# Set the font and place
font = cv2.FONT_HERSHEY_SIMPLEX
org = (50, 50)
cv2.putText(frame, str(pred_class), org, font, .5, (255,255,255),2,cv2.LINE_AA)

# now just show the frame
cv2.imshow('frame', frame)

Answered By - Abhishek Prajapat