[FIXED] Reproducing OpenCV imshow() and waitKey() with matplotlib

Issue

The following code works well on Windows but I need to run it on Ubuntu and it seems that cv2.imshow() is not supported. I'm looking for an alternative using matplotlib or any other library which runs on Ubuntu.

It works as follows:

I have a list of frames, a frame being a numpy array representing an RGB image. I loop through the frames, show the current one with cv2.imshow() and use waitKey() to register an action, either keep the frame (append it to another list) or dismiss it. I need the image to appear in the same window as the others.

The code:

"""
    frames is a list of 3-d numpy array representing an RGB image.
"""
keeped_frames = []
frame_index = 0
frames_upper_bound = len(frames)
for frame in frames:
    print('Showing frame {} / {}'.format(frame_index + 1,frames_upper_bound))
    cv2.imshow('frame', frame)
    key = cv2.waitKey(0) & 0xFF
    if key == ord('0'):
        print('Not appending frame {} / {} to list'.format(frame_index + 1, frames_upper_bound))
    elif key == ord('1'):
        print('Appending frame {} / {} to list'.format(frame_index + 1, frames_upper_bound))
        keeped_frames.append(frame)
    elif key == ord('q'):
        cv2.destroyAllWindows()
        break
    frame_index += 1

I tried with plt.imshow() on a loop but it creates new windows every time. Even if this is fixed, how can I easily register keyboard action and quickly switch to the next frame? With cv2 it works really well, I can keep the key down and frames just pass pretty fast; since I need to annotate thousands of frames this way, I'd like something similar.

Can I replicate this behavior with matplotlib or any other library which runs on Ubuntu?

Solution

Surely, you can use the imshow method of matplotlib.pyplot and event handling capabilities of matplotlib. You can try the following code and see if it works for you:

import matplotlib as mpl 
mpl.use('wxAgg')

import numpy as np 
import matplotlib.pyplot as plt

# just some random data 
frames = [np.random.random((10, 10)) for _ in range(100)]
keeped_frames = [] 
i = 0

# event listener 
def press(event):
    global i
    if event.key == '1':
        print('Appending frame')
        keeped_frames.append(frames[i % 100])
    i += 1
    imgplot.set_data(frames[i % 100])
    fig.canvas.draw()

fig, ax = plt.subplots() 
fig.canvas.mpl_connect('key_press_event', press)
imgplot = ax.imshow(frames[i % 100]) 
plt.show()

One thing that you should be aware of is that the listener behaviour depends on the backend you're using. In Qt5Agg one key press fires one key_press event whether you hold the key or not, and in wxAgg key holding fires continious press and release events (the behaviour you desire, I guess).

Answered By - Andrey Sobolev