[FIXED] Fast rendering to buffer in Matplotlib

Issue

I have a Kivy application that uses matplotlib to render figures in the application GUI. It means that the application creates a matplotlib Figure and get the Figure's buffer to display it in an Image widget.

For now, each time I want to update the figure, I recreate a Figure and draw everthing, calling refresh_gui_image.

import matplotlib.pyplot as plt

def draw_matplotlib_buffer(image, *elements):
    fig = plt.figure(figsize=(5,5), dpi=200)
    ax = plt.Axes([0, 0, 1, 1])
    ax.set_axis_off()
    fig.add_axis(ax)
    ax.imshow(image)
    for elem in elements:
        # Suppose such a function exists and return a matplotlib.collection.PatchCollection
        patchCollection = elem.get_collection()
        ax.add_collection(patchCollection)
    
    buffer = fig.canvas.print_to_buffer()
    plt.close(fig)
    return buffer

# imageWidget is a kivy Widget instance
def refresh_gui_image(imageWidget, image, *elements):
    size = image.shape()
    imageBuffer = draw_matplotlib_buffer(image, *elements)
    imageWidget.texture.blit_buffer(imageBuffer, size=size, colorfmt='rgba', bufferfmt='ubyte')
    imageWidget.canvas.ask_update()

In the code above, *elements represent multiple sets of objects. Typically, I have 2 to 4 sets which contains between 10 to 2000 objects. Each objects is represented with a patch, and each set is a PatchCollection on the Figure.

It works very well. With the current code, every patch is redrawn each time refresh_gui_image is called. When the sets becomes bigger (like 2000) objects, the update is too slow (few seconds). I want to make a faster rendering with matplotlib, knowing that some of the sets do not have to be redrawn, and that the image stays in the background, and do not have to be redrawn either.

I know blitting and animated artists can be used, this is what I tried, following this tutorial of the matplotlib documentation:

import matplotlib.pyplot as plt

# fig and ax are now global variable
# bg holds the background that stays identical 
fig = None
ax = None
bg = None

def init_matplotlib_data(image, *elements):
    global fig, ax, bg
    fig = plt.figure(figsize=(5,5), dpi=200)
    ax = plt.Axes([0, 0, 1, 1])
    ax.set_axis_off()
    fig.add_axis(ax)
    ax.imshow(image)
    fig.canvas.draw() # I don't want a window to open, just want to have a cached renderer
    bg = fig.canvas.copy_from_bbox(fig.bbox)

    for elem in elements:
            # Suppose such a function exists and return a matplotlib.collection.PatchCollection
            patchCollection = elem.get_collection(animated=True)
            patchCollection.set_animated(True)
            ax.add_collection(patchCollection)

def draw_matplotlib_buffer(image, *artists_to_redraw):
    global fig, ax, bg
    fig.canvas.restore_region(bg)

    for artist in artists_to_redraw:
        ax.draw_artist(artist)
    
    fig.canvas.blit(fig.bbox)
    buffer = fig.canvas.print_to_buffer()
    return buffer

I call init_matplotlib_data once, and the refresh_gui_image as many time as I need, with artists I need to update. The point is that I correctly get my image background, but I cannot succeed to get the patches collections on the buffer returned by fig.canvas.print_to_buffer(). I unset the animated flag of the collection and this time they appear correctly. It seems to me, after some tests that ax.draw_artist() and fig.canvas.blit() have no effect. Another behavior I do not understand is that event if I pass animated=True to ax.imshow(image), the image is still drawn.

Why does the ax.draw_artist and fig.canvas.blit functions does not update the buffer returned by fig.canvas.print_to_buffer as expected ?

Solution

Apparently, blitting is a particular feature meant for GUI. Even thought the Agg backend support blitting, it does not mean that blitting can be used solely with it.

I came up with a solution where I store every artist I want to draw, and change their data whenever I need. I then use fig.canvas.print_to_buffer(), I am not sure what it does exactly, but I thing the figure is fully redrawn. It is probably not as fast as what blitting can do, but it has the advantage to not reallocate and recreate every artists for each update. One can also remove artists from the canvas by calling the remove() method of an artist, and put it again with ax.add_artist(..).

I think this solution answer my question, since it is the fastest solution to have dynamic plotting with matplotlib while dumping the canvas into a buffer.

Answered By - Eliaz