[FIXED] How can I test if matplotlib has produced the correct plot?

Issue

I would like to write tests for code that produces matplotlib plots. Ideally the tests would be able to decide if the output was appropriate without my input.

I have decoupled the data setup into easily testable functions, but I'm unsure how I could decouple the plotting or test the outcome without visual inspection. Is this something anyone has dealt with before?

Is there an established practice for testing in situations like this?

Ideally I would like something like this:

fig, ax = plt.subplot()
ax.plot(x_data, y_data)
ax.set_xlabel('x')
ax.set_ylabel('y')
assertTrue(fig == expected_fig)

Solution

Yes, the Matplotlib developers have dealt with that. And the practice they've established is this:

• Write a test that produces a plot figure.
• Save that figure as an image file in a temporary folder.
• Assert that output image and corresponding "baseline" image are the same.

The test will fail the first time it is run. You then inspect the image, that one time, and use it as the reference for future tests simply by copying the file to the folder holding baseline images.

You may be able to re-use the Matplotlib test fixtures (source code, API documentation) and look at the Matplotlib tests to see how they are used in practice. Essentially, the comparison mechanism loads both image files via PIL, converts them to NumPy arrays, and tests that the two arrays are equal. Though there is also a way to specify a tolerance and allow minor deviations.

Answered By - John Hennig