[FIXED] How to make pandas DataFrame plot's appear at the right point in a Jupyter notebook?

Issue

I have a Jupyter notebook with %matplotlib widget as the first line. The notebook contains several markdown cells providing a header structure and some explaining texts.

Also there I am generating some plots from pandas.DataFrames, which are grouped using dynamically generated sections.

Extracted (not executable in this way), it looks like:

%matplotlib widget

import pandas
from IPython.display import display_markdown

dictionary: dict[str, pandas.DataFrame] = {
    "DataFrame 1": pandas.util.testing.makeDataFrame(),
    "DataFrame 2": pandas.util.testing.makeDataFrame(),
}
group: str
dataframe: pandas.DataFrame
for group, dataframe in dictionary.items():
    display_markdown("## %s" % (group), raw=True)
    dataframe.plot()

However, when running the notebook, it first shows me all the created sub-sections and then, after the last one, all the plots.

How can I bring them in the intended order?

For the case that this is relevant: I am using the Jupyter extension of Visual Studio Code.

Minimal exeutable/ runnable example: https://colab.research.google.com/drive/1iTefKtR93MuzStgpNB3zIxx9S0pAhAO8#scrollTo=yRqBQywrCr7T

Solution

You are seeing the plots last because of the way matplotlib and Jupyter interact. Modern Jupyter puts the plots generated in a cell as a separate entity. To interweave them with markdown produced in the course of looping as the code runs procedurally you can suppress the output using %%capture in that cell, collect the plots, and arrange to show them how you want in another cell using display for both.

Demonstration:
You can the code the follows in sessions launched from here after running %pip install ipympl in a cell first:

Top cell

%%capture
import pandas
from IPython.display import display_markdown

dictionary = {
    "DataFrame 1": pandas.util.testing.makeDataFrame(),
    "DataFrame 2": pandas.util.testing.makeDataFrame(),
}
group: str
dataframe: pandas.DataFrame
title_n_plots =[]
for group, dataframe in dictionary.items():
    #display_markdown("## %s" % (group), raw=True)
    title_n_plots.append([group,dataframe.plot()])

That should display nothing.

Next cell

# Display how they should be associated
for x in title_n_plots:
    display_markdown("## %s" % (x[0]), raw=True)
    display(x[1].figure)

---

Option(s) for still using a single cell and code more like originally posted by adding text as a plot title instead of separate markdown

Of course, an option using the original code layout along the lines of your posted MRE and not suppressing anything could be achieved by adding real titles in the plots that would have stayed associated with the appropriate plot. Like so:

import pandas
from IPython.display import display_markdown

dictionary = {
    "DataFrame 1": pandas.util.testing.makeDataFrame(),
    "DataFrame 2": pandas.util.testing.makeDataFrame(),
}
group: str
dataframe: pandas.DataFrame
title_n_plots =[]
for group, dataframe in dictionary.items():
    #ax = dataframe.plot(title = r"$\bf{" + group + "}$") 
    ax = dataframe.plot(title = r"$\bf{" + group[:-1] + "\ "+ group[-1:] + "}$") 
    #bold in title based on https://stackoverflow.com/a/44123579/8508004
    #hack to fix space showing up before number in `group` based on https://stackoverflow.com/a/34703257/8508004
    ax.title.set_size(40) # based on https://stackoverflow.com/a/67154403/8508004

Or, if you don't want the title centered, you can make it more like the 'display_markdown' example like so:

import pandas
from IPython.display import display_markdown

dictionary = {
    "DataFrame 1": pandas.util.testing.makeDataFrame(),
    "DataFrame 2": pandas.util.testing.makeDataFrame(),
}
group: str
dataframe: pandas.DataFrame
title_n_plots =[]
for group, dataframe in dictionary.items():
    #ax = dataframe.plot(title = r"$\bf{" + group + "}$") 
    ax = dataframe.plot(title = r"$\bf{" + group[:-1] + "\ "+ group[-1:] + "}$") 
    #bold in title based on https://stackoverflow.com/a/44123579/8508004
    #hack to fix space showing up before number in `group` based on https://stackoverflow.com/a/34703257/8508004
    ax.title.set_size(27) # based on https://stackoverflow.com/a/67154403/8508004
    ax.title.set_horizontalalignment("right") # based on https://stackoverflow.com/a/67154403/8508004 and 
    # https://stackoverflow.com/a/44411195/8508004 and that it shows on left-aligned when "right" supplied & vice versa

Answered By - Wayne