[FIXED] `Jupyter-book build` taking too much RAM when building 4.5K notebooks

Issue

Background

I have preprocessed over 4.5K jupyter notebooks.

I would like to use jupyter-book to build a book from the processed notebooks.

I have execute_notebooks: "off" in my jupyter-book configs.

However, when I run jupyter-book build on the notebooks, I run into an error with the RAM required: the RAM required keeps on increasing.

Questions

QS1. Is there something that I should be doing to avoid this RAM increase? Maybe something in my config/toc ymls?

QS2. Alternatively, is there any way I can estimate how much time/mem will be needed to build the book before running `jupyter-book build?

Extra

Below is a graph of the mem usage in 1hr of processing (when logs are at writing output... [ 3%] content/toi_notebooks/toi_2175)

_toc.yml:

format: jb-book
root: index
parts:
  - chapters:
    - file: content/example_loader
    - file: citation
    - file: content/toi_fits
      sections:
        - glob: content/toi_notebooks/toi_* 

_config.yml

# in _config.yml
title: "TESS Atlas"
author: the TESS Atlas community
logo: _static/atlas_logo.png
execute:
  execute_notebooks: "off"

# Define the name of the latex output file for PDF builds
latex:
  latex_documents:
    targetname: book.tex

html:
  use_edit_page_button: false
  use_repository_button: true
  use_issues_button: true

# Information about where the book exists on the web
repository:
  url: https://github.com/dfm/tess-atlas
  branch: main

gitissue: https://github.com/executablebooks/jupyter-book/issues/1571

Solution

Removing all matplotlib-inline plots and rendering images in markdown cells (with ![]()) has fixed this issue for me. The inline-plots make the notebooks much larger due to the text representation of the image.

Answered By - Avi Vajpeyi