How to use Jupyter notebooks in Sphinx

Jupyter notebooks are a popular tool to describe computational narratives that mix code, prose, images, interactive components, and more. Embedding them in your Sphinx project allows using these rich documents as documentation, which can provide a great experience for tutorials, examples, and other types of technical content. There are a few extensions that allow integrating Jupyter and Sphinx, and this document will explain how to achieve some of the most commonly requested features.

Including classic .ipynb notebooks in Sphinx documentation

There are two main extensions that add support Jupyter notebooks as source files in Sphinx: nbsphinx and MyST-NB. They have similar intent and basic functionality: both can read notebooks in .ipynb and additional formats supported by jupytext, and are configured in a similar way (see Existing relevant extensions for more background on their differences).

First of all, create a Jupyter notebook using the editor of your liking (for example, JupyterLab). For example, source/notebooks/Example 1.ipynb:

Example Jupyter notebook created on JupyterLab

Example Jupyter notebook created on JupyterLab

Next, you will need to enable one of the extensions, as follows:

conf.py
extensions = [
    "nbsphinx",
]

Finally, you can include the notebook in any toctree. For example, add this to your root document:

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   notebooks/Example 1

The notebook will render as any other HTML page in your documentation after doing make html.

Example Jupyter notebook rendered on HTML by nbsphinx

Example Jupyter notebook rendered on HTML by nbsphinx

To further customize the rendering process among other things, refer to the nbsphinx or MyST-NB documentation.

Rendering interactive widgets

Widgets are eventful python objects that have a representation in the browser and that you can use to build interactive GUIs for your notebooks. Basic widgets using ipywidgets include controls like sliders, textboxes, and buttons, and more complex widgets include interactive maps, like the ones provided by ipyleaflet.

You can embed these interactive widgets on HTML Sphinx documentation. For this to work, it’s necessary to save the widget state before generating the HTML documentation, otherwise the widget will appear as empty. Each editor has a different way of doing it:

  • The classical Jupyter Notebook interface provides a “Save Notebook Widget State” action in the “Widgets” menu, as explained in the ipywidgets documentation. You need to click it before exporting your notebook to HTML.

  • JupyterLab provides a “Save Widget State Automatically” option in the “Settings” menu. You need to leave it checked so that widget state is automatically saved.

  • In Visual Studio Code it’s not possible to save the widget state at the time of writing (June 2021).

JupyterLab option to save the interactive widget state automatically

JupyterLab option to save the interactive widget state automatically

For example, if you create a notebook with a simple IntSlider widget from ipywidgets and save the widget state, the slider will render correctly in Sphinx.

Interactive widget rendered in HTML by Sphinx

Interactive widget rendered in HTML by Sphinx

To see more elaborate examples:

Warning

Although widgets themselves can be embedded in HTML, events require a backend (kernel) to execute. Therefore, @interact, .observe, and related functionalities relying on them will not work as expected.

Note

If your widgets need some additional JavaScript libraries, you can add them using add_js_file().

Using notebooks in other formats

For example, this is how a simple notebook looks like in MyST Markdown format:

Example 3.md
---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.10.3
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

# Plain-text notebook formats

This is a example of a Jupyter notebook stored in MyST Markdown format.

```{code-cell} ipython3
import sys
print(sys.version)
```

```{code-cell} ipython3
from IPython.display import Image
```

```{code-cell} ipython3
Image("http://sipi.usc.edu/database/preview/misc/4.2.03.png")
```

To render this notebook in Sphinx you will need to add this to your conf.py:

conf.py
nbsphinx_custom_formats = {
    ".md": ["jupytext.reads", {"fmt": "mystnb"}],
}

Notice that the Markdown format does not store the outputs of the computation. Sphinx will automatically execute notebooks without outputs, so in your HTML documentation they appear as complete.

Creating galleries of examples using notebooks

nbsphinx has support for creating thumbnail galleries from a list of Jupyter notebooks. This functionality relies on Sphinx-Gallery and extends it to work with Jupyter notebooks rather than Python scripts.

To use it, you will need to install both nbsphinx and Sphinx-Gallery, and modify your conf.py as follows:

conf.py
extensions = [
    "nbsphinx",
    "sphinx_gallery.load_style",
]

After doing that, there are two ways to create the gallery:

  • From a reStructuredText source file, using the .. nbgallery:: directive, as showcased in the documentation.

  • From a Jupyter notebook, adding a "nbsphinx-gallery" tag to the metadata of a cell. Each editor has a different way of modifying the cell metadata (see figure below).

Panel to modify cell metadata in JupyterLab

Panel to modify cell metadata in JupyterLab

For example, this reST markup would create a thumbnail gallery with generic images as thumbnails, thanks to the Sphinx-Gallery default style:

Thumbnails gallery
==================

.. nbgallery::
   notebooks/Example 1
   notebooks/Example 2
Simple thumbnail gallery created using nbsphinx

Simple thumbnail gallery created using nbsphinx

To see some examples of notebook galleries in the wild:

Background

Existing relevant extensions

In the first part of this document we have seen that nbsphinx and MyST-NB are similar. However, there are some differences between them:

  • nsphinx uses pandoc to convert the Markdown from Jupyter notebooks to reStructuredText and then to docutils AST, whereas MyST-NB uses MyST-Parser to directly convert the Markdown text to docutils AST. Therefore, nbsphinx assumes pandoc flavored Markdown, whereas MyST-NB uses MyST flavored Markdown. Both Markdown flavors are mostly equal, but they have some differences.

  • nbsphinx executes each notebook during the parsing phase, whereas MyST-NB can execute all notebooks up front and cache them with jupyter-cache. This can result in shorter build times when notebooks are modified if using MyST-NB.

  • nbsphinx provides functionality to create thumbnail galleries, whereas MyST-NB does not have such functionality at the moment (see Creating galleries of examples using notebooks for more information about galleries).

  • MyST-NB allows embedding Python objects coming from the notebook in the documentation (read their “glue” documentation for more information) and provides more sophisticated error reporting than the one nbsphinx has.

  • The visual appearance of code cells and their outputs is slightly different: nbsphinx renders the cell numbers by default, whereas MyST-NB doesn’t.

Deciding which one to use depends on your use case. As general recommendations:

Alternative notebook formats

Jupyter notebooks in .ipynb format (as described in the nbformat documentation) are by far the most widely used for historical reasons.

However, to compensate some of the disadvantages of the .ipynb format (like cumbersome integration with version control systems), jupytext offers other formats based on plain text rather than JSON.

As a result, there are three modes of operation:

  • Using classic .ipynb notebooks. It’s the most straightforward option, since all the tooling is prepared to work with them, and does not require additional pieces of software. It is therefore simpler to manage, since there are fewer moving parts. However, it requires some care when working with Version Control Systems (like git), by doing one of these things:

    • Clear outputs before commit. Minimizes conflicts, but might defeat the purpose of notebooks themselves, since the computation results are not stored.

    • Use tools like nbdime (open source) or ReviewNB (proprietary) to improve the review process.

    • Use a different collaboration workflow that doesn’t involve notebooks.

  • Replace .ipynb notebooks with a text-based format. These formats behave better under version control and they can also be edited with normal text editors that do not support cell-based JSON notebooks. However, text-based formats do not store the outputs of the cells, and this might not be what you want.

  • Pairing .ipynb notebooks with a text-based format, and putting the text-based file in version control, as suggested in the jupytext documentation. This solution has the best of both worlds. In some rare cases you might experience synchronization issues between both files.

These approaches are not mutually exclusive, nor you have to use a single format for all your notebooks. For the examples in this document, we have used the MyST Markdown format.

If you are using alternative formats for Jupyter notebooks, you can include them in your Sphinx documentation using either nbsphinx or MyST-NB (see Existing relevant extensions for more information about the differences between them).