pytest-notebook

A pytest plugin for regression testing and regenerating Jupyter Notebooks.

Features

• Recognise, collect, execute (optionally output) then diff input vs. output Jupyter Notebooks.

• Provides clear and colorized diffs of the notebooks (using nbdime)

• A well defined API allows notebook regression tests to be run:

  1. Using the pytest test collection architecture.
  2. As a pytest fixtures (nb_regression.check(filename)).
  3. Using the pytest_notebook python package.

• All stages are highly configurable via:

  1. The pytest command-line interface.
  2. The pytest configuration file.
  3. The notebook and cell level metadata.

• Post-processor plugin entry-points, allow for customisable modifications of the notebook, including source code formatting with black

Installation

To install via pip from PyPI:

>> pip install pytest-notebook

To install the development version:

>> git clone https://github.com/chrisjsewell/pytest-notebook .
>> cd pytest-notebook
>> pip install -e .
>> # pip install -e .[code_style,testing,docs] # install extras for more features

Usage

See the documentation at: http://pytest-notebook.readthedocs.io/

Contributing

Contributions are very welcome.

The following will discover and run all unit test:

>> pip install -e .[testing]
>> pytest -v

Coding Style Requirements

The code style is tested using flake8, with the configuration set in .flake8, and code should be formatted with black.

Installing with pytest-notebook[code_style] makes the pre-commit package available, which will ensure these tests are passed by reformatting the code and testing for lint errors before submitting a commit. It can be setup by:

>> cd pytest-notebook
>> pre-commit install

Optionally you can run black and flake8 separately:

>> black .
>> flake8 .

Editors like VS Code also have automatic code reformat utilities, which can adhere to this standard.

License

Distributed under the terms of the BSD-3 license, pytest-notebook is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Acknowledgements

• nbdime
• nbval