pytest-doctestplus 1.2.1

Setup and Configuration

This plugin provides three command line options: --doctest-plus for enabling the advanced features mentioned above, --doctest-rst for including *.rst files in doctest collection, and --doctest-ufunc for including doctests in docstrings of Numpy ufuncs.

This plugin can also be enabled by default by adding doctest_plus = enabled to the [tool:pytest] section of the package’s setup.cfg file.

The plugin is applied to all directories and files that pytest collects. This means that configuring testpaths and norecursedirs in setup.cfg also affects the files that will be discovered by pytest-doctestplus. In addition, this plugin provides a doctest_norecursedirs configuration variable that indicates directories that should be ignored by pytest-doctestplus but do not need to be ignored by other pytest features.

Using pytest’s built-in --doctest-modules option will override the behavior of this plugin, even if doctest_plus = enabled in setup.cfg, and will cause the default doctest plugin to be used. However, if for some reason both --doctest-modules and --doctest-plus are given, the pytest-doctestplus plugin will be used, regardless of the contents of setup.cfg.

pytest-doctestplus respects the --doctest-continue-on-failure flag. If set, doctests will report all failing lines, which may be useful to detect independent errors within the same doctest. However, it is likely to generate false positives when an early failure causes a variable later lines access to remain unset or have an unexpected value.

This plugin respects the doctest options that are used by the built-in doctest plugin and are set in doctest_optionflags in setup.cfg. By default, ELLIPSIS and NORMALIZE_WHITESPACE are used. For a description of all doctest settings, see the doctest documentation.

Running Tests in Markdown Files

To run doctests in Markdown files, invoke pytest with the command line options --doctest-plus --doctest-glob '*.md'.

If you write doctests inside GitHub-style triple backtick fenced code blocks, then in order for pytest-doctest to find and run them you need to include an extra trailing newline inside your code blocks, like this:

```pycon
>>> 1 + 2
2

```

Doctest Directives

The pytest-doctestplus plugin defines doctest directives that are used to control the behavior of particular features. For general information on directives and how they are used, consult the documentation. The specifics of the directives that this plugin defines are described in the sections below.

Sphinx Doctest Directives

You can use testsetup and testcleanup in Sphinx RST to run code that is not visible in rendered document. However, due to how pytest-doctestplus works, the code within needs to be prepended by >>>. For example:

.. testsetup::

    >>> x = 42

.. testcleanup::

    >>> del x

Floating Point Comparison

Some doctests may produce output that contains string representations of floating point values. Floating point representations are often not exact and contain roundoffs in their least significant digits. Depending on the platform the tests are being run on (different Python versions, different OS, etc.) the exact number of digits shown can differ. Because doctests work by comparing strings this can cause such tests to fail.

To address this issue, the pytest-doctestplus plugin provides support for a FLOAT_CMP flag that can be used with doctests. For example:

>>> 1.0 / 3.0  # doctest: +FLOAT_CMP
0.333333333333333311

>>> {'a': 1 / 3., 'b': 2 / 3.}  # doctest: +FLOAT_CMP
{'a': 0.333333, 'b': 0.666666}

When this flag is used, the expected and actual outputs are both parsed to find any floating point values in the strings. Those are then converted to actual Python float objects and compared numerically. This means that small differences in representation of roundoff digits will be ignored by the doctest. The values are otherwise compared exactly, so more significant (albeit possibly small) differences will still be caught by these tests.

This flag can be enabled globally by adding it to setup.cfg as in

doctest_optionflags =
    NORMALIZE_WHITESPACE
    ELLIPSIS
    FLOAT_CMP

Ignoring warnings

If code in a doctest emits a warning and you want to make sure that warning is silenced, you can make use of the IGNORE_WARNINGS flag. For example:

>>> import numpy as np
>>> np.mean([])  # doctest: +IGNORE_WARNINGS
np.nan

Showing warnings

If code in a doctest emits a warning and you want to make sure that warning is shown, you can make use of the SHOW_WARNINGS flag. This is useful when warnings are turned into errors by pytest, and also because by default warnings are printed to stderr. This is the opposite from IGNORE_WARNINGS so obviously the two flags should not be used together. For example:

>>> import numpy as np
>>> np.mean([])  # doctest: +SHOW_WARNINGS
RuntimeWarning: Mean of empty slice.
RuntimeWarning: invalid value encountered in double_scalars
np.nan

Skipping Tests

Doctest provides the +SKIP directive for skipping statements that should not be executed when testing documentation.

>>> open('file.txt') # doctest: +SKIP

In Sphinx .rst documentation, whole code example blocks can be skipped with the directive

.. doctest-skip::

    >>> import asdf
    >>> asdf.open('file.asdf')

However, it is often useful to be able to skip docstrings associated with particular functions, methods, classes, or even entire files.

.. doctest-skip-all

   >>> import non_existing
   >>> non_existing.write_pseudo_code()
   All the doctests are skipped in the file below

__doctest_skip__ = ['myfunction', 'MyClass', 'MyClass.*']

__doctest_skip__  = ['.', 'myfunction', 'MyClass']

__doctest_skip__ = ['*']

__doctest_requires__ = {('func1', 'func2'): ['scipy']}

.. doctest-requires:: asdf

    >>> import asdf
    >>> asdf.open('file.asdf')

.. doctest-requires:: asdf<3

    >>> import asdf
    >>> asdf.open('file.asdf')

doctest_subpackage_requires =
    astropy/wcs/* = scipy>2.0;numpy>1.14
    astropy/cosmology/* = scipy>1.0

Remote Data

The pytest-doctestplus plugin can be used in conjunction with the pytest-remotedata plugin in order to control doctest code that requires access to data from the internet. In order to make use of these features, the pytest-remotedata plugin must be installed, and remote data access must be enabled using the --remote-data command line option to pytest. See the pytest-remotedata plugin documentation for more details.

The following example illustrates how a doctest that uses remote data should be marked:

>>> from urlib.request import urlopen
>>> url = urlopen('http://astropy.org')  # doctest: +REMOTE_DATA

The +REMOTE_DATA directive indicates that the marked statement should only be executed if the --remote-data option is given. By default, all statements marked with the remote data directive will be skipped.

Whole code example blocks can also be marked to control access to data from the internet this way:

.. doctest-remote-data::

    >>> import requests
    >>> r = requests.get('https://www.astropy.org')

Sphinx Compatibility

To use the additional directives when building your documentation with sphinx you may want to enable the sphinx extension which registers these directives with sphinx. Doing so ensures that sphinx correctly ignores these directives, running the doctests with sphinx is not supported. To do this, add 'pytest_doctestplus.sphinx.doctestplus' to your extensions list in your conf.py file.