pytest-replay 0.1.0

Features

This plugin helps to reproduce random or flaky behavior when running tests with xdist: because xdist executes order in a non-predictable manner, sometimes it is hard to reproduce a behavior seen in CI locally.

This plugin introduces a new --replay-script-dir flag where shell scripts will be recorded containing a footprint of the tests executed by each worker during the session. For example:

$ pytest -n auto --replay-script-dir build/tests/replay

Will now generate platform specific shell-scripts (.bat for Windows and .sh for Linux) in the build/tests/replay directory which record each test executed by each worker, in a ready-to-use command-line to reproduce it later. For example, it will generate a shell script for worker gw1 named .pytest-replay-gw1.sh with this contents:

# generated by pytest-replay
pytest $* \
  test_foo.py::test[1] \
  test_foo.py::test[3] \
  test_foo.py::test[5] \
  test_foo.py::test[7] \
  test_foo.py::test[8] \

So to (hopefully) reproduce a flaky behavior seen in CI one can execute that script locally to run the tests in the same order.

Installation

You can install pytest-replay via pip from PyPI:

$ pip install pytest-replay

Or with conda:

$ conda install -c conda-forge pytest-replay

Contributing

Contributions are very welcome.

Tests can be run with tox if you are using a native Python installation.

To run tests with conda, first create a virtual environment and execute tests from there (conda with Python 3.5+ in the root environment):

$ python -m venv .env
$ .env\scripts\activate
$ pip install -e . pytest-xdist
$ pytest tests

License

Distributed under the terms of the MIT license.

Issues

If you encounter any problems, please file an issue along with a detailed description.