pytest plugin to re-run tests to eliminate flaky failures
Project description
pytest-rerunfailures
pytest-rerunfailures is a plugin for py.test that re-runs tests to eliminate intermittent failures.
Requirements
You will need the following prerequisites in order to use pytest-rerunfailures:
Python 2.7, 3.4, 3.5, 3.6, PyPy, or PyPy3
py.test 2.7.3 or newer
Installation
To install pytest-rerunfailures:
$ pip install pytest-rerunfailuresRe-run all failures
To re-run all test failures, use the --reruns command line option with the maximum number of times you’d like the tests to run:
$ py.test --rerun 5Re-run individual failures
To mark individual tests as flaky, and have them automatically re-run when they fail, add the flaky mark with the maximum number of times you’d like the test to run:
@pytest.mark.flaky(reruns=5)
def test_example():
import random
assert random.choice([True, False])Note that when teardown fails, two reports are generated for the case, one for the test case and the other for the teardown error.
Output
Here’s an example of the output provided by the plugin when run with --reruns 2 and -r aR:
test_report.py RRF
================================== FAILURES ==================================
__________________________________ test_fail _________________________________
def test_fail():
> assert False
E assert False
test_report.py:9: AssertionError
============================ rerun test summary info =========================
RERUN test_report.py::test_fail
RERUN test_report.py::test_fail
============================ short test summary info =========================
FAIL test_report.py::test_fail
======================= 1 failed, 2 rerun in 0.02 seconds ====================
Note that output will show all re-runs. Tests that fail on all the re-runs will be marked as failed.
Compatibility
This plugin may not be used with class, module, and package level fixtures.
This plugin is not compatible with pytest-xdist’s –looponfail flag.
This plugin is not compatible with the core –pdb flag.
Resources
Changelog
3.1 (2017-08-29)
Restored compatibility with pytest-xdist. (Thanks to @davehunt for the PR)
3.0 (2017-08-17)
Add support for Python 3.6.
Add support for pytest 2.9 up to 3.2
Drop support for Python 2.6 and 3.3.
Drop support for pytest < 2.7.
2.2 (2017-06-23)
Ensure that other plugins can run after this one, in case of a global setting --rerun=0. (Thanks to @sublee for the PR)
2.1.0 (2016-11-01)
Add default value of reruns=1 if pytest.mark.flaky() is called without arguments.
Also offer a distribution as universal wheel. (Thanks to @tltx for the PR)
2.0.1 (2016-08-10)
Prepare CLI options to pytest 3.0, to avoid a deprecation warning.
Fix error due to missing CHANGES.rst when creating the source distribution by adding a MANIFEST.in.
2.0.0 (2016-04-06)
Drop support for Python 3.2, since supporting it became too much of a hassle. (Reason: Virtualenv 14+ / PIP 8+ do not support Python 3.2 anymore.)
1.0.2 (2016-03-29)
Add support for –resultlog option by parsing reruns accordingly. (#28)
1.0.1 (2016-02-02)
Improve package description and include CHANGELOG into description.
1.0.0 (2016-02-02)
Rewrite to use newer API of pytest >= 2.3.0
Improve support for pytest-xdist by only logging the final result. (Logging intermediate results will finish the test rather rerunning it.)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_rerunfailures-3.1-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 33a8bd3a151d6250239f72c98b52dc51e092e608f63c44da29eec067efca6f21 |
|
| MD5 | 410ac35c2ff96a0b6fd54f1b644f14da |
|
| BLAKE2b-256 | 0c0a2b205fed1601c64ff670677ac74f312af58c88b672e377c66c7acfaf31d5 |
