pytest plugin to re-run tests to eliminate flaky failures
Project description
pytest-rerunfailures
pytest-rerunfailures is a plugin for py.test that re-runs tests to eliminate intermittent failures.
Requirements
You will need the following prerequisites in order to use pytest-rerunfailures:
Python 2.7, 3.4, 3.5, 3.6, PyPy, or PyPy3
pytest 2.8.7 or newer
Installation
To install pytest-rerunfailures:
$ pip install pytest-rerunfailuresRe-run all failures
To re-run all test failures, use the --reruns command line option with the maximum number of times you’d like the tests to run:
$ pytest --reruns 5To add a delay time between re-runs use the --reruns-delay command line option with the amount of seconds that you would like wait before the next test re-run is launched:
$ pytest --reruns 5 --reruns-delay 1Re-run individual failures
To mark individual tests as flaky, and have them automatically re-run when they fail, add the flaky mark with the maximum number of times you’d like the test to run:
@pytest.mark.flaky(reruns=5)
def test_example():
import random
assert random.choice([True, False])Note that when teardown fails, two reports are generated for the case, one for the test case and the other for the teardown error.
You can also specify the re-run delay time in the marker:
@pytest.mark.flaky(reruns=5, reruns_delay=2)
def test_example():
import random
assert random.choice([True, False])Output
Here’s an example of the output provided by the plugin when run with --reruns 2 and -r aR:
test_report.py RRF
================================== FAILURES ==================================
__________________________________ test_fail _________________________________
def test_fail():
> assert False
E assert False
test_report.py:9: AssertionError
============================ rerun test summary info =========================
RERUN test_report.py::test_fail
RERUN test_report.py::test_fail
============================ short test summary info =========================
FAIL test_report.py::test_fail
======================= 1 failed, 2 rerun in 0.02 seconds ====================
Note that output will show all re-runs. Tests that fail on all the re-runs will be marked as failed.
Compatibility
This plugin may not be used with class, module, and package level fixtures.
This plugin is not compatible with pytest-xdist’s –looponfail flag.
This plugin is not compatible with the core –pdb flag.
Resources
Changelog
4.0 (2017-12-23)
Added option to add a delay time between test re-runs (Thanks to @Kanguros for the PR).
Added support for pytest >= 3.3.
Drop support for pytest < 2.8.7.
3.1 (2017-08-29)
Restored compatibility with pytest-xdist. (Thanks to @davehunt for the PR)
3.0 (2017-08-17)
Add support for Python 3.6.
Add support for pytest 2.9 up to 3.2
Drop support for Python 2.6 and 3.3.
Drop support for pytest < 2.7.
2.2 (2017-06-23)
Ensure that other plugins can run after this one, in case of a global setting --rerun=0. (Thanks to @sublee for the PR)
2.1.0 (2016-11-01)
Add default value of reruns=1 if pytest.mark.flaky() is called without arguments.
Also offer a distribution as universal wheel. (Thanks to @tltx for the PR)
2.0.1 (2016-08-10)
Prepare CLI options to pytest 3.0, to avoid a deprecation warning.
Fix error due to missing CHANGES.rst when creating the source distribution by adding a MANIFEST.in.
2.0.0 (2016-04-06)
Drop support for Python 3.2, since supporting it became too much of a hassle. (Reason: Virtualenv 14+ / PIP 8+ do not support Python 3.2 anymore.)
1.0.2 (2016-03-29)
Add support for –resultlog option by parsing reruns accordingly. (#28)
1.0.1 (2016-02-02)
Improve package description and include CHANGELOG into description.
1.0.0 (2016-02-02)
Rewrite to use newer API of pytest >= 2.3.0
Improve support for pytest-xdist by only logging the final result. (Logging intermediate results will finish the test rather rerunning it.)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_rerunfailures-4.0-py2.py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d59a6be9a7febc5948ea9dd6c912119be3709dfd3889df4f58f10006655a0652 |
|
| MD5 | 0f7055ad527bafe670b57cbccdae11be |
|
| BLAKE2b-256 | 6c3dd1b8ddfa0441cc8963e62c1d559e7a7d1eb3f3d2e3dea7ee9a9747aa00a4 |
