Skip to main content

Pytest plugin for excluding tests based on coverage data

Project description

https://api.travis-ci.org/mhallin/cov-exclude-py.svg?branch=master https://img.shields.io/pypi/v/pytest-cov-exclude.svg

Coverage-based test exclusion plugin for pytest. By looking at which lines are executed by each test, the next test run can be sped up by not executing the tests where no source files have changed.

If you have a large test suite which you often re-run, this plugin can drastically improve the iteration times.

Installation

Install with pip:

$ pip install pytest-cov-exclude

pytest-cov-exclude requires Pytest 2.8 or later. It is compatible with Python 2.7, 3.3, 3.4, 3.5, as well as PyPy. It does not work on PyPy 3 due to an unknown bug with the coverage data generated.

Usage

The first time you run your test suite, per-test coverage is enabled and the suite will take a little bit longer than usual:

$ py.test

==================== test session starts =====================

# Test output...

================ MANY passed in MANY seconds =================

Now when you re-run the test suite, all tests should have been excluded:

$ py.test

==================== test session starts =====================

# Test output...

=============== MANY deselected in FEW seconds ===============

If a test fails, it will re-run even if nothing changed in order to preserve the general failure status of the test suite.

Forcing individual test inclusion

If you have tests that depend on files not included in coverage data, such as data files or generated sources, you can mark the tests with external_dependencies. This forces them to be re-run even if no files were changed:

@pytest.mark.external_dependencies
def test_something():
    # Run tests from external data files

Known bugs

  • Changes to files during pytest’s collection phase is complete will be ignored. Test files and their dependencies are scanned as soon as possible after the test collection is complete.

    There is a test case marked xfail that highlights this issue.

Running the whole test suite

If you want to force a complete re-run of the whole test suite, you have two options: either disable the plugin, or clear pytest’s cache:

$ py.test -p no:cov-exclude  # Disable the plugin

$ py.test --cache-clear  # Clear pytest's cache

Compatibility

As stated earlier, this plugin requires Pytest 2.8 or later since it depends on the new cache module.

While PyPy is supported, the ujson library used for faster serialization/deserialization is not available, so a fallback to the default JSON implementation is used instead. Because of this, the tests might actually run slower with this plugin under PyPy.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pytest-cov-exclude-0.0.6.tar.gz (6.9 kB view details)

Uploaded Source

Built Distribution

pytest_cov_exclude-0.0.6-py2.py3-none-any.whl (11.1 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file pytest-cov-exclude-0.0.6.tar.gz.

File metadata

File hashes

Hashes for pytest-cov-exclude-0.0.6.tar.gz
Algorithm Hash digest
SHA256 8efd81dfbb1422594e8329877848934befbf380fb7dbfae8010ec51c8365c4c2
MD5 ee2c2b3cadf5b46491d25773d49f0f7c
BLAKE2b-256 24e7a60a0aa43b9707910f3bac45885703eeae1d8ca388e64ed37eae0604fc71

See more details on using hashes here.

File details

Details for the file pytest_cov_exclude-0.0.6-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for pytest_cov_exclude-0.0.6-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 dfcdb6285db1dcec6e8a8f096c332ebe71b579165a622b9a9e95f2ea4f68fa6e
MD5 b49fff3f36a19edb3f5886d6fe776c85
BLAKE2b-256 ba9cccaa36402cc9e424d2f3f268277cbcbd1e674845d827ef89d9c3ecb59e76

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page