pytest plugin to abort hanging tests
Project description
This is not the timeout you are looking for!
This plugin will time each test and terminate it when it takes too long. Termination may or may not be graceful, please see below, but when aborting it will show a stack dump of all thread running at the time. This is useful when running tests under a continuous integration server or simply if you don’t know why the test suite hangs.
The pytest-timeout plugin has been tested on Python 3.6 and higher, including PyPy3. See tox.ini for currently tested versions.
Usage
Install is as simple as e.g.:
pip install pytest-timeout
Now you can run the test suite while setting a timeout in seconds, any individual test which takes longer than the given duration will be terminated:
pytest --timeout=300
Furthermore you can also use a decorator to set the timeout for an individual test. If combined with the --timeout flag this will override the timeout for this individual test:
@pytest.mark.timeout(60)
def test_foo():
passBy default the plugin will not time out any tests, you must specify a valid timeout for the plugin to interrupt long-running tests. A timeout is always specified as a number of seconds, and can be defined in a number of ways, from low to high priority:
You can set a global timeout in the pytest configuration file using the timeout option. E.g.:
[pytest] timeout =300The PYTEST_TIMEOUT environment variable sets a global timeout overriding a possible value in the configuration file.
The --timeout command line option sets a global timeout overriding both the environment variable and configuration option.
Using the timeout marker on test items you can specify timeouts on a per-item basis:
@pytest.mark.timeout(300) def test_foo(): pass
Setting a timeout to 0 seconds disables the timeout, so if you have a global timeout set you can still disable the timeout by using the mark.
Timeout Methods
Interrupting tests which hang is not always as simple and can be platform dependent. Furthermore some methods of terminating a test might conflict with the code under test itself. The pytest-timeout plugin tries to pick the most suitable method based on your platform, but occasionally you may need to specify a specific timeout method explicitly.
If a timeout method does not work your safest bet is to use the thread method.
thread
This is the surest and most portable method. It is also the default on systems not supporting the signal method. For each test item the pytest-timeout plugin starts a timer thread which will terminate the whole process after the specified timeout. When a test item finishes this timer thread is cancelled and the test run continues.
The downsides of this method are that there is a relatively large overhead for running each test and that test runs are not completed. This means that other pytest features, like e.g. JUnit XML output or fixture teardown, will not function normally. The second issue might be alleviated by using the --boxed option of the pytest-xdist plugin.
The benefit of this method is that it will always work. Furthermore it will still provide you debugging information by printing the stacks of all the threads in the application to stderr.
signal
If the system supports the SIGALRM signal the signal method will be used by default. This method schedules an alarm when the test item starts and cancels the alarm when the test finishes. If the alarm expires during the test the signal handler will dump the stack of any other threads running to stderr and use pytest.fail() to interrupt the test.
The benefit of this method is that the pytest process is not terminated and the test run can complete normally.
The main issue to look out for with this method is that it may interfere with the code under test. If the code under test uses SIGALRM itself things will go wrong and you will have to choose the thread method.
Specifying the Timeout Method
The timeout method can be specified by using the timeout_method option in the pytest configuration file, the --timeout_method command line parameter or the timeout marker. Simply set their value to the string thread or signal to override the default method. On a marker this is done using the method keyword:
@pytest.mark.timeout(method="thread")
def test_foo():
passThe timeout Marker API
The full signature of the timeout marker is:
pytest.mark.timeout(timeout=0, method=DEFAULT_METHOD)You can use either positional or keyword arguments for both the timeout and the method. Neither needs to be present.
See the marker api documentation and examples for the various ways markers can be applied to test items.
Timeouts in Fixture Teardown
The plugin will happily terminate timeouts in the finalisers of fixtures. The timeout specified applies to the entire process of setting up fixtures, running the tests and finalising the fixtures. However when a timeout occurs in a fixture finaliser and the test suite continues, i.e. the signal method is used, it must be realised that subsequent fixtures which need to be finalised might not have been executed, which could result in a broken test-suite anyway. In case of doubt the thread method which terminates the entire process might result in clearer output.
Avoiding timeouts in Fixtures
The timeout applies to the entire test including any fixtures which may need to be setup or torn down for the test (the exact affected fixtures depends on which scope they are and whether other tests will still use the same fixture). If the timeouts really are too short to include fixture durations, firstly make the timeouts larger ;). If this really isn’t an option a timeout_func_only boolean setting exists which can be set in the pytest ini configuration file, as documented in pytest --help.
Debugger Detection
This plugin tries to avoid triggering the timeout when a debugger is detected. This is mostly a convenience so you do not need to remember to disable the timeout when interactively debugging.
The way this plugin detects whether or not a debugging session is active is by checking if a trace function is set and if one is, it check to see if the module it belongs to is present in a set of known debugging frameworks modules OR if pytest itself drops you into a pdb session using --pdb or similar.
Changelog
2.0.2
Fix debugger detection on OSX, thanks Alexander Pacha.
2.0.1
Fix Python 2 removal, thanks Nicusor Picatureanu.
2.0.0
Increase pytest requirement to >=5.0.0. Thanks Dominic Davis-Foster.
Use thread timeout method when plugin is not called from main thread to avoid crash.
Fix pycharm debugger detection so timeouts are not triggered during debugger usage.
Dropped support for Python 2, minimum pytest version upported is 5.0.0.
1.4.2
Fix compatibility when run with pytest pre-releases, thanks Bruno Oliveira,
Fix detection of third-party debuggers, thanks Bruno Oliveira.
1.4.1
Fix coverage compatibility which was broken by 1.4.0.
1.4.0
Better detection of when we are debugging, thanks Mattwmaster58.
1.3.4
Give the threads a name to help debugging, thanks Thomas Grainger.
Changed location to https://github.com/pytest-dev/pytest-timeout because bitbucket is dropping mercurial support. Thanks Thomas Grainger and Bruno Oliveira.
1.3.3
Fix support for pytest >= 3.10.
1.3.2
This changelog was ommitted for the 1.3.2 release and was added afterwards. Apologies for the confusion.
Fix pytest 3.7.3 compatibility. The capture API had changed slightly and this needed fixing. Thanks Bruno Oliveira for the contribution.
1.3.1
Fix deprecation warning on Python 3.6. Thanks Mickaël Schoentgen
Create a valid tag for the release. Somehow this didn’t happen for 1.3.0, that tag points to a non-existing commit.
1.3.0
Make it possible to only run the timeout timer on the test function and not the whole fixture setup + test + teardown duration. Thanks Pedro Algarvio for the work!
Use the new pytest marker API, Thanks Pedro Algarvio for the work!
1.2.1
Fix for pytest 3.3, thanks Bruno Oliveira.
Update supported python versions: - Add CPython 3.6. - Drop CPyhon 2.6 (as did pytest 3.3) - Drop CPyhon 3.3 - Drop CPyhon 3.4
1.2.0
Allow using floats as timeout instead of only integers, thanks Tom Myers.
1.1.0
Report (default) timeout duration in header, thanks Holger Krekel.
1.0.0
Bump version to 1.0 to commit to semantic versioning.
Fix issue #12: Now compatible with pytest 2.8, thanks Holger Krekel.
No longer test with pexpect on py26 as it is no longer supported
Require pytest 2.8 and use new hookimpl decorator
0.5
Timeouts will no longer be triggered when inside an interactive pdb session started by pytest.set_trace() / pdb.set_trace().
Add pypy3 environment to tox.ini.
Transfer repository to pytest-dev team account.
0.4
Support timeouts happening in (session scoped) finalizers.
Change command line option –timeout_method into –timeout-method for consistency with pytest
0.3
Added the PYTEST_TIMEOUT environment variable as a way of specifying the timeout (closes issue #2).
More flexible marker argument parsing: you can now specify the method using a positional argument.
The plugin is now enabled by default. There is no longer a need to specify timeout=0 in the configuration file or on the command line simply so that a marker would work.
0.2
Add a marker to modify the timeout delay using a @pytest.timeout(N) syntax, thanks to Laurant Brack for the initial code.
Allow the timeout marker to select the timeout method using the method keyword argument.
Rename the –nosigalrm option to –method=thread to future proof support for eventlet and gevent. Thanks to Ronny Pfannschmidt for the hint.
Add timeout and timeout_method items to the configuration file so you can enable and configure the plugin using the ini file. Thanks to Holger Krekel and Ronny Pfannschmidt for the hints.
Tested (and fixed) for python 2.6, 2.7 and 3.2.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for pytest_timeout-2.0.2-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | fe9c3d5006c053bb9e062d60f641e6a76d6707aedb645350af9593e376fcc717 |
|
| MD5 | 77f5023a8f5121b53ec857b3bf8314ad |
|
| BLAKE2b-256 | dd7bdd07e2f2553d8e19741d744179aa790b623abfc787a6d0f8a5f22683654a |
