testscenarios 0.2

Subclasssing

If you can subclass TestWithScenarios, then the run() method in TestWithScenarios will take care of test multiplication. It will at test execution act as a generator causing multiple tests to execute. For this to work reliably TestWithScenarios must be first in the MRO and you cannot override run() or __call__. This is the most robust method, in the sense that any test runner or test loader that obeys the python unittest protocol will run all your scenarios.

Manual generation

If you cannot subclass TestWithScenarios (e.g. because you are using TwistedTestCase, or TestCaseWithResources, or any one of a number of other useful test base classes, or need to override run() or __call__ yourself) then you can cause scenario application to happen later by calling testscenarios.generate_scenarios(). For instance:

>>> import unittest
>>> import StringIO
>>> from testscenarios.scenarios import generate_scenarios

This can work with loaders and runners from the standard library, or possibly other implementations:

>>> loader = unittest.TestLoader()
>>> test_suite = unittest.TestSuite()
>>> runner = unittest.TextTestRunner(stream=StringIO.StringIO())

>>> mytests = loader.loadTestsFromNames(['doc.test_sample'])
>>> test_suite.addTests(generate_scenarios(mytests))
>>> runner.run(test_suite)
<unittest._TextTestResult run=1 errors=0 failures=0>

Testloaders

Some test loaders support hooks like load_tests and test_suite. Ensuring your tests have had scenario application done through these hooks can be a good idea - it means that external test runners (which support these hooks like nose, trial, tribunal) will still run your scenarios. (Of course, if you are using the subclassing approach this is already a surety). With load_tests:

>>> def load_tests(standard_tests, module, loader):
...     result = loader.suiteClass()
...     result.addTests(generate_scenarios(standard_tests))
...     return result

With test_suite:

>>> def test_suite():
...     loader = TestLoader()
...     tests = loader.loadTestsFromName(__name__)
...     result = loader.suiteClass()
...     result.addTests(generate_scenarios(tests))
...     return result

On the TestCase

You can set a scenarios attribute on the test case:

>>> class MyTest(unittest.TestCase):
...
...     scenarios = [
...         ('scenario1', dict(param=1)),
...         ('scenario2', dict(param=2)),]

This provides the main interface by which scenarios are found for a given test. Subclasses will inherit the scenarios (unless they override the attribute).

After loading

Test scenarios can also be generated arbitrarily later, as long as the test has not yet run. Simply replace (or alter, but be aware that many tests may share a single scenarios attribute) the scenarios attribute. For instance in this example some third party tests are extended to run with a custom scenario.

>>> import testtools
>>> class TestTransport:
...     """Hypothetical test case for bzrlib transport tests"""
...     pass
...
>>> stock_library_tests = unittest.TestLoader().loadTestsFromNames(
...     ['doc.test_sample'])
...
>>> for test in testtools.iterate_tests(stock_library_tests):
...     if isinstance(test, TestTransport):
...         test.scenarios = test.scenarios + [my_vfs_scenario]
...
>>> suite = unittest.TestSuite()
>>> suite.addTests(generate_scenarios(stock_library_tests))

Generated tests don’t have a scenarios list, because they don’t normally require any more expansion. However, you can add a scenarios list back on to them, and then run them through generate_scenarios again to generate the cross product of tests.

>>> class CrossProductDemo(unittest.TestCase):
...     scenarios = [('scenario_0_0', {}),
...                  ('scenario_0_1', {})]
...     def test_foo(self):
...         return
...
>>> suite = unittest.TestSuite()
>>> suite.addTests(generate_scenarios(CrossProductDemo("test_foo")))
>>> for test in testtools.iterate_tests(suite):
...     test.scenarios = [
...         ('scenario_1_0', {}),
...         ('scenario_1_1', {})]
...
>>> suite2 = unittest.TestSuite()
>>> suite2.addTests(generate_scenarios(suite))
>>> print suite2.countTestCases()
4

Dynamic Scenarios

A common use case is to have the list of scenarios be dynamic based on plugins and available libraries. An easy way to do this is to provide a global scope scenarios somewhere relevant to the tests that will use it, and then that can be customised, or dynamically populate your scenarios from a registry etc. For instance:

>>> hash_scenarios = []
>>> try:
...     from hashlib import md5
... except ImportError:
...     pass
... else:
...     hash_scenarios.append(("md5", dict(hash=md5)))
>>> try:
...     from hashlib import sha1
... except ImportError:
...     pass
... else:
...     hash_scenarios.append(("sha1", dict(hash=sha1)))
...
>>> class TestHashContract(unittest.TestCase):
...
...     scenarios = hash_scenarios
...
>>> class TestHashPerformance(unittest.TestCase):
...
...     scenarios = hash_scenarios

Forcing Scenarios

The apply_scenarios function can be useful to apply scenarios to a test that has none applied. apply_scenarios is the workhorse for generate_scenarios, except it takes the scenarios passed in rather than introspecting the test object to determine the scenarios. The apply_scenarios function does not reset the test scenarios attribute, allowing it to be used to layer scenarios without affecting existing scenario selection.