kt.testing 3.1.2

Release history

Implementing fixture components

Fixture components are defined by a factory object, usually a class, and are expected to provide a slim API for the harness. Let’s look at a simple but complete, usable example:

import logging


class TestLoggingHandler(logging.StreamHandler):

    def __init__(self, stream, records):
        self.records = records
        super(TestLoggingHandler, self).__init__(stream)

    def handle(self, record):
        self.records.append(record)
        super(TestLoggingHandler, self).handle(record)


class LoggingFixture(object):

    def __init__(self, test, name=None):
        self.test = test
        self.name = name

    def setup(self):
        sio = cStringIO.StringIO()
        self.output = sio.getvalue
        self.records = []
        handler = TestLoggingHandler(sio, self.records)
        logger = logging.getLogger(self.name)
        logger.addHandler(handler)
        self.test.addCleanup(logger.removeHandler, handler)

Using this from a test fixture is straightforward:

import kt.testing


class TestMyThing(kt.testing.TestCase):

    logging = kt.testing.compose(LoggingFixture)

    def test_some_logging(self):
        logging.getLogger('my.package').error('not happy')

        record = self.logging.records[-1]

        self.assertEqual(record.getMessage(), 'not happy')
        self.assertEqual(record.levelname, 'ERROR')

Fixture components may also provide a teardown method that takes no arguments (aside from self). These are called after the tearDown method of the test case is invoked, and do not require that method to be successful. (They are invoked as cleanup functions of the test case.)

Constructor arguments for the fixture component can be provided with kt.testing.compose, but note that the test case instance will always be passed as the first positional argument:

class TestMyThing(kt.testing.TestCase):

    logging = kt.testing.compose(LoggingFixture, name='my.package')

    def test_some_logging(self):
        logging.getLogger('your.package').error('not happy')

        with self.assertRaises(IndexError):
            self.logging.records[-1]

Each instance of the test case class will get it’s own instance of the fixture components, accessible via the properties defined using kt.testing.compose. These instances will already be available when the __init__ method of the test case is invoked.

If the test class overrides the setUp method, it will need to ensure the superclass setUp is invoked so the setup method of the fixture components are invoked:

class TestSomeThing(kt.testing.TestCase):

    logging = kt.testing.compose(LoggingFixture, name='my.package')

    def setUp(self):
        super(TestSomeThing, self).setUp()
        # more stuff here

Note that the setUp didn’t invoke unittest.TestCase.setUp directly. Since kt.testing.compose can cause an additional mix-in class to be added, super is the way to go unless you’re specifically using a base class that’s known to have the right mix-in already mixed.

Multiple fixtures and test inheritance

Multiple fixture components of the same or different types can be added for a single test class:

class TestMyThing(kt.testing.TestCase):

    my = kt.testing.compose(LoggingFixture, name='my.package')
    your = kt.testing.compose(LoggingFixture, name='your.package')

    def test_different(self):
        self.assertIsNot(self.my, self.your)

Base classes that use fixture components will be properly initialized, and properties can be aliased and overridden in ways that make sense:

class TestAnotherThing(TestMyThing):

    orig_my = TestMyThing.my
    my = kt.testing.compose(LoggingFixture, name='my.another')

    def test_different(self):
        self.assertIsNot(self.my, self.your)
        self.assertIsNot(self.orig_my, self.your)
        self.assertIsNot(self.orig_my, self.my)

        self.assertEqual(self.my.name, 'my.another')
        self.assertEqual(self.orig_my.name, 'my.package')
        self.assertEqual(self.your.name, 'your.package')

kt.testing.requests - Intercession for requests

Many applications (and other libraries) use the requests package to retrieve resources identified by URL. It’s often reasonable to use mock directly to handle requests for resources in tests, but sometimes a little more is warranted. The requests library provides multiple ways to trigger particular requests, and applications usually shouldn’t care which is used to make a request.

A fixture component for requests is provided:

class TestMyApplication(kt.testing.TestCase):

    requests = kt.testing.compose(kt.testing.requests.Requests)

A default response entity can be provided via constructor arguments passed through compose. The body and content-type can both be provided:

class TestMyApplication(kt.testing.TestCase):

    requests = kt.testing.compose(
        kt.testing.requests.Requests,
        body='{"success": true, "value": "let's have some json data"}',
        content_type='application/json',
    )

If the default response entity is not defined, an empty body of type text/plain is used.

The fixture provides these methods for configuring responses for particular requests by URL:

add_response(method, url, status=200, body=None, headers={}, filter=None)

Provide a particular response for a given URL and request method. Other aspects of the request are not considered for identifying what response to provide.

If the response status indicates an entity is allowed in the response and body is provided as None, the default body and content-type will be returned. This will be an empty string unless some other value is provided to the fixture component constructor. If the status indicates no entity should be returned, an empty body will be used.

If filter is provided and not None, if must be a callable that accepts the same signature as requests.request and returns a Boolean value indicating whether than response applies to the request being made. If the result is true, the response is considered a match and will be consumed. If false, the response will not be used, but will be considered for subsequent requests.

The provided information will be used to create a response that is returned by the requests API.

add_error(method, url, exception, filter=None)

Provide an exception that should be raised when a particular resource is requested. This can be used to simulate errors such as a non-responsive server or DNS resolution failure. Only the URL and request method are considered for identifying what response to provide.

add_connect_timeout(method, url, filter=None)

Provide an exception structured the same way as it would be were the host not to connect within a reasonable time. This uses add_error, but saves having to construct the exception yourself.

add_read_timeout(method, url, filter=None)

Provide an exception structured the same way as it would be were the host to connect but not respond within a reasonable time. This uses add_error, but saves having to construct the exception yourself.

add_unreachable_host(method, url, filter=None)

Provide an exception structured the same way as it would be were the host unreachable. This uses add_error, but saves having to construct the exception yourself.

If a request is made that does match any provided response, an AssertionError is raised; this will normally cause a test to fail, unless the code under test catches exceptions too aggressively.

A test that completes without consuming all configured responses will cause an AssertionError to be raised during teardown. Test runners based on unittest will usually report this as an error rather than a failure, but it’ll require a developer to take a look, and that’s the point.

If multiple configurations are made for the same request method and URL (whether responses or errors), they’ll be provided to the application in the order configured.

kt.testing.cleanup - Global cleanup registration

Many libraries and applications end up maintaining small bits of global state. These bits may be caches, or information derived from configuration, but they need to be cleared between tests to avoid tests interfering with each other in ways that can be painful to debug.

Clearing these bits of module state in the setUp or tearDown methods of tests takes care of the problem, but each application needs to be aware of every such bit of module state that exists in the libraries and application; this can be a challenge.

Allowing each library or module to register a cleanup function makes it possible to collect everything that’s needed to ensure test cleanup can be sufficient.

This approach was built in the zope.testing package’s cleanup module, which provided functions to register and invoke cleanup functions. The kt.testing.cleanup module provides a similar API. If zope.testing.cleanup is also used, kt.testing.cleanup cooperates by sharing the behind-the-scenes registry of cleanup functions.

There are two functions which provide the kt.testing.cleanup API:

register(func, *args, **kwargs)

Register a callable that should be invoked to clean up module state. The callable will be invoked with the provided additional positional and keyword arguments.

func should be fast and simple, and must not raise an exception.

cleanup()

Invoke all registered cleanups. The cleanup functions will be invoked in the order registered. If zope.testing.cleanup was also used, cleanups registered via each API may be intermingled, according to the order of registration.

The setUp and tearDown methods of kt.testing.TestCase both invoke the cleanup function.