Skip to main content

Sampledata Generator

Project description

This package tries to do the best to support the development of sample
data generators.
A sample data generator is a pluggable tool to create data needed for tests.


Detailed Documentation
**********************

===============================
Pluggable sample data framework
===============================

Creating a good testing environment is the most important way to create
high quality software.

But most of the time it is a pain !

This package tries to do the best to support the development of sample
data generators.
A sample data generator is a pluggable tool to create data needed for tests.


There are several goals to this framework:

- provide the developers with an automatic setup that is close to
real-world use.

- provide the users with an easy setup for evaluation with plausible data

- provide the user a management interface for different sample generators

The framework is pluggable and allows the creators of generator
extensions to provide their own plugins that generate sample data for
those extensions.


Generators
----------

A generator generates sample data.

>>> from zope import interface
>>> from zope import component
>>> from z3c.sampledata.interfaces import ISampleDataPlugin

>>> class GeneratePrincipals(object):
... interface.implements(ISampleDataPlugin)
... dependencies = []
... schema = None
... def generate(self, context, param={}, dataSource=None, seed=None):
... print self.__class__.__name__
... if dataSource is not None:
... for data in dataSource:
... print '- %s'%data['login']
>>> principalPlugin = GeneratePrincipals()

For our tests we provide another generator :

>>> class GenerateSite(object):
... interface.implements(ISampleDataPlugin)
... dependencies = []
... schema = None
... def generate(self, context, param={}, dataSource=None, seed=None):
... if 'sitename' in param:
... print 'This is site %r'%param['sitename']
... else:
... print self.__class__.__name__
... return 'I am from the site'
>>> sitePlugin = GenerateSite()


Generator Manager
-----------------

A generator manager groups a collection of generators.
The manager allows to :

- define dependencies between generators

- define data connections between dependent generators

- provide default configuration data

>>> from z3c.sampledata import Manager
>>> manager = Manager('manager', '')


Generator Plugin
~~~~~~~~~~~~~~~~

For the manager our sample generators must be registered as named utilities.

>>> component.provideUtility(sitePlugin,
... ISampleDataPlugin,'z3c.sampledata.site')
>>> component.provideUtility(principalPlugin,
... ISampleDataPlugin,'z3c.sampledata.principals')


Generating Sample Data
~~~~~~~~~~~~~~~~~~~~~~

Now we can add generators to the manager.

>>> manager.add('z3c.sampledata.principals',
... dependsOn=['z3c.sampledata.site',],
... contextFrom='z3c.sampledata.site')

In addition to the "hardwired" dependencies defined by the dependencies
property in each generator it is possible to add dependencies in the generator
manager.

A manager provides it's generators.

>>> manager.generators.keys()
['z3c.sampledata.principals']

We can tell the manager to generate all samples.
There is no need to add the sample generator 'z3c.sampledata.site', it is added
automatically because of the dependency of 'z3c.sampledata.principals'.

>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals

>>> [info.name for info in infos]
['z3c.sampledata.site', 'z3c.sampledata.principals']


Parameters for the sample generators
------------------------------------

To have more control over the sample generation process it is possible to
setup parameters for the generators.

>>> manager = Manager('manager', '')

>>> manager.add('z3c.sampledata.site',
... param={'sitename':'samplesite'})

>>> manager.add('z3c.sampledata.principals',
... dependsOn=['z3c.sampledata.site',],
... contextFrom='z3c.sampledata.site')

>>> infos = manager.generate(context=None, param={}, seed='something')
This is site 'samplesite'
GeneratePrincipals

It is also possible to overwrite the parameters from the configuration.

>>> infos = manager.generate(context=None,
... param={'z3c.sampledata.site':
... {'sitename':'managers site'}},
... seed='something')
This is site 'managers site'
GeneratePrincipals


Cycles in the generator definition
----------------------------------

>>> manager = Manager('manager', '')
>>> manager.add('z3c.sampledata.principals',
... dependsOn=['z3c.sampledata.site',],
... contextFrom='z3c.sampledata.site')
>>> manager.add('z3c.sampledata.site',
... dependsOn=['z3c.sampledata.principals',])

>>> infos = manager.generate(context=None, param={}, seed='something')
Traceback (most recent call last):
...
CyclicDependencyError: cyclic dependency at 'z3c.sampledata.principals'


A test for a complex dependency.

>>> class Generator(object):
... interface.implements(ISampleDataPlugin)
... name = 'generator'
... dependencies = []
... schema = None
... def generate(self, context, param={}, dataSource=None, seed=None):
... return 'I am a generator'
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.1')
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.2')
>>> component.provideUtility(Generator(), ISampleDataPlugin,'g.3')
>>> manager = Manager('manager', '')
>>> manager.add('g.1')
>>> manager.add('g.2', contextFrom='g.1')
>>> manager.add('g.3', dependsOn=['g.2', 'g.1'], contextFrom='g.1')
>>> infos = manager.generate(context=None, param={}, seed=None)
>>> [info.name for info in infos]
['g.1', 'g.2', 'g.3']


Sample Data Source
------------------

A sample data generator usually gets its sample data from a data source.
Mostly it is necessary to have different data sources for different uses.

As an example, it is always a pain if the sample data for the tests use the
same data as the UI uses later to provide data for the customer to click
around.

>>> manager = Manager('manager', '')

>>> manager.addSource('z3c.datasource.principals',
... data=[{'login':'jukart', 'password':'trakuj'},
... {'login':'srichter', 'password':'rethcirs'}])

>>> manager.add('z3c.sampledata.principals',
... dataSource='z3c.datasource.principals',
... dependsOn=['z3c.sampledata.site',],
... contextFrom='z3c.sampledata.site')

>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals
- jukart
- srichter


It is also possible to use adapters to act as a data source.

>>> manager = Manager('manager', '')

>>> class IPrincipalDataSource(interface.Interface):
... pass

>>> def principalDataFactory(object):
... return [{'login':'jukart', 'password':'trakuj'},
... {'login':'srichter', 'password':'rethcirs'}]

>>> component.provideAdapter(
... factory=principalDataFactory,
... adapts=(ISampleDataPlugin,),
... provides=IPrincipalDataSource,
... name='testprincipals')

>>> manager.addSource('z3c.datasource.principals',
... adapterName='testprincipals',
... adaptTo=IPrincipalDataSource)

>>> manager.add('z3c.sampledata.principals',
... dataSource='z3c.datasource.principals',
... dependsOn=['z3c.sampledata.site',],
... contextFrom='z3c.sampledata.site')

>>> infos = manager.generate(context=None, param={}, seed='something')
GenerateSite
GeneratePrincipals
- jukart
- srichter


How to setup configuration for the generator manager
----------------------------------------------------

Configuration can be done using ZCML.

<configure xmlns="http://namespaces.zope.org/zope">

<configure
xmlns:zcml="http://namespaces.zope.org/zcml"
zcml:condition="have devmode">

<utility
factory=".SampleSite"
provides="z3c.sampledata.interfaces.ISampleDataPlugin"
name="z3c.site"
/>

<utility
factory=".SamplePrincipals"
provides="z3c.sampledata.interfaces.ISampleDataPlugin"
name="z3c.principals"
/>

<SampleManager
name="Site with principals"
>
<generator name="z3c.site" />
<generator
name="z3c.principal"
dependsOn="z3c.site"
contextFrom="z3c.site" />
</SampleManager>

</configure>

</configure>


Data Sources
------------

This package implements the base functionality for data generators.
A data generator is used to provide the raw data for a sample generator.
Raw data can be read from text files in different ways.

>>> from z3c.sampledata.data import DataGenerator
>>> generator = DataGenerator(55)

The generator can read data lines from files.

>>> generator.readLines('testlines.txt')
[u'Line 1', u'Another line']

The generator can read data from CSV files.

>>> generator.readCSV('testlines.csv')
[['Line 1', 'Col 2'], ['Another line', 'Another Col']]

The generator can read a list of files from a path :

>>> import os
>>> generator.files(os.path.dirname(__file__))
['...README.txt', ...]



=======
CHANGES
=======

0.1.0 (2008-02-14)
------------------

- Initial Release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

z3c.sampledata-0.1.0.tar.gz (19.1 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page