A configurable pipeline, aimed at transforming content for import and export
Project description
Transmogrifier
==============
Transmogrifier provides support for building pipelines that turn one thing
into another. Specifically, transmogrifier pipelines are used to convert and
import legacy content into a Plone site. It provides the tools to construct
pipelines from multiple sections, where each section processes the data
flowing through the pipe.
A "transmogrifier pipeline" refers to a description of a set of pipe sections,
slotted together in a set order. The stated goal is for these sections to
transform data and ultimately add content to a Plone site based on this data.
Sections deal with tasks such as sourcing the data (from textfiles, databases,
etc.) and characterset conversion, through to determining portal type,
location and workflow state.
Note that a transmogrifier pipeline can be used to process any number of
things, and is not specific to Plone content import. However, it's original
intent is to provide a pluggable way to import legacy content.
Credits
=======
Development sponsored by
Elkjøp Nordic AS
Design and development
`Martijn Pieters`_ at Jarn_
Project name
A transmogrifier_ is fictional device used for transforming one object
into another object. The term was coined by Bill Waterson of Calvin and
Hobbes fame.
.. _Martijn Pieters: mailto:mj@jarn.com
.. _Jarn: http://www.jarn.com/
.. _Transmogrifier: http://en.wikipedia.org/wiki/Transmogrifier
Change History
**************
(name of developer listed in brackets)
1.5.2 (2018-02-27)
==================
- pep 8, utf-8 headers, whitespace strip and sorted imports.
[thet]
- Ignore - but log - rare cases of construction errors. Usually this
happens for objects you do not want to migrate, but a detailed log entry is
written.
[sunew]
1.5.1 (2015-11-26)
==================
- Fix broken distribution.
[hvelarde]
1.5 (2015-10-22)
================
- Allow csvsource to read files from GS import context
[lentinj]
- Don't use traversal to avoid problems with acquisition or views.
[rpatterson]
- Add csvsource support for taking the filename from an item key.
[rpatterson]
- Add csvsource restkey handling for rows with more keys than fieldnames.
[rpatterson]
- Add a blueprint for opening and caching URLs with `urllib2`_.
[rpatterson]
- Add a source for walking a directory with `os.walk`_.
[rpatterson]
- Add support for arbitrary csvsource fmtparam options.
[rpatterson]
- Add DEBUG logging for expressions, useful for tracking changes to
items as they move through the pipeline.
[rpatterson]
- Add an XML walker source section for walking a tree of elements.
[rpatterson]
- Add a list source section for adding recursion and/or looping to pipelines.
[rpatterson]
- Add pprint support to the logger section, moved from the pprint
section used in tests to make it more useful and available in actual
pipelines.
[rpatterson]
1.4 (2013-04-07)
================
- Fix the import location of the pagetemplate engine for newer Zope versions.
[leorochael]
- Bug fix to load ZCML for GS when Products.GenericSetup is installed.
[aclark]
1.3 (2011-03-17)
================
- Added the GenericSetup import context as an annotation to the transmogrifier.
[elro]
- Added a logger to log the value of a particular key for all items. Handy
when debugging, you can see which path is failing, and good if you want
to show progress in a long import.
[regebro]
- Added a breakpoint section to break on a particular expression, which is
handy for debugging.
[regebro]
1.2 (2010-03-30)
================
- Bug fix: the constructor promises to encode paths to ASCII, but failed to
do so. Thanks to gyst for finding the discrepancy.
[mj]
1.1 (2010-03-17)
================
- Allow the CSV source to load its file from a package as well as from an
absolute or relative file path. To load from a package, pass
``package.name:filename.csv`` to the ``filename`` option.
[optilude]
- Add CMF 2.2/Plone 4 compatibility for the content constructor
[optilude]
- Use an explicit provides attribute to register the transmogrifier adapter.
Fixes the "Missing 'provides' attribute" errors when loading with
zope.annotation installed.
[mj]
- Add a required flag to the content constructor, which causes it to raise
a KeyError if the container where to construct the new item doesn't exist.
[regebro]
- Add an optional condition to the manipulator section.
[regebro]
1.0 (2009-08-07)
================
- Initial transmogrifier architecture.
[mj]
==============
Transmogrifier provides support for building pipelines that turn one thing
into another. Specifically, transmogrifier pipelines are used to convert and
import legacy content into a Plone site. It provides the tools to construct
pipelines from multiple sections, where each section processes the data
flowing through the pipe.
A "transmogrifier pipeline" refers to a description of a set of pipe sections,
slotted together in a set order. The stated goal is for these sections to
transform data and ultimately add content to a Plone site based on this data.
Sections deal with tasks such as sourcing the data (from textfiles, databases,
etc.) and characterset conversion, through to determining portal type,
location and workflow state.
Note that a transmogrifier pipeline can be used to process any number of
things, and is not specific to Plone content import. However, it's original
intent is to provide a pluggable way to import legacy content.
Credits
=======
Development sponsored by
Elkjøp Nordic AS
Design and development
`Martijn Pieters`_ at Jarn_
Project name
A transmogrifier_ is fictional device used for transforming one object
into another object. The term was coined by Bill Waterson of Calvin and
Hobbes fame.
.. _Martijn Pieters: mailto:mj@jarn.com
.. _Jarn: http://www.jarn.com/
.. _Transmogrifier: http://en.wikipedia.org/wiki/Transmogrifier
Change History
**************
(name of developer listed in brackets)
1.5.2 (2018-02-27)
==================
- pep 8, utf-8 headers, whitespace strip and sorted imports.
[thet]
- Ignore - but log - rare cases of construction errors. Usually this
happens for objects you do not want to migrate, but a detailed log entry is
written.
[sunew]
1.5.1 (2015-11-26)
==================
- Fix broken distribution.
[hvelarde]
1.5 (2015-10-22)
================
- Allow csvsource to read files from GS import context
[lentinj]
- Don't use traversal to avoid problems with acquisition or views.
[rpatterson]
- Add csvsource support for taking the filename from an item key.
[rpatterson]
- Add csvsource restkey handling for rows with more keys than fieldnames.
[rpatterson]
- Add a blueprint for opening and caching URLs with `urllib2`_.
[rpatterson]
- Add a source for walking a directory with `os.walk`_.
[rpatterson]
- Add support for arbitrary csvsource fmtparam options.
[rpatterson]
- Add DEBUG logging for expressions, useful for tracking changes to
items as they move through the pipeline.
[rpatterson]
- Add an XML walker source section for walking a tree of elements.
[rpatterson]
- Add a list source section for adding recursion and/or looping to pipelines.
[rpatterson]
- Add pprint support to the logger section, moved from the pprint
section used in tests to make it more useful and available in actual
pipelines.
[rpatterson]
1.4 (2013-04-07)
================
- Fix the import location of the pagetemplate engine for newer Zope versions.
[leorochael]
- Bug fix to load ZCML for GS when Products.GenericSetup is installed.
[aclark]
1.3 (2011-03-17)
================
- Added the GenericSetup import context as an annotation to the transmogrifier.
[elro]
- Added a logger to log the value of a particular key for all items. Handy
when debugging, you can see which path is failing, and good if you want
to show progress in a long import.
[regebro]
- Added a breakpoint section to break on a particular expression, which is
handy for debugging.
[regebro]
1.2 (2010-03-30)
================
- Bug fix: the constructor promises to encode paths to ASCII, but failed to
do so. Thanks to gyst for finding the discrepancy.
[mj]
1.1 (2010-03-17)
================
- Allow the CSV source to load its file from a package as well as from an
absolute or relative file path. To load from a package, pass
``package.name:filename.csv`` to the ``filename`` option.
[optilude]
- Add CMF 2.2/Plone 4 compatibility for the content constructor
[optilude]
- Use an explicit provides attribute to register the transmogrifier adapter.
Fixes the "Missing 'provides' attribute" errors when loading with
zope.annotation installed.
[mj]
- Add a required flag to the content constructor, which causes it to raise
a KeyError if the container where to construct the new item doesn't exist.
[regebro]
- Add an optional condition to the manipulator section.
[regebro]
1.0 (2009-08-07)
================
- Initial transmogrifier architecture.
[mj]
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Close
Hashes for collective.transmogrifier-1.5.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 12792fad3c0ef70318e466b45719b1a13159a0109532b979636ec24c7cc3c54e |
|
MD5 | 823c40cf0ccd8ef559b5e523fe9d1161 |
|
BLAKE2b-256 | c2df64077f2321da9faddc2853d66b39669d6557bd5c00f35eea2b6a6683a7ee |