redturtle.importer.base 2.1.0

Migration view

To start a migration, you only need to call @@data-migration view on site root.

In this view you can see the blueprint configuration (base and overrided), and start the process.

Pipelines customization

This tool is based on transmogrifier and works with blueprints. A blueprint is basically a config file that lists all the steps needed for the migration.

This product has a default blueprint for basic migrations, that can be used as is.

Default blueprint can be easily customized using a .migrationconfig.cfg file located in buildout root folder.

In this file you can override already present parts/variables (like pipelines into [transmogrifier] section) or add new ones (for example a new step).

For example, catalogsource step can be configured with some queries like this:

[catalogsource]
catalog-query = {'portal_type': ['Document', 'Event', 'News Item']}
...

In .migrationconfig.cfg file, under [catalogsource] section, you also need to set some settings about how to retrieve data on source site:

[catalogsource]
...
remote-url = http://localhost:8080
remote-root = /Plone
catalog-path = /Plone/portal_catalog
remote-username = username
remote-password = password
...

Before running a migration, you can check the final configuration in @@data-migration view.

catalogsource configuration

This is an example of [catalogsource] part:

[catalogsource]
...
remote-url = http://localhost:8080
remote-root = /Plone
catalog-path = /Plone/portal_catalog
remote-username = username
remote-password = password

Required options are:

• remote-url: The url of source Plone site

• remote-root: The path of Plone site that we want to migrate

• remote-username: Credentials to access to source site

• remote-password: Credentials to access to source site

Additional options are:

• default-local-path: A path where save migrate contents in destination Site. This path will replace item’s root path. Destination root path is not needed in this path.

• skip-private: Boolean to migrate or not private items into destination. Default is False.

• remote_skip_paths: A list of paths from source site that will be skipped during migration process.

• incremental-migration: Boolean value. If a content already migrate hasn’t been modified since last migration, don’t override it. Default is False.

• ignore-cache: Boolean value. If True, ignore local cache and always get content data from source site.

• cache-dir: Local folder where migration data cache will be stored. Default is /tmp/migration/migration_cache.

• migration-dir: Local fodler where migration support files (for final summary for example) will be saved. Default is ‘/tmp/migration’.

Custom types mapping

contentsmapping is the section that allows to convert one portal_type to another before object creation.

There is a plugin system based on subscribers that allows plugins to add custom mappings.

You need to register a subscriber for IPortalTypeMapping like this:

<subscriber
    factory=".types_mapping.MyCustomMapping"
    provides="redturtle.importer.base.interfaces.IPortalTypeMapping"/>

And then you need to create the class:

@adapter(IPloneSiteRoot, IBrowserRequest)
@implementer(IPortalTypeMapping)
class MyCustomMapping(object):
    order = 100

    def __init__(self, context, request):
        self.context = context
        self.request = request

    def __call__(self, item, typekey):
        """
        """
        portal_type = item[typekey]
        if portal_type == "Type-A":
            item[typekey] = "Type-B"
            ...
        return item

Custom steps for specific portal types

If you are migrating a content-type that needs some manual fixes after the creation, you can do it with an adapter.

You only need to register an adapter for your content-type like this:

<adapter
  for="my.product.interfaces.IMyInterface"
  provides="redturtle.importer.base.interfaces.IMigrationContextSteps"
  factory=".steps.MyTypeSteps"
/>

And then you need to provide a “doSteps” method in the class:

from redturtle.importer.base.interfaces import IMigrationContextSteps
from zope.interface import implementer

@implementer(IMigrationContextSteps)
class MyTypeSteps(object):

    def __init__(self, context):
        self.context = context

    def doSteps(self):
        """
        do something here
        """

2.1.0 (2022-04-10)

• Better handle datetime and date values. [cekk]

• Make transmogrifier more configurable. [cekk]

2.0.2 (2021-02-01)

• Add geolocation deserializer. [cekk]

2.0.1 (2020-12-18)

• Handle custom destination path. [cekk]

2.0.0 (2020-07-28)

• Heavy refactoring for python3 compatibility. [cekk]

• Allow to import users and groups. [cekk]

• Remove unmaintained dependencies and move needed code here. [cekk]

1.0.5 (2019-03-19)

• fixed fix_link_noreference function. [eikichi18]

1.0.4 (2019-02-08)

• Added fix for links without any references. Added dedicated report after migration. [daniele]

• Added check in schemaupdater for leave field empty when value is empty. [eikichi18]

• Fix broken links generation list. [cekk]

1.0.3 (2018-10-18)

• Added json item to adapters methods. [daniele]

1.0.2 (2018-10-11)

• Fixed mapping for link internal/external link. [eikichi18]

1.0.1 (2018-10-09)

• Fix uudi matcher after migration. [eikichi18]

1.0.0 (2018-10-04)

• Add check if Plone Site element was indexed.

• Add support for specific context steps with adapters. [cekk]

1.0a4 (2018-09-03)

• Handle cases where exlude-type is not set. [cekk]

• Generate a list of broken links in tinymce after migration, and expose them in final report view. [cekk]

1.0a3 (2018-07-19)

• Added check for element’s father data. [eikichi18]

1.0a2 (2018-07-03)

• Break migration if doesn’t find a content types. [eikichi18]

1.0a1 (2018-06-19)

• Initial release. [eikichi18]