Skip to main content

Simple metadata evolution tracking

Project description

https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336:target:https://pycqa.github.io/isort/

A simple helper for conditional metadata updates

The portal catalog of Zope / Plone sites holds socalled “brains” which have metadata attributes containing the data to be used when the catalog is searched and the search results are listed.

Those metadata columns – like index values – can be customized, and they can evolve over time, as your customization package(s) evolve(s). If your database is small, you’ll happily reindex the whole data whenenver you change something to your metadata (or indexes). Or if your metadata is really cheap.

But perhaps this is not the case.

Perhaps you like to be able to reindex the most important parts first, and the rest can simply become reindexed when changed. And then decide there are more objects which really should be reindexed without being changed themselves.

Perhaps you’d even like to be able to handle cases where objects with different generations of metadata setups occur.

This simple little package doesn’t do anything peculiarly magic; it just provides a little metadata_version column which holds the current tip number of your own metadata evolution, and it offers a little utility which helps you to reindex only those objects which don’t feature the most current metadata setup yet.

Features

  • provides a simple metadata_version metadata attribute, and

  • a simple utility to reindex objects only if their metadata_version is outdated

Examples

This is how the .utils.make_metadata_updater utility function is used.

In your policy package’s setuphandler module, you may have:

from collective.metadataversion.decorator import step
from collective.metadataversion.utils import make_metadata_updater

...
@step
def update_metadata_of_prominent_objects(context, logger):
    reindex = make_metadata_updater(context, logger, 42)
    catalog = getToolByName(context, 'portal_catalog')
    updated, skipped = 0, 0
    for brain in catalog(<my fancy query>):
        if reindex(brain):
            updated += 1
        else:
            skipped += 1
    (... logging ad libitum ...)

@step
def update_metadata_of_remaining_objects(context, logger):
    reindex = make_metadata_updater(context, logger, 42)
    catalog = getToolByName(context, 'portal_catalog')
    updated, skipped = 0, 0
    for brain in catalog({}):
        if reindex(brain):
            updated += 1
        else:
            skipped += 1
    (... logging ad libitum ...)

The reindex function returned by make_metadata_updater() will reindex every object (given by brain) which has not been recently reindexed (with metadata_version=42), and by default refresh a “cheap” selection of indexes.

Starting with the 2nd call to the first upgrade step, nothing will be actually reindexed anymore (unless your <fancy query> spans some more objects now), since everything is up-to-date; the 2nd upgrade step will update all remaining objects (which might not need to be updated so urgently), and skip all objects caught by the <fancy query>.

Notes

  • The @step decorator makes sure you have a non-None logger.

  • The factory function make_metadata_updater takes care for updating the registry value, provided the given metadata_version value is greater than the old one (or forced).

    This is currently the only utility we provide for this purpose. However, you can adjust the registry key directly yourself, if you prefer.

  • There are a few keyword-only options for customization.

Translations

This product has been translated into

  • English

  • German

Installation

Install collective.metadataversion by adding it to your buildout:

[buildout]
...
eggs =
    collective.metadataversion

and then running bin/buildout.

After your Zope instance was restarted, you’ll have the collective.metadataversion package in your extensions view (/prefs_install_products_form), or in the “Quick installer”; select and activate it.

After activation, you’ll have a (prefixed) metadata_version key in your configuration registry which you can adjust according to your needs.

Contribute

Support

If you are having issues, please let us know; please use the issue tracker mentioned above.

License

The project is licensed under the GPLv2.

Contributors

Changelog

0.2.1 (2022-03-04)

New Features:

  • New function .utils.extract_mmu_kwargs

Improvements:

  • Improved the docstring of .utils.make_metadata_updater

[tobiasherp]

0.2.0 (2021-11-17)

Bugfixes:

  • Register the metadata_version indexer, and

  • attach it to IContentish

New Features:

  • New setting default_idxs to adjust the (cheap!) indexes to be refreshed for “metadata-only” updates

[tobiasherp]

0.1.0 (2021-09-21)

  • Initial release. [tobiasherp]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

collective.metadataversion-0.2.1.tar.gz (24.2 kB view details)

Uploaded Source

File details

Details for the file collective.metadataversion-0.2.1.tar.gz.

File metadata

  • Download URL: collective.metadataversion-0.2.1.tar.gz
  • Upload date:
  • Size: 24.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.4.2 requests/2.19.0 setuptools/41.0.1 requests-toolbelt/0.8.0 tqdm/4.23.4 CPython/2.7.5

File hashes

Hashes for collective.metadataversion-0.2.1.tar.gz
Algorithm Hash digest
SHA256 9d340a4e4299758ac8e1c1e38e345362b58e5ff92a734fc4b7b5c487c409f525
MD5 49d2d99e95b65cdfdb7676691dbef6fb
BLAKE2b-256 a2db01b1a64bbd59360acc7c1e65d312180b05fac8500af2b54d2381b036f5b4

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page