wagtail-content-audit

Content audit utilities for Wagtail. Still a work in progress.

For Wagtail sites with deeply-nested blocks and a large amount of potentially old content, it can be helpful to inspect block usage and be able to search through the content as it exists in the database. This library is intended to help with these and other challenges of auditing the content in Wagtail.

• Dependencies
• Installation
• Usage
  • Block usage
    • Block usage management command
    • Block usage QuerySet
  • Page search
    • Page search management command
    • Page search QuerySet
• Getting help
• Getting involved
• Licensing

Dependencies

• Python 3.8+
• Django 3.2 (LTS)+
• Wagtail 5.2 (LTS)+

It should be compatible at all intermediate versions, as well. If you find that it is not, please file an issue.

Installation

1. Install wagtail-content-audit:

pip install wagtail-content-audit

2. Add wagtail_content_audit as an installed app in your Django settings.py:

INSTALLED_APPS = (
    ...
    "wagtail_content_audit",
    ...
)

Usage

wagtail-content-audit provides two primary audit tools at present:

• Block usage auditing
• Page field searching

For both, it provides a QuerySet-like object using queryish that returns instances of a dataclass with relevant result data.

Block usage

Block usage is intended to audit deeply-nested Wagtail Blocks to discover how much these blocks might be used, and wwithin which other blocks and fields that usage occurs.

Block usage management command

wagtail-content-audit provides a management command to run the block usage audit and output CSV results:

./manage.py block_usage

The resulting CSV can be redirected to a file:

./manage.py block_usage > block_usage_audit.csv

The command takes the following arguments:

--pagetype PAGETYPE_AND_FIELD, -p PAGETYPE_AND_FIELD

Limits the audit to the particular page type(s) and Wagtail StreamField as a dotted path. For example,

./manage.py block_usage --pagetype myapp.PageWithContent.content

Will output the blocks used in all myapp.PageWithContent pages' content field.

Block usage QuerySet

from wagtail_content_audit.query import BlockUsageQuerySet

The underlying queryish QuerySet can be used outside of the management management command as well. This QuerySet behaves like any queryish QuerySet, with a limited set of available options.

It can be filtered for page types:

filtered_queryset = BlockUsageQuerySet().filter(page_model="myapp.PageWithContent")

It can be filtered for Wagtail StreamFields:

filtered_queryset = BlockUsageQuerySet().filter(field="content")

And these can be combined:

filtered_queryset = BlockUsageQuerySet().filter(page_model="myapp.PageWithContent", field="content")

The queryset can also be sliced:

sliced_queryset = BlockUsageQuerySet()[:5]

The resulting objects in the queryset are wagtail_content_audit.query.AuditedBlock objects with the following schema:

@dataclass
class AuditedBlock:
    page_model: type
    field: str
    path: str
    block: type
    pages: list
    total_occurrences: int = 0
    pages_count: int = 0
    pages_live_count: int = 0
    pages_in_default_site_count: int = 0

Page search

Page search is intended to enable searching for specific patterns (using regular expressions) in text content in all Wagtail Page model fields.

For StreamFields specifically, it returns explicit block paths within a StreamField (i.e., 0.list.item.1.richtext for a result found in the second child list item in the first child block in the field) as well as the general block path (i.e., list.item.richtext) so that the blocks can be targetted using Wagtail StreamField migrations.

Page search management command

wagtail-content-audit provides a management command to run the page search audit and output CSV results:

./manage.py page_search -s '[tT]est'

The resulting CSV can be redirected to a file:

./manage.py page_search -s '[tT]est' > page_search_test.csv

The command takes the following arguments:

--pagetype PAGETYPE_AND_FIELD, -p PAGETYPE_AND_FIELD

Limits the search to the particular page type(s) and model field as a dotted path. For example,

./manage.py page_search -s '[tT]est' --pagetype myapp.PageWithContent.content

Will only search within the content field of myapp.PageWithContent pages.

Page search QuerySet

from wagtail_content_audit.query import PageSearchQuerySet

The underlying queryish QuerySet can be used outside of the management management command as well. This QuerySet behaves like any queryish QuerySet, with a limited set of available options.

It can be searched with any regular expression string:

search_queryset = PageSearchQuerySet().filter(search=r"[tT]est")

It can be filtered for page types:

filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", page_model="myapp.PageWithContent")

It can be filtered for model fields:

filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", field="content")

And these can be combined:

filtered_queryset = PageSearchQuerySet().filter(search=r"[tT]est", page_model="myapp.PageWithContent", field="content")

The queryset can also be sliced:

sliced_queryset = BlockUsageQuerySet()[:5]

The resulting objects in the queryset are wagtail_content_audit.query.pagesearch.PageMatch objects with the following schema:

@dataclass
class PageMatch:
    page_model: type
    page: Page
    field_name: str
    field_type: str
    stream_field_path: list
    block_type: type
    result_path: list
    matches: list

Getting help

Please add issues to the issue tracker.

Getting involved

General instructions on how to contribute can be found in CONTRIBUTING.

Licensing

1. TERMS
2. LICENSE
3. CFPB Source Code Policy