Skip to main content

Validation and data pipelines made easy!

Project description

https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required |
    f.Decimal |
    f.Min(Decimal(-90)) |
    f.Max(Decimal(90)) |
    f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
    b'\xe2\x99\xaa ',
    b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
    b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
    b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode |
    f.FilterMapper(
        {
            'birthday':  f.Date,
            'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

            'utcOffset':
                f.Decimal |
                f.Min(Decimal('-15')) |
                f.Max(Decimal('+15')) |
                f.Round(to_nearest='0.25'),
        },

        allow_extra_keys   = False,
        allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is known to be compatible with the following Python versions:

  • 3.11

  • 3.10

  • 3.9

Installation

Install the latest stable version via pip:

pip install phx-filters

Extensions

The following extensions are available:

  • Django Filters: Adds filters designed to work with Django applications. To install:

    pip install phx-filters[django]
  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install phx-filters[iso]

Running Unit Tests

Install the package with the test-runner extra to set up the necessary dependencies, and then you can run the tests with the tox command:

pip install -e .[test-runner]
tox -p

To run tests in the current virtualenv:

python -m unittest

Documentation

Documentation is available on ReadTheDocs.

If you are installing from source (see above), you can also build the documentation locally:

  1. Install extra dependencies (you only have to do this once):

    pip install '.[docs-builder]'
  2. Switch to the docs directory:

    cd docs
  3. Build the documentation:

    make html

Releases

Steps to build releases are based on Packaging Python Projects Tutorial

1. Build the Project

  1. Install extra dependencies (you only have to do this once):

    pip install -e '.[build-system]'
  2. Delete artefacts from previous builds, if applicable:

    rm dist/*
  3. Run the build:

    python -m build
  4. The build artefacts will be located in the dist directory at the top level of the project.

2. Upload to PyPI

  1. Create a PyPI API token (you only have to do this once).

  2. Increment the version number in pyproject.toml.

  3. Check that the build artefacts are valid, and fix any errors that it finds:

    python -m twine check dist/*
  4. Upload build artefacts to PyPI:

    python -m twine upload dist/*

3. Create GitHub Release

  1. Create a tag and push to GitHub:

    git tag <version>
    git push

    <version> must match the updated version number in pyproject.toml.

  2. Go to the Releases page for the repo.

  3. Click Draft a new release.

  4. Select the tag that you created in step 1.

  5. Specify the title of the release (e.g., Filters v1.2.3).

  6. Write a description for the release. Make sure to include: - Credit for code contributed by community members. - Significant functionality that was added/changed/removed. - Any backwards-incompatible changes and/or migration instructions. - SHA256 hashes of the build artefacts.

  7. GPG-sign the description for the release (ASCII-armoured).

  8. Attach the build artefacts to the release.

  9. Click Publish release.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

phx-filters-3.2.0.tar.gz (68.5 kB view details)

Uploaded Source

Built Distribution

phx_filters-3.2.0-py3-none-any.whl (39.7 kB view details)

Uploaded Python 3

File details

Details for the file phx-filters-3.2.0.tar.gz.

File metadata

  • Download URL: phx-filters-3.2.0.tar.gz
  • Upload date:
  • Size: 68.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.9

File hashes

Hashes for phx-filters-3.2.0.tar.gz
Algorithm Hash digest
SHA256 1d7611f5d855172f1a8bbad5bf30aee45e929c40dc45bc769bf33d0e945042fe
MD5 d6fe584ca2318c9156c4904e7d5c2da4
BLAKE2b-256 66a3edb7e446885077544dd7e639235a6af084aade5a220be2ae1e7ed8fd5b1e

See more details on using hashes here.

File details

Details for the file phx_filters-3.2.0-py3-none-any.whl.

File metadata

  • Download URL: phx_filters-3.2.0-py3-none-any.whl
  • Upload date:
  • Size: 39.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.9

File hashes

Hashes for phx_filters-3.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 457528dc09f4214a19f6c11e87033f31463f50fb5109976bd28ac00a388effdc
MD5 657429a0aad7608f933ed905836d79e7
BLAKE2b-256 f0f5901c3442db84ac117a8dfeaebea2a812fb30c6ac9dc23ecc8e7e35c414c2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page