Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/todofixthis/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required |
    f.Decimal |
    f.Min(Decimal(-90)) |
    f.Max(Decimal(90)) |
    f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
    b'\xe2\x99\xaa ',
    b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
    b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
    b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode |
    f.FilterMapper(
        {
            'birthday':  f.Date,
            'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

            'utcOffset':
                f.Decimal |
                f.Min(Decimal('-15')) |
                f.Max(Decimal('+15')) |
                f.Round(to_nearest='0.25'),
        },

        allow_extra_keys   = False,
        allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with the following Python versions:

  • 3.8

  • 3.7

  • 3.6

  • 3.5

Installation

Install the latest stable version via pip:

pip install phx-filters

Extensions

The following extensions are available:

  • Django Filters: Adds filters designed to work with Django applications. To install:

    pip install phx-filters[django]
  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install phx-filters[iso]

Running Unit Tests

To run unit tests after installing from source:

python setup.py test

This project is also compatible with tox, which will run the unit tests in different virtual environments (one for each supported version of Python).

Install the package with the test-runner extra to set up the necessary dependencies, and then you can run the tests with the tox command:

pip install -e .[test-runner]
tox -p all

Documentation

Documentation is available on ReadTheDocs.

If you are installing from source (see above), you can also build the documentation locally:

  1. Install extra dependencies (you only have to do this once):

    pip install '.[docs-builder]'
  2. Switch to the docs directory:

    cd docs
  3. Build the documentation:

    make html

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

phx-filters-2.0.0.tar.gz (58.6 kB view details)

Uploaded Source

Built Distribution

phx_filters-2.0.0-py3-none-any.whl (35.1 kB view details)

Uploaded Python 3

File details

Details for the file phx-filters-2.0.0.tar.gz.

File metadata

  • Download URL: phx-filters-2.0.0.tar.gz
  • Upload date:
  • Size: 58.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.35.0 CPython/3.7.1

File hashes

Hashes for phx-filters-2.0.0.tar.gz
Algorithm Hash digest
SHA256 329943eec0dd9134c6224e7a11ac1df50edf37c9ec6113a3387f9fc72f84103f
MD5 4c440bbac3a1c1f210b62e39e8407ea4
BLAKE2b-256 8da3020f32c54623e714f33047fe7fafeb53fe6f6adf7bf1e7342e40771ab6c4

See more details on using hashes here.

File details

Details for the file phx_filters-2.0.0-py3-none-any.whl.

File metadata

  • Download URL: phx_filters-2.0.0-py3-none-any.whl
  • Upload date:
  • Size: 35.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.35.0 CPython/3.7.1

File hashes

Hashes for phx_filters-2.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 75c1b77e52a286193ccc47814a6427794063c39d009eb091c10b2042965e63e6
MD5 f992cb67bdc46617a285175cc3f4abbf
BLAKE2b-256 48e0195529e15b6e5dcbea20fa21d7df449bfa763086222cb3247d1313e93b24

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page