Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Install the latest development version:

pip install https://github.com/eflglobal/filters/archive/develop.zip

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.1.5.tar.gz (28.0 kB view details)

Uploaded Source

Built Distribution

filters-1.1.5-py2.py3-none-any.whl (35.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file filters-1.1.5.tar.gz.

File metadata

  • Download URL: filters-1.1.5.tar.gz
  • Upload date:
  • Size: 28.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.1.5.tar.gz
Algorithm Hash digest
SHA256 d592291a0d479d65fd64ef40017cc61e94b0c6aefc1d29a5d48ddd67b4b78890
MD5 382ffb25d6ef4dac2756730d24b8c0ca
BLAKE2b-256 34fe6313ebd9b33ac6a883356721e73ece1cee90bd615c025cdd506269cefefb

See more details on using hashes here.

File details

Details for the file filters-1.1.5-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.1.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 dd8e7c95dfc34c2a78551f019f3e55a719e6c9f43a40b681108c02273251939d
MD5 f281dba3d580c0eac5a435987832ba43
BLAKE2b-256 37ff104e7eba5d08d61299e9fb9d198807b8bc2be10ded2e7ee641399045f039

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page