Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Install the latest development version:

pip install https://github.com/eflglobal/filters/archive/develop.zip

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.1.6.tar.gz (27.9 kB view details)

Uploaded Source

Built Distributions

filters-1.1.6-py3-none-any.whl (35.8 kB view details)

Uploaded Python 3

filters-1.1.6-py2-none-any.whl (35.8 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.1.6.tar.gz.

File metadata

  • Download URL: filters-1.1.6.tar.gz
  • Upload date:
  • Size: 27.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.1.6.tar.gz
Algorithm Hash digest
SHA256 f52a608adcd458d9608a27c6bc833cb4d333b5f397b02495e26bb79d59118153
MD5 240b5f75a2f5abfebfc47ae14ab35dde
BLAKE2b-256 5d8f83312f95c54c77d01aae7f056f7adb7f833946f39f85152ba1594f72d7c8

See more details on using hashes here.

File details

Details for the file filters-1.1.6-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.1.6-py3-none-any.whl
Algorithm Hash digest
SHA256 759f848c1969ea4717c7194711df096d0940ffe5b310dc941128caa8305186cd
MD5 62094c1836d2f106ceb3f36b95aff60f
BLAKE2b-256 5467fc45d5d211b441be32634fb1e5a70bf68fc8c2c2c6469df65a98af4f0d16

See more details on using hashes here.

File details

Details for the file filters-1.1.6-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.1.6-py2-none-any.whl
Algorithm Hash digest
SHA256 8c2ada6911e7e6736df3d9b1cc3d5e6cdc9f78e833459c4686c3e1569e272028
MD5 4e054f3c2d7cd71b45341d22d8f13a26
BLAKE2b-256 c8ce5c838cbe59703e20e8d0dc2054aa8ee067a328367a30502d06c518ff51a5

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page