Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Install the latest development version:

pip install https://github.com/eflglobal/filters/archive/develop.zip

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.1.7.tar.gz (27.6 kB view details)

Uploaded Source

Built Distributions

filters-1.1.7-py3-none-any.whl (36.4 kB view details)

Uploaded Python 3

filters-1.1.7-py2-none-any.whl (36.4 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.1.7.tar.gz.

File metadata

  • Download URL: filters-1.1.7.tar.gz
  • Upload date:
  • Size: 27.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.1.7.tar.gz
Algorithm Hash digest
SHA256 342de31320ba104e2beeca7247da53d3ab741a2dd96b6a29c665508b9b4df41e
MD5 6159bc598ec6b0579a0d52a1ae7eaaaf
BLAKE2b-256 5d53f864ccaa1206a833f710e6110c4f5e8e29c9050fd3f48a00533e39017953

See more details on using hashes here.

File details

Details for the file filters-1.1.7-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.1.7-py3-none-any.whl
Algorithm Hash digest
SHA256 e52354fa0588cfa7e4290a0b77eb54b09e068f28dff3d49560bef2f6c23b47db
MD5 b87dd5c2cd66f1aa49abd937e28710b9
BLAKE2b-256 83dc2a887463d6ad0b91faf766d9a0c56083160ffe733781829a754b5f3045aa

See more details on using hashes here.

File details

Details for the file filters-1.1.7-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.1.7-py2-none-any.whl
Algorithm Hash digest
SHA256 8b5fd8996c7908f648f9797dd8c1f5ed8aba14e6cc315b54850fee614d9b59cc
MD5 649766829c6bd8c0c0a501b93d9e84b4
BLAKE2b-256 c3d0f899ed1c8035110fd88ff8414517e6bbe63d16b61711f13091c4dd6d7d83

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page