Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Extensions

The following extensions are available:

  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install filters[iso]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.2.3.tar.gz (28.6 kB view details)

Uploaded Source

Built Distributions

filters-1.2.3-py3-none-any.whl (37.3 kB view details)

Uploaded Python 3

filters-1.2.3-py2-none-any.whl (37.4 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.2.3.tar.gz.

File metadata

  • Download URL: filters-1.2.3.tar.gz
  • Upload date:
  • Size: 28.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.2.3.tar.gz
Algorithm Hash digest
SHA256 fc0ad934875d705f1b51668ecd1e714f52ef31c3abacd6b3ff92cf6d2cd1e65a
MD5 b4eb45f23702d1bab3adf19f4a9a230f
BLAKE2b-256 accedec876bfc326d83e4301e33250f3b192c6eef5a96aed32d3df0923f9caf7

See more details on using hashes here.

File details

Details for the file filters-1.2.3-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.3-py3-none-any.whl
Algorithm Hash digest
SHA256 6fb197ed987c71e86f1617ae874be64e3675066560e992cc9def208a6dbdabd5
MD5 b6e4b899fa15d5b294468e1a032a6cd9
BLAKE2b-256 5952ccca2e93c30411161132ae026f3523abfc0be06830bbce862f3d8a3a17eb

See more details on using hashes here.

File details

Details for the file filters-1.2.3-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.3-py2-none-any.whl
Algorithm Hash digest
SHA256 e113ece27aaeabd003a2ca91a759534fbc3e315c77aeb786169d4c2c47a11bed
MD5 05ef51272ce970ed0b79c6585c93e287
BLAKE2b-256 5bbc6a9843b25f462df71c841c9a947c83d6249c0851152fcce0c7e2562a35af

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page