Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Extensions

The following extensions are available:

  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install filters[iso]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.2.1.tar.gz (28.0 kB view details)

Uploaded Source

Built Distributions

filters-1.2.1-py3-none-any.whl (36.7 kB view details)

Uploaded Python 3

filters-1.2.1-py2-none-any.whl (36.7 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.2.1.tar.gz.

File metadata

  • Download URL: filters-1.2.1.tar.gz
  • Upload date:
  • Size: 28.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.2.1.tar.gz
Algorithm Hash digest
SHA256 cc817855ad9e8d29491988c95d26f3617fd8c43dc61ccc122bae596a250489b2
MD5 0e251e56d01938291291137c51344266
BLAKE2b-256 58f83d97781e26fe4349b8d31af62abfe1312fbe962e3cfd5380122e27dcf736

See more details on using hashes here.

File details

Details for the file filters-1.2.1-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 6e620224f7374ee5c586428620322917c835be6900b1e89cb8595c4d36f06c4f
MD5 1397223a6f0ae1ae93e8713027541262
BLAKE2b-256 bdad0d1908937b8d1bfce45addde8346a73924b88fa562ac955cc04fa3419f63

See more details on using hashes here.

File details

Details for the file filters-1.2.1-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.1-py2-none-any.whl
Algorithm Hash digest
SHA256 9b9759cfbed017c0dccd1076a0340242a05ec05ea439e72c555664dab8df9712
MD5 bd5d66607f2ca62786e64edde6d33fcc
BLAKE2b-256 64047f25385aff8f911d315e881b6431ac243f9089a52142eeae1d2c70e5ef3b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page