Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Install the latest development version:

pip install https://github.com/eflglobal/filters/archive/develop.zip

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.2.0.tar.gz (27.9 kB view details)

Uploaded Source

Built Distributions

filters-1.2.0-py3-none-any.whl (36.5 kB view details)

Uploaded Python 3

filters-1.2.0-py2-none-any.whl (36.5 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.2.0.tar.gz.

File metadata

  • Download URL: filters-1.2.0.tar.gz
  • Upload date:
  • Size: 27.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.2.0.tar.gz
Algorithm Hash digest
SHA256 2a0c6c9a75924d7f7b6cc4aa2b874bfc2035bc4dde5b3ecf77732a6f4c8db9ce
MD5 afd7489d32fbba0b0c4043c742a02f2b
BLAKE2b-256 edf5fecc1ebdc137b10362984d941db9b0c470084fb06c10b3472e51d7805c31

See more details on using hashes here.

File details

Details for the file filters-1.2.0-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 55a16d7d3dfa099ba114daf8ad5e4b8a747728a46060db1a98d2adf552125749
MD5 92d3d238c88323b1593f78e88792cd30
BLAKE2b-256 693ade917634f7cd2aa621f11701ea3300ac9498ac3dd5075dc2e7549d5164a1

See more details on using hashes here.

File details

Details for the file filters-1.2.0-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.0-py2-none-any.whl
Algorithm Hash digest
SHA256 c55889f7feb30c48cea43fce70675a61d7867cd2b8112ab50476c5ea4f3cf955
MD5 136d04d32a0bff91d725a85bf661b746
BLAKE2b-256 0651b6ca7bf41126d5733ed11072f59a62383b9c9ae9a66fc3e23a9ab15090d6

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page