Skip to main content

Validation and data pipelines made easy!

Project description

https://travis-ci.org/eflglobal/filters.svg?branch=master https://readthedocs.org/projects/filters/badge/?version=latest

Filters

The Filters library provides an easy and readable way to create complex data validation and processing pipelines, including:

  • Validating complex JSON structures in API requests or config files.

  • Parsing timestamps and converting to UTC.

  • Converting Unicode strings to NFC, normalizing line endings and removing unprintable characters.

  • Decoding Base64, including URL-safe variants.

And much more!

The output from one filter can be “piped” into the input of another, enabling you to “chain” filters together to quickly and easily create complex data pipelines.

Examples

Validate a latitude position and round to manageable precision:

(
    f.Required
  | f.Decimal
  | f.Min(Decimal(-90))
  | f.Max(Decimal(90))
  | f.Round(to_nearest='0.000001')
).apply('-12.0431842')

Parse an incoming value as a datetime, convert to UTC and strip tzinfo:

f.Datetime(naive=True).apply('2015-04-08T15:11:22-05:00')

Convert every value in an iterable (e.g., list) to unicode and strip leading/trailing whitespace. This also applies Unicode normalization, strips unprintable characters and normalizes line endings automatically.

f.FilterRepeater(f.Unicode | f.Strip).apply([
  b'\xe2\x99\xaa ',
  b'\xe2\x94\x8f(\xc2\xb0.\xc2\xb0)\xe2\x94\x9b ',
  b'\xe2\x94\x97(\xc2\xb0.\xc2\xb0)\xe2\x94\x93 ',
  b'\xe2\x99\xaa ',
])

Parse a JSON string and check that it has correct structure:

(
    f.JsonDecode
  | f.FilterMapper(
      {
        'birthday':  f.Date,
        'gender':    f.CaseFold | f.Choice(choices={'m', 'f', 'x'}),

        'utcOffset':
            f.Decimal
          | f.Min(Decimal('-15'))
          | f.Max(Decimal('+15'))
          | f.Round(to_nearest='0.25'),
      },

      allow_extra_keys   = False,
      allow_missing_keys = False,
    )
).apply('{"birthday":"1879-03-14", "gender":"M", "utcOffset":"1"}')

Requirements

Filters is compatible with Python versions 3.6, 3.5 and 2.7.

Installation

Install the latest stable version via pip:

pip install filters

Extensions

The following extensions are available:

  • ISO Filters: Adds filters for interpreting standard codes and identifiers. To install:

    pip install filters[iso]

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filters-1.2.2.tar.gz (28.5 kB view details)

Uploaded Source

Built Distributions

filters-1.2.2-py3-none-any.whl (37.3 kB view details)

Uploaded Python 3

filters-1.2.2-py2-none-any.whl (37.3 kB view details)

Uploaded Python 2

File details

Details for the file filters-1.2.2.tar.gz.

File metadata

  • Download URL: filters-1.2.2.tar.gz
  • Upload date:
  • Size: 28.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for filters-1.2.2.tar.gz
Algorithm Hash digest
SHA256 6eec402687d7f821628b63219ed5c5334e66ffd535069312b824e735688bf41d
MD5 bbfb127af516017c638659ef722797ae
BLAKE2b-256 67d9c9ad5d959375f2dfd9433679f1a0ad50989a5fd4fb4327b52d22287b2404

See more details on using hashes here.

File details

Details for the file filters-1.2.2-py3-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 d9cd91a099cfebfd58f693a6993769de9bab070317502b673f0e8d2ed799a077
MD5 88414404b64b89207519c510a0685e34
BLAKE2b-256 b74211bb5123d6f6b41667fd064e778d5fbd705042914a6a96d836d828b9c20b

See more details on using hashes here.

File details

Details for the file filters-1.2.2-py2-none-any.whl.

File metadata

File hashes

Hashes for filters-1.2.2-py2-none-any.whl
Algorithm Hash digest
SHA256 0b8cad1da98b85491ebcd4e8fd12f4b64ad259ae52ea3c889f67f6e7fd449d12
MD5 6ce49f341ee30a43468b14ec73fdf451
BLAKE2b-256 4963d6b7a27a40d9ea59ac8ff25c283072f58a030868ab1dcee577d39259b083

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page