Micro-library to normalize text strings

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for occrp from gravatar.com occrp Avatar for opensanctions from gravatar.com opensanctions Avatar for pudo from gravatar.com pudo

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description

normality text cleanup

build

Normality is a Python micro-package that contains a small set of text normalization functions for easier re-use. These functions accept a snippet of unicode or utf-8 encoded text and remove various classes of characters, such as diacritics, punctuation etc. This is useful as a preparation to further text analysis.

WARNING: This library works much better when used in combination with pyicu, a Python binding for the International Components for Unicode C library. ICU provides much better text transliteration than the default text-unidecode.

Example

# coding: utf-8
from normality import normalize, slugify, collapse_spaces

text = normalize('Nie wieder "Grüne Süppchen" kochen!')
assert text == 'nie wieder grune suppchen kochen'

slug = slugify('My first blog post!')
assert slug == 'my-first-blog-post'

text = 'this \n\n\r\nhas\tlots of \nodd spacing.'
assert collapse_spaces(text) == 'this has lots of odd spacing.'

License

normality is open source, licensed under a standard MIT license (included in this repository as LICENSE).

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for occrp from gravatar.com occrp Avatar for opensanctions from gravatar.com opensanctions Avatar for pudo from gravatar.com pudo

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

2.5.0

This version

2.4.0

2.3.3

2.3.2

2.3.1

2.3.0

2.2.5

2.2.4

2.2.3

2.2.2

2.2.1

2.1.3

2.1.2

2.1.1

2.0.0

1.0.1

1.0.0

0.6.1

0.6.0

0.5.12

0.5.11

0.5.10

0.5.8

0.5.7

0.5.6

0.5.5

0.5.4

0.5.3

0.5.2

0.5.1

0.5.0

0.4.4

0.4.3

0.4.2

0.4.1

0.4.0

0.3.9

0.3.8

0.3.7

0.3.6

0.3.5

0.3.4

0.3.3

0.3.1

0.3.0

0.2.4

0.2.3

0.2.2

0.2.1

0.2

0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

normality-2.4.0.tar.gz (11.8 kB view details)

Uploaded Source

Built Distribution

normality-2.4.0-py2.py3-none-any.whl (10.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file normality-2.4.0.tar.gz.

File metadata

  • Download URL: normality-2.4.0.tar.gz
  • Upload date:
  • Size: 11.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for normality-2.4.0.tar.gz
Algorithm Hash digest
SHA256 38bbe4e1dfd737c318ffd70e981ae8ff40bd8839393f6d62f0e200e5aab9e992
MD5 860e620b837627282c0b01f1327e40ac
BLAKE2b-256 439ac5a0c1f4ef982797ebb6a5e9665daa0fd2854ee2a6161ee91879f73ff92a

See more details on using hashes here.

File details

Details for the file normality-2.4.0-py2.py3-none-any.whl.

File metadata

  • Download URL: normality-2.4.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 10.9 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.9.13

File hashes

Hashes for normality-2.4.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 8bd9dd5a0220f641ed4cc59b7ad64ab11b0ee49e57e5a70bf515445ff72574d2
MD5 c59daa67114684065f133d3189853336
BLAKE2b-256 8fa428f474c9de1ee5b461b2849e7d0aad8f692d3c7d8a7cb59cc11591bc82b9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page