Micro-library to normalize text strings
Project description
normality
Normality is a Python micro-package that contains a small set of text normalization functions for easier re-use. These functions accept a snippet of unicode or utf-8 encoded text and remove various classes of characters, such as diacritics, punctuation etc. This is useful as a preparation to further text analysis.
WARNING: This library works much better when used in combination
with pyicu, a Python binding for the International Components for
Unicode C library. ICU provides much better text transliteration than
the default text-unidecode.
Example
# coding: utf-8
from normality import normalize, slugify, collapse_spaces
text = normalize('Nie wieder "Grüne Süppchen" kochen!')
assert text == 'nie wieder grune suppchen kochen'
slug = slugify('My first blog post!')
assert slug == 'my-first-blog-post'
text = 'this \n\n\r\nhas\tlots of \nodd spacing.'
assert collapse_spaces(text) == 'this has lots of odd spacing.'
License
normality is open source, licensed under a standard MIT license
(included in this repository as LICENSE).
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file normality-2.2.1.tar.gz.
File metadata
- Download URL: normality-2.2.1.tar.gz
- Upload date:
- Size: 10.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b2091a14fc8155b39bf9c35b9d95f128cfb85c7c5ebf13739c04bfabf7da7e4d |
|
| MD5 | 612ae4a3b29be9603b206c292c8fc428 |
|
| BLAKE2b-256 | 791282c429516f331b6e1f3ffb87d6d75885fb50b6e4bb5b8eb6017c1996397f |
File details
Details for the file normality-2.2.1-py2.py3-none-any.whl.
File metadata
- Download URL: normality-2.2.1-py2.py3-none-any.whl
- Upload date:
- Size: 12.3 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.9.4
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | bc4eb549c8f51c5188a4f117dcf78c0022e332edbb4c464651cd9a065b1816ad |
|
| MD5 | d3c3d320ed36d1d3ba689eddbdb91a70 |
|
| BLAKE2b-256 | 7bede4888f38b75295703b121b513097f4b910e15c4a2bc46f0c9c807af500db |
