Micro-library to normalize text strings
Project description
normality
Normality is a Python micro-package that contains a small set of text normalization functions for easier re-use. These functions accept a snippet of unicode or utf-8 encoded text and remove various classes of characters, such as diacritics, punctuation etc. This is useful as a preparation to further text analysis.
WARNING: This library works much better when used in combination
with pyicu
, a Python binding for the International Components for
Unicode C library. ICU provides much better text transliteration than
the default text-unidecode
.
Example
# coding: utf-8
from normality import normalize, slugify, collapse_spaces
text = normalize('Nie wieder "Grüne Süppchen" kochen!')
assert text == 'nie wieder grune suppchen kochen'
slug = slugify('My first blog post!')
assert slug == 'my-first-blog-post'
text = 'this \n\n\r\nhas\tlots of \nodd spacing.'
assert collapse_spaces(text) == 'this has lots of odd spacing.'
License
normality
is open source, licensed under a standard MIT license
(included in this repository as LICENSE
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for normality-2.2.2-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 17c06fb63aa5f892eaca7fc25b93e91008b61ff64bb6ad7bd6320368f4f3fafa |
|
MD5 | 242b94ba5252b4449bcfaf8864a63193 |
|
BLAKE2b-256 | 07c893252610e6de5d45d484d08b0af0c28b21cbe338d19b6e34f695d432e359 |