Measures number of Terminal column cells of wide-character codes
Project description
Introduction
This Library is mainly for those implementing a Terminal Emulator, or programs that carefully produce output to mimick or to be interpreted by an emulator.
Problem Statement: The printible length of most strings are equal to the number of cells they occupy on the screen. However, there are categories of characters that occupy 2 cells (full-wide), and others that occupy 0 cells (zero-width).
Solution: POSIX.1-2001 and POSIX.1-2008 conforming systems provide wcwidth(3) and wcswidth(3) C functions of which this python module’s functions precisely copy. These functions return the number of cells a unicode string is expected to occupy.
Installation
The stable version of this package is maintained on pypi, install or upgrade using pip:
pip install --upgrade wcwidth
Example
Problem: given the following phrase (Japanese),
>>> text = u'コンニチハ'
Python incorrectly uses the string length of 5 codepoints rather than the printible length of 10 cells, so that when using the rjust function, the output length is wrong:
>>> print(len('コンニチハ')) 5 >>> print('コンニチハ'.rjust(11, '_')) ______コンニチハ
By defining our own “rjust” function that uses wcwidth, we can correct this:
>>> def wc_rjust(text, length, padding=' '): ... from wcwidth import wcswidth ... return padding * max(0, (length - wcswidth(text))) + text ...
Our Solution uses wcswidth to determine the string length correctly:
>>> from wcwidth import wcswidth >>> print(wcswidth('コンニチハ')) 10 >>> print(wc_rjust('コンニチハ', 11, '_')) _コンニチハ
Uses
This library is used in:
asciimatics: Package to help people create full-screen text UIs.
blessed: a simplified wrapper around curses.
curtsies: Curses wrapper with a display based on compositing 2d arrays of text.
ftfy: Fixes mojibake and other glitches in Unicode text, after the fact.
pyte: a Simple VTXXX-compatible linux terminal emulator.
python-prompt-toolkit: a Powerful interactive command line building library.
termtosvg: Terminal recorder that renders sessions as SVG animations.
wcwidth, wcswidth
Use function wcwidth() to determine the length of a single unicode character, and wcswidth() to determine the length of a several, or a string of unicode characters.
Briefly, return values of function wcwidth() are:
- -1
Indeterminate (not printable).
- 0
Does not advance the cursor, such as NULL or Combining.
- 2
Characters of category East Asian Wide (W) or East Asian Full-width (F) which are displayed using two terminal cells.
- 1
All others.
Function wcswidth() simply returns the sum of all values for each character along a string, or -1 when it occurs anywhere along a string.
More documentation is available using pydoc:
$ pydoc wcwidth
Caveats
This library attempts to determine the printable width by a fictional terminal, using the very latest Unicode specification, though some work is in progress for the ability to select a version, there is no standards conforming ability to discern what the target emulator software, version, of level of support is. Results may vary!
A crude method of determining the level of unicode support by the target emulator may be performed using the VT100 Query Cursor Position sequence.
The libc version of wcwidth(3) is often several unicode releases behind, and therefor several levels of support lower than this python library.
Developing
Install wcwidth in editable mode:
pip install -e.
Install developer requirements:
pip install -r requirements-develop.txt
Execute unit tests using tox:
tox
Use the interactive browser:
python bin/wcwidth-browser.py
This library aims to be forward-looking, portable, and most correct. The most current release of this API is based on the Unicode Standard release files:
- DerivedGeneralCategory-13.0.0.txt
Date: 2019-10-21, 14:30:32 GMT © 2019 Unicode®, Inc.
- EastAsianWidth-13.0.0.txt
Date: 2020-01-21, 18:14:00 GMT [KW, LI] © 2020 Unicode®, Inc.
Updating Tables
The command python setup.py update will fetch the following resources:
And generates the table files:
History
- 0.1.9 2020-03-22
Performance optimization by Avram Lubkin, PR #35.
Updated tables to Unicode Specification 13.0.0.
- 0.1.8 2020-01-01
Updated tables to Unicode Specification 12.0.0. (PR #30).
- 0.1.7 2016-07-01
Updated tables to Unicode Specification 9.0.0. (PR #18).
- 0.1.6 2016-01-08 Production/Stable
LICENSE file now included with distribution.
- 0.1.5 2015-09-13 Alpha
Bugfix: Resolution of “combining character width” issue, most especially those that previously returned -1 now often (correctly) return 0. resolved by Philip Craig via PR #11.
Deprecated: The module path wcwidth.table_comb is no longer available, it has been superseded by module path wcwidth.table_zero.
- 0.1.4 2014-11-20 Pre-Alpha
Feature: wcswidth() now determines printable length for (most) combining characters. The developer’s tool bin/wcwidth-browser.py is improved to display combining characters when provided the --combining option (Thomas Ballinger and Leta Montopoli PR #5).
Feature: added static analysis (prospector) to testing framework.
- 0.1.3 2014-10-29 Pre-Alpha
Bugfix: 2nd parameter of wcswidth was not honored. (Thomas Ballinger, PR #4).
- 0.1.2 2014-10-28 Pre-Alpha
Updated tables to Unicode Specification 7.0.0. (Thomas Ballinger, PR #3).
- 0.1.1 2014-05-14 Pre-Alpha
Initial release to pypi, Based on Unicode Specification 6.3.0
This code was originally derived directly from C code of the same name, whose latest version is available at http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c:
* Markus Kuhn -- 2007-05-26 (Unicode 5.0) * * Permission to use, copy, modify, and distribute this software * for any purpose and without fee is hereby granted. The author * disclaims all warranties with regard to this software.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file wcwidth-0.1.9.tar.gz
.
File metadata
- Download URL: wcwidth-0.1.9.tar.gz
- Upload date:
- Size: 24.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | ee73862862a156bf77ff92b09034fc4825dd3af9cf81bc5b360668d425f3c5f1 |
|
MD5 | b32584cd6084b4a8c81df13a2a6b4351 |
|
BLAKE2b-256 | 259d0acbed6e4a4be4fc99148f275488580968f44ddb5e69b8ceb53fc9df55a0 |
File details
Details for the file wcwidth-0.1.9-py2.py3-none-any.whl
.
File metadata
- Download URL: wcwidth-0.1.9-py2.py3-none-any.whl
- Upload date:
- Size: 19.4 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | cafe2186b3c009a04067022ce1dcd79cb38d8d65ee4f4791b8888d6599d1bbe1 |
|
MD5 | 003327fdfbb7653dda1f3ce28fc57be1 |
|
BLAKE2b-256 | f6d51ecdac957e3ea12c1b319fcdee8b6917ffaff8b4644d673c4d72d2f20b49 |