Skip to main content

Simple RTF tokenizer

Project description

RTF Tokenize

A simple RTF tokenizer.

API:

# Init:
from rtf_tokenize import RtfTokenizer
tokenizer = RtfTokenizer(rtf_string)
# Get next token (return `None` on end of string):
token = tokenizer.next_token()
# Rewind a token (next call to `next_token` will return it):
tokenizer.rewind_token(token)
# Current location (last tokenized, irrespective of rewound tokens):
position = tokenizer.lnum, tokenizer.cnum

Release history

1.0.0

  • first public release

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rtf_tokenize-1.0.0.tar.gz (4.5 kB view details)

Uploaded Source

Built Distributions

rtf_tokenize-1.0.0-cp310-cp310-win_amd64.whl (7.9 kB view details)

Uploaded CPython 3.10 Windows x86-64

rtf_tokenize-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 kB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

rtf_tokenize-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl (4.7 kB view details)

Uploaded CPython 3.10 macOS 10.9+ x86-64

rtf_tokenize-1.0.0-cp39-cp39-win_amd64.whl (7.9 kB view details)

Uploaded CPython 3.9 Windows x86-64

rtf_tokenize-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 kB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

rtf_tokenize-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl (4.7 kB view details)

Uploaded CPython 3.9 macOS 10.9+ x86-64

rtf_tokenize-1.0.0-cp38-cp38-win_amd64.whl (7.9 kB view details)

Uploaded CPython 3.8 Windows x86-64

rtf_tokenize-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 kB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

rtf_tokenize-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl (4.7 kB view details)

Uploaded CPython 3.8 macOS 10.9+ x86-64

rtf_tokenize-1.0.0-cp37-cp37m-win_amd64.whl (7.9 kB view details)

Uploaded CPython 3.7m Windows x86-64

rtf_tokenize-1.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 kB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

rtf_tokenize-1.0.0-cp37-cp37m-macosx_10_9_x86_64.whl (4.7 kB view details)

Uploaded CPython 3.7m macOS 10.9+ x86-64

rtf_tokenize-1.0.0-cp36-cp36m-win_amd64.whl (7.9 kB view details)

Uploaded CPython 3.6m Windows x86-64

rtf_tokenize-1.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.2 kB view details)

Uploaded CPython 3.6m manylinux: glibc 2.17+ x86-64

rtf_tokenize-1.0.0-cp36-cp36m-macosx_10_9_x86_64.whl (4.6 kB view details)

Uploaded CPython 3.6m macOS 10.9+ x86-64

File details

Details for the file rtf_tokenize-1.0.0.tar.gz.

File metadata

  • Download URL: rtf_tokenize-1.0.0.tar.gz
  • Upload date:
  • Size: 4.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0.tar.gz
Algorithm Hash digest
SHA256 5c3df390d00479bd7637c823bfcd6fdfb21ddd1b96ae815463de7e1ed392d608
MD5 b084170175d84b453485065faf39f9e4
BLAKE2b-256 1bc3591998d6a7e19c68933e8c4af3e8f5c0bbc17eb9c50a229c7c6afff349c4

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp310-cp310-win_amd64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp310-cp310-win_amd64.whl
  • Upload date:
  • Size: 7.9 kB
  • Tags: CPython 3.10, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 09767de3f198a4228834cc4bab01c180f65f665dbfadf22e951c0c73d123a386
MD5 c578e339a5141c24942d9c31e0b3920f
BLAKE2b-256 d79844b82f99b485fe90df6378ae122c2d1a5c6529861cacbb506e821b0d0b5b

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rtf_tokenize-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 de6d8b6c730068ebe6b441fc431ff6cc2a33f86b80f111390ad01b914d905d58
MD5 a9ae8a99d4e96a6f1518ffe7b2097443
BLAKE2b-256 4ef35240bedeb6a2af41f2f7e2d11c5024d9007bc0640a921d275a35c60d4555

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 4.7 kB
  • Tags: CPython 3.10, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp310-cp310-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 b3c2843a98536dc151751b214f81fc810a378d3d2455db6bab2f4148a648e965
MD5 81b40509f89af1962bdbe0d5967cc65d
BLAKE2b-256 7e52909c19729dca291f3fd2ea805c06d429a2ffbadf43a28737007c0ede958d

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 7.9 kB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 466ed23d9cde26cc49b0f2b3207f15bd718dd4981b8fe769b85799f209ee67f8
MD5 280371064f0dab526d110e6088bad8a4
BLAKE2b-256 447614b378e8617240195d2d85de7b1d1e98904a74ea685dc22a49d76f847f4b

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rtf_tokenize-1.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 22745a1e6585069a2a7d73595f918c6d6c276625839b013757924c76cccded3c
MD5 f7226063c7e3bbcb17f7ff5f6e7f201b
BLAKE2b-256 2226288366947abe7b9401d295506290d73563c12528cc8b32841830ed38a30a

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 4.7 kB
  • Tags: CPython 3.9, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp39-cp39-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 f9469036eed8747c62f17cbe490d47c91ccfdd4e870617bc1b68770041490498
MD5 e8cb76781b9ff730b24066ae43a4deb0
BLAKE2b-256 7e9b78b7cbdbaefb8cc5bf40ab71eb5a2382afb18a491addb41125a7778b3993

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 7.9 kB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 6d8bf0fff247864a736a4ea6a5c2658f4a37d8af39395cb97efb1a27162c384c
MD5 42760be85a4e6a7ba1974ee3c2d755ea
BLAKE2b-256 a55179a945d09ea44b7f93bdbf34b8d73acd77d554fb23f5c6e4cd520acd4228

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rtf_tokenize-1.0.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 101edd8ea379560854e46e93a5e0ccffaa021949b76828c64cc6b68c27a1d7e9
MD5 747fda13226da7b6a87a84c37a3754e2
BLAKE2b-256 868d1da68f6d0a378146a1d7468382e89f34b02b03b09c9428e033ce1556293a

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 4.7 kB
  • Tags: CPython 3.8, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp38-cp38-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 84a5369ea164bdc54ca3cbc06d257e792480b6a20f59d69ddeadfc0b93e29ca7
MD5 8225eeccc89ce6de040995087723a6e1
BLAKE2b-256 e0e383e71732d8a5799c1ce00f44055fa5a992ef5a7655fa5da0add9886910b7

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 7.9 kB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 490ec81a1d4d47957a792699b78b625a23c932ad5c3f257dc070b02a5306d5df
MD5 b877d44a813d9891e44ad84e09dd8fb7
BLAKE2b-256 ef26271766460c316c3a7654319246ef0243f67c71b49a50528d4d2522801f05

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rtf_tokenize-1.0.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d698262eb9e1be094f705ff5b52b72a57019fd4874ff42d97e693640bd7b325c
MD5 aca11b0a882d4fb6e7759d0ddcbab446
BLAKE2b-256 5f4f89ac2bcc7f0a7f54418f626b757014b20078ef3de0ac28277ce0f5bd9531

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp37-cp37m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp37-cp37m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 4.7 kB
  • Tags: CPython 3.7m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp37-cp37m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 2b6369bd28cf97f7ca65dd923bc296f259143b158168d75745169fbf9ed3671c
MD5 52e69ff4b204128fb0ce90b63635f939
BLAKE2b-256 54aaf130f186ff70a49abe069086374df6100cb341a7f229d0a36b29bc098a2f

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp36-cp36m-win_amd64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp36-cp36m-win_amd64.whl
  • Upload date:
  • Size: 7.9 kB
  • Tags: CPython 3.6m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 7840712b6e02786842c9d5528356c6400567ec2fc483c26ce677cb0f04d3a27c
MD5 b5d5215959e063878b2a52716b91ef35
BLAKE2b-256 74d1925444e4be9d556c9f5e8aad034eb1932f8b42fb464523bdb73469a1ef49

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rtf_tokenize-1.0.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c9680273dc39ccbca30539a84ff08b74eaa7b07bd9bfb3b41d754217c75aeb12
MD5 7ce66a76d34f91f1168ca4e15905cd07
BLAKE2b-256 6b5122b433130e35fd248bb05c4a05827aaae3ff46778bea5bf534a98acfd470

See more details on using hashes here.

File details

Details for the file rtf_tokenize-1.0.0-cp36-cp36m-macosx_10_9_x86_64.whl.

File metadata

  • Download URL: rtf_tokenize-1.0.0-cp36-cp36m-macosx_10_9_x86_64.whl
  • Upload date:
  • Size: 4.6 kB
  • Tags: CPython 3.6m, macOS 10.9+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.7

File hashes

Hashes for rtf_tokenize-1.0.0-cp36-cp36m-macosx_10_9_x86_64.whl
Algorithm Hash digest
SHA256 5dfbe4327b508af46603981072783895819210a7c84b83f0a2a12a2cfd677b7c
MD5 6dbf45d0e08e7cfb7a6443022e7ea773
BLAKE2b-256 4132d10dd5d6faa77e4f474bb0acd87e0b98825039f2e04f664fc153be9e7594

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page