Skip to main content

Fast and Customizable Tokenizers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install setuptools_rust
python setup.py install

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(vocab_size=20000, min_frequency=2)
tokenizer.train([
	"./path/to/dataset/1.txt",
	"./path/to/dataset/2.txt",
	"./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.10.2.tar.gz (211.2 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.10.2-cp39-cp39-win_amd64.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.10.2-cp39-cp39-win32.whl (1.8 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.10.2-cp39-cp39-manylinux2010_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ x86-64

tokenizers-0.10.2-cp39-cp39-manylinux1_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.9

tokenizers-0.10.2-cp39-cp39-macosx_10_11_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.9 macOS 10.11+ x86-64

tokenizers-0.10.2-cp38-cp38-win_amd64.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.10.2-cp38-cp38-win32.whl (1.8 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.10.2-cp38-cp38-manylinux2010_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ x86-64

tokenizers-0.10.2-cp38-cp38-manylinux1_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.8

tokenizers-0.10.2-cp38-cp38-macosx_10_11_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.8 macOS 10.11+ x86-64

tokenizers-0.10.2-cp37-cp37m-win_amd64.whl (2.0 MB view details)

Uploaded CPython 3.7m Windows x86-64

tokenizers-0.10.2-cp37-cp37m-win32.whl (1.8 MB view details)

Uploaded CPython 3.7m Windows x86

tokenizers-0.10.2-cp37-cp37m-manylinux2014_s390x.whl (3.8 MB view details)

Uploaded CPython 3.7m

tokenizers-0.10.2-cp37-cp37m-manylinux2010_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ x86-64

tokenizers-0.10.2-cp37-cp37m-manylinux1_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.7m

tokenizers-0.10.2-cp37-cp37m-macosx_10_11_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.7m macOS 10.11+ x86-64

tokenizers-0.10.2-cp36-cp36m-win_amd64.whl (2.0 MB view details)

Uploaded CPython 3.6m Windows x86-64

tokenizers-0.10.2-cp36-cp36m-win32.whl (1.8 MB view details)

Uploaded CPython 3.6m Windows x86

tokenizers-0.10.2-cp36-cp36m-manylinux2014_s390x.whl (3.8 MB view details)

Uploaded CPython 3.6m

tokenizers-0.10.2-cp36-cp36m-manylinux2010_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.6m manylinux: glibc 2.12+ x86-64

tokenizers-0.10.2-cp36-cp36m-manylinux1_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.6m

tokenizers-0.10.2-cp36-cp36m-macosx_10_11_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.6m macOS 10.11+ x86-64

tokenizers-0.10.2-cp35-cp35m-win_amd64.whl (2.0 MB view details)

Uploaded CPython 3.5m Windows x86-64

tokenizers-0.10.2-cp35-cp35m-win32.whl (1.8 MB view details)

Uploaded CPython 3.5m Windows x86

tokenizers-0.10.2-cp35-cp35m-manylinux2014_s390x.whl (3.8 MB view details)

Uploaded CPython 3.5m

tokenizers-0.10.2-cp35-cp35m-manylinux2010_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.5m manylinux: glibc 2.12+ x86-64

tokenizers-0.10.2-cp35-cp35m-manylinux1_x86_64.whl (3.3 MB view details)

Uploaded CPython 3.5m

tokenizers-0.10.2-cp35-cp35m-macosx_10_11_x86_64.whl (2.3 MB view details)

Uploaded CPython 3.5m macOS 10.11+ x86-64

File details

Details for the file tokenizers-0.10.2.tar.gz.

File metadata

  • Download URL: tokenizers-0.10.2.tar.gz
  • Upload date:
  • Size: 211.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2.tar.gz
Algorithm Hash digest
SHA256 cf7f1aad957fed36e4a90fc094e3adc03fdd45fbb058c1cde25721e3e66235f8
MD5 05421d5dd82b27b130a43e80e3d85d8a
BLAKE2b-256 0596d8f7a202cd46ac646b0aca2f3a7902bfefc70e9bd980e0371dba7fe7db85

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-win_amd64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-win_amd64.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.9, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 03056431783e72df80de68648573f97a70701d17fa22336c6d761b5d4b7be9ff
MD5 940c5873d4bd55575b1f0a8963f7bdfc
BLAKE2b-256 69fd3bc3ca34c27e71bc4736f03151f16704cc730378a14f55b9e827b3c5af99

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-win32.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-win32.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-win32.whl
Algorithm Hash digest
SHA256 c0f5bbc2e614468bcb605f2aa4a6dbcdf21629c6ff6ae81f1d9fa9683934ce8e
MD5 113620149d70e50a345325c194adff2f
BLAKE2b-256 8c113a6da4cbe8588aae1fecc256e30f0b103fcd0d7d51bb879449c621a428b0

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-manylinux2014_s390x.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-manylinux2014_s390x.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.9
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 a323d93fd5e57060428fecb6d73ab13223822f8ffa1ede282070b47a4bda2cea
MD5 f978d3cbe08107568ebacd3e95ef5362
BLAKE2b-256 4182a562eed453582fc7382523d8d36b06743dfb17346b98374c69e78a96afcf

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-manylinux2014_ppc64le.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-manylinux2014_ppc64le.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: CPython 3.9
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 39e24555d5a2d9df87fd75303e1fd9ba3f995ac8aeb543c511d601d26a54726a
MD5 e30ff3a5373944e06755cb7b8363b49a
BLAKE2b-256 b131659b65ecf2eba66d8dafd98ad6a9e808f517de2e2ed04d1275ed3657294e

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-manylinux2014_aarch64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-manylinux2014_aarch64.whl
  • Upload date:
  • Size: 3.2 MB
  • Tags: CPython 3.9
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 e3e74a9fa40b92a9817fe05ea91bf20075f45ad8cf7c0d3eb738170a27059508
MD5 d79a267ba455c4e4ea4f0673923583d2
BLAKE2b-256 3e641ab08c3a169d0b3ae10ecb97032cf5fa29b331dea7bbac8711afff5e85a1

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-manylinux2010_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.9, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 2094eb8e3608858eb4bd29c32c39969ae63ad9749d8aca9b34e82cba852acaf1
MD5 87aa09e287ac4f3a4f2956c6107fd2c1
BLAKE2b-256 5f1f5f7cf3d41ffc2a63148a0fa29370030109e226b2a1e09796d244d0755bb1

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-manylinux1_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-manylinux1_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.9
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 1ab8c467e4fe16bba33022feefcd6322642a58e4c8c123fd692c20e17f339964
MD5 2d1453debd52bf3bed4deb7a7e1abdb9
BLAKE2b-256 620eac18f273b89eb8af8e25250713706ff376189381f7ac8fc3a7b53f635ba9

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp39-cp39-macosx_10_11_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp39-cp39-macosx_10_11_x86_64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.9, macOS 10.11+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp39-cp39-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 953a4e483524fd37fd66208e21dce85d4829bfe294d8b6224d2f00f61aa9950c
MD5 343a889e78e0dded8c2ff2a10420ae91
BLAKE2b-256 81581759bdeadc2f12ce39adb77eb67a8672c9303db11b76a53feaee22233433

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-win_amd64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-win_amd64.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.8, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 6229fcc8473fd225e8e09742c354dacacd57dbdc73075e4c9d71f925cd171090
MD5 ccebe59cc588f1afc583ae88b5176cd6
BLAKE2b-256 3316b71ca271d0c5fae3490027dc154d09153d215e3e66a7b5375d4b979b90eb

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-win32.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-win32.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-win32.whl
Algorithm Hash digest
SHA256 bed7c5c2c786a2e9b3265006f15a13d8e04dcdfcf9ba13add0d7194a50346393
MD5 8bd174c96687ba1b0dbaa58de521bb79
BLAKE2b-256 703dee401a86c2597b9681f83939a5ec776febc43674eff54f222594aafbf28b

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-manylinux2014_s390x.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-manylinux2014_s390x.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 52c2479975fd5025d399493403c7aedce853da20cec04a32a829c1c12c28e2f1
MD5 f89874e144d31d4f16d961b83497cb25
BLAKE2b-256 63beb222c243a0a8bf5ae9be698bdb81d06101ef4b449cd0ea5b4d6e71fcc857

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-manylinux2014_ppc64le.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-manylinux2014_ppc64le.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 77c4c41f2147c930c66014ca43b6935133781ae1923d62e70c797e71b0ee2598
MD5 4c209e08e541cf4e7e65956b5f60653e
BLAKE2b-256 669b7d1f8c5ecd278c48048c53f94118171b6b8377252a2e8eaae83267a4e5e7

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-manylinux2014_aarch64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-manylinux2014_aarch64.whl
  • Upload date:
  • Size: 3.2 MB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f7b62497dce161babdb9197fa4b26e401bac9541b62fe0d0957134fefeb1b01c
MD5 e3da79002c94e8bf6e812b7ba6fcb026
BLAKE2b-256 efe9ac778487ace5686b387848457dbd623beeaef0d1c2b4f822def58aa6e7a3

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-manylinux2010_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.8, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 f1553e029f326eb74f36d67a38ef77a7f03068a494a0faa4e16d0d832f25b760
MD5 e8125ad00917693b93118d19d84008c4
BLAKE2b-256 1df126c9399e5e02bcb8c0ad081b0417690440634ec008676d3b10771a6233fc

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-manylinux1_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-manylinux1_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.8
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 15d9b959fd3b9e9e7c6d6d7d909bca5d7397a170a50d99ac8ce4e2ab590b137a
MD5 60dca3e399e886d4f661e2cf2220e809
BLAKE2b-256 1120270fbb0bea7f29e4aebb3d726f1c6f8de50e9ac37e784ebf20bc38da574d

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp38-cp38-macosx_10_11_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp38-cp38-macosx_10_11_x86_64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.8, macOS 10.11+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp38-cp38-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 7ba26369bc30f9d28d9ff42dcb1b57d9995157a9bb2975b95acda4220195d7aa
MD5 c98f6192f1ccb7f781fa2724e5fc6daf
BLAKE2b-256 39b66fbc36316d8adc28bfef0d73206f8334a49f959a1c6fcf2c1f5d8b16dbc1

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-win_amd64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-win_amd64.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.7m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-win_amd64.whl
Algorithm Hash digest
SHA256 aadcf38b97114d035e389f5aee4edf59e81666ad65de26c06592d76f184bb66c
MD5 e9e6f44fb3b5ee7316411a11d81e08e4
BLAKE2b-256 ed8a7fe92479404206611c16939a98a5ea50ea01fd4c31f062f9f24ec4548a03

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-win32.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-win32.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.7m, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-win32.whl
Algorithm Hash digest
SHA256 c429c25c3dfe1ea9ad6e21a49d648910335ef4188c5e8226e5aa2ba2bd13921c
MD5 7cd93fa4868e0c9d7553a572e95c29e1
BLAKE2b-256 c17c00d40ed70daa3faf1eacd0a84461d09e053b37df6317cea038abcaa103ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-manylinux2014_s390x.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-manylinux2014_s390x.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 79119578bcd1d8ec836ddd3dbb305f32084d60e9f67e93a10ca33c67eeaa89fc
MD5 a21d4555c343c02547856bd5bef2daff
BLAKE2b-256 588f63dc9f42235abd1a77bd2f0f2fea9adabe921759c68dd42f764f97a195bc

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-manylinux2014_ppc64le.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-manylinux2014_ppc64le.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 1a22bf899728eeb74ee2bb1ba9eff61898ec02e623a690ed28002762d19ab9b4
MD5 e1a73dca15c92b480cbd83a2e324089a
BLAKE2b-256 0cea7417a3137cddd1041ae44f178abc2d74c899075b02c5170328c9d961490a

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-manylinux2014_aarch64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-manylinux2014_aarch64.whl
  • Upload date:
  • Size: 3.2 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 a3de6ecfbd739ee3d59280c0c930c0c5a716df1cf0cdf68beb379066931866bd
MD5 c0f76cbeeac2caaa0dd2be51baaf8f0d
BLAKE2b-256 bec6fed66f0fb3b26bb097650ea0af01dbeb347d1e52ad2540b2ce3f9ecc6439

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.7m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 05c90ade1b9cc41aaee6056c5e460dc5150f12b602bdc6bfa3758fb965ca7788
MD5 c7d8e5d85588189aeeadad5a051582b1
BLAKE2b-256 ae045b870f26a858552025a62f1649c20d29d2672c02ff3c3fb4c688ca46467a

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-manylinux1_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.7m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 9e8f32a2ef1902f769da6215ae8beabd632676a1551fb171b5aa6d4c11fd3a02
MD5 9eb3c620d1debb37caf62e30cd4c02d3
BLAKE2b-256 0c8bce20377ecad8b4859088f03cfc6171b61833530872e5ba31b62bfe36094e

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp37-cp37m-macosx_10_11_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp37-cp37m-macosx_10_11_x86_64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.7m, macOS 10.11+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp37-cp37m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 8a575022e066878bede82bb5d5244b17c6ebda15dbb50229d86f9e8267ddd40e
MD5 fa2ca0c32913b913f1c55eaa0b6b201a
BLAKE2b-256 d143518de4fc320cf51c967202bc8fe2f2acf348a1119c7f5660f65bba6841f4

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-win_amd64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-win_amd64.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.6m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-win_amd64.whl
Algorithm Hash digest
SHA256 ea54eb0071f13fa7c6c3b88997a843d01c067158b994115759c27827e683fb82
MD5 b8c1ce8c2a622b5b79bb3867e40be9ae
BLAKE2b-256 f21ed739dd855470f1556d336283b1e3b9f61d13c0ca475aa266d21e85603ffa

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-win32.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-win32.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.6m, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-win32.whl
Algorithm Hash digest
SHA256 9124a1f77e176cb2a2571bae4c3bf8d4c40975c1681e2ba346fdca5d6a3aa843
MD5 4c5b3403149024cf7a42b9d47455267d
BLAKE2b-256 4dfee45bf972c3714b0bb8c2b1bea6ce5bea41f527f7e367746212620a6a0670

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-manylinux2014_s390x.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-manylinux2014_s390x.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 9619846026b16967465e5221206f86bdc58cf65b0f92548d048e97925361121e
MD5 cbb9c4e5f0518f9f07f4aeee44f88da3
BLAKE2b-256 a4bd0499dfad3f86d6c4a7e3373c53797c921c033a87502738b4959f2dde9fc2

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-manylinux2014_ppc64le.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-manylinux2014_ppc64le.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 86077426c615a814f7456569eade33c12c93131d02fdf548994dcedf41bdbbf1
MD5 4f5948cb79a2fd82939fcd0efefe9234
BLAKE2b-256 c0941bf51bad5b9a339c40c96fff3ab6157d5405534b92415de44c2fe3ddf342

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-manylinux2014_aarch64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-manylinux2014_aarch64.whl
  • Upload date:
  • Size: 3.2 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 bac17cceebb2a6947d380e1b7bce8fc33098a979071a1291adc456fb25434924
MD5 a8c9f25166d25f07718d8c22cdd60519
BLAKE2b-256 77a94c17d47605c6509ce539e32b5e19d1ea77f6bb44bb34d106a5276b1f9eee

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.6m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 3fb22df976701452db3ba652bd647518a043e58d4209d18273163fbc53252a3b
MD5 9a74a9c2041c6f046a6f02f1428c3694
BLAKE2b-256 b0ec029100a156a3671a385c78898cf16896f5c29a867d310895b4e221b722f5

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-manylinux1_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.6m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 bb58ad982f8f72052362a5384145e87559899dcc0b06264e71dd137869037e6e
MD5 61a0b4836836e40a2c8761d046fda5eb
BLAKE2b-256 7398f100583260f3805edc6d5b0e318b2a6a6a611078ba58928f15062b7d4cf5

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp36-cp36m-macosx_10_11_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp36-cp36m-macosx_10_11_x86_64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.6m, macOS 10.11+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp36-cp36m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 474883e8e0be431394e0ccfb70e97c1856e8c5bc80536f7b2faa3b0785d59afd
MD5 6725a1021479d72ef1f3f3f51b8174ab
BLAKE2b-256 b1b311a39fa71ffe6a218b527491d8ae3d6e556255f0933aaf433542e1bee389

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-win_amd64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-win_amd64.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.5m, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-win_amd64.whl
Algorithm Hash digest
SHA256 419bb33bb3690239b93b76b06eba1eb822aa72f4e63293d2f15c60505f6ee0d0
MD5 1f8eecf6cfa726a6fbf6e8fb133560fa
BLAKE2b-256 f18d0b3005dbb3170ec9ab19d6ca6253c69a3a88a8c9047df91ba5b538d8fb2b

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-win32.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-win32.whl
  • Upload date:
  • Size: 1.8 MB
  • Tags: CPython 3.5m, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-win32.whl
Algorithm Hash digest
SHA256 f9fe9c5556ccab03c9d42ed299bd8901c95d22373676437bfeb4656c2b5e42bc
MD5 8f8cf1214d077eb03fa785f49980a607
BLAKE2b-256 c8d4f5e4f8c5b5479c7d5ca28d104cf04f26421ee47497f54f992e1165b23d9d

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-manylinux2014_s390x.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-manylinux2014_s390x.whl
  • Upload date:
  • Size: 3.8 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 fb1dae213c8531d6af071dd021c7225be73803a0cbe609aed5074be04118aa6c
MD5 c0ce31ca7a875838812c1a285c31061e
BLAKE2b-256 dd7805eeed7e9b27d9a81c1d87a0ae6a9d19c10a826d8877afcf2f762d423f7a

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-manylinux2014_ppc64le.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-manylinux2014_ppc64le.whl
  • Upload date:
  • Size: 3.7 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 cd408266f13856dc648ed2dcc889ac17feffc28da2ebb03f1977b88935e86c9a
MD5 1c133386b99a63f24f6f88bbf15a34de
BLAKE2b-256 4143ff195a08a51f2e711723d75a93762641fab30ef3ab1876e2fd4e1e230080

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-manylinux2014_aarch64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-manylinux2014_aarch64.whl
  • Upload date:
  • Size: 3.2 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 2b592146caff20c283dadf2da99520b1dfde4af8ce964a8adcb4e990923fa423
MD5 2c6a70b69da5863853a12b097b1dbcb9
BLAKE2b-256 021800ed0d49547338a22a61d96f1cfc190cc8d953e23a5d300666318e2ce4eb

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-manylinux2010_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-manylinux2010_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.5m, manylinux: glibc 2.12+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-manylinux2010_x86_64.whl
Algorithm Hash digest
SHA256 be25827c0506d92927dc0ef4d2ce0c4653a351735546f8b22548535c3d2f7a6c
MD5 dd3d62569ce0bc21fd6234e8aaa6a64e
BLAKE2b-256 88ecbfdb67cad2ac202038d74912c1273452453f0e7e2ad7e1c335158639b3fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-manylinux1_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-manylinux1_x86_64.whl
  • Upload date:
  • Size: 3.3 MB
  • Tags: CPython 3.5m
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-manylinux1_x86_64.whl
Algorithm Hash digest
SHA256 4865d34d4897eed4ca4a758971fb14911cf5022e270b53c028fa9312fe440e2b
MD5 1353222c13bc6d5ce01917cced942a0c
BLAKE2b-256 ea8223d84f2ae49590040d8fd8bfc1af4e310233ed749e5e921107504d175e97

See more details on using hashes here.

File details

Details for the file tokenizers-0.10.2-cp35-cp35m-macosx_10_11_x86_64.whl.

File metadata

  • Download URL: tokenizers-0.10.2-cp35-cp35m-macosx_10_11_x86_64.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: CPython 3.5m, macOS 10.11+ x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.1 importlib_metadata/3.10.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.9.2

File hashes

Hashes for tokenizers-0.10.2-cp35-cp35m-macosx_10_11_x86_64.whl
Algorithm Hash digest
SHA256 8b4ae84fb410b5f5abb3a604b3274e2d6994b21f07c379b1c1659561e026bad8
MD5 4c2b1fae82ee76db09b4078918808e64
BLAKE2b-256 1ac47955de7cdf053a2627d32fd6f3aba1333265b895765a2dc2d629891d7924

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page