No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Release history Release notifications | RSS feed

0.20.3

0.20.3rc0 pre-release

0.20.2

This version

0.20.1

0.20.1rc1 pre-release

0.20.0

0.20.0rc1 pre-release

0.19.1

0.19.0

0.15.2

0.15.2rc1 pre-release

0.15.1

0.15.0

0.14.1

0.14.0

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0 yanked

Reason this release was yanked:

Breaking change with unexpected effect

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.3

0.10.2

0.10.1

0.10.1rc1 pre-release

0.10.0

0.10.0rc1 pre-release

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.9.0rc2 pre-release

0.9.0rc1 pre-release

0.9.0.dev4 pre-release

0.9.0.dev3 pre-release

0.9.0.dev2 pre-release

0.9.0.dev1 pre-release

0.9.0.dev0 pre-release

0.8.1

0.8.1rc2 pre-release

0.8.1rc1 pre-release

0.8.0

0.8.0rc4 pre-release

0.8.0rc3 pre-release

0.8.0rc2 pre-release

0.8.0rc1 pre-release

0.8.0.dev2 pre-release

0.8.0.dev1 pre-release

0.8.0.dev0 pre-release

0.7.0

0.7.0rc7 pre-release

0.7.0rc6 pre-release

0.7.0rc5 pre-release

0.7.0rc4 pre-release

0.7.0rc3 pre-release

0.7.0rc2 pre-release

0.7.0rc1 pre-release

0.6.0

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.1

0.1.0

0.0.13

0.0.12

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.20.1.tar.gz (339.6 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1-cp312-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.20.1-cp312-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.20.1-cp312-cp312-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp312-cp312-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp312-cp312-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.20.1-cp312-cp312-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.20.1-cp311-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.20.1-cp311-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.20.1-cp311-cp311-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp311-cp311-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp311-cp311-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.20.1-cp311-cp311-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.20.1-cp310-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.20.1-cp310-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.20.1-cp310-cp310-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp310-cp310-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp310-cp310-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.20.1-cp310-cp310-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.20.1-cp39-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.20.1-cp39-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.20.1-cp39-cp39-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp39-cp39-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp39-cp39-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.20.1-cp39-cp39-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.20.1-cp38-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.20.1-cp38-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.20.1-cp38-cp38-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp38-cp38-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp38-cp38-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.20.1-cp38-cp38-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.20.1-cp37-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.20.1-cp37-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ i686

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1-cp37-cp37m-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.20.1-cp37-cp37m-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.20.1.tar.gz.

File metadata

  • Download URL: tokenizers-0.20.1.tar.gz
  • Upload date:
  • Size: 339.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1.tar.gz
Algorithm Hash digest
SHA256 84edcc7cdeeee45ceedb65d518fffb77aec69311c9c8e30f77ad84da3025f002
MD5 e100aa989139037085b890d9dc24dc73
BLAKE2b-256 d7fb373b66ba58cbf5eda371480e4e051d8892ea1433a73f1f92c48657a699a6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f40df5e0294a95131cc5f0e0eb91fe86d88837abfbee46b9b3610b09860195a7
MD5 cfaa8b0cd98e808bbe6d858a6fef4473
BLAKE2b-256 d4f2ea998aaf69966a87f92e31db7cba887125994bb9cd9a4dfcc83ac202d446

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 218e5a3561561ea0f0ef1559c6d95b825308dbec23fb55b70b92589e7ff2e1e8
MD5 661fa2ec78f79cbd7649f52450f3abbc
BLAKE2b-256 cbf779a74f8c54d1232ddbd68967ce56a00cc9589a31b94bee4cf9f34af91ace

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 03b03cf8b9a32254b1bf8a305fb95c6daf1baae0c1f93b27f2b08c9759f41dee
MD5 08fdd2402a7f1a1e3221552690c666c4
BLAKE2b-256 9c2c9f04aa030ba8994d478ab35464f8c541aad264556811f12afce9369cc0d3

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 0d3caf244ce89d24c87545aafc3448be15870096e796c703a0d68547187192e1
MD5 4561f65e0914117f473b3fa6e448a921
BLAKE2b-256 c1e6ec76a7761eb7ba3cf95e2485cb2e7999a8eb0900d771616c0efa61beb1cd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 359eceb6a620c965988fc559cebc0a98db26713758ec4df43fb76d41486a8ed5
MD5 c36a1d821777fbc97707d4f5cb8f91f2
BLAKE2b-256 97e3167ca1981b3f512030a28f591b8ef786585b625d45f0fbf1c42723474ecd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 712f90ea33f9bd2586b4a90d697c26d56d0a22fd3c91104c5858c4b5b6489a79
MD5 fee55b6d56130ac2bdbcc99fd36b6241
BLAKE2b-256 37988221a62aed679aefcbc1793ed8bb33f1e060f8b7d95bb20809db1b5c0e0e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 48689da7a395df41114f516208d6550e3e905e1239cc5ad386686d9358e9cef0
MD5 ba1d204c8b7c7f6d133ba5be1c254fde
BLAKE2b-256 4b9ecf0911565ae302e4e4ed3d53bba28f2db75a9418f4e89e2434246723f01a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 de291633fb9303555793cc544d4a86e858da529b7d0b752bcaf721ae1d74b2c9
MD5 843522b66a146853b4e9b944a590314a
BLAKE2b-256 da961603264112ce0fb5da7dae215de037d21dcd9b7c6947ed575b9e81a0d05a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 e7edb8ec12c100d5458d15b1e47c0eb30ad606a05641f19af7563bc3d1608c14
MD5 9dfb052155e1a7abdea8d98b16c62187
BLAKE2b-256 2a63a640fc3e011bb0392e05c9e5f7054c039d3542426dec46f6b7c8e753d999

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c5ffe0d7f7bfcfa3b2585776ecf11da2e01c317027c8573c78ebcb8985279e23
MD5 61be5d5809b77cc1ba43a61efd8dabff
BLAKE2b-256 24e820d71cb85d3f3f39e01246f7827b42b6f9862332c500433f20307addc212

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 5dc611e6ac0fa00a41de19c3bf6391a05ea201d2d22b757d63f5491ec0e67faa
MD5 5ab63c0c439aab8146efea11bfd54aba
BLAKE2b-256 f638977e4ca1dc955fb368fa900050d08b9bf67f8d5de3bd364f0d04ed4eedfd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3e0305fc1ec6b1e5052d30d9c1d5c807081a7bd0cae46a33d03117082e91908c
MD5 16c295ab05355a278ce4b297512f86d6
BLAKE2b-256 f19fffcf52c1d0f27778ec2e5c526ee7c727c24441254ef5732d0b391583ff7b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0b7f515c83397e73292accdbbbedc62264e070bae9682f06061e2ddce67cacaf
MD5 21905e68eca5ded476f54327d6916a24
BLAKE2b-256 8f35643727d36c409f71ddd9b914513ed793dcaa56d92e4310a5f3a0d6b84138

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 89d5c337d74ea6e5e7dc8af124cf177be843bbb9ca6e58c01f75ea103c12c8a9
MD5 607c118be05c57d6a9a5cb8f49c4a0aa
BLAKE2b-256 3c89969b427a3882c029f968e44d5bb1cff14606f878fd7c7d5b6010767881d6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f861889707b54a9ab1204030b65fd6c22bdd4a95205deec7994dc22a8baa2ea4
MD5 4b9719eeb17b4918f0662b274c59772b
BLAKE2b-256 22e39038cf20799bd4832190325810329c1ea63984be0486b2c1934a22ff8327

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 62eb9daea2a2c06bcd8113a5824af8ef8ee7405d3a71123ba4d52c79bb3d9f1a
MD5 a42c5555ec1db56d2cb7ec47ae4dd47c
BLAKE2b-256 2cd01dadd1b37ac18a8056a88d88f5944eecb1d98717d8f3d74d500f1f4b9f9e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9041ee665d0fa7f5c4ccf0f81f5e6b7087f797f85b143c094126fc2611fec9d0
MD5 cfa9303c6dc0943133caa9de1dca31d2
BLAKE2b-256 6c9d8001ae42cf4ad907a97d86b5d3d43db6f9d66338d8ba2dc803855bd0005f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 cd28a8614f5c82a54ab2463554e84ad79526c5184cf4573bbac2efbbbcead457
MD5 787f1ee90774909e7fd57f553ac20f2f
BLAKE2b-256 7e1f897845e8ccf1cbfce83d0a3b5a467d56301ff461006bcf807104959ad552

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 130e35e76f9337ed6c31be386e75d4925ea807055acf18ca1a9b0eec03d8fe23
MD5 494be1bc2749e7c2f3a957874590efce
BLAKE2b-256 459c4f105c42cd29653e2f3c5c819b33f6dba067313d0430f4825a3c1c77b350

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 81970b80b8ac126910295f8aab2d7ef962009ea39e0d86d304769493f69aaa1e
MD5 1b57fe78d9f426d263205fd1c0781657
BLAKE2b-256 4d4fe7a4ea2221a7e007d9844364bc8fd7ea89b35c3c5fbc42ed7d03b12c1a86

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ebe63e31f9c1a970c53866d814e35ec2ec26fda03097c486f82f3891cee60830
MD5 4d501274c01d25096761eeee077a36b4
BLAKE2b-256 e9495e2df67ea57e92317175abc60dc81d2031dea02020dd77c216eab2a48d97

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1fbbaf17a393c78d8aedb6a334097c91cb4119a9ced4764ab8cfdc8d254dc9f9
MD5 fbe8286255c4625b298e2c5e09a83f50
BLAKE2b-256 b5b2d13823d3b44a8cf7b3f702c2a36e7d4de57bda623b5f4d1109ffab8adebf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 956f21d359ae29dd51ca5726d2c9a44ffafa041c623f5aa33749da87cfa809b9
MD5 1a5fef58ff723eff3b116e3a868d2070
BLAKE2b-256 4e506bb7f79034e4043873320aadbc36fbdd8e97748bfa7e0b18f41dc26c7296

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 31e87fca4f6bbf5cc67481b562147fe932f73d5602734de7dd18a8f2eee9c6dd
MD5 ad6340ca12c235b297aa2496bb77d8d4
BLAKE2b-256 26bdb531886b73f79ab299fd190fcce4f7d0fdf678da0cb688dc6d29d4964a70

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 62f7fbd3c2c38b179556d879edae442b45f68312019c3a6013e56c3947a4e648
MD5 60ede1788471550f03036d5bc6a0ea7d
BLAKE2b-256 3d6ead7f9c604fcc191cdd114d6e7d48e26a57deaadf5e9ef52245c2c57b1fc0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 998700177b45f70afeb206ad22c08d9e5f3a80639dae1032bf41e8cbc4dada4b
MD5 a233a69dec1874fff17a4077e64f5e89
BLAKE2b-256 dc33994f6977400c0c7991ff40157dc3663ce35286acb89c5d8997cfd1aab9da

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 08aaa0d72bb65058e8c4b0455f61b840b156c557e2aca57627056624c3a93976
MD5 b6f61e781c4705cce7a81b3f434442d5
BLAKE2b-256 2cae10d10f93b2e9fba24d038954eef5cf22a29774f65b449d23c68f642525d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 b0874481aea54a178f2bccc45aa2d0c99cd3f79143a0948af6a9a21dcc49173b
MD5 f4e865c89544044e8b2cf2916bcadf58
BLAKE2b-256 7eba18bf6a7ad04f8225b71aa862b57188748d1d81e268de4a9aac1aed237246

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp312-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp312-none-win32.whl
Algorithm Hash digest
SHA256 6d3ac5c1f48358ffe20086bf065e843c0d0a9fce0d7f0f45d5f2f9fba3609ca5
MD5 43a8b8bb5696878135e6abf184c18730
BLAKE2b-256 41ff4aeb924d09f6561209b57af9123a0a28fa69472cc71ee40415f036253203

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 3ea919687aa7001a8ff1ba36ac64f165c4e89035f57998fa6cedcfd877be619d
MD5 77d7e3dee0824bdc1ea2918523d7d836
BLAKE2b-256 bb37eaa072b848471d31ae3df6e6d5be5ae594ed5fe39ca921e65cabf193dbde

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 0c6a796ddcd9a19ad13cf146997cd5895a421fe6aec8fd970d69f9117bddb45c
MD5 441485b55d2dcb779ab4fe42dcd1b207
BLAKE2b-256 d4abceb7bdb3394431e92b18123faef9862877009f61377bfa45ffe5135747a5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 689b93d2e26d04da337ac407acec8b5d081d8d135e3e5066a88edd5bdb5aff89
MD5 a197cd10718380db449ba38b78cdb763
BLAKE2b-256 2459664121cb41b4f738479e2e1271013a2a7c9160955922536fb723a9c690b7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 bdd67a0e3503a9a7cf8bc5a4a49cdde5fa5bada09a51e4c7e1c73900297539bd
MD5 165d93e84f66d546c6c2d62c3f553be0
BLAKE2b-256 cf7dc895f076e552cb39ea0491f62ff6551cb3e60323a7496017182bd57cc314

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 e2e2d47a819d2954f2c1cd0ad51bb58ffac6f53a872d5d82d65d79bf76b9896d
MD5 d6bdf9e6e49775410610813390142353
BLAKE2b-256 a42ada72c32446ad7f3e6e5cb3c625222a5b9b0bc10b50456f6cb79f6230ae1f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 128c1110e950534426e2274837fc06b118ab5f2fa61c3436e60e0aada0ccfd67
MD5 f1526f3a780505d6462d73d6ceac57ce
BLAKE2b-256 f759185ff0bb35d46d88613e87bd76b03989ef8537ebf4f39876bddf9bed2fc1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 47c1bcdd61e61136087459cb9e0b069ff23b5568b008265e5cbc927eae3387ce
MD5 173a976099a860de266de25671048c2c
BLAKE2b-256 285192e3b25eb41be7fd65219c832c4ff61bf5c8cc1c3d0543e9a117d63a0876

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 e98eee4dca22849fbb56a80acaa899eec5b72055d79637dd6aa15d5e4b8628c9
MD5 140b579ce70f1cc4ab2c20e9590b9840
BLAKE2b-256 f46e9dfd1afcfd38fcc5b3a84bca54c33025561f7cab8ea375fa88f03407adc1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2f13a2d16032ebc8bd812eb8099b035ac65887d8f0c207261472803b9633cf3e
MD5 e82e2cba24007852e9ace9674a7e66ce
BLAKE2b-256 99c32132487ca51148392f0d1ed7f35c23179f67d66fd64c233ff50f091258b4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 407ab666b38e02228fa785e81f7cf79ef929f104bcccf68a64525a54a93ceac9
MD5 c3c7a893d0757b2865afd8a291c42938
BLAKE2b-256 8e8da051f979f955c6717099718054d7f51fea0a92d807a7d078a48f2684e54f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 899152a78b095559c287b4c6d0099469573bb2055347bb8154db106651296f39
MD5 a0d3a9279499cb7597eef8079246da70
BLAKE2b-256 f195f1b56f4b1fbd54bd7f170aa64258d0650500e9f45de217ffe4d4663809b6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp311-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp311-none-win32.whl
Algorithm Hash digest
SHA256 9af2dc4ee97d037bc6b05fa4429ddc87532c706316c5e11ce2f0596dfcfa77af
MD5 59d7f763e39f30c82c51d6174c0b4561
BLAKE2b-256 ea487d4ac79588b5b1c3651b753b0a1bdd1343d81af57be18138dfdb304a710a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 86dcd08da163912e17b27bbaba5efdc71b4fbffb841530fdb74c5707f3c49216
MD5 8711362353747d75833bb99df1cf72ff
BLAKE2b-256 d042c287d28ebcb3ba4f712e7a58d8f170a7b569528acf2d2a8fd1f684c24c0c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 ee86d4095d3542d73579e953c2e5e07d9321af2ffea6ecc097d16d538a2dea16
MD5 6dd753e3feaf18f6a5380906dd970c2d
BLAKE2b-256 d392e5b80e42c24e564ac892c9135e4b9ec34bbcd6cdf0cc7a04735c44fe2ced

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ef3f1ae08fa9aea5891cbd69df29913e11d3841798e0bfb1ff78b78e4e7ea0a4
MD5 4ca8eaafd7363ade7a04627769820d83
BLAKE2b-256 096c1b573998fe3f0e18ac5d434e43966de2d225d6837f099ce0df7df4274c87

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 5170be9ec942f3d1d317817ced8d749b3e1202670865e4fd465e35d8c259de83
MD5 c46edc4dfec9d2572501701853ea8722
BLAKE2b-256 7e0f9136bc0ea492d29f1d72217c6231dc584bccd3ba41dde12d4a85c75eb12a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 0ecaf7b0e39caeb1aa6dd6e0975c405716c82c1312b55ac4f716ef563a906969
MD5 6f22c0d5e2b99164c64d503087bf40b1
BLAKE2b-256 fef44302dce958ce0e7f2d85a4725cebe6b02161c2d82990a89317580e17469a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 9300fac73ddc7e4b0330acbdda4efaabf74929a4a61e119a32a181f534a11b47
MD5 9de6ccb4ec2bcbb73ed6faab6f1b9fe4
BLAKE2b-256 ff71b220deba78e42e483e2856c9cc83a8352c7c5d7322dad61eed4e1ca09c49

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 d10766473954397e2d370f215ebed1cc46dcf6fd3906a2a116aa1d6219bfedc3
MD5 768f216c4ef38c5f3fad7a0c466f5767
BLAKE2b-256 3188740a6a069e997dc3e96941083fe3264162f4d198a5e5841acb625f84adbd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 929c8f3afa16a5130a81ab5079c589226273ec618949cce79b46d96e59a84f61
MD5 1ef9ad2a08c5f939810c7c579be7ff88
BLAKE2b-256 2dcaf3a294ed89f2a1b900fba072ef4cb5331d4f156e2d5ea2d34f60160ef5bd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3f84dad1ff1863c648d80628b1b55353d16303431283e4efbb6ab1af56a75832
MD5 1f1bd282c57faf38bb93335b27434008
BLAKE2b-256 26a292af8a5f19d0e8bc480759a9975489ebd429b94a81ad46e1422c7927f246

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4a717dcb08f2dabbf27ae4b6b20cbbb2ad7ed78ce05a829fae100ff4b3c7ff15
MD5 313ee02da8b157daaac78ed990a6a13a
BLAKE2b-256 619abe5f00cd37ad4fab0e5d1dbf31404a66ac2c1c33973beda9fc8e248a37ab

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 f9aa93eacd865f2798b9e62f7ce4533cfff4f5fbd50c02926a78e81c74e432cd
MD5 dc6b702ec64d387638f1a432a2b3af4e
BLAKE2b-256 7adac7728bb6be0ccfbd5662f054ee28d8ba7883558cc9fcd102e6cdce07bbbf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp310-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp310-none-win32.whl
Algorithm Hash digest
SHA256 2847843c53f445e0f19ea842a4e48b89dd0db4e62ba6e1e47a2749d6ec11f50d
MD5 fb8f05d1fbf4ad04ca0198550d3dbce1
BLAKE2b-256 e85422825bc3d00ae8a801314a6d96e7e83c180b626a40299179073364c7eac7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a2ffd9a8895575ac636d44500c66dffaef133823b6b25067604fa73bbc5ec09d
MD5 1c09c3f6757e948e9cf0aa814e48ce2d
BLAKE2b-256 dff78c0ec102f0a723d09347ff6cd617c7e5e8d44efd342305f52a7fcd3e30e2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 f22dee205329a636148c325921c73cf3e412e87d31f4d9c3153b302a0200057b
MD5 fbc7982cb954c4621567f5fdea8ce107
BLAKE2b-256 dbb25e45632799d816291de4d04149decf19cf6c2faf42bb99574d80050c87bd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 fc9e95ad49c932b80abfbfeaf63b155761e695ad9f8a58c52a47d962d76e310f
MD5 776d54d44279677d6042da6edf6096e3
BLAKE2b-256 4b35326b9642307a53b3d9ae145b5c7f157aae9ecaa930888f920124412e0bd2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 e96f6c14c9752bb82145636b614d5a78e9cde95edfbe0a85dad0dd5ddd6ec95c
MD5 6b0a8730bd82a601099bad7a1da84726
BLAKE2b-256 2a393d11780b82d9ba4d8fda093daa48622ed5f2616d6ac8cb638ac290d39d95

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 299c85c1d21135bc01542237979bf25c32efa0d66595dd0069ae259b97fb2dbe
MD5 aefd9881d0436c75b51b4533b4badfa7
BLAKE2b-256 cf03801e91d41e2134a32089af2d382a6c40b3d8b932b42fa96443d77258ab28

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 d388d1ea8b7447da784e32e3b86a75cce55887e3b22b31c19d0b186b1c677800
MD5 e441096307494b8aa9dbb75f1a1af214
BLAKE2b-256 22096e0a378a35f215b40ae1c04b4d0fe43e9ddfaf3a08a2b7d7fab8953a6587

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 ec870fce1ee5248a10be69f7a8408a234d6f2109f8ea827b4f7ecdbf08c9fd15
MD5 9aebf0747febc9fafee9b9ba1f0adab8
BLAKE2b-256 5ff0c1ed45ff90088eba4f15eca9763b5e439cb86b71fc9e66a827318b61e44d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b61f561f329ffe4b28367798b89d60c4abf3f815d37413b6352bc6412a359867
MD5 41b56338eda4f4daa0c855bb32ac2c69
BLAKE2b-256 25e25046ad3b0426548b37c96cc4262a7f2ba6ac9593ee10be69effc78a91764

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 03dae629d99068b1ea5416d50de0fea13008f04129cc79af77a2a6392792d93c
MD5 6672d7f6c28bbc45f1f7d25ad6d2026c
BLAKE2b-256 24d4a529aa06db71600c1688210ce035cbff637ece919dcaca599c9235ad832d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 439261da7c0a5c88bda97acb284d49fbdaf67e9d3b623c0bfd107512d22787a9
MD5 1ab8748873866cb6b87f4c86b6025571
BLAKE2b-256 72d23c05efeeccefa833b82038ce49ee736756eed10ab66fc723ce423a747b0e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 37d1e6f616c84fceefa7c6484a01df05caf1e207669121c66213cb5b2911d653
MD5 9691fe7ba0e192d2784d556e0c313fc7
BLAKE2b-256 3195ba6461cbb58e38d725564a74ce1ec56c63cb4abf13131ec400522d4f7ce8

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp39-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp39-none-win32.whl
Algorithm Hash digest
SHA256 3d4d218573a3d8b121a1f8c801029d70444ffb6d8f129d4cca1c7b672ee4a24c
MD5 2bbcd6adbcb887d5aac73ed307bd059a
BLAKE2b-256 68bf911ca80c5928bf8728c7f4d052ab8487512dc34f5ce872dcad0f561734bc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 42c097390e2f0ed0a5c5d569e6669dd4e9fff7b31c6a5ce6e9c66a61687197de
MD5 824799155c37a426fe34e216955611c9
BLAKE2b-256 8f9ebce3fecf89901de85c12b27430aa61ada6388a0df6b11e8b1a6793399d9c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 da1001aa46f4490099c82e2facc4fbc06a6a32bf7de3918ba798010954b775e0
MD5 36d91ad43ab484566880abe661cc3378
BLAKE2b-256 00fd805bbbabe081de0f36120206e1d8c5d408336ff914f3a3b9fa1e77928794

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a908c69c2897a68f412aa05ba38bfa87a02980df70f5a72fa8490479308b1f2d
MD5 8d3548a709da47677976767399491fb1
BLAKE2b-256 63041ce87a3eae86aa4e301e863d42d6907deaf61f3e9178210d9ebe653e948c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 57b7a8880b208866508b06ce365dc631e7a2472a3faa24daa430d046fb56c885
MD5 7e543dedac51defbc82191cd2dec594b
BLAKE2b-256 9b5f9bfd63eb6e36a42af0b3f9313e66b13ec48c348ddb2e9398560daddd0206

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 ce6238a3311bb8e4c15b12600927d35c267b92a52c881ef5717a900ca14793f7
MD5 1c93083dc147b302953b1580e38084c5
BLAKE2b-256 ba6bf8d3f5eb484ba4ca92d3ce0fd050ed796aec23b28f2582e85e48c76d14d2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 0b4872647ea6f25224e2833b044b0b19084e39400e8ead3cfe751238b0802140
MD5 f19151cfbe6037556a8dd2404816a829
BLAKE2b-256 149d2eb09e02be5c581535efb550ff3920db067526d380a154cdacf3e82a3cb2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 f326a1ac51ae909b9760e34671c26cd0dfe15662f447302a9d5bb2d872bab8ab
MD5 35792aec17da1fdabc293a290e8bd05a
BLAKE2b-256 06b9029fbb476bda0e51b16e8c0bbbef759eb11d4cdbfca7d48b6b971e849270

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 02e18da58cf115b7c40de973609c35bde95856012ba42a41ee919c77935af251
MD5 3f7cd0e888b172c0f0d8db6065c852f6
BLAKE2b-256 220c18437327596682446be04597ffb6296cc472886857a057b726bbe7fc18c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a12c3cebb8c92e9c35a23ab10d3852aee522f385c28d0b4fe48c0b7527d59762
MD5 7bde280237563e88ef062ee5c33cea81
BLAKE2b-256 80eb95af53d2072c01c8a6e14114434099370d45344875b53be8df1dfaea9b6d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 a25dcb2f41a0a6aac31999e6c96a75e9152fa0127af8ece46c2f784f23b8197a
MD5 4c66e114ff6b70717f94c3ca0acad67a
BLAKE2b-256 2c7f63333364e618d60d6f82da364e6ea28e09e98f0c479ca4baf09591067cd0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 d412a74cf5b3f68a90c615611a5aa4478bb303d1c65961d22db45001df68afcb
MD5 a8158f2650ceacfd7b13faca01b7e906
BLAKE2b-256 d2a95a52a6a10064a3a0891e68fe743ee7bbf1f1389311a8b682e4ae50482f4e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp38-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp38-none-win32.whl
Algorithm Hash digest
SHA256 88b3bc76ab4db1ab95ead623d49c95205411e26302cf9f74203e762ac7e85685
MD5 d63b193a7bd3dfcbd9c764e70b62e43b
BLAKE2b-256 7a2ebb12fc4db04e7b1a5ed4e54846ca582d3ce6c51e653b46492d549be6779e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 b605c540753e62199bf15cf69c333e934077ef2350262af2ccada46026f83d1c
MD5 d903485b1a8e613a9338437960670938
BLAKE2b-256 74db403bd11c40083572a785c9a4b8dd22a5e67526dba36cc5fc2c277ddcd87d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 07c4b7be58da142b0730cc4e5fd66bb7bf6f57f4986ddda73833cd39efef8a01
MD5 1159c97ec1336bb2b78f3ea393b7383f
BLAKE2b-256 9a0f0fc165aa7521fccd5130df526657024a39777a19d30b73435053a7b65d7e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e53975a6694428a0586534cc1354b2408d4e010a3103117f617cbb550299797c
MD5 7e3e2e41250f747ea4d41f691318cc11
BLAKE2b-256 15e867dc15b6ed227358476fcfa6c2c71e4c508ff86cb6a5996c72e316ebac5b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 910b96ed87316e4277b23c7bcaf667ce849c7cc379a453fa179e7e09290eeb25
MD5 6d775c5fd02966b3ea25df3b29f0690d
BLAKE2b-256 520a70cbf45dd79f94ebf56580edb9c6d0cd04f402bba9bf8f19b2d316bea79b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 1ba72260449e16c4c2f6f3252823b059fbf2d31b32617e582003f2b18b415c39
MD5 95c7ce4ae79d89c3927a013e5c0f91c0
BLAKE2b-256 3bcc19e94d7309b0a160014f60e3c2015c3282a1abf9e9e49e91c2b3db110aeb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 7cdf379219e1e1dd432091058dab325a2e6235ebb23e0aec8d0508567c90cd01
MD5 24289cbad152bdbf07c697233b53108c
BLAKE2b-256 733ed8e5630a032b7d2f81b3bd3527a37747529d68505663e75cdf725b32ebaf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 a647c5b7cb896d6430cf3e01b4e9a2d77f719c84cefcef825d404830c2071da2
MD5 49940e627dea97d855f494c9c8cf7eda
BLAKE2b-256 6dea753829488d580b5784852c2ad56bc73f3c10b13b72483a508d13e006fb9c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 514cf279b22fa1ae0bc08e143458c74ad3b56cd078b319464959685a35c53d5e
MD5 0f2a1f9be460f420671057b583e8f110
BLAKE2b-256 0f6deb69f3c0fc3ee699794fbd56860926489e8a616aad402760f10a7eec99a4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 10be14ebd8082086a342d969e17fc2d6edc856c59dbdbddd25f158fa40eaf043
MD5 161c5db0bf426656e828f6587d1cf0ee
BLAKE2b-256 619b2786d23aa897f186f930205db836425196284784068d0fc5ca1a40ffbf36

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 741fb22788482d09d68e73ece1495cfc6d9b29a06c37b3df90564a9cfa688e6d
MD5 c76662cf30ee85608052d438aba04e41
BLAKE2b-256 f3f961e8c8836d5cbf0402630d1e9eeccaef46b91afbc71b44c0eca2391c13b7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 896195eb9dfdc85c8c052e29947169c1fcbe75a254c4b5792cdbd451587bce85
MD5 1091a44450ecd77d404a2c14bf239ed5
BLAKE2b-256 1409812a484d7cb22a043446469dfd4f81b57c5c3dd09fb8d8481993301337d7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.1-cp37-none-win32.whl
  • Upload date:
  • Size: 2.2 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1-cp37-none-win32.whl
Algorithm Hash digest
SHA256 212231ab7dfcdc879baf4892ca87c726259fa7c887e1688e3f3cead384d8c305
MD5 e5936b748a6824e05095fd45181d1f20
BLAKE2b-256 9df57d669fb93872795d663736fba03182b3ffe602d68a9e9cf3f37b7a3a75c5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 05e41e302c315bd2ed86c02e917bf03a6cf7d2f652c9cee1a0eb0d0f1ca0d32c
MD5 02ed56e155b3334b4ff909e2f5221bc4
BLAKE2b-256 b659e24ede3f75425a48974a275193367e79e2b0cfc03c677880e5bca539c768

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 9310951c92c9fb91660de0c19a923c432f110dbfad1a2d429fbc44fa956bf64f
MD5 4c81dbe2c242287dbac04b422e95ab7c
BLAKE2b-256 f1fee373a8d79f5a4c933b254e54e0e12cd3fb5375745f8eb4861cc57e4cef82

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 14e4cf033a2aa207d7ac790e91adca598b679999710a632c4a494aab0fc3a1b2
MD5 fbaad36e3b98c106f5fc937129a3ad8b
BLAKE2b-256 27db80c8bf72f2c40e693c753645149bc1f93100d0c2f62ce16195d7e136c35b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 094663dd0e85ee2e573126918747bdb40044a848fde388efb5b09d57bc74c680
MD5 1d2259017097648c1f63168d6a651e4f
BLAKE2b-256 aff778cf4837f7282dd2ef4dd45ec988308865fa420439b6d3e6bf3612400897

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 bfdad27b0e50544f6b838895a373db6114b85112ba5c0cefadffa78d6daae563
MD5 a5d0e287d90c4d6c77909d1cfa058f29
BLAKE2b-256 6064750ab25115c80d58f91d71041213e9fb0e62e1723982c1fa7f0dc36dd4e4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 4b39356df4575d37f9b187bb623aab5abb7b62c8cb702867a1768002f814800c
MD5 3789b944b29f37c0fcc636e3f3b36c57
BLAKE2b-256 6ae30ac76753e00dfbada1dd9cb0cea7a45d1b4951346c26037e267b4f6971d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 b8c0fc3542cf9370bf92c932eb71bdeb33d2d4aeeb4126d9fd567b60bd04cb30
MD5 8fb7a2a91c0b96f7b8fdcb808bc8b4ef
BLAKE2b-256 f2f4a8e86da74392855494b0f2fcf5fc92ed59f99e3359cb2ec203cdf8038922

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 17f98fccb5c12ab1ce1f471731a9cd86df5d4bd2cf2880c5a66b229802d96145
MD5 290434e6cedd74e7a99861c50425e4ef
BLAKE2b-256 500b05742acad333e816c4ffe401226f6259adb6b385286a7cadec76abbf835d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 65f34e5b731a262dfa562820818533c38ce32a45864437f3d9c82f26c139ca7f
MD5 99e9dc5f6dd4e173144b6017cd27cc36
BLAKE2b-256 27eac628b6a5884d2d580cfcc53f613143d8aba1563737ba93f7460b4315f33e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 96af92e833bd44760fb17f23f402e07a66339c1dcbe17d79a9b55bb0cc4f038e
MD5 363f5bebf30bfc8eaaef3a28b3a63ba6
BLAKE2b-256 2a2e33be045c394900cebf33233921b64d5d2a613b52493cedaa0b6f53dd786e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page