Skip to main content

No project description provided

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.15.2.tar.gz (320.3 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2-cp313-cp313-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp313-cp313-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp313-cp313-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.13 macOS 11.0+ ARM64

tokenizers-0.15.2-cp313-cp313-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.13 macOS 10.12+ x86-64

tokenizers-0.15.2-cp312-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.15.2-cp312-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.15.2-cp312-cp312-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp312-cp312-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp312-cp312-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.15.2-cp312-cp312-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.15.2-cp311-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.15.2-cp311-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.15.2-cp311-cp311-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp311-cp311-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp311-cp311-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.15.2-cp311-cp311-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.15.2-cp310-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.15.2-cp310-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.15.2-cp310-cp310-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp310-cp310-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp310-cp310-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.15.2-cp310-cp310-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.15.2-cp39-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.15.2-cp39-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.15.2-cp39-cp39-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp39-cp39-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp39-cp39-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.15.2-cp39-cp39-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.15.2-cp38-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.15.2-cp38-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.15.2-cp38-cp38-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp38-cp38-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp38-cp38-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.15.2-cp38-cp38-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.15.2-cp37-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.15.2-cp37-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ i686

tokenizers-0.15.2-cp37-cp37m-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.15.2-cp37-cp37m-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.15.2.tar.gz.

File metadata

  • Download URL: tokenizers-0.15.2.tar.gz
  • Upload date:
  • Size: 320.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2.tar.gz
Algorithm Hash digest
SHA256 e6e9c6e019dd5484be5beafc775ae6c925f4c69a3487040ed09b45e13df2cb91
MD5 0902772bdd98f05659f0c41e5c12a736
BLAKE2b-256 c044625db94e91c6196b6574359fa70bfe28e8eabf57a1b894f8f0ec69727fd1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 d857be2df69763362ac699f8b251a8cd3fac9d21893de129bc788f8baaef2693
MD5 6b9d4ce1bc9fdbdf4674b9a7b085b641
BLAKE2b-256 9f4d29e5052a11d1a9f8eb156e48c123731e6219e4f3d72cd6d7787fdf4eff7a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 4e022fe65e99230b8fd89ebdfea138c24421f91c1a4f4781a8f5016fd5cdfb4d
MD5 bf1d2cc7f9676c8408ab4fe8982ef449
BLAKE2b-256 a8cd6fe89c549d3aad886295cb9875105a75fa0d82ce80e4721cb43e6eb0830e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 67a0fe1e49e60c664915e9fb6b0cb19bac082ab1f309188230e4b2920230edb3
MD5 6ed343c4e479b418aec154ce0213f66f
BLAKE2b-256 ade1d0b441575a3ac0262c2c73773f79dd50c94e13c9dfda0d953f1c79d47ef5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 237d1bf3361cf2e6463e6c140628e6406766e8b27274f5fcc62c747ae3c6f094
MD5 e20e8b7c80a12a7c340481c01d77054c
BLAKE2b-256 89edb055d12637754471e4344f4e85c6268ef76801b0113ce1f789c5d84eaae9

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 13ca3611de8d9ddfbc4dc39ef54ab1d2d4aaa114ac8727dfdc6a6ec4be017378
MD5 b4610904a5da34a10bf237c410817033
BLAKE2b-256 fd8f1dbeaf8b2a2c00e5172d8ed000fba94edb1d424fd50dcbdcc755fbf3c0aa

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7c7d18b733be6bbca8a55084027f7be428c947ddf871c500ee603e375013ffba
MD5 f13d13c8f2c6f3cf2eee9a43b1c253cf
BLAKE2b-256 2e4f93ccada67079065f892a2c4e7159caf0ce65084fdf60253815ca964403af

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6a9b648a58281c4672212fab04e60648fde574877d0139cd4b4f93fe28ca8944
MD5 30c910634f4460ee96da6a2c189af2ad
BLAKE2b-256 a52c9c2f7a0601cccc8cf169006873ed7775ad76804e98b7236d1f345faf69f8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 7ca22bd897537a0080521445d91a58886c8c04084a6a19e6c78c586e0cfa92a5
MD5 868ef9f3a7b1a65543ebd48394df6f28
BLAKE2b-256 c993a2916bca30301fed0f388c2ff4aedb3a6539e4323a645b2b01321e680615

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 cf27fd43472e07b57cf420eee1e814549203d56de00b5af8659cb99885472f1f
MD5 8ee70562122e5f6dec188a0e17369d80
BLAKE2b-256 94b4b46f929c65fb1bb7371300ce7ad027eb537c3871e850a7b9a92cabf3c03f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ea09acd2fe3324174063d61ad620dec3bcf042b495515f27f638270a7d466e8b
MD5 0023cfd2c374ffe9b1fd6f1f6682c292
BLAKE2b-256 94d2656ff3cf666c547aaf72c7f160465603c744adbdbfeabf274b63843a1fdd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 de19c4dc503c612847edf833c82e9f73cd79926a384af9d801dcf93f110cea4e
MD5 9fdf08becc29822bb51d4687d5dcb1be
BLAKE2b-256 234b6b4bdf41b9777cd07f185c6ca015ca62d8fbccfac0e23095da3ad1315af7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 64c86e5e068ac8b19204419ed8ca90f9d25db20578f5881e337d203b314f4104
MD5 d5ddb7a81f030b0d7c9b59b7f12b20c9
BLAKE2b-256 e30c65356b710dce071234ddab78fd0f530ef318aa700e13b4b687eb738f21bf

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4b19a808d8799fda23504a5cd31d2f58e6f52f140380082b352f877017d6342b
MD5 52d9d3061b4fc7df8c94e25975b55a02
BLAKE2b-256 e7d7bc1144ba365c1929661db8c5c729addd1e9ef029106d2bbac83970585755

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e2ea752f2b0fe96eb6e2f3adbbf4d72aaa1272079b0dfa1145507bd6a5d537e6
MD5 a9cad5ef1d879ab380f40767ef6798d3
BLAKE2b-256 be53a104d802030e74cb88e3673a23c629518c5d3a585f485ac0d06d2c61fcef

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 37aaec5a52e959892870a7c47cef80c53797c0db9149d458460f4f31e2fb250e
MD5 02c6bafb4d6c97baf0264173070e8d91
BLAKE2b-256 31a3707e960c1ed0a6b85fef6abde8d0fac3283ddfa12e402116ab822e539e7c

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 d88b96ff0fe8e91f6ef01ba50b0d71db5017fa4e3b1d99681cec89a85faf7bf7
MD5 849ebfbb03aae7f1087d8d51326cce63
BLAKE2b-256 f25e748aeef53af4ba04f3ef31cb93fd9e4bd05e78985eca052d921317fed498

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b10122d8d8e30afb43bb1fe21a3619f62c3e2574bff2699cf8af8b0b6c5dc4a3
MD5 784e44eb90903d162580996c2401bd5a
BLAKE2b-256 5c0835af33e7a94da55ef6c56902e160093a74188c08d371673ddcc0c3f52b1a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f1b3b31884dc8e9b21508bb76da80ebf7308fdb947a17affce815665d5c4d028
MD5 82a820ed34d506b3886d347f9f286d71
BLAKE2b-256 e96948ddd864b3486cfa72a893ab7a66a50171be27eec872fe4da6c0034cffc4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 1cf75d32e8d250781940d07f7eece253f2fe9ecdb1dc7ba6e3833fa17b82fcbc
MD5 6510626412edf1f78d6366b67a6a6855
BLAKE2b-256 8d04460161657e342a84d835f2ea048f4cfee4798bc45d7d645ff6b9d9d60aa7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2277c36d2d6cdb7876c274547921a42425b6810d38354327dd65a8009acf870c
MD5 096b3fc3f39e734f7edc9dbceb600c86
BLAKE2b-256 1622879a69dce4110fa7f7fe2203b4f24c323e7d95a94842aee9999c3e316b9b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 fb16ba563d59003028b678d2361a27f7e4ae0ab29c7a80690efa20d829c81fdb
MD5 1644aa7b4f6ebc85d083acabd0723823
BLAKE2b-256 3893ad59fecdd7a509c871f74bbe6f0636c198916798e7f8d1f762da6343dd5e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 32e16bdeffa7c4f46bf2152172ca511808b952701d13e7c18833c0b73cb5c23f
MD5 a906b8311af153c6fd7582c2aa2e438d
BLAKE2b-256 bd8b556fcbbc257d026b0654646266733e8e8a3e4abbf20cd8ef54a2c8c47894

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 dc3ad9ebc76eabe8b1d7c04d38be884b8f9d60c0cdc09b0aa4e3bcf746de0388
MD5 01ee4581e43fb99dcc695071a83fa616
BLAKE2b-256 c3380432adf9537d461ee4a75f45b592da186093d59c48680e5d851747bba2a7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 02272fe48280e0293a04245ca5d919b2c94a48b408b55e858feae9618138aeda
MD5 b9673cec2218e1806075ce052befc7de
BLAKE2b-256 88d573d72589333e860aa63e7373c1c8fc2d19dfecdbcbbca00803533ce68e2a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 c1257f4394be0d3b00de8c9e840ca5601d0a4a8438361ce9c2b05c7d25f6057b
MD5 f15d643fec0055afa3bf79e9d6c53776
BLAKE2b-256 9db162f53d900bf620896bf7558271bb8dfd564b945abab14f1e94e345e039a3

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 64c35e09e9899b72a76e762f9854e8750213f67567787d45f37ce06daf57ca78
MD5 98e827cb94c9ccbb8fc20c66612afb59
BLAKE2b-256 2a2a0e324bf51ced98b9fb1463756775cd9ac0e8cda1b7d1b4b0388cb6db1614

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 708bb3e4283177236309e698da5fcd0879ce8fd37457d7c266d16b550bcbbd18
MD5 b9e6b66c32ec1755aff7322cbabc45ea
BLAKE2b-256 52fc54f39e68ac23b4dc0658222acb787fa0d01f0aabaa32bfd9802d785ab2b7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f33dfbdec3784093a9aebb3680d1f91336c56d86cc70ddf88708251da1fe9064
MD5 bff2ec341f5965cdc233c7bcfe36766a
BLAKE2b-256 04c68818b867611734889cd8faca1153ec5dbdd59c98e85e5f6980e7be338839

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 7c0d8b52664ab2d4a8d6686eb5effc68b78608a9008f086a122a7b2996befbab
MD5 3ab636550b9c1578b6e42852bfd3bfa9
BLAKE2b-256 9353ae4e5e49bdc61849b668263a1a4c398b4e33aea1bb9b0a59c9677bb5266b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 865c60ae6eaebdde7da66191ee9b7db52e542ed8ee9d2c653b6d190a9351b980
MD5 ac232976b378707008bab59baca3ba56
BLAKE2b-256 36ee2055fbeb590719393d29cea3016491fd3a6da10598541bff256cc3750349

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 968fa1fb3c27398b28a4eca1cbd1e19355c4d3a6007f7398d48826bbe3a0f728
MD5 0ffec88b53ffebe80ec3bb3c61b90fef
BLAKE2b-256 ed3bf9df83311475e456473958cce65a3709f07a1d1dd8ed046d4779ec4336c8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 473c83c5e2359bb81b0b6fde870b41b2764fcdd36d997485e07e72cc3a62264a
MD5 b6bd2e374ea273d9d6750aac107adbe9
BLAKE2b-256 d54c15b66eb6a47dc9345192aa77988655830c1ebd1306d2b894ecd28fbfbbca

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 2735ecbbf37e52db4ea970e539fd2d450d213517b77745114f92867f3fc246eb
MD5 a85a3039c0a404dd10167994e17d8322
BLAKE2b-256 53cd6ffc60fbc5eae02629d736d578a7c5ca5c20b2b84e9866d61a0c6395684a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 620beacc3373277700d0e27718aa8b25f7b383eb8001fba94ee00aeea1459d89
MD5 da25d9ac40e638950b2aa889a12e1616
BLAKE2b-256 008b21600349146d9fa4d341c507faf8d11b7292b7f29f8def440b81e65ad1ee

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 936bf3842db5b2048eaa53dade907b1160f318e7c90c74bfab86f1e47720bdd6
MD5 e861d8e0796ddcc21a2ff7bbe60e2d57
BLAKE2b-256 6b6cae2437a3e233298a962053c62b943ffabb38627fd6787ff8da62352333fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9c861d35e8286a53e06e9e28d030b5a05bcbf5ac9d7229e561e53c352a85b1fc
MD5 d2a3bbd388dc44e612b829bc7d875d72
BLAKE2b-256 6cb46cc305767c9b1b97b8f5bc61fc472abf42b24ad39388e8f0c57250a7c145

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp313-cp313-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp313-cp313-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 38bfb0204ff3246ca4d5e726e8cc8403bfc931090151e6eede54d0e0cf162ef0
MD5 747df6eb146e21643fa557a44f774019
BLAKE2b-256 6d04406f35822d785ccdcd740f95ba58515c739b6d57c05dd278ee64c70d1565

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 38d7ab43c6825abfc0b661d95f39c7f8af2449364f01d331f3b51c94dcff7221
MD5 bf72ff506206d298b87250333e25a9dc
BLAKE2b-256 c9ea5800f4941a713b2feed955b6a256aacc1ca68a6699916d2668622c075d38

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp312-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp312-none-win32.whl
Algorithm Hash digest
SHA256 b8fcfa81bcb9447df582c5bc96a031e6df4da2a774b8080d4f02c0c16b42be0b
MD5 adb853f9ff7ffd06ad5deba2ab7af43d
BLAKE2b-256 bfe1aed3bc98785c54bd26bf6dd3d2f54cc00de33e8b1f922a23131372eedec8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a308a607ca9de2c64c1b9ba79ec9a403969715a1b8ba5f998a676826f1a7039d
MD5 82c94e7bc91e077bbd32df6e4674423f
BLAKE2b-256 07aa66e8a81e07a791ca6ee9d74ee6de1ffbcd3985149f13aeb530bd409baba0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 a33ab881c8fe70474980577e033d0bc9a27b7ab8272896e500708b212995d834
MD5 2cae6996fee3e724300c5f59e770acaa
BLAKE2b-256 2edd7b8da304d152bb46f13bc2ba5bd545480ab6ce39d94a53eef07f7624d235

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9e0480c452217edd35eca56fafe2029fb4d368b7c0475f8dfa3c5c9c400a7456
MD5 6f602f0e712110ada7119175f1a9b20b
BLAKE2b-256 f6de3707df0c1d7bf55e6a4dba724700353bfee8e292fdd8ccfe93416549124d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 3f5e64b0389a2be47091d8cc53c87859783b837ea1a06edd9d8e04004df55a5c
MD5 a44b51ec21445f161ba61e80b9cd2419
BLAKE2b-256 6c16dad7b4aa6e34a395aef7ae7b010d8b5ebefdf3df81510de53d7f17d2f0fc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 db35825f6d54215f6b6009a7ff3eedee0848c99a6271c870d2826fbbedf31a38
MD5 a6abb9abb5ff569401abab36058e9f24
BLAKE2b-256 6fb28a965abc17fff309eb06e98ce429a19a5e04f731a669a6113b9e182f8a79

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 0143e7d9dcd811855c1ce1ab9bf5d96d29bf5e528fd6c7824d0465741e8c10fd
MD5 3f6afcacecb3424dc1e1cd8464168304
BLAKE2b-256 d642340b91f675b494c4ecc0a256c5dd88b4003dbfde05afff90b970738fdfb4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3835738be1de66624fff2f4f6f6684775da4e9c00bde053be7564cbf3545cc66
MD5 13fb5a6ff33ba47d98bfdad4ae789950
BLAKE2b-256 c0aef6a974be9b2e1615f3de3cc9e4fc2897a86357400801c58143c67cbbad2e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 d0222c5b7c9b26c0b4822a82f6a7011de0a9d3060e1da176f66274b70f846b98
MD5 b0ccead06984290b45bc3da3c8ee1007
BLAKE2b-256 82cc29bb3a25c06b90ce82bb20ef074011481de5c44413a1e1eb10cfd93080fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0774bccc6608eca23eb9d620196687c8b2360624619623cf4ba9dc9bd53e8b51
MD5 ce3ecbc9867f6990c6790c92eb6f7be0
BLAKE2b-256 f9995a55a9b6e2db274c0969ad57d989d02efae90f9e558983a561c9b2b7ea1a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f86593c18d2e6248e72fb91c77d413a815153b8ea4e31f7cd443bdf28e467670
MD5 cf8e794a0b68fda3e4d6d3f99fa6a081
BLAKE2b-256 aecaea4b5aa70d4d26f2d05620c265b07b5a249157767c1673f5753b8bfc7db1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 cc90102ed17271cf0a1262babe5939e0134b3890345d11a19c3145184b706055
MD5 b948e353444c1c8c7de1b04d0f705927
BLAKE2b-256 c10240725eebedea8175918bd59ab80b2174d6ef3b3ef9ac8ec996e84c38d3ca

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp311-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp311-none-win32.whl
Algorithm Hash digest
SHA256 6d76f00f5c32da36c61f41c58346a4fa7f0a61be02f4301fd30ad59834977cc3
MD5 4b8e3049c0220c8fc7313cd90854191e
BLAKE2b-256 d154451e96d8514b1afbef955f7420e1180e015c3f4eb085ad38189c0e83ee87

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 0ff110ecc57b7aa4a594396525a3451ad70988e517237fe91c540997c4e50e29
MD5 1e1ceae64f859c059ff4a257acdfdddd
BLAKE2b-256 c0611c26c8e54af9bab32743e0484601a60738f33797f91040da2a4104f07e70

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 107089f135b4ae7817affe6264f8c7a5c5b4fd9a90f9439ed495f54fcea56fb4
MD5 49792b750d5082f4ef6cfe8709933fdf
BLAKE2b-256 073bd8e60712e509a6f5d01bf0eb4470452b72277be4883656206d4ccd7e02de

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ccd73a82751c523b3fc31ff8194702e4af4db21dc20e55b30ecc2079c5d43cb7
MD5 aa0b518148c6c65fa6ea255fc4836757
BLAKE2b-256 150bc09b2c0dc688c82adadaa0d5080983de3ce920f4a5cbadb7eaa5302ad251

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 4620cca5c2817177ee8706f860364cc3a8845bc1e291aaf661fb899e5d1c45b0
MD5 0c18f7af2e8585e71012eb54cb229c4e
BLAKE2b-256 e539facfca8e598126a0001d4295e6b1ee670d241aa6f4fcdd97016065b43c5d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 112a1dd436d2cc06e6ffdc0b06d55ac019a35a63afd26475205cb4b1bf0bfbff
MD5 de778be2466fe98bc928f640acc7d516
BLAKE2b-256 e7d14d319a035f819af3290ec5a09482ad659d9d2a0aea33890fb5720ce81841

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 48e2b9335be2bc0171df9281385c2ed06a15f5cf121c44094338306ab7b33f2c
MD5 b0f8547bb005b933c6e5d51b1beb37c2
BLAKE2b-256 34d5e1ad46939d6de48d41bbd8b302f87ecde79847855210e75517a832b29490

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 20ea60479de6fc7b8ae756b4b097572372d7e4032e2521c1bbf3d90c90a99ff0
MD5 98725694a931de068c85056a1211337c
BLAKE2b-256 0f23e4985657ea42ad432d6dc2100b2687e70a6bae730f1f8c52f81d9e6ccf3a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 a907d76dcfda37023ba203ab4ceeb21bc5683436ebefbd895a0841fd52f6f6f2
MD5 0469f6b28adf9a7a534db48d7aed2703
BLAKE2b-256 e913b86ea87b7e3b4a2ca154220dc4eb19a56a3864ec03e9630d15d1bac10da1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cfed5c64e5be23d7ee0f0e98081a25c2a46b0b77ce99a4f0605b1ec43dd481fa
MD5 2ea7007b7dc5b53292b9f320e9828583
BLAKE2b-256 5f4fa4c12cc058a899c1caaa1e689c3df9a698e20e891d4005aa6ec2174a9339

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 89cd1cb93e4b12ff39bb2d626ad77e35209de9309a71e4d3d4672667b4b256e7
MD5 430c7e7efc895fe8526968381620f14b
BLAKE2b-256 7311933d68d395f5486d935e1c15da80bc96bf3f48595652069d19e0e9894386

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 3b919afe4df7eb6ac7cafd2bd14fb507d3f408db7a68c43117f579c984a73843
MD5 0a0d9a7c221b105a1946f850bf86c620
BLAKE2b-256 c9870bf37626c5f1ea2462e0398be88c287f3d40c696c255ba478bf525bdc852

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp310-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp310-none-win32.whl
Algorithm Hash digest
SHA256 064ff87bb6acdbd693666de9a4b692add41308a2c0ec0770d6385737117215f2
MD5 12be8eafaef03412ebbabf0ddada8358
BLAKE2b-256 abbfa804747020f1b221131b74b5f29c24b47a5d2cee4b1311ce394ca9ce242a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a47acfac7e511f6bbfcf2d3fb8c26979c780a91e06fb5b9a43831b2c0153d024
MD5 0c7aed98c09dfb04c3e97a56e3f13326
BLAKE2b-256 6a0c3435e3d54f825d4fa363a7ab2680b243314377eb2ed28e87ade70b861e7b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 5ab2a4d21dcf76af60e05af8063138849eb1d6553a0d059f6534357bce8ba364
MD5 0108ff54a35119cbb63797a479ecdc4a
BLAKE2b-256 aa0bdd9e5124fe73a01f36f5c7554ac97b9612af5e0bd401d6a606a3f52a060a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 40b6a4c78da863ff26dbd5ad9a8ecc33d8a8d97b535172601cf00aee9d7ce9ce
MD5 16eb471a8c91c2da906c342c075e3c14
BLAKE2b-256 1c5dcf5e122ce4f1a29f165b2a69dc33d1ff30bce303343d58a54775ddba5d51

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 8b9ec69247a23747669ec4b0ca10f8e3dfb3545d550258129bd62291aabe8605
MD5 0c233f26371601940062b7549be710b6
BLAKE2b-256 fa6b6b757cf6f7c30009a6759d3f7b833d974b3cd50d24d5824c695e077cb1bf

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 44f2a832cd0825295f7179eaf173381dc45230f9227ec4b44378322d900447c9
MD5 d32b1e619a537980fb802e5774abfaca
BLAKE2b-256 dcd545dd421f45b3c1a446ffd9486cef29ed568b5978f66a1803fa46a44aa9be

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 cf7fd9a5141634fa3aa8d6b7be362e6ae1b4cda60da81388fa533e0b552c98fd
MD5 6adb221c8b32f65bd7c1f060b3dd9bef
BLAKE2b-256 5f1b58e77f2b57651e8c1b4f1b7144a1250509f2e7a1f55073d12620968ae4bb

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ea621a7eef4b70e1f7a4e84dd989ae3f0eeb50fc8690254eacc08acb623e82f1
MD5 8eac98a9e890f04dca84be9463ebf210
BLAKE2b-256 31b370d3fe0ad25e065322cd902624cad4ff2647484fe823360f58af6927b48c

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 a9b9b070fdad06e347563b88c278995735292ded1132f8657084989a4c84a6d5
MD5 3f8b68aec9d8201089eebfd8a1d18f40
BLAKE2b-256 ccbf819bf4445ed68ffaf73b0f6245bcbd21a5cd58e86dabbef315a6d0b707b3

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 054c1cc9c6d68f7ffa4e810b3d5131e0ba511b6e4be34157aa08ee54c2f8d9ee
MD5 5aea89952a503ebed1fb3a444e2b3501
BLAKE2b-256 010445d88b8bddc09bf56ae1631721393255b75798af515c65c26389713a2072

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 52f6130c9cbf70544287575a985bf44ae1bda2da7e8c24e97716080593638012
MD5 f79792f3b0022e1389c1e7b394bfe011
BLAKE2b-256 a86e489419d98730b3d02381f10a8b97c5bf55b45742d1b347cdd0ffe267b827

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 524e60da0135e106b254bd71f0659be9f89d83f006ea9093ce4d1fab498c6d0d
MD5 445795226db89e7f5a645562010d8658
BLAKE2b-256 eaa5006f9b3133e71d5b41d6455dede48731c22d3ddf5371167fb3c28847581e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp39-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp39-none-win32.whl
Algorithm Hash digest
SHA256 918fbb0eab96fe08e72a8c2b5461e9cce95585d82a58688e7f01c2bd546c79d0
MD5 0fb11079cdd57fe0b36495b5c300abe9
BLAKE2b-256 51606a1d7ea4d4095753db78c250f721b9495953eb5ec88ccfc3b58604ca6544

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 c9a2ebdd2ad4ec7a68e7615086e633857c85e2f18025bd05d2a4399e6c5f7169
MD5 030ea671b5f30757f357b6f096a4bb23
BLAKE2b-256 ce7a7f260655fb27b0d47ec83239535066f5c76583c7ab4e55a71c5437f7721f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 2ef09bbc16519f6c25d0c7fc0c6a33a6f62923e263c9d7cca4e58b8c61572afb
MD5 53d0731e51f73572915fcc2684c556ae
BLAKE2b-256 d21a2d2470b36d57de78f2c2ceb3cc93d1a770f02d3a1a21cf2312b81519a6ad

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0cf6b7f1d4dc59af960e6ffdc4faffe6460bbfa8dce27a58bf75755ffdb2526d
MD5 b73c566e3af8b6c69c433feba3a28abd
BLAKE2b-256 a5bcec39dae3b0ea00724c0fea287091d62b0ccaa45c7a947004714e882d193d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 454c203164e07a860dbeb3b1f4a733be52b0edbb4dd2e5bd75023ffa8b49403a
MD5 55f9ef838b24935ae72d4cccd957a1a3
BLAKE2b-256 a29865bcde2920c022feac75ee282b3041f4d12e51c2cf955ce42a928410e591

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 3ef5dd1d39797044642dbe53eb2bc56435308432e9c7907728da74c69ee2adca
MD5 43e7827c21ffb8ce6227f6e573ee5802
BLAKE2b-256 b15049643c82a9801e09bb0d0ef9ce9cd17c0f91651cba96399af822a882a9ea

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 1ddba9a2b0c8c81633eca0bb2e1aa5b3a15362b1277f1ae64176d0f6eba78ab1
MD5 e58f8e27444daa0514c19acb4c399b23
BLAKE2b-256 eb7af0ab9e5d0ca5619d37c65e3a474d3e41cd336c939fa79c0261ebaacc387f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 158be8ea8554e5ed69acc1ce3fbb23a06060bd4bbb09029431ad6b9a466a7121
MD5 b253ce836469f39bb6b9f4b077126f9a
BLAKE2b-256 60b5b80ea7034527a268b9f8aa84c6ae9ec99edb7188aa8ffc18a42fb8546596

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 c9a09cd26cca2e1c349f91aa665309ddb48d71636370749414fbf67bc83c5343
MD5 aa389f5c699024a6911bb9cf5388736b
BLAKE2b-256 0b0b3740e5ff34da240ecef9026b0f8625edc896421fefb99bf6a8d3242f0181

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 02458bee6f5f3139f1ebbb6d042b283af712c0981f5bc50edf771d6b762d5e4f
MD5 baacf7afce57e2a9d5c67407141880f4
BLAKE2b-256 aafaf75d1c113256968833768391916ef385792409b307ae5581b7476c581963

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 82f8652a74cc107052328b87ea8b34291c0f55b96d8fb261b3880216a9f9e48e
MD5 76f35632f3759a88874ecb30a6cc94c2
BLAKE2b-256 925c7ee7e297ad9dbdcb20caae1fb506efada72dd4c7dd3aa000285901ca1c87

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 5179c271aa5de9c71712e31cb5a79e436ecd0d7532a408fa42a8dbfa4bc23fd9
MD5 4413481dd830508f9495a079d08ec729
BLAKE2b-256 96ada8be3d968937352abf2856ad4140e00751a463c915cfbc38ef25881a07ec

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp38-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp38-none-win32.whl
Algorithm Hash digest
SHA256 06cd0487b1cbfabefb2cc52fbd6b1f8d4c37799bd6c6e1641281adaa6b2504a7
MD5 1d0f3d96501fea183e7c19f27d447596
BLAKE2b-256 0c17b8414b48d248669527d3c5f18c3dfadbd0c9e345a80578d9bc08ef6e15b4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 41e39b41e5531d6b2122a77532dbea60e171ef87a3820b5a3888daa847df4153
MD5 287e5c75edaecb131c5a396679705db5
BLAKE2b-256 9092c0b5a32ebb758a57d04ba7bb75529f0fdf5fa3cbc296f61f72c7a95cbbf7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 6fc7083ab404019fc9acafe78662c192673c1e696bd598d16dc005bd663a5cf9
MD5 b92838400b6b66f918502489a1891d38
BLAKE2b-256 c0b53fe0c3e4a6c523f3e90f9d6352fa94d33940902eb7d5ef75d721e7df883f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 93268e788825f52de4c7bdcb6ebc1fcd4a5442c02e730faa9b6b08f23ead0e24
MD5 78794139f536a04cc54b26bda328b0ac
BLAKE2b-256 0ed73220a4483d240180d0b9423206cc57a4997fd4b49a8393e5ce9a2f7908a9

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 c2d60f5246f4da9373f75ff18d64c69cbf60c3bca597290cea01059c336d2470
MD5 ea433cd207d2aecbea3715301278ac14
BLAKE2b-256 45f74e0cfa7e8f2c747c94a391226ed7a5022a6a1f97c81a2918f75deca4520b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 494fdbe5932d3416de2a85fc2470b797e6f3226c12845cadf054dd906afd0442
MD5 0ee14fe47501295ef3d99f1c4e837020
BLAKE2b-256 2e2dc391cfa63e71744b5a111a5e539aa8ec7324e0641ce548bbfdef29bdc5fc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 daa348f02d15160cb35439098ac96e3a53bacf35885072611cd9e5be7d333daa
MD5 3e4eea96d1ab9ea13e112a0f93bc6599
BLAKE2b-256 c38430ad0362f65975feebdc1e9f55d7b0526783035daded63d0957cd4796432

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 508711a108684111ec8af89d3a9e9e08755247eda27d0ba5e3c50e9da1600f6d
MD5 99d2494b361a70be709a2b5a2d4ef263
BLAKE2b-256 7df5b726968de76ba440e88da9bb0d25b2d463ab999a973f055221c2a5a03d29

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 d05a1b06f986d41aed5f2de464c003004b2df8aaf66f2b7628254bcbfb72a438
MD5 501199a53a3564fa1c5a6f8a3e6df1ed
BLAKE2b-256 91471021292967eb9ca70de024d4e958ac613abf308751ca5a1384af181bab22

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4c4b89038a684f40a6b15d6b09f49650ac64d951ad0f2a3ea9169687bbf2a8ba
MD5 9b73a4b9d2d009ab0b9f43055c754768
BLAKE2b-256 0b8a208268ea0aa817ce0d1c1d219935305b1cc18d90f3a44c5f3e25dd7fe8fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4fe1f74a902bee74a3b25aff180fbfbf4f8b444ab37c4d496af7afd13a784ed2
MD5 b135d145f38604f7d1ef5ccfe2f52808
BLAKE2b-256 70a9a545a2a4797df564b17219a86bd47f050a5b0a2dabb42e6397646529e27d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 7ef789f83eb0f9baeb4d09a86cd639c0a5518528f9992f38b28e819df397eb06
MD5 a3f464053fd3824dd112dcb7191f24c4
BLAKE2b-256 a87ba87201da1e3b0b7e4db4a0f24397e7dd7a9f37f49ee13da8364f0aca1213

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.2-cp37-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2-cp37-none-win32.whl
Algorithm Hash digest
SHA256 361abdc068e8afe9c5b818769a48624687fb6aaed49636ee39bec4e95e1a215b
MD5 fe77223191407f86a819a5da90652370
BLAKE2b-256 e49238428fbca98173acb5a55385d01bcfaff64d48bf930b7ba354927b0113c4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 05a77cbfebe28a61ab5c3891f9939cc24798b63fa236d84e5f29f3a85a200c00
MD5 c31fc0b4d8b7a303063f2bbb9e8ae6a8
BLAKE2b-256 46e7948dc1ae6fc508818a1f4423026ac44591e5820cef0582c55df60428ec53

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 5645938a42d78c4885086767c70923abad047163d809c16da75d6b290cb30bbe
MD5 18ec9caa4db5ee86c15d2c38cd2692d8
BLAKE2b-256 3928b4a4e4a2b0cd5af019f75137a8df676e9ae55ce79a4ea75b7be80bac35b6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f3f40604f5042ff210ba82743dda2b6aa3e55aa12df4e9f2378ee01a17e2855e
MD5 fa173567fbaadebc957d937382cfccff
BLAKE2b-256 b31eaaa91c6a80405d83aca79d688bbe37af996674eea4e3dd263b40582d017a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 ccec77aa7150e38eec6878a493bf8c263ff1fa8a62404e16c6203c64c1f16a26
MD5 fa0ee854daa45030ad2cb85089db257d
BLAKE2b-256 44be56e1a36e9c72aef097b65778e470993604d699d8007860ca6dbe65c61337

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 9ba9f6895af58487ca4f54e8a664a322f16c26bbb442effd01087eba391a719e
MD5 95bc8ebdce9df660452ff5a0a5bd4247
BLAKE2b-256 d6dbaa6a98680c7eeed96db984f7eacb311f18ce5df47f504916cb5edc989693

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 4eeb12daf02a59e29f578a865f55d87cd103ce62bd8a3a5874f8fdeaa82e336b
MD5 2ad4240283a10ba020a88b1914fde257
BLAKE2b-256 ece701df51af8484d6a43582482ac200866d83369f40057a16ed1e65d8b99165

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 c73e2e74bbb07910da0d37c326869f34113137b23eadad3fc00856e6b3d9930c
MD5 a48cd1ee5fce4f656f56c73f0628cd23
BLAKE2b-256 077a6a13d9a6fe9ee0bc4de406359cea5fbfeaa228a9e49ace9d00fad6ff083d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0ef06b9707baeb98b316577acb04f4852239d856b93e9ec3a299622f6084e4be
MD5 a627ba3193178f2864d0329c504115f0
BLAKE2b-256 4120fc359822285d4dcab6023fcc0ea88929fa06b21e16ade6d41a5bf32ffb2e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 dce74266919b892f82b1b86025a613956ea0ea62a4843d4c4237be2c5498ed3a
MD5 f694304126d8621d6be9ad07e1827fff
BLAKE2b-256 640a62434c607bd9a116a33c7be85d6acbda62287ce1b42b35e9dee9eb6905b7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 d44ba80988ff9424e33e0a49445072ac7029d8c0e1601ad25a0ca5f41ed0c1d6
MD5 13f7f79eab70e267ccdc3b98c23d1343
BLAKE2b-256 a8b26ffddf6aafa345617bc30d18eb076e80aa7f6dd7d8471418f315ea935903

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page