Skip to main content

No project description provided

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.15.2rc1.tar.gz (320.3 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp313-cp313-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.13 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp313-cp313-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.13 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp312-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.15.2rc1-cp312-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp312-cp312-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp312-cp312-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp311-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.15.2rc1-cp311-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp311-cp311-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp311-cp311-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp310-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.15.2rc1-cp310-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp310-cp310-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp310-cp310-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp39-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.15.2rc1-cp39-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp39-cp39-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp39-cp39-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp38-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.15.2rc1-cp38-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp38-cp38-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp38-cp38-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.15.2rc1-cp37-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.15.2rc1-cp37-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ i686

tokenizers-0.15.2rc1-cp37-cp37m-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.15.2rc1-cp37-cp37m-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.15.2rc1.tar.gz.

File metadata

  • Download URL: tokenizers-0.15.2rc1.tar.gz
  • Upload date:
  • Size: 320.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.2rc1.tar.gz
Algorithm Hash digest
SHA256 1530c1480158827377cd3d219eea527844a9eccf54ded86e15dc1bf3ca5fedf2
MD5 744cc2ebff83f8709f81359c1ea4709e
BLAKE2b-256 1d1e9fa9dbc127579fe210c5655040d86ea487a6b43a0049f47a31773e9037a6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 06a5dd68027bfc62dee01c2fff079e6175cd61ed699af3c34a84686ad88f5660
MD5 d7d7a58408b305ad13e9e61fcd638ab4
BLAKE2b-256 8494e5b43644d8f191fa059b91a3da91a8d31471bb7054a7458d1abb9b9371c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 a8ee02dbb7f407b19552581fe82e8a360f4300d66cfadc6302d2a6e415208af8
MD5 6757128c66fc3fe01480f6c675cc3202
BLAKE2b-256 8c2199c19c67ece2ae854f02a5e1f1c8c557093028e04f84ad1bbe8e433c404b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 dce952dbc65188a4debc156fd751e702c3b5da88fb8a00f55ca37f91541f9863
MD5 64da9d56c6f808c36177fa760805ae47
BLAKE2b-256 1ad442a081a0870c767659115b2f77de001554fca3efd7a4c5879272c199a9b2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 d9f9cfd4da27c8048496941921929afd1273d0a4ac9a727e5f81b583b94099c1
MD5 2bba1ea4206db7e842e3935e47b1876c
BLAKE2b-256 c0ac6b30609bbd94e8cd4910bafa14ed041cacca9b67dc12ca581ea9bf3fda57

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 47d0d5490dc7688cc461b06cbe762c597ede3edcac9f6b2e9e6fb5a65e817822
MD5 a42e21c419440a8319fc45eee65cde7d
BLAKE2b-256 093d2a0f8eea09bcb7d616a2c5d45aaf459185adc629c28dc0d80c29119afe43

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cd4cae3a8ae78032c4dfebf566285aac2478bfe92eb2bda8e9188ec16f1c1971
MD5 da34fc0f6a91dd0558f6669f018cd3b8
BLAKE2b-256 854e39f81ba3eb325dbeeb314e3f31a88189d96432fac11fa634211c0d51d3d8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 9abd3d335b7f15197a5a9e64af95e73653f78fd386ccf69818415a79367ba3eb
MD5 c29dce6f2d5ea60b1a18643b5b441bd7
BLAKE2b-256 7d0f361b75888580890350aa5643f0c1efd936680bf228c063bd02b32e7553aa

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 270a702f82f00640bf79f4a44f739d923e792b89fd00f3d7d52a5b9a95b5d489
MD5 bec9341c4d55e8be360867792a2e57f4
BLAKE2b-256 a38ff9602ddf821256efafa246d8f7ed6ba7ed734b3d3deb2bcf5c8020738862

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 4d4ad48a202f26f42f372dfb61976ffe5f46449b8629f46903e3d25862680ad7
MD5 71cd2f38d6ff1c403e1ee011ff00f0c3
BLAKE2b-256 745da16b35834ffa947eea64cbb66bd1f795ed7d9d787786d720e34016b7ff25

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 232b1ff659568e6c75d78ba08339a2be95f55d4f65858b06b3e131399afa8a50
MD5 0db7b1e937d4d6f4c7f9be9e72cbf717
BLAKE2b-256 0242d1bf1a378ce511ba259a3a83f210a8efbf62fcd8f82df266c9b8f90119b8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 eb502fd24ec3699afc438e04000f6e67051adc0e689db56399a9a36c21d85bf4
MD5 f0740792435d4eed6b4aabbc6096dced
BLAKE2b-256 c8e4370b0221c4e3b990334f5fea354b5d6872b721d2218a89ca00f019b7e317

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 047cf7c2524f56d56655a6492e0900073352b4ce4c33245b88acac4092bc458c
MD5 3bdcb2baf1f381b34e9b92d546a14a82
BLAKE2b-256 ffa109e77efc3d899848e6b59bf9bdf522c385b2e253b6a8cbac06f3cdb9f305

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a2a70c8977e7fbcfb968d957ec516e21d67cb73810e7d6fa17b0815e3c5a6f09
MD5 4ae87c00ce71a1118bc123c9f8a1ad11
BLAKE2b-256 9e64ac1b302f8ef79de3270ede95f8414e264262c582bb18eecd6057c3efb64b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7bfb56becd15b77e16af0580924cdf87637db7bae309f5ebe832f6dabfa4a84f
MD5 4f479fff7dad73fb16f8c905dd116e9b
BLAKE2b-256 ff167105ac4cc426a2c342a37852885c1aafd9b25bc52fa8bea100b5fbac55fc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 d8780bb20eedbb6125cbd98c426223d992e8024b9b3172530a37e05e4d97f041
MD5 e0c3aa38a4d5d4296199016c8750d85b
BLAKE2b-256 93da8e215a847408682c2bdeb4f4e40098eb673becde48a71ac21c8195e94b9e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 75532bca192d343ff361f57ab25a8c70b9d4b0570e69c54546c2676096ca0777
MD5 e88c848e2ff87f58421a6ce9dba5b39e
BLAKE2b-256 0d1ac96e6ac3ff5df27caab108d79af09f0fc864f821ef20c4af9051f199a00b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 abd22ffa3fea5df46327057c3026fca80dd5bd59ca8142b42f239f193e14fe3b
MD5 2a469d3ad53b56e7aa8c16a6dc687b2b
BLAKE2b-256 c57d106df87c317d164933446a50865a55d55a068b9c993f0c4587befc682328

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f17cc297eb9fe925ac0c7deb050bde92dbfad5f17d2f58dfa1b4fe4c92222c8f
MD5 6c7edd3d21e892b7ff6399dfc2dd8c08
BLAKE2b-256 9787b344e4629616992e62978319d2eeb527db09e7c251e1fe52dc2f6dd37c40

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 5e60d2189adf49e757085011c66deb88f1a9de25b3d3965bffc60a9c83dea400
MD5 67d85115da24bd5b8824c9ea09c4cec0
BLAKE2b-256 d78bee2414a65a2a731abc9a0c6c69faeceef40d04fc7157cfa50bce3c4b1bec

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 36957cb2b9d1421c5e56d4e99293b47cf8430a399306281561090e20d1662444
MD5 bbe14a727e0e777ba17fe44bbb2c9ef7
BLAKE2b-256 fedc8555abb77ce4f92c36713586f43a568cd6785828f4fe60377965db20f02f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 bc48fd8acb68352ed9a08b4adb772765d4bb27ee0d2e3fb9b272e914e8ad5bda
MD5 4633becdfde1e3a15f09bbaf86ece471
BLAKE2b-256 cfd3b0231f00a4318da6f85940e0b08425e60adae44fc54ef248ae7e3f906bf8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 93c7b00b4dfe898f4f018f1361200e58713a4b745f126ceab9e37605cd11f0c9
MD5 a972e3ee4efbd43015cc0ed453fbe265
BLAKE2b-256 38b83a3f7c77ddba42b729954e726ecb092ed89197a0f8e2b7ca202e0991529b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 d75cd9db052d70fe996a284f4759a457ac39bddb3153da5f2a9e4f76bde9df92
MD5 69edcd5e957fe2d7413c987905ce3cc2
BLAKE2b-256 3f184e532e79e9fb39d666ddb3a9ca451488d2140b81aaf62ad78dda11faf6cd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9953d08595901f3a8cf4927ebb16fcfdb756c49220a7bd028e4ee1a87772c795
MD5 296cc8ded8f1a5c95470eeb307d3a175
BLAKE2b-256 43af235e70376f4877d1751182afa65db81c5830d8163f79e3e510d4e0619c6f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f03791a6ad57d98a51b84f9c824cc3e4a9b7d09821384b793971fcc66b2b7b42
MD5 d75864cc213af2c14c1c43d00063de43
BLAKE2b-256 3b040f7c4cfea471a9cbae93472ad669eaa77145a7fe5f52169aa3f4706733dd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 5e8bd98b60ef014676d4e8d5e9fdd50c1dc20c3247c14a6eb0df617cc075bc72
MD5 7f2e0a06a0008810b4959c8aa475cf7b
BLAKE2b-256 95825295c1d450f2c4c31ed6c63fff3053beafad2d8633e9be7a15fe53281ab0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 9243b5b99c34a88d6f535b3536703bf2c6397f11032c60159a419ef330fa733c
MD5 130b635e959572c50fd462ed9da52c96
BLAKE2b-256 28eb1b69bb10a4c12876eb0aa9800e75372d141c49d056af15958bc1ac462682

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6a255598e3a9affd06e53aa4c3ce3f6846f000e6ac5c037905bec2fa3452e5c1
MD5 1ce4bd9db82cefa295c3add9b123635d
BLAKE2b-256 99c2ffdb0d7c641b7a25b4646c75ef08f90b1e42f3474866f3f83333ad76e829

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 c796adb2ea22927cdef24bffeb0e3f419cc52c6a2b784feb97909f787cfff1c8
MD5 bf8de32b5210667d3f91ab2515618d81
BLAKE2b-256 2efa0022dbb6ac321a7d86bc5f4da0c67d9c75a0bc50c274340ab8c28ddb9f1b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 110800c36c7c031d11d12a0c4eefdb99a9efd846064eb3e98cf7401a5b39fa37
MD5 47b43f10b3ac29665c886a706a828711
BLAKE2b-256 fd1f9ed6ba89942c62d84a33c23d3deb1951821d3b586c8d20b1417360b40aff

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 800faa4e20f74a4913ddef995b2a237b5c48db45b0082d2d703f234f9c09970a
MD5 79af840a194243e3737e0cec80162e21
BLAKE2b-256 bb0b9b6927672fa5f3566592f3212e811428f15f555c7ad119483fcbce138230

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 115aafe1c3a37464a734447b4b51c6f0218e1834ff3d5f7478d776a0787b4e26
MD5 2b5e04152fd358141b92245972774afb
BLAKE2b-256 41eb1fa8982a0a44a6d1cec7d51d5bc18e792bf40227be2c5c13f06509602aad

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 7db34f45727d17007a6685b1d5f928a9645e3f5743c3d5ae177b2bb36407e769
MD5 253df429c6c3c499fafa99488a5d33fa
BLAKE2b-256 000f9cb8c75eccc9fe7c844f4db7d9b9476a9fd3f3d97547ec6eb9d523b3c6d5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 79f1f7d091c5f25ea12bbd62679d49ce98e41c61702346da6166b5c03fa9a0a6
MD5 d0aeddc68e06c0a28158ccf98cffd171
BLAKE2b-256 5860785d531f3c0cf0770f033719e3dc003a8596e3638e26272bbb1afeb0815e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 c721bf52215a1aa8ccaf88a3ad17954a2db0b1022ec115ea585aa73d3e1807c7
MD5 ac534ff07558e6b97a5af8193842593c
BLAKE2b-256 a5970b4771d8d73459fc89215f8d73911a9e9f8dbc0e9ee4da7076082916a4ae

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 726e689deec4fdd2b8abcdb7145e33bc09f9c7106bd007aca59ff69a03aece4f
MD5 1a5fb25ad9249cc0510f626db797b419
BLAKE2b-256 37954ac7f446d506f99c6716cf88f497be08e491c57edbecf935ac417cdfbd78

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp313-cp313-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp313-cp313-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e463f41756492ab61049b459c8cab29d53f38c1d3639a8cf383c3a0bc78fdc15
MD5 d327c51498c8bb0254e6b2c42ee09088
BLAKE2b-256 2ac55b801d4e0e313ab5d76cbc5a06be857509f6e1573cf98cf45def94aec65a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 558572bee9fd98a3d4d88a1e4433351a2f977cb88ae527cce8c06ab51da1c021
MD5 501dc0305f5fb2741dc3faf0d02e8552
BLAKE2b-256 f1c8d034612191bf94b0f18b3e82601361ebc1cef5a036ef2046fbca1bfa9163

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-none-win32.whl
Algorithm Hash digest
SHA256 fe772babd0e4e999b8501113538dcf2bca54afe01537711507c85eef755946f6
MD5 e67007c686b6bbcdc51b8f725079c84e
BLAKE2b-256 81a23715d3eb96f1770395330827e16561b9a1dafb6152092b4704389e95b776

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 5ccbf051b0b2653d331b2b0a135c073919e7010f71cbeacd1da578a441880e02
MD5 e777dd396f36488d16eeb08331b1bf0e
BLAKE2b-256 86a36df929571468f0f5c01842b4ec7c8c1f1507eecd664f1ac7339a0cbf1857

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 c5bc3df1becfd9e8723b41df3cbe1091a917d132011462793e9cdce02746b806
MD5 28b74449106ddf5c749262203d74b78e
BLAKE2b-256 881cb8a1f3a59fd1650fc180f6a5529058bcf69d1051a017bbc228741acc027f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 76064b403eca2ffd968b93e07314cb81ae565b4e577f4188fcac35acf5eedd36
MD5 88ccc276b94e3ece09af394e080e063c
BLAKE2b-256 e83f46b6cb34ac733ecdf43ee6ebd93a2a5bf57a8652aa6418f20abc23b2c7bc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 fb848c702a5a842dc52c6daf337d229c5f84807d224817e8a5b57b105f9d02b3
MD5 5960ee483d43341fda92c4fb3faec9d4
BLAKE2b-256 3fa2df909132cae092ce1d0e33354ee72aae6bb5ec101230b63fbee168e47fb5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 e21515a6c2e98f918a186ff3479bcdcae6787f6a1addc8d555fcd7e79b8efc41
MD5 44b52a200761556498de7a4e19c43f33
BLAKE2b-256 0815eab0de9949bb9e5280d57d063f9f45fe97ca1cac961eb681a18e55bc492e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 babbec5b7d45d9d867f53128c5e1dedbcefab6ae59a86db671cf8010e84794ea
MD5 d077768460872ae2ff7d1dce2c8329c8
BLAKE2b-256 a684b23b6a7ea7cb123b0be13c03bdd89a9d29e356f1a8f12df9cb7a30302367

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 01db11163302aa89c5467db66bc60f32b3995b9aa3975da0e8a034412c057823
MD5 b94e14b1722fb5e3329211a8647ce276
BLAKE2b-256 0d6b9758a32a844a84d2d666eeaa947574bae1061c073212f2a1fa57188b2c88

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 dfce0a44e453359195fd2ad2972a7387c78dc6acf6be24335cf601a85eaf60d8
MD5 72215bcb5be184c7729a30c948c9cb7b
BLAKE2b-256 50489abcaed85c8d9cb6929a9c8dd0ef5832ad50a9b9e88a079f1ce3bd2dc009

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8f717a86a80707b711aa91f497ec5325064291137bd6f7df3cc4ad0f7d7240a9
MD5 d983687b807618a6edc93ab76c026ab5
BLAKE2b-256 d663cb00ee77c0a016fb15eaa4500ea1cc31953d60ee08f0626cdcd28673b89d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 913596f1623825d8d8879303295564355dd58153015db116962bac4b8a135d67
MD5 9164c0c3bcd392275d7ec543c8170c9f
BLAKE2b-256 e73f03b138a345df6376235f4dc16742032d5ad318b721a91bc89eca3ddf6645

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 4109de47bba867a2db6cd42b038f0f4ea1ec251d8fd7314de557be73364afedb
MD5 f6e7c2abbaa38f6fd0d0586ae23bf862
BLAKE2b-256 869c198bbf93fa32ec61ec8927589871bc901fe0e9c9f7a28414d541e7bf369f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-none-win32.whl
Algorithm Hash digest
SHA256 d6050d9cd96eb9776be13cb5bd3f6d9cfd0906c4cf61f2aff29335c045acdd06
MD5 c770330a0d61151e9544233cd7c9b454
BLAKE2b-256 e048159ecf79b7f1235ef047d7b4b367ac677cae097f94c27e1f594d65db8f0b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 8dffd697f9996a91f2a0d5806a1dbb6a503163e8ba83afcdca0fe3c56d41325d
MD5 8ea75fa404dfa5e8ae1977ed9bc8c337
BLAKE2b-256 04e373a646fd10f960d5fca48ceffa32905603a4f02e23cf1ca9093ca315af20

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 2aad14eaf43ef883bad9bfbaaf07ced35e0a2418885d9f7ea5b44bba462a074e
MD5 a1b86d1b737f49e53f44b69c8798bdcf
BLAKE2b-256 5ff8728800afb7c303253b7542583281c3ab88968f4d2757213c49c81d64f526

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b19a6cde6b8442c104362f11a00c30b58052bc84df134c831f5a5218a0b80b58
MD5 689cabb8556366d2d52f60d4969aaa0d
BLAKE2b-256 3e98f4e8232a6568d6a4142a142632f9e0e40df1b6bc51436858d61dd7e48f02

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 79d03441dfcf8960f8e2f2e12d896bafb7fa67effc4b67fef644673554dd4d92
MD5 83405b30b564db162414688547e7df71
BLAKE2b-256 636c66a65062eac281be661af42c056bc8c76358f3fd75d55176266eea428541

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 676221de8abf6cd3386d7aa72af8ea23dde0d43a394b613466f393db31a0db4e
MD5 6208e5b7cc2887b29d8058a374129d64
BLAKE2b-256 9487358e16dad26ce915aceb9fcec289201d8ad38a82e96b043e443e63f7235a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 94a2360419745f31652d6ac17f26dffa2d434c851185113c016a1b8d618e1871
MD5 256e4447c8a7e92514d50f5711e1cf53
BLAKE2b-256 c70f5da111110780a75d5f75e1c4f36be7f9a3a70a495568aea4cb622328c07f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 a880243f72f9514b58ae30976abb1b40fcd159dc3a83b9f0e9fca889547788f4
MD5 577ae68c52e5c66fea3142243a09e37b
BLAKE2b-256 d1d85dff9c322b1508c6d7559b87b1a9dce93d8225e238c29e3afe97c7166d5b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 18b4f3ab46853d4284b6762adc88f3bd177d90f8ccdf39fba7ac7c5f3d10acc8
MD5 789868c60f7ebcef1dd5fe6d1d5ff017
BLAKE2b-256 c7396dbd2aced90e1ca16c6c6b998012a4f7e2590c54fd79c33fc16d6dcae270

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b440e7f72a4017eaf079314b89aab03b5b4c9a956329a10348dac74a1cb9aced
MD5 37f86e1984873614f42c504954591073
BLAKE2b-256 3b7eef4289193453e8547c6e4196c4ea7ab2410d9ded4d97c8d131de300a22fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4c29cc760e0aa69abb87214ebffdda243d7a11148f54b2521d7607d10e5a876c
MD5 d93f2f0b3fe5be5116102b1563d12b25
BLAKE2b-256 0e00494b904a9cfc963fa1dbc36b6a95b5ab5bbf2e5b1717ab4a4ede468cb50d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 86e7d3873a4a30d251aa215dcd2aea89f8dc3fbff45c6726c1fccbaa90f5c7a0
MD5 9c2f1a35ca246d0e710b8b07f6c63510
BLAKE2b-256 258b9017a8f44febd979c0da5a267af07696ea4a0f597c08693ca0299674b4ad

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-none-win32.whl
Algorithm Hash digest
SHA256 80cafb5c8d952561f59e3625e5a497d078f31d9c691375e3623204799628a63c
MD5 db5fc2f409b1f22cec696f888eb2779f
BLAKE2b-256 6c05de15975d3fece5275689c0114464b6cc36861d4293535381bad31d319d17

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 2f389545ff0ef3b3f9a30c09bd7fd3e9d4261ab8c928e0f24f05e54847f42cbd
MD5 c9266f4a59dcb79e0abb5e0cf315769c
BLAKE2b-256 6cb0db804bce212f27f93c2302a632a4e4bc605f75dea861c0789583456aedc1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 35ac784aa073282271083a6fc9ffc8886aa50759eb01078d181404c3fe7534dc
MD5 e24d6eed6bd035c3b56c6f76b6d106e0
BLAKE2b-256 03b744c951928793c559b3b1200350fe4d472721ceb58332edc5485a2577183f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 67dc81ee09027434be5719528bf2c8bfc7b7862343cb3f344cb5ce22efdf01a0
MD5 56f39c58d7a7fcfefe1c50cfa7cf7cb3
BLAKE2b-256 43a136b2b5b34ecb07fd3f8691d4cb0bbe17dd435b880783a22d516446a2e8b1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 c092eabc80452a0ccfbd8221dbb74c47f90f01dcbb5c914d022037b3e9f10d8b
MD5 3a1b1f5bb64e2143b27e1eacd48df1f3
BLAKE2b-256 3275e52af227607b119c5d9449d4575ceecec6469a73daab1b402a5a277a30f1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 4a647d5ce889f0e8955816ae7fd0edfdfeb753c683d42898ccf101ccf65428e7
MD5 0bcd869db69670e841a8471c25ec10e7
BLAKE2b-256 4daeb69e3aa4b4f0cc32ddb8ad28e0412748e7999238ffbcb08d93e0262fc9f0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 07d1307652bc5002b837f2e22bb258c86b536411e2a7c4ed8aef326cd382ce17
MD5 83d207f908f06efa04edd98faad1331e
BLAKE2b-256 eba51f8e545b4ee94a580e7cd92c9682520291f3546663c226f6a8f2feb5899f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 2097f2f1cee58ba551ebf9395f409082b039d6b17546e1db18ed354c17cd86d5
MD5 17bdcea47554ca38933797f70f7b13e6
BLAKE2b-256 8b8ca816e1830eab5312c8ffd1401b67675c18756985e3b0c57a6bc83fec390b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 39c439299cbe838b3f2e7902521a560627da97473749d15fcceb169b19d9715b
MD5 2d6663e4bdda4dc146bce9aa37f540ed
BLAKE2b-256 f475550e4bdfdd6be8e628c784c3ba18b2d25fcf3328770441b5a8c10ecd3dcd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 29689c169d28ac1468ce098abb6f252a54d7d8925193424fc42cad1505395aaf
MD5 8b4470620676d76f1c97fcb49e94f439
BLAKE2b-256 cdbd42ba90d0baea5923be06f77004baecc15ef6b10220e843568d514dbc32e1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 8025e3dbf9d8e16e69af857996da382ee83886756638b14944523191a38e0205
MD5 c8d13813c1902f2d7b852df565800b86
BLAKE2b-256 0e46709af048eef31576b02707894c0f8af0230ac2687fc91ac101194a022e77

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 5fc917b89a74a13bd88891e04a90c73ccef05b8157afa6a4090024fd4f726c50
MD5 bc5b9bf3238501d72c21f555ca4fd758
BLAKE2b-256 8c937d69ea101fd912dab84c70dbd2bd3e1a3eb35e4642eefb4611bc966ba8cc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-none-win32.whl
Algorithm Hash digest
SHA256 fd7b1a2239f3fe052bf2271397b670de8760f34411ccc86e49dcef86c946357e
MD5 79660de3abce4451baa7fbe5629d0eba
BLAKE2b-256 80272a6f9a3d04bc4396e49d86e28b32ee266a2baf8d800a65242b51dc145c53

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 2ebe165d6bde2fe27cf01e59920979743c90b8b770d6675023a5be5fd84871d5
MD5 33dff1d7eb29f93b7607391085e95729
BLAKE2b-256 74b19c9d0cf3cd3573968137d4e570429ff726619964dfe9c4796d8e06ede9c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 77f75ec1706e6740d735e30dc2361ec107e8586e66fb9de7b718a7ed6dcbad8d
MD5 5aa6b643799f0e1326bab80259c23d39
BLAKE2b-256 be1f490b066f3630648b20032b3e9b5beb2ba773704138a70ef613a261705933

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 287a38e3ef1fa5124043557854e85f2b25bc3afbb02c4e1ea17b740bd637720e
MD5 0695ae93fa53d502ea6f2219486ada9c
BLAKE2b-256 46983e09854f8cefe8c5a711c64950d50ba6b17ac77ede575ba8f1c5fd99d42e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 8eb3f464775e33fba4ec9381523cd610bc2ee7c4d1190dfeed078bf5e2075cfa
MD5 77ed510b200e62a199340f85361bdb84
BLAKE2b-256 ffda4af6a71b4ce0f573a1aff917dfb0acc4afc18fc01a904fb787550e68f590

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 5ad72b81d9ad9966d16711cbd8720dbeb5c870a32d1bc04eeb4e287a38bcb381
MD5 74c362035f50a5cc57e00ee918827e8e
BLAKE2b-256 9388751d437d0ac2e3b636aeb558879cf4db68def92f9c1407d9813ef4d56df4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 1d3b0276ce52c85818663ebd448497d727be1bbb0f8d282d37b7af129352b418
MD5 5d0353b76170e7ad778bd4d186a67f28
BLAKE2b-256 f76a9a2c341316a46994f5e50ff69a90279647c842436c4fcc91f83f7caed31f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 70ad03c2017634c40b41525b0cb4b2dee028373c3455e73bcef2720d159cac70
MD5 731ec16ed91af66a7dc4139d023c48ba
BLAKE2b-256 fa86e54ab89a9144655e95120cf5fcf94bf6435aae10531e55dec9f90ca457c2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 5180889f50f0e3cae49c5bd90b7b96997b2bd0e92d60a85c4cc16ba3b5d0a042
MD5 0bc2ce6b02de1a076d68901b88521fdc
BLAKE2b-256 212d81b1f2215d2ae0f74a602b4a16ad4c105214a7b6b3755b1da73f7b719b61

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b79efa9efb94847871758cda5ae263b6a5aed7ce5d75b7f6e34662c9bd39c054
MD5 08251ed9c0bda0805974d2f4fadc232d
BLAKE2b-256 f0186b327d00b6eaf0668688e7c62b423476c288dc843f61709a8f9a687183ac

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 9615211f0c7ff78fc971b215f5b7ca99127b8034744881396fde157437313a84
MD5 7c91165942e9678d8d5b71f816193bec
BLAKE2b-256 12b450a84db9a2c39e8bd95f8e50989a396b5e5a12ef2a0ced1b4083fb82cdf7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 e56c3a2c604c45637402de82f566c805e1541e9e1555761e32b29cdd5d46f5d2
MD5 5ad086d6a8f8da1acecd798d150e68d0
BLAKE2b-256 84bce56a9c7e7acf5380de22a3a9db1e66ce8f5f4b086c4086fe1bab5c4751ab

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-none-win32.whl
Algorithm Hash digest
SHA256 c23814b3612657c90b4c0f1d2b136d672532ee047e59ba844135c3297a71bff7
MD5 a419220346aae5375bcf86a2b554cf29
BLAKE2b-256 138754918786c357826224b089d0d17b420ddb45174068227342746825b2a618

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 af8d147818136dc94d682fb20efc4250a535e0d8504be171c6a4e74d3f0b0af5
MD5 506c23f6d459f973fd7ecc78ca43b955
BLAKE2b-256 1c0bffafd5dfa78e1454ad0e58e5fd355f400ebb61f336da8559f6f77a45bfa4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 3caf24b353c18b80542a89b559c82c78864773f54bfeed754b5ac02418baaa8d
MD5 e4cc78a5b8d5ddcbf47cc1c9d53dc95c
BLAKE2b-256 a16bbdee75671dc776d3ee0695ce3f28a913cee3bc6549654bd9530a0621b3c2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5dd4c1b80af2a5270c2b5825f6ebeaaa60ceee55eb1234c31a97217ed1d24dcf
MD5 d75e7f60885891bcc0881602a66667ca
BLAKE2b-256 91b3c2a931d29c0135d416f3971d2cd9b2a998a7e82e72f03afa489cd788e2ed

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 f046c3cdd7370d7861e52cc7ca17b1dca71ab0b462e793e00e4affbcd2efe94b
MD5 3a43fc11ac65f739894027cee3314de6
BLAKE2b-256 ffff396b09749ccb29cb3b231e01ebcf4e215f2da62cd268e68cc40f6af6dbe0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 ce6c506030fad740f8a74ec8bae4828edaa2f605db2b1cd2f0b2be7255be47af
MD5 a4b7c681be8f6186f39d1c501f498dad
BLAKE2b-256 8279b489ab0eabbafc4a81050f6b08cc80c6bce03ec23dd749d4eb0eccf0ffaf

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 e897544c9fe75752afa36cce4f9a7675252c74608c97cd0c38be0d5e0520c0d8
MD5 24ce709393034688f1ae531127169072
BLAKE2b-256 67f7baffc56858904eb622ebf2fc545ba711a8054fcac0dcbf6e7561ae70e1d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 8f93b82c793c8be5bebc048d1f916000ad5c011f2b5943657f0504fd502c360b
MD5 4e10683a5b725b6a8e7218e66247b6cb
BLAKE2b-256 241368551a915de7acf497e184db1802bec13c374632992717e888364689dda5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 5e32ee9e75dde0096bba51e3d174fbaff9d87733297d0a500eae3c7e9fae6947
MD5 a0b5b1b06f896a5993d67b8bf8457543
BLAKE2b-256 152ed985437949c36d3a49fc8c4a247d917a3f8cc4933c72d27bbe1e60edad6e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 c8e369ad82bcb2269777f41c71e84e13e32ed3e3149b309e0e8b0c4ce2ab4064
MD5 b98065bb3149ccaf489d02c6d898fff8
BLAKE2b-256 0a76355231201f7086ffbf81092fec9b1f0cf47ee67a41d591621a2129f78377

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 beb787465e5e9e946dc5a671bc7bc692a08673489482af6726ce23f69e5cda5e
MD5 0fb1455c5055b8e2fbbcc42e833f4ce5
BLAKE2b-256 a6525c39e39fcf305b9f2e00cb1fefac48ab16731eec137e50c2f7880f1011ac

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 76b8c7bef19e55c7f3a4389d29c6357ec1504c5b2e8bdb6c8b49776b7ad699bb
MD5 55effa1cb1adc67b974af473df947651
BLAKE2b-256 084cf6a6d475e6c9bc884bc16ecb75e4bde125f5c7f11b4ccf3c744f2631b924

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-none-win32.whl
Algorithm Hash digest
SHA256 f01ff8c3fdd3a45f88288037546646b5aa9cb27574d9e1d87a6e667957c60fd5
MD5 ea9a1b6f9a2cad517a4f0365c5f24b9c
BLAKE2b-256 9dc21f582d0ff54702a18a717223f191169db78cf7d675759fa3aea306faf85a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 2884131d2db76ce8f8816c2362badc6aba208c567f1daa094fb4d1e037deb49a
MD5 177872d5e889ad04eb2cc46e63b15a35
BLAKE2b-256 88657929e1be588252eafd511731ce728304e9029e2a28d5c1b577f18e21dcde

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 c50623e5f0727b78bcfe5491c3a3dc0eca6874c0f4cda59c45474709926f3c47
MD5 37abdee72fb83cfee2ecd65ad99c3337
BLAKE2b-256 ff7087c88b481e15c40365a705f319b3a167ab671d0baa24aae39b194d30c7fd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 862d43b15b57e8120ea7beab3f14ed854681605571c6f2a8dbe7532c5e9f4b67
MD5 da5fba4045c5b7152a06207699de4f8b
BLAKE2b-256 a7f793a363eab491efaba17cab0a431e209f62eb147806e55165e4739cd6eef6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 d230d3b03038919fde08133d09fbb18897ba8aa6e32fc7d7d26bea06b99e7480
MD5 6574ed344c6991062de35be68b85a6e6
BLAKE2b-256 e3715f182e5b3c22972330f5e6ff36e44186e8c5373ebb8cb992c9c574d6c26e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 594b9f8c30dc08a331d72e253151aaa1419692ae4ffbcaed368642eb85829df1
MD5 075926eb982cd779ed91940831d74094
BLAKE2b-256 3553cfa835d1561b940589291301c086b0c619ca19844efc4e5d9543a14db821

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 01cc67a99948aa75e388ee5bf260d1eafcda9f3751c920a2fbdad0d1be7e2475
MD5 afe8380dfc3f301fd7a9088489aeaf05
BLAKE2b-256 977bb942aafe196940b69cb0a5cfbad7dc573a9a68e890e5c62e3704d2629601

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 7cbeeaabde5822b715d5e643590a05cff1fe9fb8d291f5487010b715fd096463
MD5 d3f6dfc89548e485ecc229a7a8b5bba3
BLAKE2b-256 fb3a15ec3e8a50f27c9b554f68b7c163c557822a72fe047b1fb4d3bc366112d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0d7572d31518084f387cd090f881e0411c2aa48bc7c434656a7d69dc6896a168
MD5 d9af60adff36b9ec4ba30c3d62994e9e
BLAKE2b-256 ae0faad573834bd9db66db2d1f3025fc7a80278aaa22c3ec1720d405f16f0e78

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 19010582687ff5f178b418cf4896b17c0c29540728f7ba63a5a3ed428d1b8691
MD5 99c2d14b2c5f01b207e795770259c362
BLAKE2b-256 f9cc915094c1b98679b2329a612837c319e87ec27dbc6c398c0b042eb5b5f0ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.2rc1-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.2rc1-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7523e95fb3bb4fd1d257a23cb8314a66f137a2c893a5783fca0a74a331fe2865
MD5 827573125bb5385b5b6fd70739acf2b8
BLAKE2b-256 a6c48a7bbf1d0f40833d2d6705749105ffa38c0de97d2b053f52b0c8431d82ac

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page