Skip to main content

No project description provided

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.15.1.tar.gz (320.4 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.15.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.15.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.15.1-cp313-cp313-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp313-cp313-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.13 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp313-cp313-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.13 macOS 11.0+ ARM64

tokenizers-0.15.1-cp313-cp313-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.13 macOS 10.12+ x86-64

tokenizers-0.15.1-cp312-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.15.1-cp312-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.15.1-cp312-cp312-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp312-cp312-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp312-cp312-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.15.1-cp312-cp312-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.15.1-cp311-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.15.1-cp311-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.15.1-cp311-cp311-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp311-cp311-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp311-cp311-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.15.1-cp311-cp311-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.15.1-cp310-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.15.1-cp310-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.15.1-cp310-cp310-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp310-cp310-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp310-cp310-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.15.1-cp310-cp310-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.15.1-cp39-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.15.1-cp39-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.15.1-cp39-cp39-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp39-cp39-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp39-cp39-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.15.1-cp39-cp39-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.15.1-cp38-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.15.1-cp38-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.15.1-cp38-cp38-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp38-cp38-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp38-cp38-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.15.1-cp38-cp38-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.15.1-cp37-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.15.1-cp37-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_aarch64.whl (9.7 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.5 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.15.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ i686

tokenizers-0.15.1-cp37-cp37m-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.15.1-cp37-cp37m-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.15.1.tar.gz.

File metadata

  • Download URL: tokenizers-0.15.1.tar.gz
  • Upload date:
  • Size: 320.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1.tar.gz
Algorithm Hash digest
SHA256 c0a331d6d5a3d6e97b7f99f562cee8d56797180797bc55f12070e495e717c980
MD5 8d0a724e9cb7074ef53006a0f702b6b9
BLAKE2b-256 c99006bd4bd6c33b9a8d4b4a184e2e575906234d3ba0d6f9e48890b6aa06d13b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 d82951d46052dddae1369e68ff799a0e6e29befa9a0b46e387ae710fd4daefb0
MD5 b583a3801765b463f2476d25454ec779
BLAKE2b-256 50af57ab3633715bbe95f1c5a4d5acd9296ea6dd982173f6b80934ced410e7c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 244dcc28c5fde221cb4373961b20da30097669005b122384d7f9f22752487a46
MD5 98ff52fcc4b4c161facd39db8e3e3e74
BLAKE2b-256 b2d4b0db6f61c9f34068953ffba668fd2fae1390790a06f72dbf8e3e94804366

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 1e563ac628f5175ed08e950430e2580e544b3e4b606a0995bb6b52b3a3165728
MD5 4dbdfbd0b4203d850ad32fa1def782e6
BLAKE2b-256 308d456d52cac1538d435ef46be4b19c610aeb388afa04d4058a4377aae597b8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 089138fd0351b62215c462a501bd68b8df0e213edcf99ab9efd5dba7b4cb733e
MD5 db65e570c46804bcf3819ff72461f303
BLAKE2b-256 25f187290f6440fc6fee4a59069bfdde32b1b147b05c29530a788981e5c78b34

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 1d0e463655ef8b2064df07bd4a445ed7f76f6da3b286b4590812587d42f80e89
MD5 3fa31bb565e552d2fbf585982d71deeb
BLAKE2b-256 baea69d7d51da1fc6a12062823225d6886c1919521ccec1a479ddc26d6e14138

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f3d4176fa93d8b2070db8f3c70dc21106ae6624fcaaa334be6bdd3a0251e729e
MD5 bf2f8a8ba576ff0837668ee8d84b5222
BLAKE2b-256 8ad0f1b173fac8fa520481849bfce7f4cf24f5b0989b97075f0fb7b07c315f80

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 b3aa007a0f4408f62a8471bdaa3faccad644cbf2622639f2906b4f9b5339e8b8
MD5 fffe0ab12c61677655b3babf1e276b01
BLAKE2b-256 366d7c3411d0518c8bc36562435a2dbe6d9634a7110ee761a890fba7a324499b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1922b8582d0c33488764bcf32e80ef6054f515369e70092729c928aae2284bc2
MD5 6bdf692c97469c57fa8867d3601f3b3e
BLAKE2b-256 09d8489001f4d0512951f83aa36bb9f202720b7ae2a418515bfce7264cc92870

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 b41dc107e4a4e9c95934e79b025228bbdda37d9b153d8b084160e88d5e48ad6f
MD5 9cda1ad8dbbfc4bb5b7e4bee24a24748
BLAKE2b-256 33bd45fd686f0fb20f059518db103d7f74f1acef43d2510c4817aa4e76b2dd24

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 4eaff56ef3e218017fa1d72007184401f04cb3a289990d2b6a0a76ce71c95f96
MD5 5ffedce22fa140dd770e5824eef28cf6
BLAKE2b-256 8e67752c7b52287f4863684bc5d591270bb542637e393e464667ee8fdc99f23b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 80e45dc206b9447fa48795a1247c69a1732d890b53e2cc51ba42bc2fefa22407
MD5 cee69aafbbfbe589801e95a6987b2048
BLAKE2b-256 24cb90ac409d1b1d2367b6299a5e11987d38e4b8eab2f2581d552a06ab1eef12

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 2a0fd30a4b74485f6a7af89fffb5fb84d6d5f649b3e74f8d37f624cc9e9e97cf
MD5 29261fda284609d6589fb98f842b2a61
BLAKE2b-256 1edc362fe421648d566ddb4d6161962f9a83cd1bf2d88e32850966e65e392d57

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1e9d1f046a9b9d9a95faa103f07db5921d2c1c50f0329ebba4359350ee02b18b
MD5 61538e29525f5d4703e94cc574090984
BLAKE2b-256 f8a544eeb57a7705bbf71eb3fb240fd38d065d6c4c1059ca5b082e0141d533ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6ac22f358a0c2a6c685be49136ce7ea7054108986ad444f567712cf274b34cd8
MD5 dcbb51e38e8c033bbb32a0161e1d3af5
BLAKE2b-256 67ba23f6b2bc3c2cfdedff81294828489b329a47e8e100014ace5aed0033e9f5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 b87a15dd72f8216b03c151e3dace00c75c3fe7b0ee9643c25943f31e582f1a34
MD5 aa4d52d49be2540dd152f1bdc91241ed
BLAKE2b-256 60c93c6b3825442f2ccc759d81bc6a7026e4b0e6d3f76251645ba24652e82570

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 1bfd95eef8b01e6c0805dbccc8eaf41d8c5a84f0cce72c0ab149fe76aae0bce6
MD5 fff797ffcbaa76c291abd4087c3d4432
BLAKE2b-256 91ad3ac770a94c19c6ec07879a2dff2b6bd0fedd64a461280f18f2117cbe98ec

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 194ba82129b171bcd29235a969e5859a93e491e9b0f8b2581f500f200c85cfdd
MD5 8dc975e2fd3e154d65176bbaa9928f5c
BLAKE2b-256 5a3e93277369b6bebbdfafccfaf09c0ff43b0b59eaa12c41a52707b3f84adbe8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 d8550974bace6210e41ab04231e06408cf99ea4279e0862c02b8d47e7c2b2828
MD5 7b5f9f3ebe508f729069bf19f92e6fc0
BLAKE2b-256 98679520158f6fe1471ebeee62b97765d0acd2d3786e566920bde36a1d0871f2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 2417d9e4958a6c2fbecc34c27269e74561c55d8823bf914b422e261a11fdd5fd
MD5 7d7bb9e199207fb8e8275abb68ed0425
BLAKE2b-256 89b13b0d1756b1bf1c28b3ab11af3ac4bd79116b581c3a50077ddd7088a83dce

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6a63a15b523d42ebc1f4028e5a568013388c2aefa4053a263e511cb10aaa02f1
MD5 bad4d32cb55f1f8c0f507503f8338d2f
BLAKE2b-256 a5b50fd280d862fbd05fd5b98ce2245e9eb948fd8bcb2b23ec09a02906faf9f8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 5f7e37f89acfe237d4eaf93c3b69b0f01f407a7a5d0b5a8f06ba91943ea3cf10
MD5 0c37f0cfa5f4b41a778c4dcc43682e1d
BLAKE2b-256 7110e94f1371788f762bee405ccf4dcf261881427544121468a7b30d9f467afb

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 60067edfcbf7d6cd448ac47af41ec6e84377efbef7be0c06f15a7c1dd069e044
MD5 0bd71e402d06f41167e42e1d3180f74f
BLAKE2b-256 c351bca122365bebdb590b88c143855972b9fa9a9f9e720d5071f77ba1e30a41

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 385e6fcb01e8de90c1d157ae2a5338b23368d0b1c4cc25088cdca90147e35d17
MD5 4d904bf324bd0eb63a46f6a833e6ac95
BLAKE2b-256 0ea1e8d26a4195fc66ad403032b4cbca9f3e5eb6d51ecd20b9cf323490ef632a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 85288aea86ada579789447f0dcec108ebef8da4b450037eb4813d83e4da9371e
MD5 ee07237ba63801adacabad1943fc0410
BLAKE2b-256 59870852be754a271814e2f28d6de4709230b72b3abbd904e45f55b5fa9e8fe2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 083f06e9d8d01b70b67bcbcb7751b38b6005512cce95808be6bf34803534a7e7
MD5 47deeb5e929aa564eb39b3afe83e3a98
BLAKE2b-256 db509b0f66f4710127e4410228c2e50d2868196dedc5f632e86dab081b5a717e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0309357bb9b6c8d86cdf456053479d7112074b470651a997a058cd7ad1c4ea57
MD5 1a9af5b6b2cb1f5e297f1666c4515b88
BLAKE2b-256 bf1eccbffc49b502d9cc8ba519c7ec0256e9dff4089e0606635857198a6d74d4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7b14296bc9059849246ceb256ffbe97f8806a9b5d707e0095c22db312f4fc014
MD5 566d9bb940ca73b739a261966b2d8d36
BLAKE2b-256 eb35912d73938bf2babd081d6c141b43835bae9be6f15fdd6b1c46abb6ecbe27

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 3c5573603c36ce12dbe318bcfb490a94cad2d250f34deb2f06cb6937957bbb71
MD5 6139dbfa24219a7264723ca91939405b
BLAKE2b-256 3e778b3cbff06abe7da3eff00835c653bd498a80b464e12813f6bd59398ae112

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 325212027745d3f8d5d5006bb9e5409d674eb80a184f19873f4f83494e1fdd26
MD5 69f7bc9ceadf369536ad41b93f5da38a
BLAKE2b-256 1db246ee7a127c0ec5a1fd089d4591d7e291894adc23bfa3621fe266706250df

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 01c28cc8d7220634a75b14c53f4fc9d1b485f99a5a29306a999c115921de2897
MD5 c0b133d1547d9873f7ff1bbc9ec51ad3
BLAKE2b-256 aa6f7678e75e2f1e3bb877adf9b2358d033764f5cb4b14aad421bdbbdf9ec662

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 191d084d60e3589d6420caeb3f9966168269315f8ec7fbc3883122dc9d99759d
MD5 dd7208905d8caefd240a2889e95ae1be
BLAKE2b-256 b452ec6c2e43fc89e45b3a1a7dd914c09fe2f445feb1b2d7903eae2b88b28421

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 485e43e2cc159580e0d83fc919ec3a45ae279097f634b1ffe371869ffda5802c
MD5 25050f0766289e649c27dd7d8e616be3
BLAKE2b-256 0d7ba352c08ade1dad9a76d22f41e8a277d50854bd348be62fe2eaa2f06a9412

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 97194324c12565b07e9993ca9aa813b939541185682e859fb45bb8d7d99b3193
MD5 f445f3f5e6756aeded800c0be08ef4da
BLAKE2b-256 6890290b22891db1f9791984152025b0a50263832ffc483fb1080eed05033e59

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cdbd9dfcdad4f3b95d801f768e143165165055c18e44ca79a8a26de889cd8e85
MD5 72a3270b6995368b8d1d4d5e5090db60
BLAKE2b-256 d30aa5fdaf07a3b5ac7554c1786e1816c95996c43a9b37e504e559ee1629f53c

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0faebedd01b413ab777ca0ee85914ed8b031ea5762ab0ea60b707ce8b9be6842
MD5 b57df9647c5a50ae605b264bd87fc566
BLAKE2b-256 a8ad93a2959d97d855806b535d28b8b71fddcce63b946411de8022440efc1384

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 89b24d366137986c3647baac29ef902d2d5445003d11c30df52f1bd304689aeb
MD5 f6c073bb3f82372dff29259f995823cd
BLAKE2b-256 8526d27de71d115995577fb6639aca2ba67418694ec82b63aa0402c96a24db9e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp313-cp313-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp313-cp313-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2ce9ed5c8ef26b026a66110e3c7b73d93ec2d26a0b1d0ea55ddce61c0e5f446f
MD5 af57c036f113f234199f299baf3d9e65
BLAKE2b-256 e2de44d13e026da36926a6637bb465f0392908c4da396c565409a36b3e5e8b67

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 9abe103203b1c6a2435d248d5ff4cceebcf46771bfbc4957a98a74da6ed37674
MD5 5525e3b3f1bcdf4cf01e3469205b6fda
BLAKE2b-256 5e8ce32b066f3a2c924235e6a2ecad5b22c3c64b569f192975815c887b4bcfdf

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp312-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp312-none-win32.whl
Algorithm Hash digest
SHA256 f2272656063ccfba2044df2115095223960d80525d208e7a32f6c01c351a6f4a
MD5 e69736e361e04a904bdeee17d1b17c5d
BLAKE2b-256 75eae5592cb8a3d3e1ed39cfbbac0b470e4fb456cb7df98a7b084245ad7df40f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 681ac6ba3b4fdaf868ead8971221a061f580961c386e9732ea54d46c7b72f286
MD5 31fad157995723d4c23bf7f12ff0cf79
BLAKE2b-256 3fe253e6726c52e8651fcef3d509f5fcdb8f28ee560d7e7383e5747f12c6013e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 425b46ceff4505f20191df54b50ac818055d9d55023d58ae32a5d895b6f15bb0
MD5 c6aa244831748747c93859968744d525
BLAKE2b-256 02ccc5ec8ea0cc761595f062d7f54144b6abf50f6ee529d110e62ac9e580dd66

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 82c1f13d457c8f0ab17e32e787d03470067fe8a3b4d012e7cc57cb3264529f4a
MD5 baa2e06827fe1070f4de7e848f1ad0f6
BLAKE2b-256 89d7bf4d68971e553289b5276acbaaaf8ab9e17e6d9c3ef8ed38450ad46480db

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 1b173753d4aca1e7d0d4cb52b5e3ffecfb0ca014e070e40391b6bb4c1d6af3f2
MD5 2b6ffb84f9de081341fd42f06afed69b
BLAKE2b-256 d33d1367dbfb761f7fe79cbb37a5b577e1c767bcb114e6d9405a4e197dd6e416

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 760334f475443bc13907b1a8e1cb0aeaf88aae489062546f9704dce6c498bfe2
MD5 aacc325ac09d57b0f0df06cf2563c5d5
BLAKE2b-256 73c1fd7658ab2762614d0ca7536f2824c43f1b594d27b3ac18f1dcd6133dbe9f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 d8e322a47e29128300b3f7749a03c0ec2bce0a3dc8539ebff738d3f59e233542
MD5 aca2b4fe97b9b335fef31020e92588e2
BLAKE2b-256 5f2d224449f2639e50ddd537d6ea05829a5274eda3928b2502dc2bd1f704e491

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 61ae9ac9f44e2da128ee35db69489883b522f7abe033733fa54eb2de30dac23d
MD5 fe7da2b0001a29fdb5c7f6d0658fb16d
BLAKE2b-256 3f41c8ac87d95a1ab8e16cb8a4a7a9f77d4b478695b012621f2bd9f9767d7cff

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 d6d28e0143ec2e253a8a39e94bf1d24776dbe73804fa748675dbffff4a5cd6d8
MD5 fd47516ca42f4391825e50746fe015ca
BLAKE2b-256 8a690416d588d661a1283bd909194ee2f3551b7e2675ecfb73f4425b5bfaeb94

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7d870ae58bba347d38ac3fc8b1f662f51e9c95272d776dd89f30035c83ee0a4f
MD5 3a159c74fef0e21dcf9356eeb19216a2
BLAKE2b-256 e8b59f84fb80fce8fbf3657a30e1922231e7ddd4ab5c2ed53fdb8650533487e4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 0f3a379dd0898a82ea3125e8f9c481373f73bffce6430d4315f0b6cd5547e409
MD5 748813f2bcc3f1a897e1579af4aa1f9a
BLAKE2b-256 8d8e54b57ca9f1b8e757ee910bd5053ef8cb75ae8f8024d89c872e0f108d1bef

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 30f689537bcc7576d8bd4daeeaa2cb8f36446ba2f13f421b173e88f2d8289c4e
MD5 fe4985465cf43ed420584fdb5ec108af
BLAKE2b-256 be721be614ce1e7da45d7bddaa50f5f491bc56c586d0acd2e30e4575c4b42aa3

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp311-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp311-none-win32.whl
Algorithm Hash digest
SHA256 a4f03e33d2bf7df39c8894032aba599bf90f6f6378e683a19d28871f09bb07fc
MD5 9c4d84a338fd331e92c53eb5dc42ee7b
BLAKE2b-256 5a6f152141cbfa26cf5342f5972a5d1d4e19cc922213dd221b43a7440854821b

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 671268f24b607c4adc6fa2b5b580fd4211b9f84b16bd7f46d62f8e5be0aa7ba4
MD5 8c4034735b0c46c9ca9d3443b44a9b5c
BLAKE2b-256 d0caa36cb7ce2ddfa2e4caa4a1c420cf78f74e7c6a1cd25d6a1f7fec1addc6e1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 0ab1a22f20eaaab832ab3b00a0709ca44a0eb04721e580277579411b622c741c
MD5 013f4e7eedea3b284fe0ae1f7d13187e
BLAKE2b-256 493aac445e7e7eb629bc003d72299e0c389b239f9c06583bf7927a8c97e6f868

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f49068cf51f49c231067f1a8c9fc075ff960573f6b2a956e8e1b0154fb638ea5
MD5 2e7f5ef64dda0681625cd76e7b737bc9
BLAKE2b-256 3c339b6dbae3f7f1e25d6bf06cd511d4cd9694c805465aafbceb77f306f7826d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 c2921a53966afb29444da98d56a6ccbef23feb3b0c0f294b4e502370a0a64f25
MD5 479390a78b67c2f235e1a4f4d76120d9
BLAKE2b-256 25e6519080dcfc68f189121a1790a078fee98f9b42dfe728be0c5d8164a2dc3e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 0f3f4a36e371b3cb1123adac8aeeeeab207ad32f15ed686d9d71686a093bb140
MD5 782e1c2e0d87b2320984fe72516d729d
BLAKE2b-256 6fcea1ac12b045f796bc16b96bc12e3858a5fb4c0278bfda9121e3c4d5763b85

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 bd244bd0793cdacf27ee65ec3db88c21f5815460e8872bbeb32b040469d6774e
MD5 8d40e5c726ece1d57033c63e0ea642d0
BLAKE2b-256 195e6bf22965e1718d15b8de253b1f6ec977526b774326aa71ca2a226dc7eca5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 6a7d44f656320137c7d643b9c7dcc1814763385de737fb98fd2643880910f597
MD5 faf503f30e7a91d6d17798b47d2986a2
BLAKE2b-256 0666b2b0c2ed43762b9b006723cec867d35b47eb17bf507841b3f19598bc08ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 b85d6fe1a20d903877aa0ef32ef6b96e81e0e48b71c206d6046ce16094de6970
MD5 05015af1e3e691a3da5edea02527385c
BLAKE2b-256 491a9df2805a8035732bd90855a3fb7a963480efa7b5e06f0c5aeec860daee2a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ea9ede7c42f8fa90f31bfc40376fd91a7d83a4aa6ad38e6076de961d48585b26
MD5 7bdf494e3c7a8787326fe2a3f324c354
BLAKE2b-256 80f5219718ee1fa3ffafc4a3f7d116eb240349abe3d080be5d1f1bb4ca6896e8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 8ad034eb48bf728af06915e9294871f72fcc5254911eddec81d6df8dba1ce055
MD5 077da634bf0c46776cc81bc9a55bf7c5
BLAKE2b-256 0d50717e56e31b15dcd1e600528207f5faceb83e89178600f3e0bcd292329ae1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 39d06a57f7c06940d602fad98702cf7024c4eee7f6b9fe76b9f2197d5a4cc7e2
MD5 5f0ff11db03a894b046808e3aa1ac88d
BLAKE2b-256 cc7f3d1d65fb2555f51635ed99386dc36fe1e28f058893bb277a3c85e0e57ce4

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp310-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp310-none-win32.whl
Algorithm Hash digest
SHA256 dd999af1b4848bef1b11d289f04edaf189c269d5e6afa7a95fa1058644c3f021
MD5 603ea673b6ef04073f4e86b55052fe7b
BLAKE2b-256 f1ca4e92085a632d87857b5c89bc990af846573209ccc881d7d45c9ed5063ef2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1c9df9247df0de6509dd751b1c086e5f124b220133b5c883bb691cb6fb3d786f
MD5 f7362f56d9cae96a809ede53fc5d5b9f
BLAKE2b-256 fd4308a270778d28a5f25ba2c7b4ae9af881f4d66d95482464de84c7b5dafb2d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 7ed5e35507b7a0e2aac3285c4f5e37d4ec5cfc0e5825b862b68a0aaf2757af52
MD5 8ed6e65d05bc567e12f48ef51cc472a0
BLAKE2b-256 fe5754ea42e2b9faada7a00a3d76573f00428a07cc7aada9d1f4801ad0681fbf

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7061b0a28ade15906f5b2ec8c48d3bdd6e24eca6b427979af34954fbe31d5cef
MD5 9fbeda58256e0b4f39e1c98d87969ce8
BLAKE2b-256 1e54a3451cdf47760a0f671e07ae69fdff00a1dc074db869a1655096c592fd0f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 506555f98361db9c74e1323a862d77dcd7d64c2058829a368bf4159d986e339f
MD5 5f0357d5d31b0296cbe4b219714ed1ea
BLAKE2b-256 0565e2674ef996c443422c9cdb9ec7a3bb1e001d65bbcc72122efd18a48a26dc

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 22bf28f299c4158e6d0b5eaebddfd500c4973d947ffeaca8bcbe2e8c137dff0b
MD5 c025aa058872cbdeacf8a8ce239c1845
BLAKE2b-256 689247e0cc90f6b0b80110ee9055e149f25fbf376f3011c9699b752d4cf0795a

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 8cc575769ea11d074308c6d71cb10b036cdaec941562c07fc7431d956c502f0e
MD5 a8d9882ad36cbd873ddfc70c3434de6e
BLAKE2b-256 fd4a93bf2e53d4633fd5134dbd95824e482f0d8d368c74bc336cd4fc234a1dd8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 e84b3c235219e75e24de6b71e6073cd2c8d740b14d88e4c6d131b90134e3a338
MD5 0e29fed53d3c76140bfc3960de5ba27a
BLAKE2b-256 73d38a70b946878700cc8ee1cae4a226b77c19cc513da85f8bd333328c91e7d7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0049fbe648af04148b08cb211994ce8365ee628ce49724b56aaefd09a3007a78
MD5 39b21600154d391c01ca1d44bf99cf8b
BLAKE2b-256 353c376115ef0608a96458775ddad2b1a873ff912f2ce0af499299d6ea69fdd2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 29a1b784b870a097e7768f8c20c2dd851e2c75dad3efdae69a79d3e7f1d614d5
MD5 c0c635de79e1d8069aebdedccafc1115
BLAKE2b-256 53fb3587edbb9bdf935e011a6bf54660709ecc3e482ab2dd94fd0e373c6aefc2

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 32c9491dd1bcb33172c26b454dbd607276af959b9e78fa766e2694cafab3103c
MD5 552374d109dcb4629ce95caffa1c52b6
BLAKE2b-256 3c5a1507f19b3240f4b9b2e4bc15e93659a606deb6634716b723d2f1f6da8511

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 9ff499923e4d6876d6b6a63ea84a56805eb35e91dd89b933a7aee0c56a3838c6
MD5 27c00e255be2686c512bfc2361eea733
BLAKE2b-256 ba45a419698bd1b5ddb99b2b49e2cad967a900ed95ff48b54acab1f16a3423f6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp39-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp39-none-win32.whl
Algorithm Hash digest
SHA256 74d1827830f60a9d78da8f6d49a1fbea5422ce0eea42e2617877d23380a7efbc
MD5 a0a44195aba417a2720a3149889454ae
BLAKE2b-256 339745e9a17c990ef589e61671d15afc2839ad1473c846b57295a7f7d7c67e91

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 5a3c5d8025529670462b881b7b2527aacb6257398c9ec8e170070432c3ae3a82
MD5 dc54b9eff864b36e3dee4daa3eea8300
BLAKE2b-256 e9cfc546bdabe0617ef292a4d40beec695295b6ba7241b1fdab655903fcd5051

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 68f30b05f46a4d9aba88489eadd021904afe90e10a7950e28370d6e71b9db021
MD5 6bfcfeb8598e394424c16cfeab8755e0
BLAKE2b-256 718b976cb7002ba3b9c81baceeb821ba39ebbf6d1e5428986e2e3d16b67a2481

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 241482b940340fff26a2708cb9ba383a5bb8a2996d67a0ff2c4367bf4b86cc3a
MD5 3af9e62b8208cbc6cb0bd9e886258d9f
BLAKE2b-256 83ef284349b3987642a3a29b5e5eb183d7f888822a78dea95a56502fc60e5941

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 0a1aa370f978ac0bfb50374c3a40daa93fd56d47c0c70f0c79607fdac2ccbb42
MD5 869c6071d8eb9d6d27976ec11b6ec586
BLAKE2b-256 65acfd6a406b5da520b396a28f3acc7132bbc9d85b363389c396cb5f5aacdd67

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 0cd33d15f7a3a784c3b665cfe807b8de3c6779e060349bd5005bb4ae5bdcb437
MD5 29c50bd56cbeb0a9f29241b04f067e6c
BLAKE2b-256 cb16fae068af2ba6bd7097446f49e699ed966dddc560965ffd32ac85a3764016

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 4eda68ce0344f35042ae89220b40a0007f721776b727806b5c95497b35714bb7
MD5 42b4108c70ec6f2d229a410f66ff401c
BLAKE2b-256 986e516a34a7824ff854c0ee89b6f6c47338c796e9cd708dab52d3167e7eea8c

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 58d4d550a3862a47dd249892d03a025e32286eb73cbd6bc887fb8fb64bc97165
MD5 07ac9958b1ce5b008e55e6b89424954a
BLAKE2b-256 11fb7c59995cdaa0ed0e4271caa373c59dbd8c7e87b2439e0768c33e2e9194bd

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 777286b1f7e52de92aa4af49fe31046cfd32885d1bbaae918fab3bba52794c33
MD5 344f79e60523f8a14aba83ec431a6bf4
BLAKE2b-256 94eae4cfc0032328b72fb0182ebf2d6456e12f37e6c13624968a0e46faf69a40

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 38dbd6c38f88ad7d5dc5d70c764415d38fe3bcd99dc81638b572d093abc54170
MD5 8c38ec974c6e6042d8e1eee5a8b66029
BLAKE2b-256 2485f319fd3d9855fd8b247a68d1c7c9d93bfc388369a35408ceb75ec2670ff5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 8d3f18a45e0cf03ce193d5900460dc2430eec4e14c786e5d79bddba7ea19034f
MD5 28c97549f4fa81a2070073ed0401b1e3
BLAKE2b-256 f3d141ba481c90b1c759059dc6c066bb54c2384578180a029598c80d23f348a6

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 a4fa0a20d9f69cc2bf1cfce41aa40588598e77ec1d6f56bf0eb99769969d1ede
MD5 77e62e173ba4146f045f3ec0fadd2928
BLAKE2b-256 a11238407f6da9656b78c3be1f42b4e2fdf824c3c0b72eae194447353d75a11e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp38-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp38-none-win32.whl
Algorithm Hash digest
SHA256 614f0da7dd73293214bd143e6221cafd3f7790d06b799f33a987e29d057ca658
MD5 4555b47012eb3163fc616b73cba4cca6
BLAKE2b-256 08d1c071184268c9ff857470749508b2538d7d7c5de79d7bc05c31dd3464fd01

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6456e7ad397352775e2efdf68a9ec5d6524bbc4543e926eef428d36de627aed4
MD5 bfba7c481870eb3589f512ba0c985d87
BLAKE2b-256 dd53e657cd61c1c38ad23fca4fe93ed55a06db829f511d6823f2300fbc339d92

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 587e11a26835b73c31867a728f32ca8a93c9ded4a6cd746516e68b9d51418431
MD5 145608fb805cb4df823fc10f2f36e0a8
BLAKE2b-256 bbf74c779ba3f32d408f4a2e866c5434dc06575c0c9468a6ca49a835a164403f

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8a6f238fc2bbfd3e12e8529980ec1624c7e5b69d4e959edb3d902f36974f725a
MD5 c6f7f262c5da80aa39bd2397c92c75b6
BLAKE2b-256 a4037cb53875787383f4c12304bb45e06fd97a3ccb87aa7bf52abcc97e0c17b8

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 aca16c3c0637c051a59ea99c4253f16fbb43034fac849076a7e7913b2b9afd2d
MD5 ac3c23eed4d2b8b63822b9467dc7b0a0
BLAKE2b-256 ceaac4a4c73b367f55285382340607a04d881da309dcbab6a62618461f54828e

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 574ec5b3e71d1feda6b0ecac0e0445875729b4899806efbe2b329909ec75cb50
MD5 7a30534780ba55ae12fc4147c19a18fa
BLAKE2b-256 08c2c49c8f67e0b706d324a8409de89b8e265a96d5c313dbb8c13960befe3b53

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 fac011ef7da3357aa7eb19efeecf3d201ede9618f37ddedddc5eb809ea0963ca
MD5 5ae0dff94cb85fe61e0096c0a7cb5a97
BLAKE2b-256 560bf6f9165e62e28030fa15f2a2af039ead50005ec68b388394f0f54c4d4394

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 fef90c8f5abf17d48d6635f5fd92ad258acd1d0c2d920935c8bf261782cfe7c8
MD5 28092ae338e8e0360e4b23281321e647
BLAKE2b-256 d0e34729f647b86b3d40e8f004c63533a20a19551742824d0131da2c82f4a4ba

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 3d190254c66a20fb1efbdf035e6333c5e1f1c73b1f7bfad88f9c31908ac2c2c4
MD5 1c86bf8dc1e3b5a577024bcf62f67f08
BLAKE2b-256 fe84710af59c21d0ea7e8c527a9aa4d5f19ebe0e096ab92a18961e93ffc328d9

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 48fe21b67c22583bed71933a025fd66b1f5cfae1baefa423c3d40379b5a6e74e
MD5 a6b3c10aee0694584528a01de4df196f
BLAKE2b-256 fbea758ee7bb67e00c5d1d545da3a23ee9dc0b58f2753d46de81488d5ec2e22c

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 59b3ca6c02e0bd5704caee274978bd055de2dff2e2f39dadf536c21032dfd432
MD5 342e3c431567f403a10c0604f04394e2
BLAKE2b-256 b961bcffd93f63fe58a1748377df15c84eb14b75f155aee3a3fcd3157638deed

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 d2bd7af78f58d75a55e5df61efae164ab9200c04b76025f9cc6eeb7aff3219c2
MD5 c58917dbd6c6fe2c938b36f6ef5045b8
BLAKE2b-256 0bc340b51a43c9d8d7dd3a6770e8b939e6990b461c104c4060c4556d635df60d

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.15.1-cp37-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.12.1

File hashes

Hashes for tokenizers-0.15.1-cp37-none-win32.whl
Algorithm Hash digest
SHA256 cd6caef6c14f5ed6d35f0ddb78eab8ca6306d0cd9870330bccff72ad014a6f42
MD5 70aa144e7646bda4a9edaa99d5878a96
BLAKE2b-256 b8fd9de9f1c418d2abb36057e64ce716be9f5eff757502d112d213bc6aaaa566

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 bedd4ce0c4872db193444c395b11c7697260ce86a635ab6d48102d76be07d324
MD5 5c45d33701d37d156aced76a5a1d88b2
BLAKE2b-256 b99fc4feb95ca6c9335476b42d2b2c0d33d2c9d8617e021f803960fb6eb913e7

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 ceb5c9ad11a015150b545c1a11210966a45b8c3d68a942e57cf8938c578a77ca
MD5 d1d0765111b71e52e4586fca43077e38
BLAKE2b-256 76acf47f8a4c5494a75ca11d652d1d55c4285e5bab0305c4e842f5da935e3289

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8c5b6f633999d6b42466bbfe21be2e26ad1760b6f106967a591a41d8cbca980e
MD5 0f9ffd44209586aa3e57ec587821a53f
BLAKE2b-256 a94e6652107425e6f78d88da0cd54b0fa5384315886e679db0a139de558b9272

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 25f5643a2f005c42f0737a326c6c6bdfedfdc9a994b10a1923d9c3e792e4d6a6
MD5 421050e467da403e90bbfeffa7700e0f
BLAKE2b-256 f0a778adff9620e86517cac3afe3d0a8d518f62d13af7a5784974d85dc18bc26

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 b72c658bbe5a05ed8bc2ac5ad782385bfd743ffa4bc87d9b5026341e709c6f44
MD5 20aa0d53d51ebee7d93ab600d670f243
BLAKE2b-256 2c0541c841e9280944c6e13ee9783f85eadea4fc67ff0687f11879b061047075

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 c52606c233c759561a16e81b2290a7738c3affac7a0b1f0a16fe58dc22e04c7d
MD5 2ad64208d7a7a2c6324311f40e22a353
BLAKE2b-256 2bc0b346ca7dd6087c48f127bc9bc73a2752d36b68c668aac9b4e2f973b0e9e9

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 401df223e5eb927c5961a0fc6b171818a2bba01fb36ef18c3e1b69b8cd80e591
MD5 f4c8ab380f3dca4bb9e06ced982284b6
BLAKE2b-256 13d622bf31a1d64b1935402543108962644d98f213e5e276d6852dad908ca7a5

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 e76959783e3f4ec73b3f3d24d4eec5aa9225f0bee565c48e77f806ed1e048f12
MD5 883fd27847adfa63d83cdbf97c5b3c43
BLAKE2b-256 6c1268ac0777f4c1ba92fc82ba062bcc80e02008bdeb5c8feca2222dfcd85476

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 382a8d0c31afcfb86571afbfefa37186df90865ce3f5b731842dab4460e53a38
MD5 2cc62b3cc081a6e3c9264aeab3ac900e
BLAKE2b-256 2b916e396cec8fe6b19f2e8c99733780ee137065131b3be33e5ad973535372b1

See more details on using hashes here.

File details

Details for the file tokenizers-0.15.1-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.15.1-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 1441161adb6d71a15a630d5c1d8659d5ebe41b6b209586fbeea64738e58fcbb2
MD5 8aa1f25b9c9499cbd5c650f8bceaea96
BLAKE2b-256 043ec4aeec7ca17d5f0ce3c74a1bb928d243e5cc7e1ae416a9afe9961c0358d9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page