No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Release history Release notifications | RSS feed

0.20.3

0.20.3rc0 pre-release

0.20.2

0.20.1

This version

0.20.1rc1 pre-release

0.20.0

0.20.0rc1 pre-release

0.19.1

0.19.0

0.15.2

0.15.2rc1 pre-release

0.15.1

0.15.0

0.14.1

0.14.0

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0 yanked

Reason this release was yanked:

Breaking change with unexpected effect

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.3

0.10.2

0.10.1

0.10.1rc1 pre-release

0.10.0

0.10.0rc1 pre-release

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.9.0rc2 pre-release

0.9.0rc1 pre-release

0.9.0.dev4 pre-release

0.9.0.dev3 pre-release

0.9.0.dev2 pre-release

0.9.0.dev1 pre-release

0.9.0.dev0 pre-release

0.8.1

0.8.1rc2 pre-release

0.8.1rc1 pre-release

0.8.0

0.8.0rc4 pre-release

0.8.0rc3 pre-release

0.8.0rc2 pre-release

0.8.0rc1 pre-release

0.8.0.dev2 pre-release

0.8.0.dev1 pre-release

0.8.0.dev0 pre-release

0.7.0

0.7.0rc7 pre-release

0.7.0rc6 pre-release

0.7.0rc5 pre-release

0.7.0rc4 pre-release

0.7.0rc3 pre-release

0.7.0rc2 pre-release

0.7.0rc1 pre-release

0.6.0

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.1

0.1.0

0.0.13

0.0.12

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.20.1rc1.tar.gz (339.6 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp312-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.20.1rc1-cp312-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp312-cp312-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp312-cp312-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp311-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.20.1rc1-cp311-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp311-cp311-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp311-cp311-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp310-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.20.1rc1-cp310-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp310-cp310-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp310-cp310-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp39-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.20.1rc1-cp39-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp39-cp39-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp39-cp39-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp38-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.20.1rc1-cp38-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp38-cp38-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp38-cp38-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.20.1rc1-cp37-none-win_amd64.whl (2.4 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.20.1rc1-cp37-none-win32.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl (3.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ i686

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.20.1rc1-cp37-cp37m-macosx_11_0_arm64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.20.1rc1-cp37-cp37m-macosx_10_12_x86_64.whl (2.7 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.20.1rc1.tar.gz.

File metadata

  • Download URL: tokenizers-0.20.1rc1.tar.gz
  • Upload date:
  • Size: 339.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokenizers-0.20.1rc1.tar.gz
Algorithm Hash digest
SHA256 9080da4513534a0da2d25fbeede6bdc1bab1fd2dce6e0bf0dd1d38cf087fce57
MD5 587e48913785198a2c16e9bfdfaed860
BLAKE2b-256 f4ad79a71eb09d8a7f58d127665a2cde7c3e890fefa835b5b79c2fd7310cab2a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 d6c2b97e241e4592dc5fb9971297b8dd91cec6ed843c9fb1c8107f62ea2ce696
MD5 43eb35255eb334a00be2d1bf740c83a5
BLAKE2b-256 d79e3c4603725f5e12e2a355b6cbd68bb155cfa1afb271b9558b37e67a75e872

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 30a3b6fe6ccb2048b471ae84cf135c790e0fa5eeb99d18166fcd5d379c0f7394
MD5 9365c620256c55edcd869aa1127c5758
BLAKE2b-256 e9ac3fc410dbbf1eb6dfc342b4a63e4b396a412541f9c8ae3501b8f1b5509a33

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d77a27827a7366f0a8497032caccdf35469dc9d057f29147d266b4ac9b067c98
MD5 dd932f76fa98cfe502e58de781830651
BLAKE2b-256 5ef23c9cb88dd42749650a87582520b4f400ad9ea4b61b8647aabfa6a6e63a14

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 7c3c9c155868303a54f3bdc59c76107d4a65febf6392ee497ae4a121af7b8dda
MD5 3c885520d85f69cbb61b488a4b0d0e62
BLAKE2b-256 26ffec46a5c582e739654717d90adb771a0f97478c78f450c5c33745484b5b53

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 7e987c605da9612583f0fadc8d4ec23afa63dbfc9e0fd8c5b8bb1875f16e9bfd
MD5 54e47df4c38140699f5e92dd1cabfa52
BLAKE2b-256 bc68b34ddb4a4f7aba20c1b61d332a4c609ae6f61861108b97fb30fbc1fa4c91

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ececa1fc2b665d38c35229a0b028a4ec16ae6a71fd3b5dfdf05b964101650ed0
MD5 75e20d33a5f51b66bf9ca468cee0ccf8
BLAKE2b-256 1086cf0ef357b83bc7e42a399154667c4a939016517bcae06b7b9055c2f5ba8b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 bb6e46dee79bad75f489b82a80a4f00833ac799517e6bba8a4216316b9d0a17a
MD5 03242e4319611d40921a78bf06b867e2
BLAKE2b-256 5b62dbd712c9bbc6e9f54ec16d12530df4baa8a542584f0a82240e6e8860e196

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 4120a156b2dabaee56a800e5da90d96e40b69366b1de00b88ad3ddf93e819b13
MD5 d57e1e5aca70efe0fea62d865c58ff9c
BLAKE2b-256 a74b43faf835104528902b5666588c8f61d7774f7532f2bdb94581a403e47292

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 2e933f91eb1a05fc592861f8c3ef3efce48bc118c42b46768c33682260b70f46
MD5 1a8b02b4582ead9da21644a8ff70dee1
BLAKE2b-256 18b38beffb3bae98b2a95b22e689739ab069e055eef6f15070caffc39eb88799

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b3b631d32316c4e24979c8e4fd1920aea5485a45e15b79850eee004f0dc9d1c5
MD5 134cd7f3249958f7f69b14b457c16852
BLAKE2b-256 c528622b2acb5f8f17cb3d5d99153d8ececfaf3769558d75d5940b298c788ab7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 65265d8f64d91eab75b1a1d19844bed73f1960d7d5d360891f99d7fdc96ca8eb
MD5 c69517d5af8ec729409de6535d8ccc2e
BLAKE2b-256 f27b89eff20c06b063a76f1359737115302038f33c8881bb3cd969d84bd290dc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 db0dd460f9bcc6e62ac62cc591f395880c2337df3c6591087795a059d3c950cc
MD5 2e7c2d88fe864a57d2be9f897bcbe162
BLAKE2b-256 3d78cc146d68f9bf04270db7888fa5077bc8e23b3ec3e07bd732f58a37336139

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 012b6035512d00dc21e02a8cc83e3c6f415faf9975c8a567d0d529679665f4b2
MD5 d3e2bd37d942a9ede50eff89faecbec2
BLAKE2b-256 133183ec4fa135807fe3d0b11547ee9a2d9fbde98752ed602e4488d19091e71e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 d465dc671c8a81a3a4f90ad4510f3c65f5d46bcf697441bd03d6aad6c53655fd
MD5 aeb29aeeb3f7f498ef0bf77cbec10609
BLAKE2b-256 438d9231420f21b8740bd14458235c8130763ab65c7860505194a7f498cf257a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 35c968ed55cbfe8e73258abe9488298c11f3d1b0ad49d791fb0c3b7552d140f2
MD5 1bd795588eb2ca0c2dc0407e1c0968df
BLAKE2b-256 7a5e60309361ae17782ace7c1331f27c3d649ef8b13a915d1e096c7850fa2337

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 3a56b037c44848d061f3ab48fca46d72eb3d2589dbf77783da64fb626d1e8df9
MD5 bf392e00e45612f409b92efa336cfc05
BLAKE2b-256 b16a19cce67808c2f545540ed38ddf8c89749f2ca67a330caa118b3526e6deb1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 23d7244581af3b56bffe2efb8143b4223a141ee7b3c11a74a6e283d633b2ff25
MD5 9a9e0136e5207c76e9f1c3e9bbd27bf5
BLAKE2b-256 f9261a2aa81ec836aab0ff81ec0720b548c6d244713908889045f6946b5ba2bb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 98e602a92b4b331843ed737291db4ecc8d291bf67c3901d514fb90bd20db5ad1
MD5 5422442ce4dcae32207f2ee21fee54ce
BLAKE2b-256 7c2439a46e767ed69245c54dd7fd6b63a6a56d54266a88fcb0ff0f93e90a34a2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b4aa8035a03a4c18701f7b30b060e150a97d8e3110a087b3042634a4a5bc0616
MD5 ba0c39e49026cc0ab20d0501f5a485c4
BLAKE2b-256 a44167163f9707e2d88c44ed4dacfec2d13cdb905d93b8af94216d6778f59c9a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7c8396788340fb0edb2e7c56bf7115ea8098afbf657db7c59309dcf50a0333bb
MD5 ba4671aa75063e24a000556c39b5390a
BLAKE2b-256 1c65b7560d9237c4841ba175f25ab519794a58124c57073a88d4e8c0e0155aa1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 86910c4144adfa6ddb51ca3b7e2daaf0eec47b3614b62e1e33d2ca2fa73d654d
MD5 c9ae1a2369de0aa90cfee6a68a8e6868
BLAKE2b-256 85457d1ebedafc69566566c791715d076ce75b131fb4ddbfdabaceb9b2cd840d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 07cc1c3f46907e1ffe8f58d6eae023fe42be5388e81228577aab87f0d786ebb6
MD5 581263749ede864ec3eca2ab37190338
BLAKE2b-256 52e28df2c71f67a45547b84df9352885326c0e2f71d9ae080ff40f8324b1d633

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 cc4b9c6b3a4003be39a117fafd965994f5f968b1638feefb38f14797e39b8026
MD5 8efa61b4811629eb39981b156401a41a
BLAKE2b-256 272921695e646f950fa6821556b347609831adb00f0dd56ed7a411f6cbcde472

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 313086091c164b2a433229dc19641c39574e801f10ce0d7dd9269a1c6c359b08
MD5 b8660dcf6b353aa69460f128fe8f413a
BLAKE2b-256 0c327d8c58b21712b4a215cca3b5d9d3a1cd6fd6c299b60e7c98afea7c6360ad

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 948e454c8934f81bcecfb711ac84bffcb3921024478b754ba2991dc6e33ccaec
MD5 788e6a5ca8f843248307dc74f892af3a
BLAKE2b-256 9ddf584f7c9786407000e1d44ce434497c348988c4c03c836409fd6680835f35

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 86bb4aad30e1c18856139dd4ab682008ec982859d54da7465555dc0b4ff1b978
MD5 a4d2da598e423027f69f90f8b9f7dfa5
BLAKE2b-256 42a717900035a83c997881e0ce05a623ea109a5c117ea22a53a79abe20d692bc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e1c34ac14a608c3a6614724adfbfa58ed310bd6dc90277f50e6a829036a63f1f
MD5 c03d5d46f78c8efdf21057e646988b27
BLAKE2b-256 4467a477a3a1bafe8d97dabf3840dfa06da805039ddf72268c325bafdd738dc2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 43485cd039ed1e43a5c598d04194fcb5603e2fe73926b122a72dc9b91d084fad
MD5 ffdea800b5f3f4c075ff96b5261fc294
BLAKE2b-256 ad5a517149740a5d8737101b7f1186a6760f60db1466bf2e999476c9b2d0c73c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-none-win32.whl
Algorithm Hash digest
SHA256 dd522c9e7261c4b0253f1dc549dc947656eb1ca4263ecba1ad7c93cbb34b9dd6
MD5 deebd368f41e0eba90bfb68c6337ced8
BLAKE2b-256 13a775dfe783d914108f2fd60416b2347c38a092fc8b7fff5a8b4a0d05de5b5c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6090e3841ac4bff88c00405e65bd4812339e4d2760d436cbb45a6a34e2bb07b6
MD5 7ac369d2c1224cd1f8a127a9173fd462
BLAKE2b-256 f38db97b0f972b607a1c96abed637c9b3f27599706d539e3910694c5f69dde2c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 b688738262bcabeccebe6e74ddf87b727a9e01b96ac0aead90e475085b542018
MD5 7ab8333bd9ef8bf6f06d112c3da59d05
BLAKE2b-256 e5993d17f2903b22313846d64815421d099fca840248a476bc5b525cff9ca279

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0d65065a5d732276f3d654e6b11317758c6d37ff39d666ff22c868b606835321
MD5 4c61482481bbe38d61f4e54b077e8613
BLAKE2b-256 e4efafa6582c18e2b03b078d0d5d7047b7bf2d73c1078415709a380bf54d4e65

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 02e17254b27826668bd1740c89a31d98fa9f7cbd5e9750c7f8cd56ac30dfb38c
MD5 242fb8d7cbba577b38c393b387f3fe6b
BLAKE2b-256 57d76684e75f2637791ab0214381cb463a807b93d8bccd7d8d3c98d3a424e997

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 9320f3119bebedd1d13d3433beb2b8fc6af8a1649ab0e5177fdc2980327e5ae6
MD5 503b950a80e9c503b986a1c9abddb6fc
BLAKE2b-256 c1d914c0db14a8941ad0b4d50ac48691f9fe5ae6d56e4521aa128b2ecdb17f1a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 ebf44ebf79812ba2cc17c885c3afeceab5e17ef45ed8b6c4812bef761ed6965d
MD5 06734b71a4fb9a16cbb11fd89749a12d
BLAKE2b-256 307e903fba3c5d0b63a7eae2bbc040f6e3c12323f86dee0c01a45b22592499cc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 c74e76a91feb089a10eed318e5c6f38c619e89cabb42d40606afca015b996c67
MD5 2d36ca78b68bd142ff3dc470ea1f1002
BLAKE2b-256 4635fbe94113d56c057df26d8c54c803698085802487057a701667158f1a7cf8

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 9c7054c55cfcff5c41a1d047648c2c6d7b5eb47b940dbdb766f41685f51aa230
MD5 dc02591a3138a834adad361214e012e9
BLAKE2b-256 5369beb3f1742e71bd688d72cee9cbfeec3d96822adde2dc94097a1dd5df028d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1f84bc30ae2c590ec5996f56ac63803b8dcf0c93a30f9cee0950da8d5d207f79
MD5 afb781e04267ea0101a16e2240ba5011
BLAKE2b-256 159e2f9cadbb04561e15f17e60087b25e95b2a41de5bc062b2a334b07183ce01

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 505383e259912941a3b9f883f82dc7bef003ee2bd44534e1f96ce7258c3abcbd
MD5 963c3b4f65a879bbddff005d0352b116
BLAKE2b-256 e40e1d51ca208fd91e72c12c17b56fed72720fcd82479f8ce253eee755cb1cdb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 17f25316f21d0320092d113e04731f5f76732b965c5de36ab9e831cd35d455f1
MD5 9576bb3144824ff260a905e7cb674b88
BLAKE2b-256 62c60ce2dc9d739461a637775f06ee17f82c473997a5a4b5e09abfdb21f484d4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-none-win32.whl
Algorithm Hash digest
SHA256 f4cf2288f543b8c010fa6cb3e155f75591e9010785520faedb369d72fb23d5aa
MD5 434452b075359986499f3c03f97b8bf3
BLAKE2b-256 25522c7e691a3390e5b2bed61b34e089c2ce22527566e1b2c0e3a5827737d939

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 7dbb507b4b77d2e9471ac2a26bc1cb7c771673efb81efd17183511766d6a3645
MD5 d99e94cf7e563d03660b96bc3e6aa41b
BLAKE2b-256 546021d87996c5a92f5b3d009b01b021a93080ef56cb15b284b355988ce6aa98

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 38605220f5cae7cc676626452d05f4cf6fe5ac7e549eb1fe34b565e3459ba970
MD5 d0cfcd9b7de2f771992bf5ca30f6338c
BLAKE2b-256 c518e7b56946b1e6bb9dcc49322f4c4af5928c10450c4cbffefecad4bc6783f5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 44f9290e752840e6030b97bea40b1291858439acf45c149ac6d06ce20fd1c59d
MD5 4817596815ca945ba1285295ec3d6fda
BLAKE2b-256 8e0b9ff480c4f9bb3c542c8c4cc0c7b77a354322d19b48157bf772a98f68a7eb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 76a15ad1b59223378bdbbb0b65a13df4704501e74fa2d5bb7712d16710767c68
MD5 07c972ab3c7cba16a4a917a89c4e88ab
BLAKE2b-256 68fc46a95b437e791a2127c083f98ca7ad9e64b0bc1f4f7d529785b26aa009d1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 022cf98ba896d9001a5bd7ff6ed08eb5d340b4d118d4f8d93531316cdd194e03
MD5 9d6f2ecdaee25f8d509a9691b04dfd31
BLAKE2b-256 28978d81eca83f13db1b7acac7ad449967c1775b410a00fc782e51ae03e2fe77

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 9bc790aaa1ff63016ddbc22f984aedbb67c4ef124aaa541f0c42b295804c31d3
MD5 0f337bd9c22ac7816f96396a810ab665
BLAKE2b-256 ed1e60cc151b11f42d961f27a1a477bcac860ac6d5a48f9d1d02eb68c30878f7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 b579d2908f70be0cf4f13822f14f23856fbc8d35ccbadcaba3c161d58340954d
MD5 f6bddfbb799727236eca75e1c23773d8
BLAKE2b-256 aeea6e56f732790c7d54add62053e9cd0618bd4dc607869e03caead861c7e1f3

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b192bb56b378552ab9db5d1efee91c155dd2b8532d4e1267588a7661dddbdd12
MD5 eceb725eca1cd1d09d30d6be081e55dd
BLAKE2b-256 1f99df4467adbaa48586148233badf75c7196b81adbfd9bb87462aec9270ee70

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f408678f760eabdb0aa04b501a743a78b2faf4918e598f479ffbf8059937f0af
MD5 32688b29a8474f0b792f4ec1dbfc9920
BLAKE2b-256 0554d0bde48eadeea293790b03f13ea86be9e3f4c5c24f3b0af90dd7c1bfbb6c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 46d5a8d18003133fa3f082ca309bf142b18001142f1c37708cfc39492712839b
MD5 10ff63aad3752df05d752549024ea7a1
BLAKE2b-256 cd83e8caac6751e813652646966bbcc7e692007610ea92a0f611ddcbdb917a47

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 d446fe5673339e3fd53e418bce19d4b3cd216386cef760a6915aaad84aca7b92
MD5 c46802af2d61cf3e074967a56477e990
BLAKE2b-256 19769cba76bfe566dd06bd8c9f3fc89f489f15bf5c77566d55c2fdb7f555511c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-none-win32.whl
Algorithm Hash digest
SHA256 b66459056bfd4dbb9937feb65d09b1b77d7ea9ee34bb3cbc4a1d05a1c6d868e4
MD5 2277bfbdfce4ef0acc236a4a919a62c7
BLAKE2b-256 a0e92ff5b135fe178b0e9b240c3a57b3113fb4c9f76e614ebebcf6b590c6c569

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 a4ecc576bf0016b849938fc0da9a61aee71e6c1f0c45d00cd67ae3b95cfbc438
MD5 2cb21c35e6d66b9cc946d3835f9912c7
BLAKE2b-256 cee9394e02206e4bf807a7dfd67abe78300c740db4d7b5a08e6407404ec146a3

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 78c0bab2815df08c1337a5609ec1240fb55a896dd25440b2364eadc8363c1c6b
MD5 cc57f0ab60930a7d5305e4f31856cb22
BLAKE2b-256 4cdca643ebcf047c2580b176e48e9bab347c6532382ec9d0943cdf95fe6a55b0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ad515a242d9fadbb066ca920a7c430c133e7f7b2b44d4e6b48d346758b643fb6
MD5 345841946f0f9a30f103fc0aa1bcfd4d
BLAKE2b-256 1eb7cd468ded24e13a79ac1eb22e766bf8c39e789a2c54a9fdb1f18c262af92f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 b3bfb45c114b414040a2043be182bdff8e8b49a80ae192335629269995e8ec6c
MD5 5ab2d775c44575c4bdeaebb70069f674
BLAKE2b-256 0d9e22c46f69c46bca0e977d0d258707bcd67edbb52d5006617a9b08485fd028

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 3e557b5df5f7197268811d71135d47802b06ebaf0686913a3a289cfb24d453db
MD5 d984074f18174a47a25a2261d6921235
BLAKE2b-256 b4b3ea26abef0dd821923f77321f3bcc695dee43c78ecfc22a2a8353547edef0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 fa8f00234a61b5a06e90a69f6d827470305c243a54c982db738c51cbcfa3c9ae
MD5 c98e4015ffaaeb3e07d6af0ee728a71e
BLAKE2b-256 58b59d5aa8955653be84720175128ab8623e9801f432a702153a4508e30c367d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 b7b1e6e6accb6171ce3a68c709aa46011121dbdd3c33fc272bbc3aa4105dcfb2
MD5 413b9217782986e80b595b0f39cff0a2
BLAKE2b-256 c6b02112809e013b5186f82f24cfbe381b3b5aac49971d83b2230d518a3e0386

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f697dceb857cec62da601236a80d48fb5f7926df682346cc39d22239d2732ce4
MD5 9725ff5f296cca0e39ce2e62f91e0768
BLAKE2b-256 5f9c83d856bc714dbbd0b0e6043dd481bf20a97b4d390de95c97b3101332bbcf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b9fe82053b945db10c7b02c498219af1e1113a1b577de0933e6a5aa2f60b3802
MD5 04f2737bea13bfe56fec9e21e74845a4
BLAKE2b-256 f67f25fbb14f80a69d01255621f6918b8231415635dbbcc4a7b8b906845de95e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2a825a87e36d9d1bb7ff3e06adb6234c4ee88f4629b5941e88a77805f77184ab
MD5 5ffa4b1a2e35d278fd176b67afee1846
BLAKE2b-256 7fa0793c934a455e7a29494ee9473ef4bf5527fa57217e05396f82d5e1227e21

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 d35a444e18b9d46edc1fd8987b35c2f42aaa63bfb2d2ddf58804411af9497a28
MD5 e73c90b81578b89b16ccb965250e9c58
BLAKE2b-256 00067c85c1232ad2ee21d1a82162af36fa56e951eb765092a65738df5beb1c2b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-none-win32.whl
Algorithm Hash digest
SHA256 3984cd4a62d31abc45dbfd5b13491c8fb8cccff94923b89d58adb2a9466884e5
MD5 0cbb80806e2ce945eb94a7c58097176d
BLAKE2b-256 8b07d819baa789bc00a0bca9fbb3b07a70bc6acd8f3650d9f6683dfd12558f8a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 e696f65f2acf340b1f18aa933f3dba3b1a7060bb577f7a13dc37d484ba123568
MD5 8eda0e0c8a57876852352f22a82a1d9f
BLAKE2b-256 5204a44ccc1a6fb069d283b3031a99940f24a03e8015b720e7757a62a0c3f224

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 4942ca9fa749c890ebd42956b6b191ec4abde2b53e6b21f644dfeb651d51877d
MD5 e1bbbe695f19ec8d21d9d1ab6b6316a3
BLAKE2b-256 62c04d2ef08ffa8a65cdb77892465cb38eb892c8036e3dbdcef2fea8ec7f6522

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8442cac68a383153d6e9e43695e789c231fae4803bcf4afbf69d5c5429541a17
MD5 1ffc96fee428bbda3060db26e64e5f1c
BLAKE2b-256 e1bd79cf97d496736fde7d8d09edc610930d64d2a58858fa836b26d5c5d8d38a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 de9cde5aefc9e57f657d98eda7e542b291d32fded0711bff6665db81b337716a
MD5 732e1a222a56fd8fa16cf3d5d743d285
BLAKE2b-256 d8696bec03a96e3c8b72072b9d6e7fcfa9a9fb209d1538aa47d97dec1e403262

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 723eb130519c2fa8cdf559692ceb259969fc5ad1334a34f5976b166be6a17b92
MD5 b28266963694d564991f45aacd8a85ea
BLAKE2b-256 85f4a0eeb982daaeb511e96e7f24d0f5f3141b0a0351d4156cd9398c52a908c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 5fed884455a195b2899c19fa8834fd7782d3c0d97e59e3ca783e5006fd0a66d7
MD5 d175fd6e78e4dfc2f48040974620ea58
BLAKE2b-256 33ee584095726a4f3f076f1425c428cd353f09c4571509558318fd85a0ea5bc7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 96287d8dd02bcaddf706853b37c52d1a89137a86c0ae0a38331c67f13b28e29a
MD5 63ba4e1b616edb15c7f9b2507e93d4fb
BLAKE2b-256 d7fb89a2d16cf640126716f18549c9772a64eee7577a92bc5c3b807464922ab5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 69c09847742e3b17a56bc9ef4bd8452fea9997648a9b51023a2993f422fcc7a5
MD5 39d535619b3d579cde2758c066fe81cb
BLAKE2b-256 36b9ceacd0e17480944d3d24607dde677246ca4a50e91b9d84d1ea88a0eb346a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 97d0eaa4018a883dbb019ff641ab9ec3dc8c0fcabcec609d5e67988228c620d4
MD5 0765fa608c26fae655bc21983c828b4a
BLAKE2b-256 c861ace0882763062bde2416cf3758a35760cea25e16a5ca5732da1ef35c05fc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 a03f303ca24540d4bbe80809433fe2f0fcc9d3059967f319c6641068fc0a1669
MD5 986f2b4e84195cbca1b658907794a30c
BLAKE2b-256 3b9b89ed602808fbd418138e7429d921f76fbccded53f72cf00e51ea09426f2e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 a7579311c05373ab02075b5ff6fe06f93a02893f27abe3bfaba1cce9c98b26f5
MD5 9d4a849c7b9d8c8e6c237acca89d051d
BLAKE2b-256 507998ea974413588adf270cd87390f06927294c36f5dab83e27b429284c92a5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-none-win32.whl
Algorithm Hash digest
SHA256 9e6f7ef1155bd4195114104a2d4d4aa726d7858daa6e8f66db98b149b88a1f19
MD5 1d6dde9d40e6dd049be442ba26df8343
BLAKE2b-256 b41f5ed94a52ec5772f3b7e56112bb914b9c9450086fb5d3694689ca2c307c2b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 627a7e244a6ebc829e839f6d3cea2d28473bfa230e58ab54990ecf1c2416a479
MD5 98f98231f1d2d266b33dd19ee7c679d1
BLAKE2b-256 a21235e006e865eb7aff6ea8917bcca2e0acf1ec287e35e0f1b46577d3a9121a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 7652576dcab2c80b737d12f59bc8ef985cffcd393d77d9233bb9d8698879b1b4
MD5 40efb19618eef9add4d79ae22eb33742
BLAKE2b-256 9d2db396b599a3f0325d379fac22c138063c94d29cb2a027e96f883f2dcba7a7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 86cdf08e92698295a2735c4af1ed783aa12cf7845fb48df69fea1abc6ebe938f
MD5 b0c0c33448eb920164419d23aac79639
BLAKE2b-256 c939afa93d196d41e2cb8c8135b67b4a3125379841fa52a83aca111ce83db2ce

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 86fad99814f7b48d523f694d4b72d6f5a9d21c5b313e493460eb64671405e2bb
MD5 f332cff3139db240ca0aa1b6700d2bbf
BLAKE2b-256 92652d44fae7ba760430816f2083a8aad56d7fd63c5edaa38148ff821ab317fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 33e4e99e84ac336d5aa133f3f43e76840fb35716574302403944bc495e1d1cc0
MD5 2e57cb9a6c519938df81bd9eed214d85
BLAKE2b-256 b3ecbd8eb3e7a7ea79cf0170e29035b7ef57135afc38fbca3030721355e45384

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 0a5571b2f75a49d2332debf53dd5cb7a10c9199c1c558e50483539beeeee7679
MD5 ced06f37173b9ba12b251c42dd5b34ac
BLAKE2b-256 f99fe90b656f269c4850705568bf1fb4a1ab1e929a13dc3178e85f3bb1e945e1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 6dedb1b2798ab91419d0ca8b07b61c419eeb04fd32cee3fafb4385f5b3077eda
MD5 79ab0b2348c12c9d682f4cc28bbc45f0
BLAKE2b-256 fc7a8359cc063964f360f41ef0184392e97bfe7c7a40af6314a72893705a3c2b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 83118d66c00544d974d353772739156257e0cd57d109f23296fc181b079f4605
MD5 2db1a1da061642f04583d68d4fa8cd1a
BLAKE2b-256 190f9e2169b1a3a2a806a841eb2a4133e835dd12524f4f8731513f676a92cf31

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f2927482390de945f1a97f33b8b4091070a9e9e760bc4fc4a8deddcbd1419660
MD5 a5f335123bea6787530c3666db543d77
BLAKE2b-256 e75c029ff3d6d2b965faefef6f75f385c7ecc6d7990bc9ab95eaf3bc33028faa

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6f1eca6035dba115b85ccaea07539abca88edd6aad3658eab26882d4685a160c
MD5 ae2838cefa8a1ac903b312cc3afbce53
BLAKE2b-256 cb20ee88347cb5e9bfc3be7f541c5f7b484cee06ac3e1cf5b0115b03b1311c38

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 4732f542d1c934edb3d67f1691d473bc59bbf06c2a348cd6361fa5c46ac31288
MD5 c093d25fed941577bcb0882c5f800904
BLAKE2b-256 a8991c1d2abfa1609d868ac5169d1b6daf0300f824891f26df2da9964d6ef391

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-none-win32.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-none-win32.whl
Algorithm Hash digest
SHA256 045a56a32cdb51586bf1a650365fc3d7714ed94c2e5cd4fd878d2e8fabbad85f
MD5 4b41473ae82c1201eb76165c9873db0c
BLAKE2b-256 69a6eb1b5eb4001731c0575a010329be3636681de8dce97115c088c2965e5c90

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 e97e91f80cbe7ca5f49b91c89e57924a6a22d03d5d40e240f7d99d9f6300a47d
MD5 d625c5f5c6cf6f143865fd0a12862920
BLAKE2b-256 c31e0d8000e026f20d5cc12fcefefc8cf0541b2fc3b8df496781b344a07ebfaf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 8f400cccd1e07d8adcb3c33eb30e67bd54379e4cfe8ea0b8a030ee9781a614c7
MD5 0ed6dac664ac9e7943be4e55593f65e5
BLAKE2b-256 80cb732d55a28cc5a97a72ea7b0343f1bbb5c8c732827550611cf76e1480aa8d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 c19246de6d9ab964e2ba3cb952ab4ef5b029bbec3473b3d3692d822d7457f18f
MD5 eff51398086e37f625fa2bc0ad17d7ed
BLAKE2b-256 793b02178011976fcef81cd73e0e2452437488f42a7e640756349db3216b7317

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 54dc062d08b743b1ebcb7f55043239956dbbaa653f7ab9063a093192943bcce3
MD5 782bf59accb472b7b2ae9c3a046d1932
BLAKE2b-256 dda72763d5b63bc0070c251381322d08f44a9d5be5ae41c277c1ec809a0f167e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 8429b4a8612004fcaaee2b99e322a183e48148033b0cd318bae77c267fd0318d
MD5 b4da55a49970ec4d3b535a906f21db6b
BLAKE2b-256 f69b99d1a1322472f0ca5018e97ffeabb78d8d9ed5d7fc89f20b70cff3da9604

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 bd807c6088629bef1cb36ae82c087d3dbb805c9096480b25109e03adbcf0cdd3
MD5 ad126255105f42fb45cf986ab20f4972
BLAKE2b-256 1a39bc5bd3005d2abb07eb9fb29b843cdab501528b14c0dda12372c62f45292f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 0b0424fa07d3b3e12b704e5e32235dcef271f2d99e809eeae3ff1ffeb7392600
MD5 f6637638aad693139cd17d4eab008a27
BLAKE2b-256 c93b3cdeedac88c3c0f9c31a00a4a30135c57e11d95c20d882606617a90a9c99

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 88fa8f078ea84bb6a8cf1c062c37ae8804231ee25914a06673a05ca442b7a0c2
MD5 a5c80051ed71f4fb4f2b18c168eb448a
BLAKE2b-256 650dffae7463fafaa91458eba1a106522b097c0f2bb76fc5c1e741eb642f2d22

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0b4ffae0b5e3670128fcb08d977132cf76171fbac0de64d9dc016b9558adff68
MD5 5f0db4004b29197988af8d566b6a99ed
BLAKE2b-256 d445a7e544fa0f948bc7d444d1267d4483fbb5593121df46a7863d67c854ccee

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.1rc1-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.1rc1-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7810ed75f1182781f0dd437159fbf58ad0a964f9dc43384768bcd4a7a346262b
MD5 8d2329e06c81748b6d8e557eb93b5875
BLAKE2b-256 a891b1e6482b24a175adb7751c0aa2d581eee6ad8daaa4340c63f14f8f82f6dd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page