No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
  • License: Apache Software License
  • Author: Anthony MOI <m.anthony.moi@gmail.com>
  • Tags NLP, tokenizer, BPE, transformer, deep learning
  • Requires: Python >=3.7
  • Provides-Extra: testing, docs, dev
Classifiers

Release history Release notifications | RSS feed

0.20.3

0.20.3rc0 pre-release

0.20.2

0.20.1

0.20.1rc1 pre-release

This version

0.20.0

0.20.0rc1 pre-release

0.19.1

0.19.0

0.15.2

0.15.2rc1 pre-release

0.15.1

0.15.0

0.14.1

0.14.0

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0 yanked

Reason this release was yanked:

Breaking change with unexpected effect

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.3

0.10.2

0.10.1

0.10.1rc1 pre-release

0.10.0

0.10.0rc1 pre-release

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.9.0rc2 pre-release

0.9.0rc1 pre-release

0.9.0.dev4 pre-release

0.9.0.dev3 pre-release

0.9.0.dev2 pre-release

0.9.0.dev1 pre-release

0.9.0.dev0 pre-release

0.8.1

0.8.1rc2 pre-release

0.8.1rc1 pre-release

0.8.0

0.8.0rc4 pre-release

0.8.0rc3 pre-release

0.8.0rc2 pre-release

0.8.0rc1 pre-release

0.8.0.dev2 pre-release

0.8.0.dev1 pre-release

0.8.0.dev0 pre-release

0.7.0

0.7.0rc7 pre-release

0.7.0rc6 pre-release

0.7.0rc5 pre-release

0.7.0rc4 pre-release

0.7.0rc3 pre-release

0.7.0rc2 pre-release

0.7.0rc1 pre-release

0.6.0

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.1

0.1.0

0.0.13

0.0.12

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.20.0.tar.gz (337.4 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.20.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ i686

tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.20.0-cp312-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.20.0-cp312-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.20.0-cp312-cp312-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp312-cp312-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp312-cp312-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.20.0-cp312-cp312-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.20.0-cp311-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.20.0-cp311-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.20.0-cp311-cp311-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp311-cp311-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp311-cp311-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.20.0-cp311-cp311-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.20.0-cp310-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.20.0-cp310-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.20.0-cp310-cp310-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp310-cp310-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp310-cp310-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.20.0-cp310-cp310-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.20.0-cp39-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.20.0-cp39-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.20.0-cp39-cp39-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp39-cp39-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp39-cp39-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.20.0-cp39-cp39-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.20.0-cp38-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.20.0-cp38-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.20.0-cp38-cp38-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp38-cp38-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp38-cp38-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.20.0-cp38-cp38-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.20.0-cp37-none-win_amd64.whl (2.3 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.20.0-cp37-none-win32.whl (2.1 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_x86_64.whl (9.3 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_aarch64.whl (9.0 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (3.3 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (3.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl (3.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ i686

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (2.8 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (2.9 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.20.0-cp37-cp37m-macosx_11_0_arm64.whl (2.5 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.20.0-cp37-cp37m-macosx_10_12_x86_64.whl (2.6 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.20.0.tar.gz.

File metadata

  • Download URL: tokenizers-0.20.0.tar.gz
  • Upload date:
  • Size: 337.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0.tar.gz
Algorithm Hash digest
SHA256 39d7acc43f564c274085cafcd1dae9d36f332456de1a31970296a6b8da4eac8d
MD5 16a475d69d5aebb667f764899128b1a4
BLAKE2b-256 023a508a4875f69e12b08fb3dabfc746039fe763838ff45d6e42229ed09a41c2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 6324826287a3fc198898d3dcf758fe4a8479e42d6039f4c59e2cedd3cf92f64e
MD5 2b18069e2d3079d13881c0ae4fcc9f24
BLAKE2b-256 9ffa075959c7d901a55b2a3198d0ecfbc624c553f5ff8027bc4fac0aa6bab70a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 a1158c7174f427182e08baa2a8ded2940f2b4a3e94969a85cc9cfd16004cbcea
MD5 448a8e731277e5085bf08d9f043c9c91
BLAKE2b-256 bd7afde367e46596855e172c466655fc416d98be6c7ae792afdb5315ca38bed0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e26e6c755ae884c2ea6135cd215bdd0fccafe4ee62405014b8c3cd19954e3ab9
MD5 867e72c229720c18f941ecbcd637ddf1
BLAKE2b-256 775aa59c9f97000fce432e3728fbe32c23cf3dd9933255d76166101c2b12a916

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 b3dc750def789cb1de1b5a37657919545e1d9ffa667658b3fa9cb7862407a1b8
MD5 170dbdcf1e71c3a43c1c2517ef2df82f
BLAKE2b-256 97107b74d7e5663f886d058df470f14fd492078533a5aee52bf1553eed83a49d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f830b318ee599e3d0665b3e325f85bc75ee2d2ca6285f52e439dc22b64691580
MD5 95859c5007aa0229c35107a746b297a3
BLAKE2b-256 182f35f7fdbf1ae6fa3d0348531596a63651fdb117ff367e3dfe8a6be5f31f5a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 23f9ecec637b9bc80da5f703808d29ed5329e56b5aa8d791d1088014f48afadc
MD5 155c83fc4776fef5314aaff67fa7d7fe
BLAKE2b-256 79e6eb28c3c7d23f3feaa9fb6ae16ff313210474b3c9f81689afe6d132915da0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 d68e15f1815357b059ec266062340c343ea7f98f7f330602df81ffa3474b6122
MD5 873ab0b659d697db802a407262b50ca5
BLAKE2b-256 cd99dba2f18ba180aefddb65852d2cea69de607232f4cf1d999e789899d56c19

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 3e14cdef1efa96ecead6ea64a891828432c3ebba128bdc0596e3059fea104ef3
MD5 5b089cd9ed28c158c16d4ed6792af4e5
BLAKE2b-256 465c6d9b722228cb25b144db64da330efa429ca90398eac564865584d9237f3d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 1e86594c2a433cb1ea09cfbe596454448c566e57ee8905bd557e489d93e89986
MD5 38057647e88e1ecc7ea1f62f39cbf025
BLAKE2b-256 e500b50015d92416bc51a740f038e082ad4695cd80f6cec81ade42755120d75c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0b6466e0355b603d10e3cc3d282d350b646341b601e50969464a54939f9848d0
MD5 159e2920bba14425bbfc926f8347352b
BLAKE2b-256 b20e007b9bfc95213183534fe941507b238e39e1f65e97bfbd9da0f5319aa1f7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 0994b2e5fc53a301071806bc4303e4bc3bdc3f490e92a21338146a36746b0872
MD5 6c0934ee77c870cc9c4c470b69db1156
BLAKE2b-256 2efe408cc043e5b89c39a3850d91e2424ed63fd63486d0516ed6659bdafedefd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 a07eaa8799a92e6af6f472c21a75bf71575de2af3c0284120b7a09297c0de2f3
MD5 250d2edb0a1a97c73609d41190b2d86d
BLAKE2b-256 ce643428b3da1a718ca160deae1fc1cc12a25a81e48e6833f13d826f4318d70d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9afbf359004551179a5db19424180c81276682773cff2c5d002f6eaaffe17230
MD5 8b0eca4aefa967068ac3019afae23698
BLAKE2b-256 17369e427afeaf8390ca1d3d054f17e75e5b84836f3c195c986e74bdb9c1f400

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e47a82355511c373a4a430c4909dc1e518e00031207b1fec536c49127388886b
MD5 82ffe48bdc7a4b87d95bd89d21f43bf5
BLAKE2b-256 b35b13fb7714f5ac5cc1f86340ae1fced8ff7b6fa4a47cfefbf3d461d9d78b22

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 c8842c7be2fadb9c9edcee233b1b7fe7ade406c99b0973f07439985c1c1d0683
MD5 ecc40c72b1fc64cf8e3a2b989c0bb7c1
BLAKE2b-256 a8862815e4e11c7b5ddb44f9aa364f0f115a1d54eb5cbe70ea92d4c4618b0170

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 e738cfd80795fcafcef89c5731c84b05638a4ab3f412f97d5ed7765466576eb1
MD5 6762c06d1ad62f2d335e58843d4f71b5
BLAKE2b-256 fc70b2c51b95278d0f0b7c296d918b0e56c38a2c2a70d9c2db4fdc6e06dafd0b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 df6b819c9a19831ebec581e71a7686a54ab45d90faf3842269a10c11d746de0c
MD5 f2e9584cfb3981527a27b651ca4abfc6
BLAKE2b-256 20c720650756d693acf4ee42a1cc45cb2c56f6fe5ab64e5d6a1de360384736ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 983a45dd11a876124378dae71d6d9761822199b68a4c73f32873d8cdaf326a5b
MD5 03729c36834b51ca6c08c6b1fc29b5b4
BLAKE2b-256 c5f9b0a259506adf0609e08536fa27fcac6ad4dbca6e967cd98b1dda28df6fd3

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 2b8cf52779ffc5d4d63a0170fbeb512372bad0dd014ce92bbb9149756c831124
MD5 a188e693675169cf955ea593d6e20810
BLAKE2b-256 0f92a9d708814c2e515dbb764acf3b38d737e04d5b44b9902d443d78da3d96b4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b52aa3fd14b2a07588c00a19f66511cff5cca8f7266ca3edcdd17f3512ad159f
MD5 8d5125a40d3a125bd433dc6e3614fd36
BLAKE2b-256 b104da7f222c1b2b50bf8935108697ca93b4623d1f9618b4225485c14a9736bb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 e3144b8acebfa6ae062e8f45f7ed52e4b50fb6c62f93afc8871b525ab9fdcab3
MD5 ea8dc24495c6c16bb0d6c76edfec15d1
BLAKE2b-256 44649c0b2f946ead477e94c7c390c9664bd155c9f11b718370eeff4282493467

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 055ec46e807b875589dfbe3d9259f9a6ee43394fb553b03b3d1e9541662dbf25
MD5 c471938de69789c70a17c38b18788ae6
BLAKE2b-256 ca93af21d827b51cc1682a0c65047dfd354771c756912f5f815485fdc3c4e414

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 302f7e11a14814028b7fc88c45a41f1bbe9b5b35fd76d6869558d1d1809baa43
MD5 ec92d1a707779d7a39cc9fb378fc23f7
BLAKE2b-256 88319b3deac5e9d90ed0a47c181d597c0b3becd5013c33292f08b1663bcde10c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 da06e397182ff53789c506c7833220c192952c57e1581a53f503d8d953e2d67e
MD5 d9cb107a62288bf7c9c4c11f2768bf87
BLAKE2b-256 6792c490f16f1124d62ef018c34ad0f461cb4a36fccb52d6b10eb7e978315469

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 07bef67b20aa6e5f7868c42c7c5eae4d24f856274a464ae62e47a0f2cccec3da
MD5 430816514c1d913fec935c2c57741bb5
BLAKE2b-256 53fc612fa40f1d1e95d45a613aa48f8b97462877d53c1a3cab1ffa95ba8ac694

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b8a2dc1e402a155e97309287ca085c80eb1b7fab8ae91527d3b729181639fa51
MD5 2beb3ca0fb14beb352776f295e9fb4ad
BLAKE2b-256 b85a49787f6659541b65ad6c6549044a5d090662edc259a91f21346f4aca0b66

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7d8653149405bb0c16feaf9cfee327fdb6aaef9dc2998349fec686f35e81c4e2
MD5 28d07f09ab190d5c93584b022ae62ae4
BLAKE2b-256 57531f72bee530bbd3a18d4159b8240701203de491338d59d72aff54ae33beb2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 f18661ece72e39c0dfaa174d6223248a15b457dbd4b0fc07809b8e6d3ca1a234
MD5 164ec41e7df5ef32b0c98fb9eeddc760
BLAKE2b-256 a5459c19187645401ec30884379ada74aa6e71fb5eaf20485a82ea37a0fd3659

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp312-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp312-none-win32.whl
Algorithm Hash digest
SHA256 ec53e5ecc142a82432f9c6c677dbbe5a2bfee92b8abf409a9ecb0d425ee0ce75
MD5 9a3bbb9240032baec3eaa9553b29bc64
BLAKE2b-256 0075426a93399ba5e6e879215e1abb696adb83b1e2a98d65b47b8ba4262b3d17

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 e5e56df0e8ed23ba60ae3848c3f069a0710c4b197218fe4f89e27eba38510768
MD5 2a098e441cc5053b4b523f354b5a0f67
BLAKE2b-256 e6eb3a1edfc1ffb876ffc1f668c8fa2b2ffb57edf8e9188af49218cf41f9cd9f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 baf5d0e1ff44710a95eefc196dd87666ffc609fd447c5e5b68272a7c3d342a1d
MD5 f35d8c5de1d87cb918ee9d6c5fef75cb
BLAKE2b-256 06e878f1c0f356d0a6e4e4e450e2419ace1918bfab875100c3047021a8261ba0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e48afb75e50449848964e4a67b0da01261dd3aa8df8daecf10db8fd7f5b076eb
MD5 7c7e0540ac466421431644e6eeb3309a
BLAKE2b-256 3220a8fe63317d4f3c015cbd5b6dec0ce08e2722685ca836ad4a44dec53d000f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 3d7e559436a07dc547f22ce1101f26d8b2fad387e28ec8e7e1e3b11695d681d8
MD5 869a892732f434076486b9fb11464577
BLAKE2b-256 627c4e3cb25dc1c5eea6053752f55007071da6b33a96021e0cea4b45b6ef0908

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 9c1b88fa9e5ff062326f4bf82681da5a96fca7104d921a6bd7b1e6fcf224af26
MD5 6aac02e46cae319124daf62efd8f0eeb
BLAKE2b-256 160b099f5e5b97e8323837a5828f6d21f4bb2a3b529507dc19bd274e48e15825

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 626403860152c816f97b649fd279bd622c3d417678c93b4b1a8909b6380b69a8
MD5 344b2bb631b6588438bac194528ea739
BLAKE2b-256 0fb2f212ca05c1b246b9429905c18a4d68abacf2a35214eceedb1d65c6c37831

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 8768f964f23f5b9f50546c0369c75ab3262de926983888bbe8b98be05392a79c
MD5 9de3d95e07230b09b6e405f44a387310
BLAKE2b-256 2de659abfc09f1dc23a47fd03dd8e3bf3fce67d9be2b8ba15a73c9a86b5a646c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 06caabeb4587f8404e0cd9d40f458e9cba3e815c8155a38e579a74ff3e2a4301
MD5 214346cd15539fb4c5a489370210cce9
BLAKE2b-256 24fa77f0cf9b3c662b4de18953fb06126c424059f4b09ca2d1b720beabc6afde

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 6a531cdf1fb6dc41c984c785a3b299cb0586de0b35683842a3afbb1e5207f910
MD5 7ef2089f9f276de4f56e6c17e13fec0e
BLAKE2b-256 6e99594b518d44ba2b099753816a9c0c33dbdcf77cc3ec5b256690f70d7431c2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 15c81a17d0d66f4987c6ca16f4bea7ec253b8c7ed1bb00fdc5d038b1bb56e714
MD5 11e1403d12a8595b5340bd01e12766d0
BLAKE2b-256 ced4152f9964cee16b43b9147212e925793df1a469324b29b4c7a6cb60280c99

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 2b709d371f1fe60a28ef0c5c67815952d455ca7f34dbe7197eaaed3cc54b658e
MD5 90053205d138b54d43c625daea32a382
BLAKE2b-256 c03c9228601e180b177755fd9f35cbb229c13f1919a55f07a602b1bd7d716470

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp311-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp311-none-win32.whl
Algorithm Hash digest
SHA256 a4bb8b40ba9eefa621fdcabf04a74aa6038ae3be0c614c6458bd91a4697a452f
MD5 2f9bee8c67aef57fa302f85e6a0ac598
BLAKE2b-256 ba7870f79f939385579bb25f14cb14ab0eaa49e46a7d099577c2e08e3c3597d8

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 c3124fb6f3346cb3d8d775375d3b429bf4dcfc24f739822702009d20a4297990
MD5 f2248dd18f34c13e637dc0f86ac9c806
BLAKE2b-256 fb71b9626f9f5a33dd1d80bb6d3721f0a4b0b48ced0c702e65aad5c8c7c1ae7e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 31e087e9ee1b8f075b002bfee257e858dc695f955b43903e1bb4aa9f170e37fe
MD5 da735a9b045033983132bc3c6fce2b49
BLAKE2b-256 b908ac9c8fe9c1f5b4ef89bcbf543cda890e76c2ea1c2e957bf77fd5fcf72b6c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 0b5c7f906ee6bec30a9dc20268a8b80f3b9584de1c9f051671cb057dc6ce28f6
MD5 33f3a86d1c2694642f1186fd96be7ec3
BLAKE2b-256 a50ee4421e6b8c8b3ae093bef22faa28c50d7dbd654f661edc5f5880a93dbf10

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 2a5ad94c9e80ac6098328bee2e3264dbced4c6faa34429994d473f795ec58ef4
MD5 0d2263634487218f5f25f14b625ff7a0
BLAKE2b-256 71aec7fc7a614ce78cab7b8f82f7a24a074837cbc7e0086960cbe4801b2b3c83

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 ca6557ac3b83d912dfbb1f70ab56bd4b0594043916688e906ede09f42e192401
MD5 b4be1fe411326059c451236f525e1cb5
BLAKE2b-256 bc14193b7e58017e9592799498686df718c5f68bfb72205d3075ce9cdd441db7

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 06c0ca8397b35d38b83a44a9c6929790c1692957d88541df061cb34d82ebbf08
MD5 ed352425df4614a203316891bf79faab
BLAKE2b-256 227c02431f0711162ab3994e4099b9ece4b6a00755e3180bf5dfe70da0c13836

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 07bbeba0231cf8de07aa6b9e33e9779ff103d47042eeeb859a8c432e3292fb98
MD5 84642c6ab1c3bf5b6f941cd6e31b1cc1
BLAKE2b-256 028022ceab06d120df5b589f993248bceef177a932024ae8ee033ec3da5cc87f

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 cce124264903a8ea6f8f48e1cc7669e5ef638c18bd4ab0a88769d5f92debdf7f
MD5 80610d03b265faed34c664a0dcad017c
BLAKE2b-256 951e800e0896ea43ab86d70cfc6ed6a30d6aefcab498eff49db79cc92e08e1fe

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5ec603e42eaf499ffd58b9258162add948717cf21372458132f14e13a6bc7172
MD5 c02d38cf1644e59d38cf9929aa5fbf97
BLAKE2b-256 628bdab4d716e9a00c1581443213283c9fdfdb982cdad6ecc046bae9c7e42fc8

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6636b798b3c4d6c9b1af1a918bd07c867808e5a21c64324e95318a237e6366c3
MD5 c7b9929bbaecefd63c7ebc246cf6f72b
BLAKE2b-256 a4f6ae042eeae413bae9af5adceed7fe6f30fb0abc9868a55916d4e07c8ea1fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 0a61a11e93eeadbf02aea082ffc75241c4198e0608bbbac4f65a9026851dcf37
MD5 aa028f068e4eb67c7f91f953fb0514c2
BLAKE2b-256 639084534f81ff1453a1bcc049b03ea6820ca7ab497519b79b129d7297bb4e60

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp310-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp310-none-win32.whl
Algorithm Hash digest
SHA256 95ee16b57cec11b86a7940174ec5197d506439b0f415ab3859f254b1dffe9df0
MD5 d87afb88f77f4418cee90d57f1c78008
BLAKE2b-256 32165eaa1405e15ca91a9e0f6c07963cd91f48daf8f999ff731b589078a4caa1

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 ca4d54260ebe97d59dfa9a30baa20d0c4dd9137d99a8801700055c561145c24e
MD5 c4e447979441700964a44f6305811026
BLAKE2b-256 27cb76636123a5bc550c48aa8048def1ae3d86421723be2cca8f195f464c20f6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 8029ad2aa8cb00605c9374566034c1cc1b15130713e0eb5afcef6cface8255c9
MD5 3d41c024ce3858fcd9b2dbe7ce1eb6db
BLAKE2b-256 ea3dd573173b0cd78cd64e95b5c8f268f3a619877bc6a484b649d98af4de24bf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d7dfd796ab9d909f76fb93080e1c7c8309f196ecb316eb130718cd5e34231c69
MD5 0b83600520abcf9d27a9ad3fbe7169f6
BLAKE2b-256 fc7e794850f99752d1811952722c18652a5c0125b0ef595d9ed069d00da9a5db

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 9a81cd2712973b007d84268d45fc3f6f90a79c31dfe7f1925e6732f8d2959987
MD5 cabc7ceef909c5fd97ea31e20a917fce
BLAKE2b-256 874352b096d5aacb3eb698f1b791e8a6c1b7ecd39b17724c38312804b79429fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 a11435780f2acd89e8fefe5e81cecf01776f6edb9b3ac95bcb76baee76b30b90
MD5 0535a75c3bb3878d3a21bdc5846bb698
BLAKE2b-256 c198f4a9a18a4e2e254c6ed253b3e5344d8f48760d3af6813df4415446db1b4c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 3d7074aaabc151a6363fa03db5493fc95b423b2a1874456783989e96d541c7b6
MD5 d1cba864f1e66a5172a0125ee4de8e7e
BLAKE2b-256 d0619f3def0db2db72d8da6c4c318481a35c5c71172dad54ff3813f765ab2a45

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 053c37ecee482cc958fdee53af3c6534286a86f5d35aac476f7c246830e53ae5
MD5 820ccacd086350f95ef22eff13a5259f
BLAKE2b-256 35077004003098e3d442bba9b9821b78f34043248bdf6a78433846944b7d9a61

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 68cc7de6a63f09c4a86909c2597b995aa66e19df852a23aea894929c74369929
MD5 2aa17c43d8ed52608aac91f44dd80000
BLAKE2b-256 4ef51087cb5100e704dce9a1419d6f3e8ac843c98efa11579c3287ddb036b476

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 62a56bf75c27443432456f4ca5ca055befa95e25be8a28141cc495cac8ae4d6d
MD5 2ca40051a49d5eeaf3191fcf197bc7d6
BLAKE2b-256 fce574c6ab076de7d2d4d347e8781086117889d202628dfd5f5fba8ebefb1ea2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 6cff5c5e37c41bc5faa519d6f3df0679e4b37da54ea1f42121719c5e2b4905c0
MD5 81c0643f59c763e79d1631e5a9740474
BLAKE2b-256 0d4788f92fb433fe2fb59b35bbce28455095bcb7b40fff385223b1e7818cec38

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 d90188d12afd0c75e537f9a1d92f9c7375650188ee4f48fdc76f9e38afbd2251
MD5 7114e36249d9d87c2f2436789eeab8f4
BLAKE2b-256 3a184baa182c8a110c3d36582a135007afd5d1b065405d0b623545d8d6f27c80

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp39-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp39-none-win32.whl
Algorithm Hash digest
SHA256 8a3d2f4d08608ec4f9895ec25b4b36a97f05812543190a5f2c3cd19e8f041e5a
MD5 8e1f5ba0875faea792ae8a57f4e2c846
BLAKE2b-256 74ca3ab1eec8fe65d42daf27384aebde8bee3fc8cb8c06af680d87f6e47ddc1a

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 7c157550a2f3851b29d7fdc9dc059fcf81ff0c0fc49a1e5173a89d533ed043fa
MD5 77d059329ac23cada37e2802999ab600
BLAKE2b-256 f7b6fc5162f6465310470676d786e882a13946ea78d553c9c753ad4238c130bc

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 980f3d0d7e73f845b69087f29a63c11c7eb924c4ad6b358da60f3db4cf24bdb4
MD5 4efdac4882be1091ed537315a2c8c46d
BLAKE2b-256 e04a1613e7063c56cdf052bc166d14f4dc7ad7eb5cc0b59e526fc9aab1372544

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6eec0061bab94b1841ab87d10831fdf1b48ebaed60e6d66d66dbe1d873f92bf5
MD5 659aec8ff69238d263a87b017275cd6d
BLAKE2b-256 0c5224180c93a1a3d0f386d66a7f90e0e597c238667fd6d1689b3688409b2dd6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 ef703efedf4c20488a8eb17637b55973745b27997ff87bad88ed499b397d1144
MD5 d0e694b4f374a366e54e55ada9a343fd
BLAKE2b-256 d0f5809a4a8d84164fb44068d293ae41b04344e9b2593d9915268b4e6bcc27ea

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 9859e9ef13adf5a473ccab39d31bff9c550606ae3c784bf772b40f615742a24f
MD5 19e6299fcb2de23a205700b9ebfbe0eb
BLAKE2b-256 e335ccde410799e1493fae49dd4e50c81e08668409fd65dd808dce15969a17cd

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 4806b4d82e27a2512bc23057b2986bc8b85824914286975b84d8105ff40d03d9
MD5 d03a6580a39adac1381f602dbfbd371f
BLAKE2b-256 7f2d7a7a9cfdd0a7051baa837577caa3a5c5f3c7517035e2bc2ace5113234d40

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 2b69e55e481459c07885263743a0d3c18d52db19bae8226a19bcca4aaa213fff
MD5 0e43277f096d0d22710470dc4b398232
BLAKE2b-256 af214dfbba26751f5853d704cf84941bf02e4901f13f7c2746655dd78b4c816e

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 fc1e1b791e8c3bf4c4f265f180dadaff1c957bf27129e16fdd5e5d43c2d3762c
MD5 ccde78673a179fb18b17ce8cc809b879
BLAKE2b-256 f4fd66831ff46aea9befcb1e5d0921cce1c4ea5421a8f2136ada118e71bbcacf

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b5ef0f814084a897e9071fc4a868595f018c5c92889197bdc4bf19018769b148
MD5 e7b48515f2bd4b44a72a0cdd6d311a58
BLAKE2b-256 b82045c46c8ad54e3b5bee585fd3e3033ba3de2543d4f65894967c36f94555b9

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 aa2d4a6fed2a7e3f860c7fc9d48764bb30f2649d83915d66150d6340e06742b8
MD5 ddc9f6ebacfd9ab0792d31c4444f5035
BLAKE2b-256 7329b00f299052ee2e5b9e9903df44efa1de3e3bf14d9721a6fe07ef7c7a932c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 30ffe33c5c2f2aab8e9a3340d0110dd9f7ace7eec7362e20a697802306bd8068
MD5 e62f55baeafbdd3d06129339ec98b85e
BLAKE2b-256 8037d30bc7e93cdaebf48545acb653451c7d589cfcdeca5c01e7298e5403ba25

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp38-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp38-none-win32.whl
Algorithm Hash digest
SHA256 55a393f893d2ed4dd95a1553c2e42d4d4086878266f437b03590d3f81984c4fe
MD5 7b0ff5669742e4f2fc28d8de13481a5d
BLAKE2b-256 c0984fe299b711541c1772333726ac3e77fd016c4b8e1d4b4aa7c1561660f37c

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 392b87ec89452628c045c9f2a88bc2a827f4c79e7d84bc3b72752b74c2581f70
MD5 34978202a5c5f0d49e63c7a6173955a9
BLAKE2b-256 09cccf3520af1c2356e039f20ff03d9944037fd8f3137b21f4d7e5c217f1511b

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 459f7537119554c2899067dec1ac74a00d02beef6558f4ee2e99513bf6d568af
MD5 677e768b076f6a2ccfd29c2abd4fcef4
BLAKE2b-256 3db7e25f197c57dc58ae8313b1f3fea5dcdc3e2cff37c8f38d3b000293919656

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d98b01a309d4387f3b1c1dd68a8b8136af50376cf146c1b7e8d8ead217a5be4b
MD5 d948a4a1fe5877341f2a07e3f4e24429
BLAKE2b-256 1e8f0daf4b105571f35adb2e7feee2f87b23d22f0aad5cb4c823a8b4c0a22c14

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 e359f852328e254f070bbd09a19a568421d23388f04aad9f2fb7da7704c7228d
MD5 5d6609bc49e1e35a9266eda9739adad7
BLAKE2b-256 73b703541a39095754cff4ca2589012c44d713df673cefce37f3e1d571aeea06

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 c375c6a889aeab44734028bc65cc070acf93ccb0f9368be42b67a98e1063d3f6
MD5 71291cc7d748c4f007172f113acb7c22
BLAKE2b-256 df854b2f37e499f6322fd2ab0a8516eb893b782df0b14493fb96536ae50f5f76

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 9a9643d9c8c5f99b6aba43fd10034f77cc6c22c31f496d2f0ee183047d948fa0
MD5 9bb29c105ea6e4eb6236d989a5449b71
BLAKE2b-256 7bb771d12e4aa2fbd57adb0c78c2d5785fb9654354de4d2c73c3cadf48de2511

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 8f868600ddbcb0545905ed075eb7218a0756bf6c09dae7528ea2f8436ebd2c93
MD5 969f23b70f2e712746b47af983c24251
BLAKE2b-256 1d869c459d9b5415a756b8c002f13a4dab175424a9c1089faddd5a3c140b73ee

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 9253bdd209c6aee168deca7d0e780581bf303e0058f268f9bb06859379de19b6
MD5 92b408e67ed1857570f2deb2695be28f
BLAKE2b-256 ff6c041d843113a9ecf4d04f126df2b4dd19987e216f6de72c7e50f234f596d6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 68012d8a8cddb2eab3880870d7e2086cb359c7f7a2b03f5795044f5abff4e850
MD5 a81bc9bba75d543275137d465bda297b
BLAKE2b-256 11067b93cee5d490c98be382ca6b2880f700c698a22fedbc43c04eacc5db20d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 099c68207f3ef0227ecb6f80ab98ea74de559f7b124adc7b17778af0250ee90a
MD5 9a0fdbe8fb5384c4a8f93e93a5edb463
BLAKE2b-256 e1044d53c8012e2e253f717a0d5fd344180ad6a1131d0a83facf80120b3713f8

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 7d29aad702279e0760c265fcae832e89349078e3418dd329732d4503259fd6bd
MD5 927d12d7da2510eee64bd765d4d90672
BLAKE2b-256 fe20ccd624738c048764994b997d472584603c35575e2eedb0b1599f6a0758b4

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.20.0-cp37-none-win32.whl
  • Upload date:
  • Size: 2.1 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.12.4

File hashes

Hashes for tokenizers-0.20.0-cp37-none-win32.whl
Algorithm Hash digest
SHA256 fada996d6da8cf213f6e3c91c12297ad4f6cdf7a85c2fadcd05ec32fa6846fcd
MD5 427d6caad29fe4c833c127f596aa488e
BLAKE2b-256 446605b9ab23875445d7bf0031cb42df91e18803356752910da0007bd26eed1d

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 c8da0fba4d179ddf2607821575998df3c294aa59aa8df5a6646dc64bc7352bce
MD5 f8f1aea5ab28a18f9406253034fb89e7
BLAKE2b-256 4d8d25583483d7ef85d04bf936e6c13592f7fb121bbcbec561cca0fe8592edd6

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 39fa3761b30a89368f322e5daf4130dce8495b79ad831f370449cdacfb0c0d37
MD5 df316de8d7423f35ff33e6189bc26c49
BLAKE2b-256 b725f69bdfa56b84b696b67a244f003c188bdde552159d890bcf8ef480db3c41

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 fab3cf066ff426f7e6d70435dc28a9ff01b2747be83810e397cba106f39430b0
MD5 0ff6337a5c4922bfac2b691487848d8c
BLAKE2b-256 784d0adfc444c189801254e43c69d37fd24a9ca6ceb0954165c455d4f7aca1e2

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 e13eb000ef540c2280758d1b9cfa5fe424b0424ae4458f440e6340a4f18b2638
MD5 bb3ed4d7c1929e7f9a9c902ce5cd3a89
BLAKE2b-256 80e8b9b48d38d89a7eaf7c19da442fb32fadfe0610fce7d6f84ec8d91aa64281

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 98d88f06155335b14fd78e32ee28ca5b2eb30fced4614e06eb14ae5f7fba24ed
MD5 7372240497ca9c4aea14f1fda40a8e32
BLAKE2b-256 e99c20f629fdd0782c0d00f3ad6686c1daae4027e516028d5784d0ee5fe21355

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl
Algorithm Hash digest
SHA256 010ec7f3f7a96adc4c2a34a3ada41fa14b4b936b5628b4ff7b33791258646c6b
MD5 ca14d49fb957daab0e307c9e08a73e6f
BLAKE2b-256 fc99c6cdbb804c25537e87f22eb6ddae4d86d6f811114514c43ebbf7e679d056

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 7f9baa027cc8a281ad5f7725a93c204d7a46986f88edbe8ef7357f40a23fb9c7
MD5 932c539700d6bbd2b13a0f5a642d2f2e
BLAKE2b-256 0e69108baecf22bfae65479f3dca2e609b797ba16493f92b792be94cedac8776

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 799b808529e54b7e1a36350bda2aeb470e8390e484d3e98c10395cee61d4e3c6
MD5 81db004ce6329f9fd748db5ab4050e45
BLAKE2b-256 aa182160c7d94dbfbaeec4a99015d3f73e70c0a53e56a4cd4bfdb1997f8d07b5

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e5d4069e4714e3f7ba0a4d3d44f9d84a432cd4e4aa85c3d7dd1f51440f12e4a1
MD5 e68d40a867af878c4ea8365d7407bfcd
BLAKE2b-256 99eb9f131835dbfefd5a6a5fce70e4925601e855a1c082befe49501bfef864ce

See more details on using hashes here.

File details

Details for the file tokenizers-0.20.0-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.20.0-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f7065b1084d8d1a03dc89d9aad69bcbc8415d4bc123c367063eb32958cd85054
MD5 797896e116eb68e0d425083eebb26b5b
BLAKE2b-256 44896aee9409ecc244d9802f7577ff99b5b884e295a2af337793382bf511cd79

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page