No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

0.20.3

0.20.3rc0 pre-release

0.20.2

0.20.1

0.20.1rc1 pre-release

0.20.0

0.20.0rc1 pre-release

This version

0.19.1

0.19.0

0.15.2

0.15.2rc1 pre-release

0.15.1

0.15.0

0.14.1

0.14.0

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0 yanked

Reason this release was yanked:

Breaking change with unexpected effect

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.3

0.10.2

0.10.1

0.10.1rc1 pre-release

0.10.0

0.10.0rc1 pre-release

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.9.0rc2 pre-release

0.9.0rc1 pre-release

0.9.0.dev4 pre-release

0.9.0.dev3 pre-release

0.9.0.dev2 pre-release

0.9.0.dev1 pre-release

0.9.0.dev0 pre-release

0.8.1

0.8.1rc2 pre-release

0.8.1rc1 pre-release

0.8.0

0.8.0rc4 pre-release

0.8.0rc3 pre-release

0.8.0rc2 pre-release

0.8.0rc1 pre-release

0.8.0.dev2 pre-release

0.8.0.dev1 pre-release

0.8.0.dev0 pre-release

0.7.0

0.7.0rc7 pre-release

0.7.0rc6 pre-release

0.7.0rc5 pre-release

0.7.0rc4 pre-release

0.7.0rc3 pre-release

0.7.0rc2 pre-release

0.7.0rc1 pre-release

0.6.0

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.1

0.1.0

0.0.13

0.0.12

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.19.1.tar.gz (321.0 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.1-cp312-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.19.1-cp312-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.19.1-cp312-cp312-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp312-cp312-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp312-cp312-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.19.1-cp312-cp312-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.19.1-cp311-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.19.1-cp311-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.19.1-cp311-cp311-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp311-cp311-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp311-cp311-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.19.1-cp311-cp311-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.19.1-cp310-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.19.1-cp310-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.19.1-cp310-cp310-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp310-cp310-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp310-cp310-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.19.1-cp310-cp310-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.19.1-cp39-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.19.1-cp39-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.19.1-cp39-cp39-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp39-cp39-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp39-cp39-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.19.1-cp39-cp39-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.19.1-cp38-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.19.1-cp38-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.19.1-cp38-cp38-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp38-cp38-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp38-cp38-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.19.1-cp38-cp38-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.19.1-cp37-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.19.1-cp37-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.19.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ i686

tokenizers-0.19.1-cp37-cp37m-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.19.1-cp37-cp37m-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.19.1.tar.gz.

File metadata

  • Download URL: tokenizers-0.19.1.tar.gz
  • Upload date:
  • Size: 321.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1.tar.gz
Algorithm Hash digest
SHA256 ee59e6680ed0fdbe6b724cf38bd70400a0c1dd623b07ac729087270caeac88e3
MD5 d2d6ec7d3b16f8d1ce2d859a17ef52fb
BLAKE2b-256 48042071c150f374aab6d5e92aaec38d0f3c368d227dd9e0469a1f0966ac68d1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 43350270bfc16b06ad3f6f07eab21f089adb835544417afda0f83256a8bf8b75
MD5 782036f040226dbf8ed0f4132db295e0
BLAKE2b-256 45b636c1bb106bbe96012c9367df89ed01599cada036c0b96d38fbbdbeb75c9f

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 35583cd46d16f07c054efd18b5d46af4a2f070a2dd0a47914e66f3ff5efb2b1e
MD5 ed8b89c80e552637b23d6db010ad46d1
BLAKE2b-256 83ac26bc2e2bb2a054dc2e51699628936f5474e093b68da6ccdde04b2fc39ab8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ce05fde79d2bc2e46ac08aacbc142bead21614d937aac950be88dc79f9db9022
MD5 1421e174545e7637d20512037af4ad7e
BLAKE2b-256 d8582e998462677c4c0eb5123ce386bcb488a155664d273d0283122866515f09

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ca407133536f19bdec44b3da117ef0d12e43f6d4b56ac4c765f37eca501c7bda
MD5 9487252d2dda7e2c4113097f6eaac26f
BLAKE2b-256 ce2bdb8a94608c392752681c2ca312487b7cd5bcc4f77e24a90daa4916138271

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 e8d1ed93beda54bbd6131a2cb363a576eac746d5c26ba5b7556bc6f964425594
MD5 a47d0f0778ac8c800fe30ae114d4f448
BLAKE2b-256 19e0f9e915d028b45798723eab59c253da28040aa66b9f31dcb7cfc3be88fa37

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d26194ef6c13302f446d39972aaa36a1dda6450bc8949f5eb4c27f51191375bd
MD5 69c2a33b81c7fb7f147baaf38973f183
BLAKE2b-256 1d0d2c452fe17fc17f0cdb713acb811eebb1f714b8c21d497c4672af4f491229

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 3b11853f17b54c2fe47742c56d8a33bf49ce31caf531e87ac0d7d13d327c9334
MD5 aafe41d67c39a1e96c85347ddfa155b5
BLAKE2b-256 cf7b38fb7207cde3d1dc5272411cd18178e6437cdc1ef08cac5d0e8cfd57f38c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 f8a9c828277133af13f3859d1b6bf1c3cb6e9e1637df0e45312e6b7c2e622b1f
MD5 e0624d0fd1ca5eabf499f4a5a477da90
BLAKE2b-256 f12a5ac32ef70d6f9464155c4c4239139dc5aa9297052180b171f5ae22fd7b7a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 ed69af290c2b65169f0ba9034d1dc39a5db9459b32f1dd8b5f3f32a3fcf06eab
MD5 058462a54ed949d247ec654de5b5b247
BLAKE2b-256 a5024944f51c7248ae78c9758266f4e92d72fe98cf58f3c973949bcdede0313a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 a2b718f316b596f36e1dae097a7d5b91fc5b85e90bf08b01ff139bd8953b25af
MD5 2c847be827a8a40b3695c55a5fcdaf30
BLAKE2b-256 6d61f8b59cc2580297ca78a7b5b2cefc8996b8417dc6cb9abb6a1d303973156b

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ec11802450a2487cdf0e634b750a04cbdc1c4d066b97d94ce7dd2cb51ebb325b
MD5 cfe9e46eb48a588da3b65638ca71faf4
BLAKE2b-256 085c54419545d61c085d7adcbd54f5711815ffbb1164d6132209172c984320be

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 6c330c0eb815d212893c67a032e9dc1b38a803eccb32f3e8172c19cc69fbb439
MD5 d254ecbed7a18300fb7ece8c54275686
BLAKE2b-256 7242e18b67ab9fd31e433171cf447d85bf5dede8009db04a46f3905bff5ca715

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f9939ca7e58c2758c01b40324a59c034ce0cebad18e0d4563a9b1beab3018243
MD5 48fe0a85854aa09a24c73b0023584f5c
BLAKE2b-256 c8594dbebca9ef6b61d10a94cbf404d3abf509dfedb52cdcf2fe7ed1fb52460d

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 56ae39d4036b753994476a1b935584071093b55c7a72e3b8288e68c313ca26e7
MD5 f33873c60347f02cea5e0d3acfe5a524
BLAKE2b-256 aa9cdeed1e549b767832cc4ee5b386d1660bde3408bbd6d1ab48352fb61c54e2

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 eddd5783a4a6309ce23432353cdb36220e25cbb779bfa9122320666508b44b88
MD5 80615f9666c557c65b6bfc48c5d3744d
BLAKE2b-256 9d0f4ddf31367eb45f32a66d8f2649cbbe7ae4a393679e35ff26c670736e1fa6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 dd26e3afe8a7b61422df3176e06664503d3f5973b94f45d5c45987e1cb711876
MD5 4c95d87ea7c2fe15ce8aa0136c4bbf68
BLAKE2b-256 11710f11a0d21ebaa968a90f09623f9c9bad563fed03ad47361854c657824424

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 63c38f45d8f2a2ec0f3a20073cccb335b9f99f73b3c69483cd52ebc75369d8a1
MD5 030b8834843ea7c96b2b3dd11d4e04c0
BLAKE2b-256 9d36df75fc03b4cfd3b35c818e3e8e8134485f1e1041a065671bc3be7fd38e60

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 6f8a20266e695ec9d7a946a019c1d5ca4eddb6613d4f466888eee04f16eedb85
MD5 f1504b83c4650a54dba2a0cc3905340b
BLAKE2b-256 cd2f40d26e0e9426be4ab95255088224640a012a47cb0469e91eade16243e8db

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 0b5ca92bfa717759c052e345770792d02d1f43b06f9e790ca0a1db62838816f3
MD5 f885bd602f16c94fbc6feeb6d3462a5c
BLAKE2b-256 61fbed3aa4f0ffeb177a72394c99cd642fb5b257eb103829ff2858ba523b7bd5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 0e64bfde9a723274e9a71630c3e9494ed7b4c0f76a1faacf7fe294cd26f7ae7c
MD5 b34576f26c87c114406a3e763a38c34c
BLAKE2b-256 d9db81d0b09855d09b658eeed3ba4d22ad0c6a3e38bfc6a92d339f0a7be5613a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 b2edbc75744235eea94d595a8b70fe279dd42f3296f76d5a86dde1d46e35f574
MD5 1f0f21361e11882f9b1eb2a40902e135
BLAKE2b-256 53ed0f8d5b3913ad21688bf7070a928be81320db3a5b2e46bf508daffb24ec52

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 89183e55fb86e61d848ff83753f64cded119f5d6e1f553d14ffee3700d0a4a49
MD5 e6e0a6bcdaf02ed0f902da794a4de0e3
BLAKE2b-256 d4c2de9ca55958ce750dddd309fdb1af65e8b5459d20e358fdc7abbb69d65c7d

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 b279ab506ec4445166ac476fb4d3cc383accde1ea152998509a94d82547c8e2a
MD5 f66e9c438cf0dead865fed4f610aaf5f
BLAKE2b-256 3e1a24d27c8a8a55589277923c760e6e15e6a9745beb94a64039c89cf4375331

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 ecb2651956eea2aa0a2d099434134b1b68f1c31f9a5084d6d53f08ed43d45ff2
MD5 ad24b469da60abe191cb8661b6027ab9
BLAKE2b-256 58b26545bc89e5686b6f6d31fef88bc65c22379e4afe998132362c83c191942c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 bcd266ae85c3d39df2f7e7d0e07f6c41a55e9a3123bb11f854412952deacd828
MD5 23a56debf295f8075c41f9b194b4edfb
BLAKE2b-256 0b1dd745cb583aba94094923eba9117879d6e7d1a2182e76395f9292c6582acf

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 6852c5b2a853b8b0ddc5993cd4f33bfffdca4fcc5d52f89dd4b8eada99379285
MD5 39cfb91bc3bd6eff3166b287cdba9ea5
BLAKE2b-256 fd6b10cc91903b97a4fb8f5813b4793684c3e4e25317d0ec2a2fc3ab9a762664

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 b4399b59d1af5645bcee2072a463318114c39b8547437a7c2d6a186a1b5a0e2d
MD5 c8536c0cc456707a41f938ebd51fac88
BLAKE2b-256 cc61e36885f692b4bd15644593b18bfaf26167c97e023ea5d66c548ea4a3783f

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 b70bfbe3a82d3e3fb2a5e9b22a39f8d1740c96c68b6ace0086b39074f08ab89a
MD5 b655c2e82880368cbecd02933322cdd8
BLAKE2b-256 0c9780bff6937e0c67d30c0facacd4f0bcf4254e581aa4995c73cef8c8640e56

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp312-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp312-none-win32.whl
Algorithm Hash digest
SHA256 01d62812454c188306755c94755465505836fd616f75067abcae529c35edeb57
MD5 959a1f66ab8e0cb8e172dfb674cdfdc8
BLAKE2b-256 2a94ec3369dbc9b7200c14c8c7a1a04c78b7a7398d0c001e1b7d1ffe30eb93a0

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 952b80dac1a6492170f8c2429bd11fcaa14377e097d12a1dbe0ef2fb2241e16c
MD5 67705fd9613c8926205a8d93be12254f
BLAKE2b-256 ac3d2284f6d99f8f21d09352b88b8cfefa24ab88468d962aeb0aa15c20d76b32

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 a179856d1caee06577220ebcfa332af046d576fb73454b8f4d4b0ba8324423ea
MD5 d419c0c78b571a261dbcd383664e961b
BLAKE2b-256 52b71e8a913d18ac28feeda42d4d2d51781874398fb59cd1c1e2653a4b5742ed

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7c9d5b6c0e7a1e979bec10ff960fae925e947aab95619a6fdb4c1d8ff3708ce3
MD5 ffe0493b2b926d9b416a3953c173824c
BLAKE2b-256 805412047a69f5b382d7ee72044dc89151a2dd0d13b2c9bdcc22654883704d31

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 3174c76efd9d08f836bfccaca7cfec3f4d1c0a4cf3acbc7236ad577cc423c840
MD5 43ed77468fee2374194754d98a3821be
BLAKE2b-256 0ab6f7b7ef89c4da7b20256e6eab23d3835f05d1ca8f451d31c16cbfe3cd9eb6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 c5c2ff13d157afe413bf7e25789879dd463e5a4abfb529a2d8f8473d8042e28f
MD5 d813622799dcb2d682af016dd5a4b055
BLAKE2b-256 f24baae61bdb6ab584d2612170801703982ee0e35f8b6adacbeefe5a3b277621

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 04ce49e82d100594715ac1b2ce87d1a36e61891a91de774755f743babcd0dd52
MD5 5cc446d41ca886ff63f5ddbce27587fd
BLAKE2b-256 6714e7da32ae5fb4971830f1ef335932fae3fa57e76b537e852f146c850aefdf

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 9620b78e0b2d52ef07b0d428323fb34e8ea1219c5eac98c2596311f20f1f9266
MD5 31eb9eab587a57ff1941a9f58ac0e402
BLAKE2b-256 0540fa899f32de483500fbc78befd378fd7afba4270f17db707d1a78c0a4ddc3

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 4f3fefdc0446b1a1e6d81cd4c07088ac015665d2e812f6dbba4a06267d1a2c95
MD5 a6f947435d87b448a65e8299795e3194
BLAKE2b-256 ed3089b321a16c58d233e301ec15072c0d3ed5014825e72da98604cd3ab2fba1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d924204a3dbe50b75630bd16f821ebda6a5f729928df30f582fb5aade90c818a
MD5 dee05246d56521783cc529827e6acf82
BLAKE2b-256 74d1f4e1e950adb36675dfd8f9d0f4be644f3f3aaf22a5677a4f5c81282b662e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 621d670e1b1c281a1c9698ed89451395d318802ff88d1fc1accff0867a06f153
MD5 42eed543e4a641027f8b2bc37943bd75
BLAKE2b-256 63902890cd096898dcdb596ee172cde40c0f54a9cf43b0736aa260a5501252af

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 ad57d59341710b94a7d9dbea13f5c1e7d76fd8d9bcd944a7a6ab0b0da6e0cc66
MD5 17c0000754eb1df2fe6053f7d31f5f90
BLAKE2b-256 658e6d7d72b28f22c422cff8beae10ac3c2e4376b9be721ef8167b7eecd1da62

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp311-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp311-none-win32.whl
Algorithm Hash digest
SHA256 9ed240c56b4403e22b9584ee37d87b8bfa14865134e3e1c3fb4b2c42fafd3256
MD5 fb73604c4d08e7f649b293f15fc1f74b
BLAKE2b-256 db1131be66710f1d14526f3588a441efadeb184e1e68458067007b20ead03c59

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 16baac68651701364b0289979ecec728546133e8e8fe38f66fe48ad07996b88b
MD5 82bae0ff96d8c4b7fc2b257deec87e93
BLAKE2b-256 25508f8ad0bbdaf09d04b15e6502d1fa1c653754ed7e016e4ae009726aa1a4e4

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 706a37cc5332f85f26efbe2bdc9ef8a9b372b77e4645331a405073e4b3a8c1c6
MD5 5172af6b7578f7439b2978cf2e4f952a
BLAKE2b-256 5bcd0385e1026e1e03732fd398e964792a3a8433918b166748c82507e014d748

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 d16ff18907f4909dca9b076b9c2d899114dd6abceeb074eca0c93e2353f943aa
MD5 58aa86e1256b2ad4c4c4ad2e9c64a34f
BLAKE2b-256 a703fb50fc03f86016b227a967c8d474f90230c885c0d18f78acdfda7a96ce56

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 b342d2ce8fc8d00f376af068e3274e2e8649562e3bc6ae4a67784ded6b99428d
MD5 7060c1188ede557fc2611a0145de7eb0
BLAKE2b-256 4c8aa166888d6cb14db55f5eb7ce0b1d4777d145aa27cbf4f945712cf6c29935

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 76951121890fea8330d3a0df9a954b3f2a37e3ec20e5b0530e9a0044ca2e11fe
MD5 e4a3fb5af5b5588961042fcf66764e82
BLAKE2b-256 06dbc0320c4798ac6bd12d2ef895bec9d10d216a3b4d6fff10e9d68883ea7edc

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 ac11016d0a04aa6487b1513a3a36e7bee7eec0e5d30057c9c0408067345c48d2
MD5 7f7f17aad33ebad3a4ca9b9bae041b99
BLAKE2b-256 afef3c1deed14ec59b2c8e7e2fa27b2a53f7d101181277a43b89ab17d891ef2e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 dfedf31824ca4915b511b03441784ff640378191918264268e6923da48104acc
MD5 f69d98e37af910d2019a4b84df49b917
BLAKE2b-256 36c6537f22b57e6003904d35d07962dbde2f2e9bdd791d0241da976a4c7f8194

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 dadc509cc8a9fe460bd274c0e16ac4184d0958117cf026e0ea8b32b438171594
MD5 9d2b564309110806dcafc80f408fd779
BLAKE2b-256 c7282d11c3ff94f9d42eceb2ea549a06e3f166fe391c5a025e5d96fac898a3ac

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ddf672ed719b4ed82b51499100f5417d7d9f6fb05a65e232249268f35de5ed14
MD5 1f0705e2d594bf1084e61e83bdb85b3c
BLAKE2b-256 9079d17a0f491d10817cd30f1121a07aa09c8e97a81114b116e473baf1577f09

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 5c88d1481f1882c2e53e6bb06491e474e420d9ac7bdff172610c4f9ad3898059
MD5 5cd34189d886eda58f482e32ed60551f
BLAKE2b-256 c8d66e1d728d765eb4102767f071bf7f6439ab10d7f4a975c9217db65715207a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 bea6f9947e9419c2fda21ae6c32871e3d398cba549b93f4a65a2d369662d9403
MD5 4adcc7ad724cb444dd1d481d59d2acdb
BLAKE2b-256 f485d999b9a05fd101d48f1a365d68be0b109277bb25c89fb37a389d669f9185

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp310-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp310-none-win32.whl
Algorithm Hash digest
SHA256 7ff898780a155ea053f5d934925f3902be2ed1f4d916461e1a93019cc7250837
MD5 81d7879a3d4847a17d73d0f3a4b4fc22
BLAKE2b-256 7ae726bedf5d270d293d572a90bd66b0b030012aedb95d8ee87e8bcd446b76fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 2e8a3dd055e515df7054378dc9d6fa8c8c34e1f32777fb9a01fea81496b3f9d3
MD5 520d8c2d19ce7e2c60eef1305769d730
BLAKE2b-256 2ebedebb7caa3f88ed54015170db16e07aa3a5fea2d3983d0dde92f98d888dc8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 7fb297edec6c6841ab2e4e8f357209519188e4a59b557ea4fafcf4691d1b4c98
MD5 62fabe77fe828a8c2efc2c279f10dd3a
BLAKE2b-256 f5f8141dcb0f88e9452af8d20d14dd53aab5937222a2bb4f2c04bfed6829263c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8b01afb7193d47439f091cd8f070a1ced347ad0f9144952a30a41836902fe09e
MD5 ad127afbf190713479ec8e9182e8b37e
BLAKE2b-256 404feb78de4af3b17b589f43a369cbf0c3a7173f25c3d2cd93068852c07689aa

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 e28cab1582e0eec38b1f38c1c1fb2e56bce5dc180acb1724574fc5f47da2a4fe
MD5 8453e2d502806f51cf64400c091b5336
BLAKE2b-256 6ae15dbac9618709972434eea072670cd69fba1aa988e6200f16057722b4bf96

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 b07c538ba956843833fee1190cf769c60dc62e1cf934ed50d77d5502194d63b1
MD5 31d778744425faa24196a593e7a1c3df
BLAKE2b-256 658e5f4316976c26009f1ae0b6543f3d97af29afa5ba5dc145251e6a07314618

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 02e81bf089ebf0e7f4df34fa0207519f07e66d8491d963618252f2e0729e0b46
MD5 af37024b72c0ca795ecbf134c2ddd1af
BLAKE2b-256 7579158626bd794e75551e0c6bb93f1cd3c9ba08ba14b181b98f09e95994f609

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 453e4422efdfc9c6b6bf2eae00d5e323f263fff62b29a8c9cd526c5003f3f642
MD5 863773622df5c0b8055c158a57e740c6
BLAKE2b-256 504e2e5549a26dc6f9e434f83bebf16c2d7dc9dc3477cc0ec8b23ede4d465b90

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 f03727225feaf340ceeb7e00604825addef622d551cbd46b7b775ac834c1e1c4
MD5 d811b1e3b1be9d4742f76f644bed6be6
BLAKE2b-256 e403b2020e6a78fb994cff1ec962adc157c23109172a46b4fe451d6d0dd33fdb

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 82c8b8063de6c0468f08e82c4e198763e7b97aabfe573fd4cf7b33930ca4df77
MD5 2ab6e3386315145e64df7476f98c81d1
BLAKE2b-256 4c129cb68762ff5fee1efd51aefe2f62cb225f26f060a68a3779e1060bbc7a59

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 952078130b3d101e05ecfc7fc3640282d74ed26bcf691400f872563fca15ac97
MD5 7db5f4b616a06b1a93e0719e3158d1cb
BLAKE2b-256 c16091cac8d496b304ec5a22f07606893cad35ea8e1a8406dc8909e365f97a80

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 f97660f6c43efd3e0bfd3f2e3e5615bf215680bad6ee3d469df6454b8c6e8256
MD5 702e2fc944c5244a1a764396256a2b86
BLAKE2b-256 25aac6992cdc0a74bcbb666e7c00ada6826f5b49fc4cbdafc50db0d1369503fe

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp39-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp39-none-win32.whl
Algorithm Hash digest
SHA256 61b7fe8886f2e104d4caf9218b157b106207e0f2a4905c9c7ac98890688aabeb
MD5 78531090857eb9bc58955ab39dd32c5c
BLAKE2b-256 81b2bf9a0f9136964df5e94dd9854ba071480c5425ff0db6d1ad9a6a8e683d55

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 c2a0d47a89b48d7daa241e004e71fb5a50533718897a4cd6235cb846d511a478
MD5 bfbefe22000444ce3f39eab0032a4dc9
BLAKE2b-256 d94697f8e84ba6a9133e34b148631d2933fda2a6ad8e0767b6e07ad0af9d83c2

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 39c1ec76ea1027438fafe16ecb0fb84795e62e9d643444c1090179e63808c69d
MD5 9b70100369d6a0b4475890530f904ad6
BLAKE2b-256 7605badd3a66571ad257270b38c33b9a7470afd2ae12e409c7c74baedf16f2ef

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b4c89aa46c269e4e70c4d4f9d6bc644fcc39bb409cb2a81227923404dd6f5227
MD5 c2bc45a39179f702ddae9c0fd06e188e
BLAKE2b-256 0fcb8fc733c8f251bac1e5c4ae52458c353b3faa98f41d734c226cad3783da03

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 cbf001afbbed111a79ca47d75941e9e5361297a87d186cbfc11ed45e30b5daba
MD5 f5b838fd44a769380e63ab83d34e28a9
BLAKE2b-256 e04a59a0aa37b8bfe1888a72f75bbf24b94c888a1aa333aab2910ae22c369e23

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 85aa3ab4b03d5e99fdd31660872249df5e855334b6c333e0bc13032ff4469c4a
MD5 800ea08dc597180eda49b47fbe6c4474
BLAKE2b-256 3374fa1f86d161db482e10c92d83e924600b691210c5d676fa323738c91d8dba

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 670b802d4d82bbbb832ddb0d41df7015b3e549714c0e77f9bed3e74d42400fbe
MD5 446c98e46a1bfa6e3a28b68a554518ef
BLAKE2b-256 3b6b98383dff29416127c73dc196844ed23e29d790f1ad4b4ecf69d45e03841d

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 c85cf76561fbd01e0d9ea2d1cbe711a65400092bc52b5242b16cfd22e51f0c58
MD5 304a600a99bab6bc38838244cd10dc14
BLAKE2b-256 ba26139bd2371228a0e203da7b3e3eddcb02f45b2b7edd91df00e342e4b55e13

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 6258c2ef6f06259f70a682491c78561d492e885adeaf9f64f5389f78aa49a051
MD5 a5f4812ee1713ec04b51965458e779f7
BLAKE2b-256 c528ac2a277bd23b631e1ff986182c4fcb9028ccc7ff7c07743ef906fa5389e7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4692ab92f91b87769d950ca14dbb61f8a9ef36a62f94bad6c82cc84a51f76f6a
MD5 e552628b554f83116e79ac227e72f4d5
BLAKE2b-256 0a2b4e5718e806ff23e5e758e02bd4b34967b5218f085b0c189335fd27c14dc1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 0b9394bd204842a2a1fd37fe29935353742be4a3460b6ccbaefa93f58a8df43d
MD5 8994c7bff7f995d4baa9903ffa97b88a
BLAKE2b-256 1aed42801618bab16c79d6bd222977c212dba5770e6c935ba53728b731653a3d

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 0bcce02bf1ad9882345b34d5bd25ed4949a480cf0e656bbd468f4d8986f7a3f1
MD5 2d82c67ce5a09a0006ea63fa4eea6179
BLAKE2b-256 850e9ac0883a0f76d07de4d45be6442bb23febb26e5f4e22fdc0445d45d4ef24

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp38-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp38-none-win32.whl
Algorithm Hash digest
SHA256 1de5bc8652252d9357a666e609cb1453d4f8e160eb1fb2830ee369dd658e8975
MD5 13a43bdfad2cae057afcbc6afdd7a65d
BLAKE2b-256 7ed644b91c8bcfb41677c62f1b3d1d12017b702382ad3b1c0c87da8e3cc34c34

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 08a44864e42fa6d7d76d7be4bec62c9982f6f6248b4aa42f7302aa01e0abfd26
MD5 deb8d8b5e958474ef0982b1134c211eb
BLAKE2b-256 f66b37bad4b34cc5892a2784e291bcbae197ccc8935a3b64e761d1ef5c009333

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 8a6298bde623725ca31c9035a04bf2ef63208d266acd2bed8c2cb7d2b7d53ce6
MD5 9bc7fd2a77eb04282e50da814e0c492a
BLAKE2b-256 5583b02cb52e580672f23141f5c0aba433395187327904063606ad870586fa8e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bac0b0eb952412b0b196ca7a40e7dce4ed6f6926489313414010f2e6b9ec2adf
MD5 72a3d825eb342578782e1f6a10df80cc
BLAKE2b-256 180dee99f50407788149bc9eddae6af0b4016865d67fb687730d151683b13b80

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 c27b99889bd58b7e301468c0838c5ed75e60c66df0d4db80c08f43462f82e0d3
MD5 89ec01aa2cba894e781a080654d71cda
BLAKE2b-256 6dc39148e2053d3910eab319bb8052964d7af2b6f622192642ef13c2095c4a21

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 e83a31c9cf181a0a3ef0abad2b5f6b43399faf5da7e696196ddd110d332519ee
MD5 c1301883c1ce3c62e16d8ae78a04aa28
BLAKE2b-256 bbab26810f92b3db96c73f184fa9199910ec80c1b371023e3f29a5effc61c951

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 427c4f0f3df9109314d4f75b8d1f65d9477033e67ffaec4bca53293d3aca286d
MD5 1920365808914e8f3014a5b40adc72b3
BLAKE2b-256 2b6eeb5a943dbe04ffe3970df2f44aaf936d4facd3daa36704ef356190005de5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 4ad23d37d68cf00d54af184586d79b84075ada495e7c5c0f601f051b162112dc
MD5 3b2a8bcce7a64a7bff8bfbab9009df05
BLAKE2b-256 2c9615fe0fa8637bbd4467cf05f67958afc148a1edcac33f2966ac71a2923958

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 6309271f57b397aa0aff0cbbe632ca9d70430839ca3178bf0f06f825924eca22
MD5 837d15ad7ec8b57687247dca0c9808b9
BLAKE2b-256 d48e6da0899098ee6b8358f80bf6507241d38f7afbe46a93df6c97f5cc2f8bbc

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 10a707cc6c4b6b183ec5dbfc5c34f3064e18cf62b4a938cb41699e33a99e03c1
MD5 108bc85b4109540c45fa0be23f132907
BLAKE2b-256 85de5e53efc7ba183e89ed73d6cc3bc1a08f0ee73770574694c8fb361dbb35e9

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 07f9295349bbbcedae8cefdbcfa7f686aa420be8aca5d4f7d1ae6016c128c0c5
MD5 b56c8d21b9568d0f2d44c9a1c72c5301
BLAKE2b-256 6759477125f604cffd2f272301634d3426e258164aebf20b9631eb8141311794

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 f3bbb7a0c5fcb692950b041ae11067ac54826204318922da754f908d95619fbc
MD5 9164af0378c107c372b2b5f5cf4a3f36
BLAKE2b-256 cbeae9bf17a3f492d756b54c87ffcb99a606078c3224e276c8263c5eb69db423

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.1-cp37-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.3

File hashes

Hashes for tokenizers-0.19.1-cp37-none-win32.whl
Algorithm Hash digest
SHA256 72791f9bb1ca78e3ae525d4782e85272c63faaef9940d92142aa3eb79f3407a3
MD5 b492b1c79cf0d7d2b2f20132a0856278
BLAKE2b-256 b919320fcad0a094a2d3621725a23d08750fa33adc460cd2aaec009b267db0d1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 78e769eb3b2c79687d9cb0f89ef77223e8e279b75c0a968e637ca7043a84463f
MD5 138507936173bd1d9548a3c1fa385807
BLAKE2b-256 c6138ffe956006673030cf69f1e2e696ecb20a4b1feecde3abf085508323df91

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 638e43936cc8b2cbb9f9d8dde0fe5e7e30766a3318d2342999ae27f68fdc9bd6
MD5 f42221cf4b3768378ef0115007017fde
BLAKE2b-256 c1afd209248152647cc3f007d0614d25177feb503346aeec4b658a1b47068571

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6b2da5c32ed869bebd990c9420df49813709e953674c0722ff471a116d97b22d
MD5 eaf0b8a5692af8771a3ff9b9e99a80e1
BLAKE2b-256 b8c2e9c23c9263d0d0b0c366400325b5f472ae7cf8d54cdec3c74f96ab5309ae

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 d8c5d59d7b59885eab559d5bc082b2985555a54cda04dda4c65528d90ad252ad
MD5 2816239592dcc1d30fb504450f3600ae
BLAKE2b-256 17b3beb7b8e6e2e104029e0fb4c3d073f0b70e24c7b1464349e255a7f30502e7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 e742d76ad84acbdb1a8e4694f915fe59ff6edc381c97d6dfdd054954e3478ad4
MD5 759247d117c738caffa10d3b1f47ee02
BLAKE2b-256 9085e0c86a7d1e51db5ef2635cc80466dc6e830bb35fcafec104718437b2b9a4

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 e8ff5b90eabdcdaa19af697885f70fe0b714ce16709cf43d4952f1f85299e73a
MD5 41005ba653848163eaeb2d414b5715a8
BLAKE2b-256 ddd187b954f3ab1f28e158bbe2afbf854c2850b60b199cd6a7d790895ab15639

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b82931fa619dbad979c0ee8e54dd5278acc418209cc897e42fac041f5366d626
MD5 bc24491f9548a4c8daee291c7dacbed5
BLAKE2b-256 9cb8747bc61ac331c551429a373621bfa3ba506474a7205504cd06cf718fceed

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 71e3ec71f0e78780851fef28c2a9babe20270404c921b756d7c532d280349214
MD5 27e9a318e3a9c3c516ed7abd5cd68bd5
BLAKE2b-256 02b43961dd02ed16ade7c667409ec7f71be52a36d5d55c5aa35611320bcf4ea0

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1f0360cbea28ea99944ac089c00de7b2e3e1c58f479fb8613b6d8d511ce98267
MD5 867cc6ce8c083f1675dd38d93cc9da5e
BLAKE2b-256 ca0c76f2340099c3058eb3ca72a33f1b74a545ad4f9445c894faff574eb5e8e5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.1-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.1-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 bb9dfe7dae85bc6119d705a76dc068c062b8b575abe3595e3c6276480e67e3f1
MD5 74c0a5166e8a3ac6adc316d3a7067cc9
BLAKE2b-256 3f16fbef1dabec1a5e83aad7aa28d19863c5ab9db3cf714bc96df5531a0e94db

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page