No project description provided

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Project description



Build GitHub


Tokenizers

Provides an implementation of today's most used tokenizers, with a focus on performance and versatility.

Bindings over the Rust implementation. If you are interested in the High-level design, you can go check it there.

Otherwise, let's dive in!

Main features:

  • Train new vocabularies and tokenize using 4 pre-made tokenizers (Bert WordPiece and the 3 most common BPE versions).
  • Extremely fast (both training and tokenization), thanks to the Rust implementation. Takes less than 20 seconds to tokenize a GB of text on a server's CPU.
  • Easy to use, but also extremely versatile.
  • Designed for research and production.
  • Normalization comes with alignments tracking. It's always possible to get the part of the original sentence that corresponds to a given token.
  • Does all the pre-processing: Truncate, Pad, add the special tokens your model needs.

Installation

With pip:

pip install tokenizers

From sources:

To use this method, you need to have the Rust installed:

# Install with:
curl https://sh.rustup.rs -sSf | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"

Once Rust is installed, you can compile doing the following

git clone https://github.com/huggingface/tokenizers
cd tokenizers/bindings/python

# Create a virtual env (you can use yours as well)
python -m venv .env
source .env/bin/activate

# Install `tokenizers` in the current virtual env
pip install -e .

Load a pretrained tokenizer from the Hub

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_pretrained("bert-base-cased")

Using the provided Tokenizers

We provide some pre-build tokenizers to cover the most common cases. You can easily load one of these using some vocab.json and merges.txt files:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
vocab = "./path/to/vocab.json"
merges = "./path/to/merges.txt"
tokenizer = CharBPETokenizer(vocab, merges)

# And then encode:
encoded = tokenizer.encode("I can feel the magic, can you?")
print(encoded.ids)
print(encoded.tokens)

And you can train them just as simply:

from tokenizers import CharBPETokenizer

# Initialize a tokenizer
tokenizer = CharBPETokenizer()

# Then train it!
tokenizer.train([ "./path/to/files/1.txt", "./path/to/files/2.txt" ])

# Now, let's use it:
encoded = tokenizer.encode("I can feel the magic, can you?")

# And finally save it somewhere
tokenizer.save("./path/to/directory/my-bpe.tokenizer.json")

Provided Tokenizers

  • CharBPETokenizer: The original BPE
  • ByteLevelBPETokenizer: The byte level version of the BPE
  • SentencePieceBPETokenizer: A BPE implementation compatible with the one used by SentencePiece
  • BertWordPieceTokenizer: The famous Bert tokenizer, using WordPiece

All of these can be used and trained as explained above!

Build your own

Whenever these provided tokenizers don't give you enough freedom, you can build your own tokenizer, by putting all the different parts you need together. You can check how we implemented the provided tokenizers and adapt them easily to your own needs.

Building a byte-level BPE

Here is an example showing how to build your own byte-level BPE by putting all the different pieces together, and then saving it to a single file:

from tokenizers import Tokenizer, models, pre_tokenizers, decoders, trainers, processors

# Initialize a tokenizer
tokenizer = Tokenizer(models.BPE())

# Customize pre-tokenization and decoding
tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=True)
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel(trim_offsets=True)

# And then train
trainer = trainers.BpeTrainer(
    vocab_size=20000,
    min_frequency=2,
    initial_alphabet=pre_tokenizers.ByteLevel.alphabet()
)
tokenizer.train([
    "./path/to/dataset/1.txt",
    "./path/to/dataset/2.txt",
    "./path/to/dataset/3.txt"
], trainer=trainer)

# And Save it
tokenizer.save("byte-level-bpe.tokenizer.json", pretty=True)

Now, when you want to use this tokenizer, this is as simple as:

from tokenizers import Tokenizer

tokenizer = Tokenizer.from_file("byte-level-bpe.tokenizer.json")

encoded = tokenizer.encode("I can feel the magic, can you?")

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for ArthurZucker from gravatar.com ArthurZucker Avatar for McPotato from gravatar.com McPotato Avatar for Nicolas.Patry from gravatar.com Nicolas.Patry Avatar for xn1t0x from gravatar.com xn1t0x

Unverified details

These details have not been verified by PyPI
Project links
Meta
Classifiers

Release history Release notifications | RSS feed

0.20.3

0.20.3rc0 pre-release

0.20.2

0.20.1

0.20.1rc1 pre-release

0.20.0

0.20.0rc1 pre-release

0.19.1

This version

0.19.0

0.15.2

0.15.2rc1 pre-release

0.15.1

0.15.0

0.14.1

0.14.0

0.13.3

0.13.2

0.13.1

0.13.0

0.12.1

0.12.0 yanked

Reason this release was yanked:

Breaking change with unexpected effect

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.3

0.10.2

0.10.1

0.10.1rc1 pre-release

0.10.0

0.10.0rc1 pre-release

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.9.0rc2 pre-release

0.9.0rc1 pre-release

0.9.0.dev4 pre-release

0.9.0.dev3 pre-release

0.9.0.dev2 pre-release

0.9.0.dev1 pre-release

0.9.0.dev0 pre-release

0.8.1

0.8.1rc2 pre-release

0.8.1rc1 pre-release

0.8.0

0.8.0rc4 pre-release

0.8.0rc3 pre-release

0.8.0rc2 pre-release

0.8.0rc1 pre-release

0.8.0.dev2 pre-release

0.8.0.dev1 pre-release

0.8.0.dev0 pre-release

0.7.0

0.7.0rc7 pre-release

0.7.0rc6 pre-release

0.7.0rc5 pre-release

0.7.0rc4 pre-release

0.7.0rc3 pre-release

0.7.0rc2 pre-release

0.7.0rc1 pre-release

0.6.0

0.5.2

0.5.1

0.5.0

0.4.2

0.4.1

0.4.0

0.3.0

0.2.1

0.2.0

0.1.1

0.1.0

0.0.13

0.0.12

0.0.11

0.0.10

0.0.9

0.0.8

0.0.7

0.0.6

0.0.5

0.0.4

0.0.3

0.0.2

0.0.1

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokenizers-0.19.0.tar.gz (320.8 kB view details)

Uploaded Source

Built Distributions

tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded PyPy macOS 11.0+ ARM64

tokenizers-0.19.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded PyPy musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded PyPy musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded PyPy manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded PyPy manylinux: glibc 2.12+ i686

tokenizers-0.19.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded PyPy macOS 10.12+ x86-64

tokenizers-0.19.0-cp312-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.12 Windows x86-64

tokenizers-0.19.0-cp312-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.12 Windows x86

tokenizers-0.19.0-cp312-cp312-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp312-cp312-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.12 musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp312-cp312-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.12 macOS 11.0+ ARM64

tokenizers-0.19.0-cp312-cp312-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.12 macOS 10.12+ x86-64

tokenizers-0.19.0-cp311-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.11 Windows x86-64

tokenizers-0.19.0-cp311-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.11 Windows x86

tokenizers-0.19.0-cp311-cp311-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp311-cp311-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.11 musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp311-cp311-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.11 macOS 11.0+ ARM64

tokenizers-0.19.0-cp311-cp311-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.11 macOS 10.12+ x86-64

tokenizers-0.19.0-cp310-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.10 Windows x86-64

tokenizers-0.19.0-cp310-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.10 Windows x86

tokenizers-0.19.0-cp310-cp310-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp310-cp310-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp310-cp310-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.10 macOS 11.0+ ARM64

tokenizers-0.19.0-cp310-cp310-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.10 macOS 10.12+ x86-64

tokenizers-0.19.0-cp39-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.9 Windows x86-64

tokenizers-0.19.0-cp39-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.9 Windows x86

tokenizers-0.19.0-cp39-cp39-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp39-cp39-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp39-cp39-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.9 macOS 11.0+ ARM64

tokenizers-0.19.0-cp39-cp39-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.9 macOS 10.12+ x86-64

tokenizers-0.19.0-cp38-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.8 Windows x86-64

tokenizers-0.19.0-cp38-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.8 Windows x86

tokenizers-0.19.0-cp38-cp38-musllinux_1_1_x86_64.whl (10.0 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp38-cp38-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp38-cp38-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.8 macOS 11.0+ ARM64

tokenizers-0.19.0-cp38-cp38-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.8 macOS 10.12+ x86-64

tokenizers-0.19.0-cp37-none-win_amd64.whl (2.2 MB view details)

Uploaded CPython 3.7 Windows x86-64

tokenizers-0.19.0-cp37-none-win32.whl (2.0 MB view details)

Uploaded CPython 3.7 Windows x86

tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_x86_64.whl (9.9 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_aarch64.whl (9.6 MB view details)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl (4.1 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ s390x

tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl (4.0 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ppc64le

tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (3.4 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARMv7l

tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.6 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

tokenizers-0.19.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl (3.7 MB view details)

Uploaded CPython 3.7m manylinux: glibc 2.12+ i686

tokenizers-0.19.0-cp37-cp37m-macosx_11_0_arm64.whl (2.4 MB view details)

Uploaded CPython 3.7m macOS 11.0+ ARM64

tokenizers-0.19.0-cp37-cp37m-macosx_10_12_x86_64.whl (2.5 MB view details)

Uploaded CPython 3.7m macOS 10.12+ x86-64

File details

Details for the file tokenizers-0.19.0.tar.gz.

File metadata

  • Download URL: tokenizers-0.19.0.tar.gz
  • Upload date:
  • Size: 320.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0.tar.gz
Algorithm Hash digest
SHA256 84c429210cacccff695e9bbcc331f9a234978dc7074d7da3845c2d02fc488118
MD5 8339f98db0bc9b1539155670b845e1fa
BLAKE2b-256 599a7ba038f101d74fea0861c8f82e188441fe99b2a26fb0991da35f2850f9f3

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 ef73c6ab4fb283835fe664e6886a435ec9842188458003cf633d25d8023d9ab4
MD5 313ff68ce1512a5fc23f3cff431ec595
BLAKE2b-256 4ac827c26c5eb5db8506ef600f2722d189abf04856b5d471f0e0247112fad849

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 ea0738daea29cd9e1eb1a740e73ace352719cc548eb4b05e42b4fc81f7543281
MD5 bc29121e10ede4762fe6cdfb9e0463a5
BLAKE2b-256 09816aa0f8a417629f7c9d12185d1e41e96dbcd62ab1811f4b05fb6313100cd5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f8e63e665185abd0edd27db5a38ffa6443960ed685e66366a44066333e430a18
MD5 a8dc069b57f2662227cccbd70178ba94
BLAKE2b-256 dc522958ac2085059cfd06198dc6b01ac6c0faccad793fc907e13a42a9bc9fae

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ac3fe69c1cf4ff4cd820b19d4e6e87c210d7acffd6ce743f771a9a02104ad458
MD5 d5b257c3dda7d653673776494090c050
BLAKE2b-256 adc9c3d975a11e27e2d28f4938073502bf078e36878294d02a81c886e69cb478

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 efb1b7fec18ae3c4145588890deb2db43a0cd9889823f01898ce8fa916b099bb
MD5 9dae83d220963b1aed9ec48f46a69a71
BLAKE2b-256 ea89d622aa125392a273d0c9f347532af22952065eb405420db7c5c88e9dd643

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 e19ca14d4ed97a37272732dc2d5a873754604a7dcdbb8f3dbdbd8b3117d9abae
MD5 49d36000f89a43639a98cae243e97864
BLAKE2b-256 4690b771267fe784af9b240e3a838d8d4d5dc4260c87d994d3a83b9f042d952e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp310-pypy310_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 19a4b2a35a8d46f2857ee668eb125772bc78cb4328de80448b8fa7a1e2bd6992
MD5 45ac82d0572e832543ad2bc0288dcb47
BLAKE2b-256 5a91b2f9454b8e5e5b8b6502aa867a22fa960fb40a927663afac36b2dfb86fe1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 9c1dcbe9369b328f3df5b2fd1161787010e1000f669ca92239f8ba18c1ae3583
MD5 03366bbd5520ffacc3298f3cb0f347cf
BLAKE2b-256 2c74af2f28f396ebda1253399a0556ed7e0506c28f25d816465b3961707518fb

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 b98a9b6ab035fd29b312ddc8809f5a50da8cf7353cc4e94423cbfd74f3fe7cf1
MD5 49f0a6dc9d4dc37fa090efc125276acb
BLAKE2b-256 6877f0931d00d2dce326b160a827d7563a1c377c7fc172200daf7d6b47e24dc2

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 b83fdfa7a56d63281760261e1e3f93a2a2f9d8af003149b21bf3f29883c77333
MD5 4a65633894ad407f23478e607df8db5c
BLAKE2b-256 ac9bf0775dde2c6266ef635ed98508290d07c4985722bab845f56903bc204db6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 3eabfaa19a5e2b9c7df44614515b314e74ea1cf4a2de5b6d3d7d39980584d27d
MD5 2b1d2359fbbb8caec371cc36307b0f08
BLAKE2b-256 86dda23818e4de5c8c183db882902ed45553718222eb68bd5568ce435e2d5ce2

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 f51114e2330e36120e943d90adc58dfd7a3afe8002fcd91d9cb9ecc0a74b84d9
MD5 05bd70ae3430f4e2d2bf34cc303b1e62
BLAKE2b-256 57fa43e1d9799ad9dd4f31b15fad05e3a44337c20545a114e3244bc72743e660

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a95e7f23f31e8470cf050282f0f6fb20d992d77c9e924a8fa7cf455c51bdb16a
MD5 6a0547ad4844a1d07ab74c056a64c172
BLAKE2b-256 f6d813ad091dc0b31f86cbab5fce7a47a618d51a9d195dfcb1ca281daf759c01

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp39-pypy39_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 fd9ced829c378baee1a61d188e12d77300561e74ccc0642bbf7002ae3bf75069
MD5 af27855fc7f15fa7a702c8f4ade5f0a9
BLAKE2b-256 0ab2e250f57ba15bc18a76dd75587ee2357d086c63c20fef6a3707c5df465f43

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 cbfdcd14290ada21da051911e2bbaf3f9479ed27eb182c260a6ad3ae36c098d0
MD5 23f291594df4b976b2fe1cd55bb299cd
BLAKE2b-256 c6be3750b6b66344b22d4abee33793c48fce0499360d1f2bda48c84f5d5dbc2b

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 5cf44cada5d0728aa9c227d62286898e26559d300869d720e3a03e09e672def3
MD5 a60e6629893a75468840f05b0ec5c4ae
BLAKE2b-256 da182acd0c791f45f8c2011e9b1efecf23c4968e82236cc9661c96cb619f5250

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 152138747e2904e3a426286b40debd1de3f3082d3cd1ee983a63357249b34551
MD5 7f8d4eae55520549ba2212bb87e100fd
BLAKE2b-256 54f6da88bba22087132b4f0befee53e0f0e69b2117dcd00e90f139f11f1d8510

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 68a1d219d231a187c46ad16c526535b4e836f7a670e29f286a24bd9b091f1fda
MD5 fa5971f566ba207668b7373371bdb482
BLAKE2b-256 7dd8ae19be79e77b4f8ee40999f60f12c1f9b9d20b7d8fd01ff63b68aefed3cd

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 b271bc36f3797640f9079733d37a15c6c9d1aeacba4dac52946edfec960c3d88
MD5 43c6840c9b27baf0042dd8a534681959
BLAKE2b-256 556598f867615b95786c5d5a141e7ec660e60503e39f10206e8202d37b0d7dcc

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d4a0fd0672ec2ef9d48e66281d3c685824c4d1de067eb7e901765e3f17eac431
MD5 269a32c745ff68c5e7f15d8449820dbb
BLAKE2b-256 94cfd33c87788423f7cb3ef5e7440802bcefe4528bad94910d7b57e875387aba

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp38-pypy38_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 f00c7f30db69c780634cfb46dccfd36bd54ff6f117617f1c382d075fb0a8a040
MD5 217e0993cf418fd69dcf0c8fe406c084
BLAKE2b-256 b6d335f711f4caf68d3c2dc2ca1cf22c282ac31567014d845f813bf55dcf79e1

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 8e0e63d1dc368edf91289575d32512fcd66ee9fd21eab5fd28bf6be3ff2b5773
MD5 3ac769725468f25dc62f05eff6aa5f33
BLAKE2b-256 947bb40012b1b32a5510a999d1b9ef0a0fa781f8f122a6df3d26176077247785

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 598956a46a337bf744143176c75618b987b71e03c3ac0727e69e2b9c82dba7e0
MD5 e5dd1a1c7ffc361e0ffc3541d5fb858e
BLAKE2b-256 df2e4511263504ef1041b31f0bd4df9110455203311c273023b48c9f8ab8ef85

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 e412175d98c3d8cda70fa0b113937db2640d04eb3317c1c7da99fe090cdecc2f
MD5 dfad4fdad8ea8c302f60c5bdb841986d
BLAKE2b-256 8d7e2b98ef98ad131925a8c924e1ac0e9be6c3f3e3ffa15c96331bc925465887

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 1808400645f2d9d29c919e62821d7fe27fd2f665552d9ce52de39f2d67a2e7a0
MD5 7a948b4da16b463ce5aaf6c6f627155f
BLAKE2b-256 9240352b543ca191d5114d3bfac1fb91be2807570c4684ad9c9c238102d000c6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 033ac935f31f4447b8f83f0ebf41e84f8d16d15180460564c32a39c67061137e
MD5 4d072d63d01514e091e62a78fada9aa3
BLAKE2b-256 93869b54a9fca51d76b59c041217d8b74966d3719fc56b566a606cc5db99d8d0

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-pp37-pypy37_pp73-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 171ba7264ad240a59f33eab7ace2b6190318b21b4b556373f938cc4597135c02
MD5 52c65a756a9d4503462cded3fc2b1a7d
BLAKE2b-256 596bb53d07c3cc55d20922d58bb3631640c0dff8e82421eacdaf9175e67bb0ae

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-none-win_amd64.whl
Algorithm Hash digest
SHA256 6df21dd98029243ef2348892af0495e5c99b7952bcce684bb08a65a2abd33b98
MD5 4d1f1d37ed8917083c372832620f6ef2
BLAKE2b-256 53441443ebb5cf9d1be8f5988b162824404381640611cc111f3841256b509cd7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp312-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.12, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp312-none-win32.whl
Algorithm Hash digest
SHA256 51a968b09d4e50dd9a535c7d69f1bba0506f3b6deba37a2345078f7de9c49612
MD5 877e8a9262b1beeac22d38e1b95d64e2
BLAKE2b-256 36af0a69c9bb55cd5d3e0726428728ff611267bf9d3705b102aadeb7bc800052

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 4df701de210706bc21f8c4e5a88d81b1d51b16e0ce1ab34327a1f84801647bd2
MD5 90f8b4bc270f840b07a69211711e2667
BLAKE2b-256 42e10905ab25e178aaa0e2fb85a430a444028b9f287c7c26ad0edc55d9e0166c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 b78efe06f196bdaadd76621b9f39e9693db98eb9c102719a61c1b4b7ef7eaf7a
MD5 070042bd3bbbfd1bc6e02877b610af64
BLAKE2b-256 f5003aa2c486c10e9fd92e4726689af358d4c8152cd93a4b2acdca072e3021b3

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 5cd2599389c35d7c188ad671af9d4df0abf4b513f7395928cf821429d0b306f7
MD5 21e7a9d098c6537058a44a0d420bd692
BLAKE2b-256 088dba233e3d330f11d05e7fddc721db8524ef57bc04aa1f4a28ac0476bb4fc8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 71391118454cc3e95c84645fb6d62ba3b158d04a8dc64578915af13df3807868
MD5 0f0658d975f432f5b5461939530738b7
BLAKE2b-256 bcc8c5b22e59ec825008b4114a3a8637992b0a0b5154fb2d144dd0e7dfc6fa9f

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 a99bd36ce63ced3107aad55c1ae6567106dc171a5dcde6a2d228b1b0b8396430
MD5 75c53b94e11c0d518253eb92b6850ac9
BLAKE2b-256 f5213872604420a76721101808d41733207a8bf45666ffaa1d7e993639a59fba

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 1a92d7dfc09501b9e402e3f0bacac9c548a658453f04852293271177b2997629
MD5 d39c4a5e91fbcf0c9d77030d41b02300
BLAKE2b-256 667bbf4b2ba19a88324aff94f5fc09b0021241df62c08a39ed5cd6b476f07e70

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 6da85a36fb71787cea9f48ba35030b7356fddc155f23030902982d99f3eb13e5
MD5 7a7dee0d8fc4c7e2af100a3522ea2d3e
BLAKE2b-256 6d21343649ca375fc32e1b7a745a7919c7bfd1b98c2f74d25e897e24b90bc150

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 23e99574c946b1f1af6efc33fcd7d6f804586037e900444558660c1a597b7238
MD5 62c9b565d452edc01cafad0e0f44482f
BLAKE2b-256 e2e4c532368dac3638b7091d87028869ff717a6a95152fa64dd692a9b9135ae2

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 02e520b3588b6cb9abecdcf02538123668d2c369a4e80665ad493c5c258d30ea
MD5 60970dbe5b392594b351c7fe885bf8da
BLAKE2b-256 6e566b157a7a16cd78f3ca322204fb7e30752a5a8085ef98e420d37fe8b4925b

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 0a4845cf991e35fdaa246d0edf0e9509ceb95f0ab04335cfb01be018bb011204
MD5 076a5e22a239ded608dccec0009a9ec4
BLAKE2b-256 b35ecabfc55ded1fda96b806d9517091d0746444c16a013a1e4415143c2faa99

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-none-win_amd64.whl
Algorithm Hash digest
SHA256 d80e6c4eef77226712209682d57b68ed0e95ad901a6a64eada268d7ff3754117
MD5 5d5f8ef48a9d8b4c875a6134b8422070
BLAKE2b-256 13f2ab1c221b42725af331121db6a8dc0cbd699a14457600133cbfd5a3b92a1a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp311-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.11, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp311-none-win32.whl
Algorithm Hash digest
SHA256 f1a42fcf8c15ff44a06638f59366464d899ba5b9ef0304130e960b9319b55c13
MD5 00e0f87182c948916a05e8dbc3768438
BLAKE2b-256 70353f74ad86c7ee22169ef42b2562c896b061d79a9843f054020c845b45ba50

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 34484c9cc4bb7debc939273441b9b6bebaf00a3b0b56ea1619023005e34dc7ac
MD5 53438fe36ea61d353aa09f748d50fc6f
BLAKE2b-256 b32cbe8fd55c69ca743a4fd58f919647390b55cf9c470ba97ccf6b06fa7339c9

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 33c0ce3fe18e870e9ec99f71f5a1e48aca6381d5aa1af3c8c91df3ebc3d6830d
MD5 6ceb0dab6cff1ebd8e62d48b33be4ef3
BLAKE2b-256 5c016fc42ca99da604130db500c867bfdea718a84c9d536985c3fa9d0f98e92e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 7b0e6fa7f782b3b1c63430516d38945ae8e62af8134a6f1af1cffb527159e4a8
MD5 65ffb24dedad96805f7eb54887f2817c
BLAKE2b-256 910000bea057523d685ca054fcc9846755759c70d5085389262c2753597c8dd9

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 086d6d42d05b011a9a1404410e54c950927f0f02a441f263559371f005f9a5db
MD5 7c795a8848f9db4f54f0014620c0c4b2
BLAKE2b-256 a7e6ee7eb78a74cf8a35c972f0e5c077850979f799f59b9cd67099a5e1890a68

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 a19d662874a8776e81dd3f24947ca1af59a3008c4b8d0fbf7f0c216d10878ccd
MD5 33616bcf4027e46195a1488b39feb4b0
BLAKE2b-256 f12fde90078b548eb10b69b407591b1d27e515ae75d4574abaf97dc26ec1be63

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 62abdbf4c689d0e3e09d3459ba5b963038aca3666f7b1e12b1b571546e5eb33a
MD5 8b682d441eb64c987b304c2c25b90d9c
BLAKE2b-256 dbe95e4689c31718394b2e7054cd669dea96bf1c1b9915acd776190ec12efab5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 a0c68c969c4ec2a9242e9152a11f1dfd68a1b440fee2aafbfb82aa956f19d9fa
MD5 ebf35b7e4ecd1e359eb8871fd4af8f14
BLAKE2b-256 1d569b7c93d95bf3b4bdbc757ea246feeb02d6677a94c4d793e00a7af63a4b36

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 df9e1c8758568bd73f7eef6a7966e0136870fb9e10739dd868970d6fe4a7f848
MD5 45f108eca9df46d5677b8543efccc384
BLAKE2b-256 669e702bdd4370157807d8a5c63e1e4fd4979c8e3765f907f88e87ed4e73c3b7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 81c9bb656dedd2a4fc65447b0efbe667b64f7f9c6ed847d14e4d948ade4d50b9
MD5 a1d55f985801f38838175fb4acac0e6a
BLAKE2b-256 6f0cf845b0a664f01be6fdad165fc01944952ce9cc78cccb2f5ae20cee906d87

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp311-cp311-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp311-cp311-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 4e66a4f59539355e600d42dbea22b621864793a3f9ce6085ced812e0cbfbf8c4
MD5 85df7aa1e3b225ea83d4e5d55517733e
BLAKE2b-256 158b98199b4abf9ae756d506eb67ad7fbc8929ae49cbe096fa579955d77f7f25

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-none-win_amd64.whl
Algorithm Hash digest
SHA256 8534ae69418941864cb036814e2396bba18c9b79dd115d59b285ef99c093a5b5
MD5 c7cc36a61a247a8aa11320f4049cc002
BLAKE2b-256 f88a44be7c019f8fe8af7779150972c08bef38194fa5505b035ddb277891cbb6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp310-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.10, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp310-none-win32.whl
Algorithm Hash digest
SHA256 8cabde5b770b4e1157940d6934fd870f7f3c4e23ca9b3f9e2d6a727b46d02dde
MD5 c688267be201248e05a67f8d3d502b27
BLAKE2b-256 a6c0d81cca6c3fe2d604f2db6ff1fb4cf26d8e5ef3f7d07bd383b276162cf299

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 fab81d96a1b9a15c9272aa5869ffd6bec6dc001af6e0e142f58aba09fed746d5
MD5 dd22c6562db2cb47a52bed97377f0496
BLAKE2b-256 924df9a87781cd0732c18ce698dfd7664bccc6e73c289d21a9ec8f86d47c7d3e

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 0f4feb893d6ae7cbd62b9aa5a306485b53bc47774d1b45b0cba2451287e9db80
MD5 1fd11d802222428c55f92de58192b759
BLAKE2b-256 c9500a9f1b2e17f3592b28ae7fa6c7b28b0a710dc6375064cebc9acf7d8b5c89

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 06a56acdfe6c5d51c03ebfc6838f727fcf231c035b94f2460cca68947f6799dc
MD5 4c5d531696ed6b6d4f72a8f630e287b1
BLAKE2b-256 11f98c77a471469ea7d1b52f2a25607385109c954d6444a9b0df19796beba461

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 6dda0b56caa7ef6ba98552a39864e225e6cd5bb5b0c31d2a4b1c31842bba996e
MD5 dd43026f8098e8e64147b1ea154de5b6
BLAKE2b-256 df8c9e4847d728639ca8d168becbdb49dd3acf6022a1c719260e52b53a07937c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 048e7709f48352b87250410e2516084ed844ca1e352f7ded72dac1b86eca8852
MD5 b5b8f1e44384362014957c93e85d12ed
BLAKE2b-256 02da161a981464154b4ca83b9f830f38d0d2131c5f943684235eb0acc788ac83

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 c92a9e45ef373d7b0d8de4e1cb4eee750cb1b55b509cd60fd474b0b6a2985bf4
MD5 723b05b0aecc83e720fc34d595e7356f
BLAKE2b-256 5072a3026c0afec76e338033ee56867d5f6e08dec2dd5f9169eb3f1b9dead4ea

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 0c57eac1f107db3c838a7dd7e20730981a5445dc10c195aea6af12c102daf21c
MD5 28cc66af3a56e78458210edcf3552522
BLAKE2b-256 3ec2cd38fc1ff3418df6fcd33d48ea07ac69f5a9452a7ebf7be2d2e9f07568fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 fa245378691f3b1f4cd9bc2b55aa620fde4486db889e2584eecc953b7d84c0b2
MD5 d10e2644ac23cda51526ef3edefe3d7a
BLAKE2b-256 62869b124967976bb6a6b8369c8dd2ed8c4849426e10569b65c37730472dcfd4

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 903d50ab57b63a5e1c555b1f54ca27f32cb6b68b872098efc82b216a2dbab241
MD5 b7e83f2e485a93fe8ca4cddf90894d3a
BLAKE2b-256 61051fd05290f7cd8411b91eb552cbaa51bc99f450afd13624818e327c855ac4

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp310-cp310-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp310-cp310-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 8e1c2ab2e501d52c39fa61fecb1270ff5ece272beab9b893792176c6e077116a
MD5 7d6d2f3538b111b13a31481eabeb0008
BLAKE2b-256 da4dd2616c4c10605003f4e32366f96e540d41092da9512b8a75389b5015b3fa

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-none-win_amd64.whl
Algorithm Hash digest
SHA256 adfbe2cac4b6b4773d7d4324f1932e98f4cf296b25542457ba692d50fc82a567
MD5 bf1afe47cea1eb159254858b1da952aa
BLAKE2b-256 fd1ddecef5841566b14d76ef29a4693663bc02692853882e5b21403b0188a9ce

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp39-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.9, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp39-none-win32.whl
Algorithm Hash digest
SHA256 c5ef11bd74460b1cb81b131d7229f80101e6bb5ff1c09777a927732e11237dae
MD5 f962304d3b28d1d3a3ac26438b77433d
BLAKE2b-256 52b81087c6a45f19ee2c2711f90f4c83b82766b3211c38b1ce41ee00ce695de8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 16cc108e5907ca9ed78363981c347deed4a835ae373b6d7cb73bacae6ed90add
MD5 d71f97d4def8d4e9dd76632d1e3e0903
BLAKE2b-256 0316801b0f9d5580cdeb855f40922d9216adb4ce1e9025e34483b006e09fc869

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 c6f1eb7a52f984dbfe641e4425e6809d89b2aa2122bfe3f62c8cca561371b677
MD5 88d3e7c7d54c63c8695ffd0f95216adf
BLAKE2b-256 f26845ed487b15455500ca694af367b3a8a7116e7937554afd471020a5e716b6

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 af6d09c3023b9acac7ab7eceae558dd35782f488938e27e4adedf1cfe4685b34
MD5 51657cbf2b9905b092017ff627b99b79
BLAKE2b-256 cb9dd17bb416f800333fdb863fd2665f91c288583deb8fa7c3299bb37f5ac9d5

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 8ed09d43119c19f3770582e1e43578e38215829110458ccb1761189c856d2c5a
MD5 048b51e090ee7d523611c200dfc8a5c1
BLAKE2b-256 529658b0cc50b65699ff786ef898b29b8d037476593a8bdcfe59e9569bdacbfc

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 c4025a6d665d96d3ebff290bba865736452c21d5917dc93cd975f55cde463f64
MD5 db5c61a979aae65ffc8ef7abc069d382
BLAKE2b-256 7999f2f99132a5a904af59b302a79df9d1be7200fd89cf459bf3b0ac34c8a42a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 702df084665e6c44fd7a5c4193a62f574db405c5962871073031b33d2523a63e
MD5 6cd93e58432bce13845bf882ecf72226
BLAKE2b-256 35323722b188008ea14b0cf5062dcce3e41a36ccf14738fb50d400a674cbccef

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 bb06a003a4eab32eab33e13114d4f656df08579ee17452066d6b576997b016e8
MD5 ac3508b8c83a71c13153c6dfa5253954
BLAKE2b-256 0faf9cc7bf6c9cae7d6ac3494cf8b9818fd47d6b857760e516fc8ea09e32a4ff

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 8390ae7558924d2168826fc20ea845cc1b1d7545d8471b9f781d5b042da3b8cd
MD5 9e1eb65f7ae623db6efdb20f91349248
BLAKE2b-256 542301b3aa0c8b21f94336c00f568d9f81716e5e3a6e66aca9ed2bc87c57bf78

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 79e48fe564408ec1eb28a5a975748509fd4d5ca018aa41c530df143e14720e15
MD5 bcd9d641b88dbf04258f2d0a703b63f6
BLAKE2b-256 f86c9f755a568ce7d9dfde769ca60e32b34d9c1f82dced82ad7549695d439496

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp39-cp39-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp39-cp39-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2b61cbed700617aa9cd5c6344317ca0147f60d3253432820a945544fd47c7e01
MD5 7dd6905fe5d2a758be6cdfca2e93d939
BLAKE2b-256 0ea1b66886e2d9a23e2e88d0b5e694adf4351edadbd1c6e0207c134fd556b63b

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-none-win_amd64.whl
Algorithm Hash digest
SHA256 62d6b5643ca3ce0b6ea06a6c482e5582a75ea7d283e065a4f285e4d685a57169
MD5 1b962ee2b3d1cb5a6ce969c130cd3c91
BLAKE2b-256 4faa62429287c247b3ca0c627f356505c31734e895662943ced4daba84a1bfbf

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp38-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.8, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp38-none-win32.whl
Algorithm Hash digest
SHA256 ae8859bb723193c423f34b3ad8e593cc5db8d4bfcb3a8ffc8148a2271602ce07
MD5 ba589bd8f07519a42d71a54c5b9cfa87
BLAKE2b-256 cbb39f056a1cfdcf922d14f838f8cf8987347954192041f5f7419c1d80fed858

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 1fb896ceb1fbaa1aa2d2256b3919b2321f02518cee0d00bce7066722b14475b8
MD5 a7e39b0625f05277202bb32ded79b262
BLAKE2b-256 a27a0780b3f3eab16dd2bca0ab2a2f23f649ea9138255b54a78fe5a4e983f60c

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 2facccd44b6669af859b3b43331d70b69b4c217218022a22dab6b194839b8a0e
MD5 85abfcb75e2808639c656d5ebd154dac
BLAKE2b-256 ed3b544e1a9a76dccfa4b97189fed303b426d42a83088f8e9cba0b2ff7aa4d11

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 aeb79f83cdc1f91b462186636c7e2771b197a94d22ffd42fd4f01b12c6e50be0
MD5 8d133b7866ce7436c654549ceedc6629
BLAKE2b-256 e27e495d414990d8869e38c0add7d7c729562d3d2106a312ab8036d1f0af51de

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 b3296fdae7ce368c6b32241fc3672763e3e95918292f4df0993f0eaa9d045585
MD5 de96c3ed09eab80de476a1822a9eb6e5
BLAKE2b-256 b58713e04fce8cc1584fa689518220c9cfdde3e912a9cf95f45785e87c2394c7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 1d8a4a669f7ba5c332f48ae6b4b858fdad51558f8027595545910ca9cd9c9d08
MD5 0d1219ad50aa4eb1efb581375b964492
BLAKE2b-256 f4665b043c23f3f8eb97df49d5e38c0b4151f0f4f724d61ca58c626407ee3e9a

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 bd9f5ae759ac1c0d4f6a9a401b6aed84c1d6c971c453123412458feddae5acc4
MD5 c304b63f6276cb4789b9fb5ae088956c
BLAKE2b-256 f2ca5a93144ebf1fd1b7e07ba133d924349f4b670041c7ea02b5ab944d8277b8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 c35e4c72f296f07d0d74f448446fdbaec36d5395ba2f9fbc1f06841c9f4ca933
MD5 bb18ca2832b85d90f9e1dfd4f64c7136
BLAKE2b-256 b7ed21e71872cf2ac8723379ae899a5fdf904aedf797336e994bde15fef46335

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 318aaeb2440294cdaddbb9a1a555f931dee1c5174c94a625de3b0fd3d2f69b10
MD5 45f06bc2c391f2e2f7837f6d7085d0df
BLAKE2b-256 dddb29b6b321307be392be51b39c50f89ecf963073f8fa5c027eacb7f756ddb7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 896a8c3fa973842e2080d61bcaa0a4d08d1a1732c68ae7e7bafabb0af02241d1
MD5 a9285b47df010bfc0cbb17ecad94f99f
BLAKE2b-256 8a10c433cb42a0983f6e7f38733a1ee27413c1b1450f98e62ad0f2d88b7f2dff

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp38-cp38-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp38-cp38-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 1fcff5cdc3480cdaf594fb4714eb3f2d8a49f071f56546a6f67bc98924e8d04a
MD5 66c459690e3a4fbacc819ff1f8041770
BLAKE2b-256 2b0b5550eac88bf2efef15f6879c096f0e4d69d36485d8d4c80de8ae0dd59209

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-none-win_amd64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-none-win_amd64.whl
Algorithm Hash digest
SHA256 bffa7582de944f56d985f46c12207c2fa8efbdd9958e59479fde2406067351b0
MD5 54b7ebfc1504ac6a822b36a78abf084e
BLAKE2b-256 a1d182eb4e9fcf60878b9da74c9a6e6e9027824bed80b30f04027f3f73f4328f

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-none-win32.whl.

File metadata

  • Download URL: tokenizers-0.19.0-cp37-none-win32.whl
  • Upload date:
  • Size: 2.0 MB
  • Tags: CPython 3.7, Windows x86
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.12.2

File hashes

Hashes for tokenizers-0.19.0-cp37-none-win32.whl
Algorithm Hash digest
SHA256 93d0effb6de4d1fe63418e34360b922a0091a1ba4553e76caf44abf096ff4697
MD5 83740067ed9554f6c80160c57c760c86
BLAKE2b-256 e413c51d23af4297b2f07e0186e8b43bf43c9eb28726597e717b1235eaa9d525

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_x86_64.whl
Algorithm Hash digest
SHA256 4cd8cc13594b9e41860154a6ce7de4dffd88828bf92e0a162b83d8c9f114016f
MD5 3e70ec9fe22d25d7e7725fed1f1e0081
BLAKE2b-256 ba890fafb9eb96483f4d4bd47f2b10cd0411679bbcd6b98062af802de89236d7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-musllinux_1_1_aarch64.whl
Algorithm Hash digest
SHA256 aefc2519372919cc5582e50737cc7068c568d394fae5585461bd9cd51c840359
MD5 465b08a2f8b6fcfe4f08a80639150121
BLAKE2b-256 a3a089e81489b64448f95c31fb9b5d32f288328b810a1c82fcffb99454b8ff91

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 8ea672ac91c8f92d749c427358ff57961ffa911fed2c9927626fdf11bde6033f
MD5 2eef59cd917f3c578a48328e821e78b4
BLAKE2b-256 5d31ad38e4bf618ba17b5b051f3ed8b0ff8b8c3812cf4f8455ab1168aeef48f8

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl
Algorithm Hash digest
SHA256 ca8eb3857bf9ed5bf3939644a5a9aa5435d941c9bf6969ddfb5e17f547c14c7e
MD5 f5a3f99dc81426222255bb39c8e15900
BLAKE2b-256 f60ffd4432f696ba4b39bb150020b21c3abde457e5bce795e54b4def8b3b5ea7

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
Algorithm Hash digest
SHA256 156d9d19d9bee19db1a14e876167a525eaf5da144b48b9fa437a5facc00d6cb7
MD5 f1a62475d365870d1f7c820fc0eb8da5
BLAKE2b-256 e2eaf3fa3160a031a91bf3c5b6e878baa3514d6a10fbb2862948852cddfefadb

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 5e9e665ebbae2899cbd55122742bd360b56d866c49cff622f0125b48fedc7eaa
MD5 aa8280414c248aad51f94a519bb4e1c2
BLAKE2b-256 2475a0d7c9d181ddf454c1b468b8ba9a96eee9ff09ca60a73914b6c7290c2c03

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 be0d76702d0e281fcba49e6b9514e4453b02b04c3f435febf8573911ea6db1aa
MD5 8688a7815cf504dcf7839b7ebec87863
BLAKE2b-256 9816f372280ad307bb356041b29ac6b7ce09f46875b3c0cee3b798052950545b

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.whl
Algorithm Hash digest
SHA256 c4f2baec5e9dbcf8cd15e071759c5742d087119122aeb9ede47dc0ea7e63459b
MD5 e6d3d68b96aaa0b7dce8a833593a69da
BLAKE2b-256 b0a6cc26a7209e71d9dc74e83ddb3de19a712e42e06552a9a95500c93f278b27

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 a956da5b55eaf919b9e00eb9cb6b6f371271b84d782be56e42aac837eaa78cda
MD5 67a4de336aa24e8941c5fde6f600d53c
BLAKE2b-256 785789c41828fb0c91fef116aa6cd2c9ccbb84de97da78e3910591da1cf871ba

See more details on using hashes here.

File details

Details for the file tokenizers-0.19.0-cp37-cp37m-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for tokenizers-0.19.0-cp37-cp37m-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 2076139acb57a63844ec1570fedf82ad61b4cecace597f94660fec08d6da9b5a
MD5 b1dc76686da128bf535f2a975e1ec195
BLAKE2b-256 282456783707958103d5df4c6367689aa333e674275fe613ab85ed40079468dc

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page