Skip to main content

Finite-state grammar compilation

Project description

OpenGrm Pynini

This is a a Python extension module for compiling, optimizing and applying grammar rules. Rules can be compiled into weighted finite state transducers, pushdown transducers, or multi-pushdown transducers. It uses OpenFst finite-state transducers (FSTs) and FST archives (FArs) as inputs and outputs.

This library is primarily developed by Kyle Gorman.

If you use Pynini in your research, we would appreciate if you cite the following paper:

K. Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In Proc. ACL Workshop on Statistical NLP and Weighted Automata, 75-80.

(Note that some of the code samples in the paper are now out of date and not expected to work.)

Dependencies

  • A standards-compliant C++17 compiler (GCC >= 7 or Clang >= 700)
  • The compatible recent version of OpenFst (see NEWS for this) built with the grm extensions (i.e., built with ./configure --enable-grm) and headers
  • Python 3.6+ and headers

Installation instructions

There are various ways to install Pynini depending on your platform.

Windows

While Pynini is neither designed for nor tested on Windows, it can be installed using the Windows Subsystem for Linux (WSL). Simply enter the WSL environment and follow the Linux instructions below.

MacOS

The pre-compiled library can be installed from conda-forge by running conda install -c conda-forge pynini.

Alternatively, one can build from source from PyPI by running pip install pynini.

Finally, one can use Bazel to build from source by running bazel build //:all anywhere in the source tree.

Linux

The pre-compiled library can be installed from conda-forge by running conda install -c conda-forge pynini.

Alternatively, one can install a pre-compiled manylinux wheel from PyPI by running pip install pynini. This will install the pre-compiled manylinux wheel (if available for the release and compatible with your platform), and build and install from source if not. Unlike the conda-forge option above, which also installs OpenFst and Graphviz, this does not install the OpenFst or Graphviz command-line tools. See the enclosed Dockerfile for instructions for building and deploying manylinux wheels.

Finally, one can use Bazel to build from source by running bazel build //:all anywhere in the source tree.

Testing

To confirm successful installation, run pip install -r requirements, then python tests/pynini_test.py. If all tests pass, the final line will read OK; a successful run will log some errors to STDERR (this is working as expected).

Python version support

Pynini 2.0.0 and onward support Python 3. Pynini 2.1 versions (onward) drop Python 2 support. The current release supports Python 3.8--3.13.

License

Pynini is released under the Apache license. See LICENSE for more information.

Interested in contributing?

See CONTRIBUTING for more information.

Mandatory disclaimer

This is not an official Google product.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pynini-2.1.6.post1.tar.gz (790.4 kB view details)

Uploaded Source

Built Distributions

pynini-2.1.6.post1-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl (98.8 MB view details)

Uploaded PyPy manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl (98.8 MB view details)

Uploaded PyPy manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp313-cp313-manylinux_2_28_x86_64.whl (154.7 MB view details)

Uploaded CPython 3.13 manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp312-cp312-manylinux_2_28_x86_64.whl (154.7 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp311-cp311-manylinux_2_28_x86_64.whl (154.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp310-cp310-manylinux_2_28_x86_64.whl (154.5 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp39-cp39-manylinux_2_28_x86_64.whl (154.5 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.28+ x86-64

pynini-2.1.6.post1-cp38-cp38-manylinux_2_28_x86_64.whl (154.7 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.28+ x86-64

File details

Details for the file pynini-2.1.6.post1.tar.gz.

File metadata

  • Download URL: pynini-2.1.6.post1.tar.gz
  • Upload date:
  • Size: 790.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.12.2

File hashes

Hashes for pynini-2.1.6.post1.tar.gz
Algorithm Hash digest
SHA256 d831ab53abb22c862fb56471b12edf26e77d2b02d1956f45f0259e6402493084
MD5 4980c77c4c392ee178c8551a27202007
BLAKE2b-256 b87906049a733359a2da244c21a88df90828cf27eb0f7947ebb8b00653c7e93c

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a5a38e074b9eaf7aa08c1cd286d5b7990ef9ccc28f4e5e8d31f09ba10729431d
MD5 fe69cfb142df9398ce366884455bfafa
BLAKE2b-256 e2236a487e8de16b151f5b9bfbd15d5b7969976d8bc5e79eaba8a33513dda6d8

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 8bb8b57896df3ae01bc3a7e4018b90847d16b7a1f0430c1c0bc8a73d1e23baf6
MD5 bbedfa2fb6e5e8bf8e2d0e9bb4440525
BLAKE2b-256 82227d3c5fc5ea2aabe87ce09cb14415cbf822c97bda283065f29da6de9aa23a

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp313-cp313-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp313-cp313-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 afcbcd23eaed541c5619c47b20031055af7ded44980fe96c7e6d939fac81b91f
MD5 9265b142ea413d7ee713a6b26d3ba840
BLAKE2b-256 1e057b2fd6fcaa33362e1bc99cf371273d1500e4b105c9f26ad0c65a43c68551

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ca567a454fefa7c9bf5180d4d9b4cb5d62f9f99fa326554076fe2a945dc2f776
MD5 32164462045dea980aa9a52a6c33071b
BLAKE2b-256 62cf67f560932cbb12a0d7d69e14ed08e83e0ee1ca0ea1c019e486b21048e9b7

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 957a8b0d7b27a6c7ac15aae1e0e384d92e3ea2ef4e673188a8ccb92a3224e895
MD5 69693b7e4faa70a0b9594ec8b0fe8bb3
BLAKE2b-256 b2595c2499971384041aa9a2ad16c02c5d70f6802d49cac08e234f265cd1bc67

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 10f0717fc6bf09330c41978e4d958a000b790b9b344cbc1117d33b008edf4fde
MD5 46a2e22b4b0576ec9b783fc56b2b6111
BLAKE2b-256 ad3abf7d70e896d3099999c0350c5deb4c7d404738f34b422ddb1481321009b2

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 3c77d24d2111b7aeabfbdac69fbd52d512eb67481fa348b3e8e1e688cb63f792
MD5 e8d7fc697d9c523508537bc456f9bc57
BLAKE2b-256 995c71df7fd882bfe38be0c74a52ba6748d91f49a30ccf155e0b3ea9203b1a84

See more details on using hashes here.

Provenance

File details

Details for the file pynini-2.1.6.post1-cp38-cp38-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pynini-2.1.6.post1-cp38-cp38-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 89c282fc6d6e61781f1ab1ebd614b3db95f0d39cdbf662bdb2a78853d04dbdb0
MD5 5a722f9104e03f0ca0ad8af178047e48
BLAKE2b-256 6be4294de71581e44d350df42a22d1f4225ebfbab8f10942485df52a74fbc2fc

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page