ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI

Project description

# ONNX Runtime generate() API

Run LLMs such as Phi, Llama, Mistral, and Gemma with ONNX Runtime.

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI

Release history Release notifications | RSS feed

0.5.1

0.5.0

0.4.0

0.3.0

0.3.0rc2 pre-release

This version

0.3.0rc1 pre-release

0.2.0

0.2.0rc7 pre-release

0.2.0rc6 pre-release

0.2.0rc5 pre-release

0.2.0rc4 pre-release

0.2.0rc3 pre-release

0.2.0rc2 pre-release

0.2.0rc1 pre-release

0.1.0

0.1.0rc4 pre-release

0.1.0rc3 pre-release

0.1.0rc2 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.3.0rc1-cp312-cp312-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai-0.3.0rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.9 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc1-cp311-cp311-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai-0.3.0rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.9 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc1-cp310-cp310-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai-0.3.0rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.9 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc1-cp39-cp39-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.9 Windows x86-64

onnxruntime_genai-0.3.0rc1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.9 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc1-cp38-cp38-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.8 Windows x86-64

onnxruntime_genai-0.3.0rc1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.9 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 6f6cfcaa711d49266881e978ee9dc4ba1402d96b4fa8d1f5766a1b6d076e19da
MD5 8f0fa4539fa854c1534bad280dff33e0
BLAKE2b-256 46ec91d56f3435f78444c65f6718ca9e297e69108959dd6f7010f59c84ce890a

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 0d67784ef38fff19aabf5218bbb1fbd836f06412084b32cc70990289fb6ff52c
MD5 aecb785060c02c70b53a42120a045d38
BLAKE2b-256 f3088bfe6fb867ed2233f30819614bc30b52f38794b8a54f8bbaa7a8ff296e80

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 18380992cfd0b18b00aeb8f0e35c71daf90177fa8ee23a13a5c903519985a633
MD5 ca2b716fa1877611b636ff7a96c8b9b0
BLAKE2b-256 f5f6239ddcff299af5e7acfd09c1807b0fb8b0c06328ca77fbc332d8b8f318fa

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 07bcccb25c8656b41514f5d959ab247365923b8a7661e2b30eb33a36111ef86e
MD5 1e28c5cad03dfa96c189201f16029a8b
BLAKE2b-256 68a39ba469561a2f7d66f9263c45bc5b834d41686993ad6b7487f4a93b10df31

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 e2db53c7b21b197c10658c169191ee0b57122089fed3caf6b7b826c99b00d7a6
MD5 e534f32e38a0c66684ff6e36b718f76b
BLAKE2b-256 9cc3aae067c5d569f8031f4204c19973d47ebd31125b88351a6f7df6299237a9

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 2a017283de302304fc74856a4b6b2f6ea603b337b7f0e299f7f45c06ade295b7
MD5 03c9e45351483b391483bd714f2b6ab0
BLAKE2b-256 4cc9fc7b335fcf3e224014ed710057e85e796a63ddfc1da832a4a88d54282c05

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp39-cp39-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 a34c820307a70ea055428614410438393ab565025d8035c1e52b00d8fce946e2
MD5 59e30996f42a7bd91018ea3f856c7f02
BLAKE2b-256 16fa0ee9d28a5715d134a60887f210881aa56f7d603102c2d73d4f42a61c57df

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 eef3174b8abc924e5e099ce6dd2a1820be09356c132f632d16cc73ce355fd31b
MD5 1b85d47757c999fdebcb8074d6689fa6
BLAKE2b-256 3ee1505254ed204f6c7629409d3d3451919349ad261b8bcc90a74cab9afd2f3f

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp38-cp38-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 3a5ada2351a3269840fa0912feebdceda917431e799ca91d061ab07e457e4aef
MD5 adb63648a821c2ab912b8a0d9e05b17e
BLAKE2b-256 6114044abf9d1e58fd4aa13953420106adf3615e312a0a7a91459c12a9322682

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 47bee04e48a609f6302d950bcbbc718ece1e5b3de915aeab8c818602cee5b75a
MD5 d951d16617ae683d85a1488ededc087e
BLAKE2b-256 54a5334f4bbd23193d9fd7fe979a236552adcac85ed397cae88e429b4ef720ec

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page