ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI

Project description

# ONNX Runtime generate() API

Run LLMs such as Phi, Llama, Mistral, and Gemma with ONNX Runtime.

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI

Release history Release notifications | RSS feed

0.5.1

0.5.0

0.4.0

0.3.0

This version

0.3.0rc2 pre-release

0.3.0rc1 pre-release

0.2.0

0.2.0rc7 pre-release

0.2.0rc6 pre-release

0.2.0rc5 pre-release

0.2.0rc4 pre-release

0.2.0rc3 pre-release

0.2.0rc2 pre-release

0.2.0rc1 pre-release

0.1.0

0.1.0rc4 pre-release

0.1.0rc3 pre-release

0.1.0rc2 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.3.0rc2-cp312-cp312-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai-0.3.0rc2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.8 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc2-cp311-cp311-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai-0.3.0rc2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.8 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc2-cp310-cp310-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai-0.3.0rc2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.8 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc2-cp39-cp39-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.9 Windows x86-64

onnxruntime_genai-0.3.0rc2-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.8 MB view details)

Uploaded CPython 3.9 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.3.0rc2-cp38-cp38-win_amd64.whl (5.0 MB view details)

Uploaded CPython 3.8 Windows x86-64

onnxruntime_genai-0.3.0rc2-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (12.8 MB view details)

Uploaded CPython 3.8 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 3d794bba2637047c5f9ce04a9cfb09c5bb2be16831791c91a53a63cfe91b360e
MD5 d74a9503765d8a42875fcbcc6d4e6c27
BLAKE2b-256 fd276a3165c46f52a35c5348783e83d86da71fcc2926b9b8123960c6f59efc48

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 9634f6578f4ffcf714e761b0de7cc3650e20b441ad20f714d5b592aae5bd7a41
MD5 8997e1b278c184ec73427c506783912b
BLAKE2b-256 22c422d7fe283831a97b73fde2601ebab7a3a2538e874a70077c635134eba847

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 14b99495d15ab0d3818602310eecc1c8fee437f73607de1a058135df9ea29f35
MD5 52c3c552101c3392a21e4edde2159dfd
BLAKE2b-256 712bbb6103f141ef96ba68f6ae293357ac9f40aae6d8416604e7e8a5e6bfb8e3

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 b1f6f48f8bd73c9862007919ce2b847a470ddc3a4ffd3fc99f7f5f12c86943e8
MD5 43d976317d4d7663e5b6863cdd846671
BLAKE2b-256 b4879ef5e519060af38759be88d38c2d4b08290ef1287952dd277d25f987bba3

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 dd360f545f157b4094b1d71ec6337441a5b8cb601773ffdf8d92d9e066ada6da
MD5 d81d75f201ab7d47c031ba99419c24ed
BLAKE2b-256 fd4264f0a3f7d5c7f941d93a97bab740d99cfbf1d8bd61d95f3e81d715c37247

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 fbf96d34f6bbfcd2bb83f763c550592199b1c69cab615f17a3cf147b2027c850
MD5 0008c8f380c84d25dd5526ff1c7c2fff
BLAKE2b-256 93c4b6df1df0daa39ed60223de021e5b5311b77182226b34627442b236ce3e64

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp39-cp39-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp39-cp39-win_amd64.whl
Algorithm Hash digest
SHA256 63d2ebdd1f004f5ba34de22f24b18fd8bb4cbfb5d425df60d310e751d7e269fc
MD5 40094c20fbae60c5146de8583df4e824
BLAKE2b-256 6d4b82ae6599cbf54c1970532626020762909764de58c383fffdd81000ccf0f9

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a04fde5e513245c18a6fd41d1ce4813eb9324f1dd54243beb55592fa068a273b
MD5 edf0efaa2727e7f00a67786423599f27
BLAKE2b-256 7afa4fc7ae2c2ac88bdf38e35e80d7489acad185547ed53858a18c3d1522f57c

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp38-cp38-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp38-cp38-win_amd64.whl
Algorithm Hash digest
SHA256 bbd7c65255c26afde30e1b641b55853d6a9b4d5bd10491dfd2fd241ff4cedf31
MD5 773e41ab67608db69a630beccb9bf228
BLAKE2b-256 a4fcea9cdbe7ab94d5b6f220ea1c08237cc840f862b8515715a1d4bd5bd96227

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.3.0rc2-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.3.0rc2-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 d825b3989b0211314f62602821dbbba7bef3fda0b2cabc173f2fe020971b4b77
MD5 268c815fa7413d7bab2655c3a1c6fab3
BLAKE2b-256 cf587747b6f2256aa7cfb05bb2bc9744cd5d83c9a043cbae0c41f8ce0a2894dc

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page