ONNX Runtime generate() API

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Project description

ONNX Runtime generate() API

Run SLMs/LLMs and multi modal models on-device and in the cloud with ONNX Runtime.

Model architectures supported so far (and more coming soon): Gemma, Llama, Mistral, Phi (language and vision).

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for microsoft from gravatar.com microsoft Avatar for natke from gravatar.com natke

Unverified details

These details have not been verified by PyPI
Meta

Release history Release notifications | RSS feed

This version

0.5.2

0.5.1

0.5.0

0.4.0

0.3.0

0.3.0rc2 pre-release

0.3.0rc1 pre-release

0.2.0

0.2.0rc7 pre-release

0.2.0rc6 pre-release

0.2.0rc5 pre-release

0.2.0rc4 pre-release

0.2.0rc3 pre-release

0.2.0rc2 pre-release

0.2.0rc1 pre-release

0.1.0

0.1.0rc4 pre-release

0.1.0rc3 pre-release

0.1.0rc2 pre-release

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.5.2-cp312-cp312-win_arm64.whl (751.2 kB view details)

Uploaded CPython 3.12 Windows ARM64

onnxruntime_genai-0.5.2-cp312-cp312-win_amd64.whl (777.0 kB view details)

Uploaded CPython 3.12 Windows x86-64

onnxruntime_genai-0.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_x86_64.whl (871.5 kB view details)

Uploaded CPython 3.12 macOS 12.0+ x86-64

onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_arm64.whl (771.1 kB view details)

Uploaded CPython 3.12 macOS 12.0+ ARM64

onnxruntime_genai-0.5.2-cp311-cp311-win_arm64.whl (751.6 kB view details)

Uploaded CPython 3.11 Windows ARM64

onnxruntime_genai-0.5.2-cp311-cp311-win_amd64.whl (776.3 kB view details)

Uploaded CPython 3.11 Windows x86-64

onnxruntime_genai-0.5.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_x86_64.whl (869.1 kB view details)

Uploaded CPython 3.11 macOS 12.0+ x86-64

onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_arm64.whl (769.9 kB view details)

Uploaded CPython 3.11 macOS 12.0+ ARM64

onnxruntime_genai-0.5.2-cp310-cp310-win_amd64.whl (776.3 kB view details)

Uploaded CPython 3.10 Windows x86-64

onnxruntime_genai-0.5.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (1.4 MB view details)

Uploaded CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_x86_64.whl (869.1 kB view details)

Uploaded CPython 3.10 macOS 12.0+ x86-64

onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_arm64.whl (769.9 kB view details)

Uploaded CPython 3.10 macOS 12.0+ ARM64

File details

Details for the file onnxruntime_genai-0.5.2-cp312-cp312-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp312-cp312-win_arm64.whl
Algorithm Hash digest
SHA256 315b23cb04749202c9cc3eb34f281bb4943de477a5aa46c99b940603b6a5d272
MD5 e21c93b3f16da16fe4c495a494732858
BLAKE2b-256 08c1a69aeba29f40febd8d70d45044d4eb97905beb37fc8491b1628c8714ecc1

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 422e9af27f182247378e9423f5745becfaffcdf7a4f452da17fd5d9390770ca7
MD5 95275e916811762238def332fc04f6a7
BLAKE2b-256 9f7b53b217ed0db401877fafa2f63d2ce7de754899f2bdf4cb415931e2019f18

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 bb5b9650512e21a71d965e582d616b33df07978b0c3ecbd5bef0912a7b5f7832
MD5 d8d5df04f63fd8aa861cb19becbf9c39
BLAKE2b-256 793d43211c8a66d7ce54dea137ad7bec30767e3f2dc5e1e22befdcca290ebbe0

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 d7bffb799d44656b2615fc43130a1a287d57e8893b80523e560924cf05770f1d
MD5 020a3d0e29b1dee8971ba5cb54c343c8
BLAKE2b-256 8c7273c95e357ada258025236437fb2b4d56fb7e8594db6361f4560ea97ca06c

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp312-cp312-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 6b438d7f4901081b8f3ff99db6c6ea15a3fcc107abce79859ff635e1278e26b0
MD5 75b4bcf59eaef9cb8c0226079708c7d9
BLAKE2b-256 cf72259de19e93e72b14d0a3910f1025f71da006a8dfc76c97792646b335a8a3

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp311-cp311-win_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp311-cp311-win_arm64.whl
Algorithm Hash digest
SHA256 893be15d2113438e60b8a1c0095892e0fd4f2b01dd470d6197337db2a5778c88
MD5 b15f02827628a57945077a7e1d698915
BLAKE2b-256 2257d249827c3e37abe528674bfa97de4c61b18afb452d2afced690a745e0866

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp311-cp311-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp311-cp311-win_amd64.whl
Algorithm Hash digest
SHA256 09b697f955616156948f21366d13d02884a15521926f68a259722d9fa4437db4
MD5 74a8307bc1a9f3171c82a791d83e705b
BLAKE2b-256 59b1acb1daf1a08c8098c828e7ea9e187b9728a8fc151a4df4911f988c08a874

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 88edb36c9e2d670316f1e6e4ce27a86f212648a92053a94a31f88b1f4d6c0935
MD5 cf674d3ccc588174ee08dbb07dc35f31
BLAKE2b-256 6a17a29c0cf89d90374234b8e510fcb970f2e043b42689b5ea23cbdab5a414b6

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 6194aabd589b3ffb571b325f504266ac47c33c434abfd87575c30d7a3e1179c9
MD5 6842e46f204baa152a4323aa9a2207eb
BLAKE2b-256 331366ffa143cc82f8352ec87ba0501bc21e05dd9e84fbbad530e74a705ac911

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp311-cp311-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 217c748f8ebd1a5082e1ad8ee8fc90fc1a4e9ce7839189f4c2c2545d1390af15
MD5 30b292d68335f6a88664fc2eff5bec27
BLAKE2b-256 3e3de2d8f89c05c6cf35e2ade2b335b1b97725327591b8fb141d266ab98615f9

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp310-cp310-win_amd64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp310-cp310-win_amd64.whl
Algorithm Hash digest
SHA256 4d2968df6d8064664a5f095006c70520f4ca689204b695e88951f088477bc1e0
MD5 02ff230628a2f088ca38315f78ead84a
BLAKE2b-256 57d691e486424f924c2a99e8f1bd201180979101ecc09bee1ca7f53dae1c8a38

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 bd7954f9dc829e69dabd7f676443529ac18171ec8077438c16364d381733070e
MD5 c76008e86ab4ffa0f2f8e0ae1ad15521
BLAKE2b-256 125b6f08f9435f0c3977046cb4292ab1e836c22cd7d56fc87ace4d2a90dfb828

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_x86_64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_x86_64.whl
Algorithm Hash digest
SHA256 79d721a33e80a9664aeeb87c0ceec75801fc81e48e8ff7940e3658d0b28f25cc
MD5 4f6f4295c58837d6cb7e19e3af357e87
BLAKE2b-256 7f1b5166ed4a73c5e9f92e6db4d7838923ffd595cea164661fae20d82e3a6966

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_arm64.whl.

File metadata

File hashes

Hashes for onnxruntime_genai-0.5.2-cp310-cp310-macosx_12_0_arm64.whl
Algorithm Hash digest
SHA256 cd322ead0027fbfa309e7be76c4512157ad369dc189ab3334a58a199b4f58a02
MD5 2a15780c14b4a37165325d0da524c35b
BLAKE2b-256 523522a421f852eb14f47c33a4dd4c3ef58a2f3d5a96be8bb6d6cc271b2a0e83

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page