onnxruntime-genai

ONNX Runtime generate() API

Project description

# ONNX Runtime generate() API

Run LLMs such as Phi, Llama, Mistral, and Gemma with ONNX Runtime.

For more details, see: docs https://onnxruntime.ai/docs/genai and repo: https://github.com/microsoft/onnxruntime-genai

Project details

Release history Release notifications | RSS feed

0.5.2

Nov 25, 2024

0.5.1

Nov 13, 2024

0.5.0

Nov 7, 2024

0.4.0

Aug 21, 2024

0.3.0

Jun 21, 2024

0.3.0rc2 pre-release

May 29, 2024

0.3.0rc1 pre-release

May 22, 2024

This version

0.2.0

May 20, 2024

0.2.0rc7 pre-release

May 14, 2024

0.2.0rc6 pre-release

May 4, 2024

0.2.0rc5 pre-release

May 4, 2024

0.2.0rc4 pre-release

Apr 25, 2024

0.2.0rc3 pre-release

Apr 24, 2024

0.2.0rc2 pre-release

Apr 23, 2024

0.2.0rc1 pre-release

Apr 23, 2024

0.1.0

Apr 8, 2024

0.1.0rc4 pre-release

Mar 25, 2024

0.1.0rc3 pre-release

Mar 18, 2024

0.1.0rc2 pre-release

Mar 15, 2024

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

onnxruntime_genai-0.2.0-cp312-cp312-win_amd64.whl (4.5 MB view details)

Uploaded May 20, 2024 CPython 3.12 Windows x86-64

onnxruntime_genai-0.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (11.8 MB view details)

Uploaded May 20, 2024 CPython 3.12 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.2.0-cp311-cp311-win_amd64.whl (4.5 MB view details)

Uploaded May 20, 2024 CPython 3.11 Windows x86-64

onnxruntime_genai-0.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (11.8 MB view details)

Uploaded May 20, 2024 CPython 3.11 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.2.0-cp310-cp310-win_amd64.whl (4.5 MB view details)

Uploaded May 20, 2024 CPython 3.10 Windows x86-64

onnxruntime_genai-0.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (11.8 MB view details)

Uploaded May 20, 2024 CPython 3.10 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.2.0-cp39-cp39-win_amd64.whl (4.5 MB view details)

Uploaded May 20, 2024 CPython 3.9 Windows x86-64

onnxruntime_genai-0.2.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (11.8 MB view details)

Uploaded May 20, 2024 CPython 3.9 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

onnxruntime_genai-0.2.0-cp38-cp38-win_amd64.whl (4.5 MB view details)

Uploaded May 20, 2024 CPython 3.8 Windows x86-64

onnxruntime_genai-0.2.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (11.8 MB view details)

Uploaded May 20, 2024 CPython 3.8 manylinux: glibc 2.27+ x86-64 manylinux: glibc 2.28+ x86-64

File details

Details for the file onnxruntime_genai-0.2.0-cp312-cp312-win_amd64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp312-cp312-win_amd64.whl
Upload date: May 20, 2024
Size: 4.5 MB
Tags: CPython 3.12, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp312-cp312-win_amd64.whl
Algorithm	Hash digest
SHA256	`e3fddf2ebe52a82fc6e4f9e686c46acf5f51060d24636c71f8b95e4b21b4378d`
MD5	`ef34e80b0c975b6716459f8584facee4`
BLAKE2b-256	`fe7553d245a7e7fae6b1a56c205ce5c22cba3dffe77a6fd77d8e6d82e2980b02`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Upload date: May 20, 2024
Size: 11.8 MB
Tags: CPython 3.12, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`71e0e30cdfc59b8022da6bded5db6845a5754cef17fccfc50cdb030d14428e69`
MD5	`62990612c83a0a81d51b2a500df2cc8d`
BLAKE2b-256	`98442ee313069ec537556af3c0aee317c1e0e05c241d51441b977f9a5c6c5171`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp311-cp311-win_amd64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp311-cp311-win_amd64.whl
Upload date: May 20, 2024
Size: 4.5 MB
Tags: CPython 3.11, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp311-cp311-win_amd64.whl
Algorithm	Hash digest
SHA256	`26f7a8c58f07071073418eb8ff815800e71837c4c74ca693dab120d8aea73095`
MD5	`f362efe655462f263e0529f93f06235d`
BLAKE2b-256	`81dd444364bc0523afbb10b9dafb8277ac1cc9f31feb227359feb786758d6186`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Upload date: May 20, 2024
Size: 11.8 MB
Tags: CPython 3.11, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`74a664759158689d73bf52dbf870790d9e050f212b963a4ee041be3eed110e50`
MD5	`48c91fc63c25e740b59d4a9745718b9c`
BLAKE2b-256	`12346a9f7445eef4d3eeff38b2337ff14ab396c02c2835bb204b6bc5adf44183`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp310-cp310-win_amd64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp310-cp310-win_amd64.whl
Upload date: May 20, 2024
Size: 4.5 MB
Tags: CPython 3.10, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp310-cp310-win_amd64.whl
Algorithm	Hash digest
SHA256	`faa62939284075ef1d8092290e921ba3d871610b33c18c84793c7ee41171c302`
MD5	`37f5344dcb6c4ff857f791575f09884e`
BLAKE2b-256	`88f11bf5e5d49a236808f6e6fbc229918be11de30286df8185b09bfd1797f01d`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Upload date: May 20, 2024
Size: 11.8 MB
Tags: CPython 3.10, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`c0fd28e576ca7ed5a86c170fb99c9ca8cb0688a77f36f4a69fd6d797874ed966`
MD5	`33873004e1a931713ef1a28742c291a5`
BLAKE2b-256	`75e6cdd1f02f36c5275fa84fc62fd6367cfb70e10d3b74d0260037a2b98e2e53`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp39-cp39-win_amd64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp39-cp39-win_amd64.whl
Upload date: May 20, 2024
Size: 4.5 MB
Tags: CPython 3.9, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp39-cp39-win_amd64.whl
Algorithm	Hash digest
SHA256	`80762d0fbd78302d416a7eac12c6c5ee70e7a0c115394dcf3ec6048adeb0c69f`
MD5	`b39cfaf0ea080bea85be2ce5fad05968`
BLAKE2b-256	`037ce10a937a85a07c69fedc8ae254364de472ffa941d5a0fb1efff6f676686d`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Upload date: May 20, 2024
Size: 11.8 MB
Tags: CPython 3.9, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`2f1e5d5c805a4586315dba8c16564451e6af28bc20c67038f388c86e7ee7b14c`
MD5	`67df63bbb3598d087fa14ec8bdcf6cbf`
BLAKE2b-256	`248a7fd357cb3a817e98e8206f3e604001d4fcdde08e54829faef5099eaa6c5b`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp38-cp38-win_amd64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp38-cp38-win_amd64.whl
Upload date: May 20, 2024
Size: 4.5 MB
Tags: CPython 3.8, Windows x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp38-cp38-win_amd64.whl
Algorithm	Hash digest
SHA256	`fd312518852ceb1981899cdc5c8525ec5c0d9d68a0a9dba7a12fcd2cafa3cb99`
MD5	`784d1ed23d4a1d524694d91d1159be4a`
BLAKE2b-256	`0afd1567cc5277cfe8c163fe90ac86d811aca3b257e5bfeddf2a61227275ccf6`

See more details on using hashes here.

File details

Details for the file onnxruntime_genai-0.2.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.

File metadata

Download URL: onnxruntime_genai-0.2.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Upload date: May 20, 2024
Size: 11.8 MB
Tags: CPython 3.8, manylinux: glibc 2.27+ x86-64, manylinux: glibc 2.28+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: RestSharp/106.13.0.0

File hashes

Hashes for onnxruntime_genai-0.2.0-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Algorithm	Hash digest
SHA256	`3e82fc5ebe242a709cd29e1a27358a0e994df8c375196942334e3b982efaf25e`
MD5	`0d557008aec23467e86e50d19ed0cf34`
BLAKE2b-256	`ffc94e3296819b54316613a63ff31f93e4767ef90496d2150e6692c0d9e47262`