Not Diamond Python SDK
Project description
notdiamond-python
Not Diamond is an AI model router that automatically determines which LLM is best-suited to respond to any query, improving LLM output quality by combining multiple LLMs into a meta-model that learns when to call each LLM.
Key features
- Maximize output quality: Not Diamond outperforms every foundation model on major evaluation benchmarks by always calling the best model for every prompt.
- Reduce cost and latency: Not Diamond lets you define intelligent cost and latency tradeoffs to efficiently leverage smaller and cheaper models without degrading quality.
- Train your own router: Not Diamond lets you train your own custom routers optimized to your data and use case.
- Python, TypeScript, and REST API support: Not Diamond works across a variety of stacks.
Installation
Python: Requires Python 3.10+. It’s recommended that you create and activate a virtualenv prior to installing the package. For this example, we'll be installing the optional additional create dependencies, which you can learn more about here.
pip install notdiamond[create]
Apple Silicon
Make sure to install libmagic if you intend to use the create or rag extras:
brew install libmagic
Setting up
Create a .env file with your Not Diamond API key and the API keys of the models you want to route between:
NOTDIAMOND_API_KEY = "YOUR_NOTDIAMOND_API_KEY"
OPENAI_API_KEY = "YOUR_OPENAI_API_KEY"
ANTHROPIC_API_KEY = "YOUR_ANTHROPIC_API_KEY"
Sending your first Not Diamond API request
Create a new file in the same directory as your .env file and copy and run the code below (you can toggle between Python and TypeScript in the top left of the code block):
from notdiamond import NotDiamond
# Define the Not Diamond routing client
client = NotDiamond()
# The best LLM is determined by Not Diamond based on the messages and specified models
result, session_id, provider = client.chat.completions.create(
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Concisely explain merge sort."} # Adjust as desired
],
model=['openai/gpt-3.5-turbo', 'openai/gpt-4o', 'anthropic/claude-3-5-sonnet-20240620']
)
print("ND session ID: ", session_id) # A unique ID of Not Diamond's recommendation
print("LLM called: ", provider.model) # The LLM routed to
print("LLM output: ", result.content) # The LLM response
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file notdiamond-0.3.30.tar.gz.
File metadata
- Download URL: notdiamond-0.3.30.tar.gz
- Upload date:
- Size: 59.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | fa30ebd82870589cbd8d69991a465ec072afab9db1b12524839a7131d3c4e97d |
|
| MD5 | 267e617b717a45ea8c125760ce3ba46c |
|
| BLAKE2b-256 | 2c7e9679b1f0fa499a6cb7ca36139e9b4e3d4a33cfc524a192a0b17fa99af45a |
File details
Details for the file notdiamond-0.3.30-py3-none-any.whl.
File metadata
- Download URL: notdiamond-0.3.30-py3-none-any.whl
- Upload date:
- Size: 68.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 686813164a1bb16fc44b192afd16d4242e1c3c62d438ef26cc05cbe394492483 |
|
| MD5 | 48cc479792ebf275e706bdb998ab4e7e |
|
| BLAKE2b-256 | 49f12e0659ae1c34c9ecfce497a56d4693fd3074a1663c14638f08b9061040a3 |
