Skip to main content

No project description provided

Project description

gpt-json

JSON is a beautiful format. It's both human readable and machine readable, which makes it a great format for structured output of LLMs (after all - LLMs are somewhere in the middle). gpt-json is a wrapper around GPT that allows for declarative definition of expected output format when you're trying to parse results into a downstream pipeline.

Specifically it:

  • Relies on Pydantic schema definitions and type validations
  • Allows for defining both dictionaries and lists
  • Includes some lightweight manipulation of the output to remove superfluous context and fix broken json
  • Includes retry logic for the most common API failures
  • Adds typehinting support for both the API and the output schema

Getting Started

pip install gpt-json

Here's how to use it to generate a schema for simple tasks:

import asyncio

from gpt_json import GPTJSON, GPTMessage, GPTMessageRole
from pydantic import BaseModel

class SentimentSchema(BaseModel):
    sentiment: str

SYSTEM_PROMPT = """
Analyze the sentiment of the given text.

Respond with the following JSON schema:

{json_schema}
"""

async def runner():
    gpt_json = GPTJSON[SentimentSchema](API_KEY)
    response = await gpt_json.run(
        messages=[
            GPTMessage(
                role=GPTMessageRole.SYSTEM,
                content=SYSTEM_PROMPT,
            ),
            GPTMessage(
                role=GPTMessageRole.USER,
                content="Text: I love this product. It's the best thing ever!",
            )
        ]
    )
    print(response)
    print(f"Detected sentiment: {response.sentiment}")

asyncio.run(runner())
sentiment='positive'
Detected sentiment: positive

The json_schema is a special keyword that will be replaced with the schema definition at runtime. You should always include this in your payload to ensure the model knows how to format results. However, you can play around with where to include this schema definition; in the system prompt, in the user prompt, at the beginning, or at the end.

You can either typehint the model to return a BaseSchema back, or to provide a list of Multiple BaseSchema. Both of these work:

gpt_json_single = GPTJSON[SentimentSchema](API_KEY)
gpt_json_single = GPTJSON[list[SentimentSchema]](API_KEY)

If you want to get more specific about how you expect the model to populate a field, add hints about the value through the "description" field. This helps the model understand what you're looking for, and will help it generate better results.

from pydantic import BaseModel, Field

class SentimentSchema(BaseModel):
    sentiment: int = Field(description="Either -1, 0, or 1.")
sentiment=1
Detected sentiment: 1

Other Configurations

The GPTJSON class supports other configuration parameters at initialization.

Parameter Type Description
model GPTModelVersion | str (default: GPTModelVersion.GPT_4) - For convenience we provide the currently supported GPT model versions in the GPTModelVersion enum. You can also pass a string value if you want to use another more specific architecture.
auto_trim bool (default: False) - If your input prompt is too long, perhaps because of dynamic injected content, will automatically truncate the text to create enough room for the model's response.
auto_trim_response_overhead int (default: 0) - If you're using auto_trim, configures the max amount of tokens to allow in the model's response.

Comparison to Other Libraries

A non-exhaustive list of other libraries that address the same problem. None of them were fully compatible with my deployment (hence this library), but check them out:

jsonformer - Works with any Huggingface model, whereas gpt-json is specifically tailored towards the GPT-X family. GPT doesn't output logit probabilities or allow fixed decoder templating so the same approach can't apply.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gpt_json-0.1.1.tar.gz (11.5 kB view details)

Uploaded Source

Built Distribution

gpt_json-0.1.1-py3-none-any.whl (13.6 kB view details)

Uploaded Python 3

File details

Details for the file gpt_json-0.1.1.tar.gz.

File metadata

  • Download URL: gpt_json-0.1.1.tar.gz
  • Upload date:
  • Size: 11.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.10.11 Linux/5.15.0-1036-azure

File hashes

Hashes for gpt_json-0.1.1.tar.gz
Algorithm Hash digest
SHA256 fef3f96595fea3fcb233aa48c4911ccd3d2530e44287821fbfbb86b4b3b7734f
MD5 238d9cc930245bf98cc045f93d6430db
BLAKE2b-256 7b52da1dafd7d1b272a022903a0bd243134e9089fa442ef05ec9448a8c300ea2

See more details on using hashes here.

File details

Details for the file gpt_json-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: gpt_json-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 13.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.4.2 CPython/3.10.11 Linux/5.15.0-1036-azure

File hashes

Hashes for gpt_json-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 9ace7b011f139c01bc76bf3987dabbcb7f07cee35a6e8c0634a1f0e68c4fdcc7
MD5 547417a165e9617c86378c1d5cef9207
BLAKE2b-256 0be5a45625c2520fba0721e0d558f48b1ca06bf3cf52c353bd7b07863a0fff13

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page