permchain

Get started

pip install permchain

Overview

PermChain is an alpha-stage library for building stateful, multi-actor applications with LLMs. It extends the LangChain Expression Language with the ability to coordinate multiple chains (or actors) across multiple steps of computation. It is inspired by Pregel and Apache Beam.

Some of the use cases are:

• Recursive/iterative LLM chains
• LLM chains with persistent state/memory
• LLM agents
• Multi-agent simulations
• ...and more!

How it works

Channels

Channels are used to communicate between chains. Each channel has a value type, an update type, and an update function – which takes a sequence of updates and modifies the stored value. Channels can be used to send data from one chain to another, or to send data from a chain to itself in a future step. PermChain provides a number of built-in channels:

• LastValue: stores the last value sent to the channel, useful for input values, and single-value outputs
• Inbox: stores an ephemeral sequence of values sent to the channel, useful for sending data from one chain to another
• UniqueInbox: same as Inbox, but deduplicates values sent to the channel
• Archive: stores a persistent sequence of values sent to the channel, useful for accumulating data over multiple steps
• UniqueArchive: same as Archive, but deduplicates values sent to the channel
• BinaryOperatorAggregate: stores a persistent value, updated by applying a binary operator to the current value and each update sent to the channel, useful for computing aggregates over multiple steps. eg. total = BinaryOperatorAggregate(int, operator.add)
• Context: exposes the value of a context manager, managing its lifecycle. Useful for accessing external resources that require setup and/or teardown. eg. client = Context(httpx.Client)

Chains

Chains are LCEL Runnables which subscribe to one or more channels, and write to one or more channels. Any valid LCEL expression can be used as a chain. Chains can be combined into a Pregel application, which coordinates the execution of the chains across multiple steps.

Pregel

Pregel combines multiple chains (or actors) into a single application. It coordinates the execution of the chains across multiple steps, following the Pregel/Bulk Synchronous Parallel model. Each step consists of three phases:

• Plan: Determine which chains to execute in this step, ie. the chains that subscribe to channels updated in the previous step (or, in the first step, chains that subscribe to input channels)
• Execution: Execute those chains in parallel, until all complete, or one fails, or a timeout is reached. Any channel updates are invisible to other chains until the next step.
• Update: Update the channels with the values written by the chains in this step.

Repeat until no chains are planned for execution, or a maximum number of steps is reached.

Example

from permchain import Channel, Pregel
from permchain.channels import LastValue

grow_value = (
    Channel.subscribe_to("value")
    | (lambda x: x + x)
    | Channel.write_to(value=lambda x: x if len(x) < 10 else None)
)

app = Pregel(
    chains={"grow_value": grow_value},
    channels={"value": LastValue(str)},
    input="value",
    output="value",
)

assert app.invoke("a") == "aaaaaaaa"

Check examples for more examples.

Near-term Roadmap

• Iterate on API
  • do we want api to receive output from multiple channels in invoke()
  • do we want api to send input to multiple channels in invoke()
  • Finish updating tests to new API
• Implement input_schema and output_schema in Pregel
• More tests
  • Test different input and output types (str, str sequence)
  • Add tests for Stream, UniqueInbox
  • Add tests for subscribe_to_each().join()
• Add optional debug logging
• Implement checkpointing
  • Save checkpoints at end of each step
  • Load checkpoint at start of invocation
  • API to specify storage backend and save key
• Add more examples
  • human in the loop
  • combine documents
  • agent executor
  • run over dataset
• Fault tolerance
  • Retry individual processes in a step
  • Retry entire step?
• Pregel.stream_log to contain additional keys specific to Pregel
  • tasks: inputs of each chain in each step, keyed by {name}:{step}
  • task_results: same as above but outputs
  • channels: channel values at end of each step, keyed by {name}:{step}