No project description provided
Project description
LangServe 🦜️🔗
Overview
LangServe is a library that allows developers to host their Langchain runnables /
call into them remotely from a runnable interface.
Examples
For more examples, see the examples directory.
Server
#!/usr/bin/env python
from fastapi import FastAPI
from langchain.prompts import ChatPromptTemplate
from langchain.chat_models import ChatAnthropic, ChatOpenAI
from langserve import add_routes
from typing_extensions import TypedDict
app = FastAPI(
title="LangChain Server",
version="1.0",
description="A simple api server using Langchain's Runnable interfaces",
)
# Serve Open AI and Anthropic models
LLMInput = Union[List[Union[SystemMessage, HumanMessage, str]], str]
add_routes(
app,
ChatOpenAI(),
path="/openai",
input_type=LLMInput,
config_keys=[],
)
add_routes(
app,
ChatAnthropic(),
path="/anthropic",
input_type=LLMInput,
config_keys=[],
)
# Serve a joke chain
class ChainInput(TypedDict):
"""The input to the chain."""
topic: str
"""The topic of the joke."""
model = ChatAnthropic()
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
add_routes(app, prompt | model, path="/chain", input_type=ChainInput)
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="localhost", port=8000)
Client
from langchain.schema import SystemMessage, HumanMessage
from langchain.prompts import ChatPromptTemplate
from langchain.schema.runnable import RunnableMap
from langserve import RemoteRunnable
openai = RemoteRunnable("http://localhost:8000/openai/")
anthropic = RemoteRunnable("http://localhost:8000/anthropic/")
joke_chain = RemoteRunnable("http://localhost:8000/chain/")
joke_chain.invoke({"topic": "parrots"})
# or async
await joke_chain.ainvoke({"topic": "parrots"})
prompt = [
SystemMessage(content='Act like either a cat or a parrot.'),
HumanMessage(content='Hello!')
]
# Supports astream
async for msg in anthropic.astream(prompt):
print(msg, end="", flush=True)
prompt = ChatPromptTemplate.from_messages(
[("system", "Tell me a long story about {topic}")]
)
# Can define custom chains
chain = prompt | RunnableMap({
"openai": openai,
"anthropic": anthropic,
})
chain.batch([{ "topic": "parrots" }, { "topic": "cats" }])
Installation
# pip install langserve[all] -- has not been published to pypi yet
or use client extra for client code, and server extra for server code.
Features
- Deploy runnables with FastAPI
- Client can use remote runnables almost as if they were local
- Supports async
- Supports batch
- Supports stream
Limitations
- Chain callbacks cannot be passed from the client to the server
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
langserve-0.0.1.tar.gz
(10.1 kB
view details)
Built Distribution
langserve-0.0.1-py3-none-any.whl
(10.2 kB
view details)
File details
Details for the file langserve-0.0.1.tar.gz.
File metadata
- Download URL: langserve-0.0.1.tar.gz
- Upload date:
- Size: 10.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 97c7c2c2da7ae09aa02dbee535ebca64f9ac79372728a9911665d69649b700cc |
|
| MD5 | 64b9d251a7e790c963b6a85dfd038f4c |
|
| BLAKE2b-256 | 08be312bcd13526f1083bfd18f5663d1b8ea49ee37629bfe77885a63fb05e6ca |
File details
Details for the file langserve-0.0.1-py3-none-any.whl.
File metadata
- Download URL: langserve-0.0.1-py3-none-any.whl
- Upload date:
- Size: 10.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/4.0.2 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 292297b90beb2454e6be79eafe6eafd02a3b477bdd953c2ce8b3a001c1d0aab2 |
|
| MD5 | 94abbf7c593b11f4f54f299eb05bccfc |
|
| BLAKE2b-256 | 2fba9c3667490d39d3cb8c51fd0faf476f2bdfb727600fa2a91e0d7e18c0801f |
