Skip to main content
Open In ColabOpen on GitHub

ChatOutlines

这将帮助您入门 Outlines 聊天模型。要查看所有 ChatOutlines 特性和配置的详细文档，请访问 API 参考。

Outlines 是一个用于约束语言生成的库。它允许您使用各种后端的大规模语言模型（LLMs），并在生成的输出中应用约束。

概览​

集成细节​

Class包本地序列化JS支持Package downloadsPackage 最新版本
ChatOutlineslangchain-community✅❌❌PyPI - DownloadsPyPI - Version

模型特性​

工具调用结构化输出JSON 模式图像输入音频输入视频输入Token级流式传输原生异步Token 使用对数概率
✅✅✅✅❌❌✅❌❌❌

设置​

要访问 Outlines 模型，您需要连接互联网以从 huggingface 下载模型权重。根据所需的后端，请安装相应的依赖项（请参见Outlines 文档）

Credentials​

没有内置身份验证机制用于Outlines。

安装​

The LangChain Outlines集成存在于langchain-community包中，并需要outlines库:

%pip install -qU langchain-community outlines

Instantiation​

现在我们就可以实例化我们的模型对象并生成聊天完成内容:

from langchain_community.chat_models.outlines import ChatOutlines

# For llamacpp backend
model = ChatOutlines(model="TheBloke/phi-2-GGUF/phi-2.Q4_K_M.gguf", backend="llamacpp")

# For vllm backend (not available on Mac)
model = ChatOutlines(model="meta-llama/Llama-3.2-1B", backend="vllm")

# For mlxlm backend (only available on Mac)
model = ChatOutlines(model="mistralai/Ministral-8B-Instruct-2410", backend="mlxlm")

# For huggingface transformers backend
model = ChatOutlines(model="microsoft/phi-2")  # defaults to transformers backend
API 参考:ChatOutlines

Invocation​

from langchain_core.messages import HumanMessage

messages = [HumanMessage(content="What will the capital of mars be called?")]
response = model.invoke(messages)

response.content
API 参考:人类消息

流式传输​

ChatOutlines 支持令牌流式传输:

messages = [HumanMessage(content="Count to 10 in French:")]

for chunk in model.stream(messages):
    print(chunk.content, end="", flush=True)

链式调用​

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | model
chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)

约束生成​

ChatOutlines 允许您对生成的输出应用各种约束条件：<br/>

Regex Constraint​

model.regex = r"((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)"

response = model.invoke("What is the IP address of Google's DNS server?")

response.content

Type Constraints​

model.type_constraints = int
response = model.invoke("What is the answer to life, the universe, and everything?")

response.content

Pydantic 和 JSON 模型​

from pydantic import BaseModel


class Person(BaseModel):
    name: str


model.json_schema = Person
response = model.invoke("Who are the main contributors to LangChain?")
person = Person.model_validate_json(response.content)

person

Context Free Grammars​

model.grammar = """
?start: expression
?expression: term (("+" | "-") term)*
?term: factor (("*" | "/") factor)*
?factor: NUMBER | "-" factor | "(" expression ")"
%import common.NUMBER
%import common.WS
%ignore WS
"""
response = model.invoke("Give me a complex arithmetic expression:")

response.content

LangChain的结构化输出​

您也可以使用LangChain的结构化输出与ChatOutlines:

from pydantic import BaseModel


class AnswerWithJustification(BaseModel):
    answer: str
    justification: str


_model = model.with_structured_output(AnswerWithJustification)
result = _model.invoke("What weighs more, a pound of bricks or a pound of feathers?")

result

API 参考​

详细文档包含了所有ChatOutlines功能和配置，请参阅API参考：https://python.langchain.com/api_reference/community/chat_models/langchain_community.chat_models.outlines.ChatOutlines.html

全文档概述：​

https://dottxt-ai.github.io/outlines/latest/