ChatLiteLLM 和 ChatLiteLLMRouter

LiteLLM 是一个简化调用 Anthropic、Azure、Huggingface、Replicate 等服务的库。

这个笔记本介绍了如何使用 Langchain + LiteLLM I/O 库入门。

此集成包含两个主要类：<br/>

• ChatLiteLLM: 主要的Langchain封装，用于基础使用LiteLLM（文档）。
• ChatLiteLLMRouter: 一个利用LiteLLM的Router (文档) 的包装器。

Table of Contents

1. 概览
   • 集成详情
   • 模型功能
2. 设置
3. 权限信息
4. 安装
5. 瞬时化
   • ChatLiteLLM
   • ChatLiteLLMRouter
6. 调用
7. 异步与流式功能
8. API 参考

概览

集成细节

Class	包	本地	序列化	JS支持	Package downloads	Package 最新版本
ChatLiteLLM	langchain-litellm	❌	❌	❌
ChatLiteLLMRouter	langchain-litellm	❌	❌	❌

模型特性​

工具调用	结构化输出	JSON 模式	图像输入	音频输入	视频输入	Token级流式传输	原生异步	Token 使用	对数概率
✅	❌	❌	❌	❌	❌	✅	✅	✅	❌

设置

要访问ChatLiteLLM和ChatLiteLLMRouter模型，您需要安装langchain-litellm包并创建一个OpenAI、Anthropic、Azure、Replicate、OpenRouter、Hugging Face、Together AI或Cohere账户。然后，您必须获取API密钥并将其导出为环境变量。

Credentials

您需要选择一个LLM提供商并注册以获取其API密钥。

Example - Anthropic

请访问 https://console.anthropic.com/ 注册 Anthropic 并生成 API 密钥。完成这一步后，请设置 ANTHROPIC_API_KEY 环境变量。

示例 - 开放AI

前往 https://platform.openai.com/api-keys 注册 OpenAI 并生成一个 API 密钥。完成此步骤后，请设置 OPENAI_API_KEY 环境变量。

## Set ENV variables
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

安装

The LangChain LiteLLM 集成在 langchain-litellm 包中可用:

%pip install -qU langchain-litellm

Instantiation

ChatLiteLLM

您可以使用提供的由LiteLLM支持的名称model实例化一个ChatLiteLLM模型。

from langchain_litellm import ChatLiteLLM

llm = ChatLiteLLM(model="gpt-4.1-nano", temperature=0.1)

ChatLiteLLMRouter

您还可以通过按照指定的方式定义您的模型列表，来利用LiteLLM的路由功能，请参阅这里。

from langchain_litellm import ChatLiteLLMRouter
from litellm import Router

model_list = [
    {
        "model_name": "gpt-4.1",
        "litellm_params": {
            "model": "azure/gpt-4.1",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
    {
        "model_name": "gpt-4o",
        "litellm_params": {
            "model": "azure/gpt-4o",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
]
litellm_router = Router(model_list=model_list)
llm = ChatLiteLLMRouter(router=litellm_router, model_name="gpt-4.1", temperature=0.1)

Invocation

无论您是实例化了一个ChatLiteLLM还是一个ChatLiteLLMRouter，现在都可以通过Langchain的API使用ChatModel。

response = await llm.ainvoke(
    "Classify the text into neutral, negative or positive. Text: I think the food was okay. Sentiment:"
)
print(response)

content='Neutral' additional_kwargs={} response_metadata={'token_usage': Usage(completion_tokens=2, prompt_tokens=30, total_tokens=32, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)), 'model': 'gpt-3.5-turbo', 'finish_reason': 'stop', 'model_name': 'gpt-3.5-turbo'} id='run-ab6a3b21-eae8-4c27-acb2-add65a38221a-0' usage_metadata={'input_tokens': 30, 'output_tokens': 2, 'total_tokens': 32}

Async和流式功能

ChatLiteLLM 和 ChatLiteLLMRouter 也支持异步和流式功能:

async for token in llm.astream("Hello, please explain how antibiotics work"):
    print(token.text(), end="")

Antibiotics are medications that fight bacterial infections in the body. They work by targeting specific bacteria and either killing them or preventing their growth and reproduction.

There are several different mechanisms by which antibiotics work. Some antibiotics work by disrupting the cell walls of bacteria, causing them to burst and die. Others interfere with the protein synthesis of bacteria, preventing them from growing and reproducing. Some antibiotics target the DNA or RNA of bacteria, disrupting their ability to replicate.

It is important to note that antibiotics only work against bacterial infections and not viral infections. It is also crucial to take antibiotics as prescribed by a healthcare professional and to complete the full course of treatment, even if symptoms improve before the medication is finished. This helps to prevent antibiotic resistance, where bacteria become resistant to the effects of antibiotics.

API 参考

详细介绍了所有ChatLiteLLM和ChatLiteLLMRouter功能和配置的文档，请参阅API参考：https://github.com/Akshay-Dongare/langchain-litellm

相关

• 聊天模型 概念指南
• 聊天模型 使用指南