ChatLiteLLM 和 ChatLiteLLMRouter

LiteLLM 是一个简化调用 Anthropic、Azure、Huggingface、Replicate 等的库。

本笔记本介绍了如何开始使用 Langchain + LiteLLM I/O 库。

此集成包含两个主要类：

ChatLiteLLM：LiteLLM 基本用法的主要 Langchain 包装器（文档）。
ChatLiteLLMRouter：一个ChatLiteLLM利用 LiteLLM 的 Router 的包装器（文档）。

概述

集成详细信息

类	包	本地化	序列化	JS 支持	软件包下载	最新包装
ChatLiteLLM	langchain-litellm	❌	❌	❌
ChatLiteLLMRouter	langchain-litellm	❌	❌	❌

模型特点

工具调用	结构化输出	JSON 模式	图像输入	音频输入	视频输入	令牌级流式处理	本机异步	Token 使用情况	日志
✅	❌	❌	❌	❌	❌	✅	✅	✅	❌

设置

要访问ChatLiteLLM和ChatLiteLLMRouter模型，您需要安装langchain-litellm打包并创建 OpenAI、Anthropic、Azure、Replicate、OpenRouter、Hugging Face、Together AI 或 Cohere 帐户。然后，您必须获取 API 密钥并将其导出为环境变量。

凭据

您必须选择您想要的 LLM 提供商并与他们注册以获取他们的 API 密钥。

示例 - Anthropic

前往 https://console.anthropic.com/ 注册 Anthropic 并生成 API 密钥。完成此作后，设置 ANTHROPIC_API_KEY 环境变量。

示例 - OpenAI

前往 https://platform.openai.com/api-keys 注册 OpenAI 并生成 API 密钥。完成此作后，设置 OPENAI_API_KEY 环境变量。

## Set ENV variables
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

安装

LangChain LiteLLM 集成在langchain-litellm包：

%pip install -qU langchain-litellm

实例

ChatLiteLLM

您可以实例化ChatLiteLLM模型，方法是提供model名称。

from langchain_litellm import ChatLiteLLM

llm = ChatLiteLLM(model="gpt-4.1-nano", temperature=0.1)

ChatLiteLLMRouter

您还可以通过定义此处指定的模型列表来利用 LiteLLM 的路由功能。

from langchain_litellm import ChatLiteLLMRouter
from litellm import Router

model_list = [
    {
        "model_name": "gpt-4.1",
        "litellm_params": {
            "model": "azure/gpt-4.1",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
    {
        "model_name": "gpt-4o",
        "litellm_params": {
            "model": "azure/gpt-4o",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
]
litellm_router = Router(model_list=model_list)
llm = ChatLiteLLMRouter(router=litellm_router, model_name="gpt-4.1", temperature=0.1)

调用

您是否已实例化了ChatLiteLLM或ChatLiteLLMRouter，您现在可以通过 Langchain 的 API 使用 ChatModel。

response = await llm.ainvoke(
    "Classify the text into neutral, negative or positive. Text: I think the food was okay. Sentiment:"
)
print(response)

content='Neutral' additional_kwargs={} response_metadata={'token_usage': Usage(completion_tokens=2, prompt_tokens=30, total_tokens=32, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)), 'model': 'gpt-3.5-turbo', 'finish_reason': 'stop', 'model_name': 'gpt-3.5-turbo'} id='run-ab6a3b21-eae8-4c27-acb2-add65a38221a-0' usage_metadata={'input_tokens': 30, 'output_tokens': 2, 'total_tokens': 32}

异步和流式处理功能

ChatLiteLLM和ChatLiteLLMRouter还支持异步和流式处理功能：

async for token in llm.astream("Hello, please explain how antibiotics work"):
    print(token.text(), end="")

Antibiotics are medications that fight bacterial infections in the body. They work by targeting specific bacteria and either killing them or preventing their growth and reproduction.

There are several different mechanisms by which antibiotics work. Some antibiotics work by disrupting the cell walls of bacteria, causing them to burst and die. Others interfere with the protein synthesis of bacteria, preventing them from growing and reproducing. Some antibiotics target the DNA or RNA of bacteria, disrupting their ability to replicate.

It is important to note that antibiotics only work against bacterial infections and not viral infections. It is also crucial to take antibiotics as prescribed by a healthcare professional and to complete the full course of treatment, even if symptoms improve before the medication is finished. This helps to prevent antibiotic resistance, where bacteria become resistant to the effects of antibiotics.

API 参考

有关所有ChatLiteLLM和ChatLiteLLMRouter功能和配置，请前往 API 参考：https://github.com/Akshay-Dongare/langchain-litellm

聊天模型概念指南
聊天模型操作指南

目录

概述