Skip to main content
Open In Colab在 GitHub 上打开

AzureMLChatOnlineEndpoint

Azure 机器学习是用于构建、训练和部署机器学习模型的平台。用户可以在 Model Catalog 中探索要部署的模型类型，该目录提供来自不同提供商的基础和通用模型。

通常，您需要部署模型才能使用其预测（推理）。在Azure Machine Learning，在线终端节点用于部署具有实时服务的这些模型。它们基于以下理念Endpoints和Deployments这允许您将 production 工作负载的接口与为其提供服务的实现分离。

此笔记本介绍了如何使用托管在Azure Machine Learning Endpoint.

from langchain_community.chat_models.azureml_endpoint import AzureMLChatOnlineEndpoint

建立

您必须在 Azure ML 上或 Azure AI Studio 上部署模型，并获取以下参数：

  • endpoint_url：端点提供的 REST 端点 URL。
  • endpoint_api_type：用endpoint_type='dedicated'将模型部署到专用终端节点（托管托管基础设施）时。用endpoint_type='serverless'使用 Pay-as-you-go 产品 （Model as a service） 部署模型时。
  • endpoint_api_key：端点提供的 API 密钥

内容格式化程序

这content_formatterparameter 是一个处理程序类，用于转换 AzureML 终结点的请求和响应以匹配所需的架构。由于模型目录中有各种各样的模型，每个模型处理数据的方式可能彼此不同，因此ContentFormatterBase类允许用户根据自己的喜好转换数据。提供了以下内容格式化程序：

  • CustomOpenAIChatContentFormatter：为遵循 OpenAI API 请求和响应规范的模型（如 LLaMa2-chat）设置请求和响应数据的格式。

注意：langchain.chat_models.azureml_endpoint.LlamaChatContentFormatter被弃用并替换为langchain.chat_models.azureml_endpoint.CustomOpenAIChatContentFormatter.

您可以实现特定于从类派生的模型的自定义内容格式化程序langchain_community.llms.azureml_endpoint.ContentFormatterBase.

例子

以下部分包含有关如何使用此类的示例：

示例：使用实时端点完成聊天

from langchain_community.chat_models.azureml_endpoint import (
    AzureMLEndpointApiType,
    CustomOpenAIChatContentFormatter,
)
from langchain_core.messages import HumanMessage

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/score",
    endpoint_api_type=AzureMLEndpointApiType.dedicated,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter(),
)
response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")]
)
response
AIMessage(content='  The Collatz Conjecture is one of the most famous unsolved problems in mathematics, and it has been the subject of much study and research for many years. While it is impossible to predict with certainty whether the conjecture will ever be solved, there are several reasons why it is considered a challenging and important problem:\n\n1. Simple yet elusive: The Collatz Conjecture is a deceptively simple statement that has proven to be extraordinarily difficult to prove or disprove. Despite its simplicity, the conjecture has eluded some of the brightest minds in mathematics, and it remains one of the most famous open problems in the field.\n2. Wide-ranging implications: The Collatz Conjecture has far-reaching implications for many areas of mathematics, including number theory, algebra, and analysis. A solution to the conjecture could have significant impacts on these fields and potentially lead to new insights and discoveries.\n3. Computational evidence: While the conjecture remains unproven, extensive computational evidence supports its validity. In fact, no counterexample to the conjecture has been found for any starting value up to 2^64 (a number', additional_kwargs={}, example=False)

示例：使用即用即付部署的聊天完成（模型即服务）

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter,
)
response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")]
)
response

如果需要向模型传递其他参数，请使用model_kwargs论点：

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter,
    model_kwargs={"temperature": 0.8},
)

参数也可以在调用期间传递：

response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")],
    max_tokens=512,
)
response