Skip to main content
Open In ColabOpen on GitHub

AzureMLChatOnlineEndpoint

Azure 机器学习 是一个用于构建、训练和部署机器学习模型的平台。用户可以在模型目录中探索可以部署的不同类型的模型，该目录提供了来自不同提供商的基础性和通用目的模型。

在一般情况下，你需要部署模型以消耗其预测（推理）。在Azure Machine Learning中，在线端点被用来使用实时服务来部署这些模型。它们基于Endpoints和Deployments的思想，允许你将生产工作负载的接口与其提供的实现解耦。

这个笔记本介绍了如何使用托管在Azure Machine Learning Endpoint上的聊天模型。

from langchain_community.chat_models.azureml_endpoint import AzureMLChatOnlineEndpoint

设置​

您必须将模型部署到Azure ML或Azure AI工作室，并获取以下参数：部署到Azure ML 或 部署到Azure AI工作室。

  • endpoint_url: 由端点提供的REST端口url。
  • endpoint_api_type: 当将模型部署到专用端点(托管管理基础设施)时，请使用endpoint_type='dedicated'。当使用按需付费(模型即服务)方案进行部署时，请使用endpoint_type='serverless'。
  • endpoint_api_key: 由端点提供的API密钥

Content Formatter​

The content_formatter参数是一个处理程序类，用于将AzureML端点的请求和响应转换为所需的模式。由于模型目录中的模型种类繁多，每种模型可能以不同的方式处理数据，因此提供了一个ContentFormatterBase类允许用户根据需要转换数据。以下是一些提供的内容格式器：

  • CustomOpenAIChatContentFormatter: 格式化请求和响应数据，适用于遵循OpenAI API规范的模型，例如LLaMa2-chat。

注意：langchain.chat_models.azureml_endpoint.LlamaChatContentFormatter 正在弃用并将被替换为 langchain.chat_models.azureml_endpoint.CustomOpenAIChatContentFormatter。

您可以实现自定义内容格式器，针对您的模型继承自类langchain_community.llms.azureml_endpoint.ContentFormatterBase。

示例​

The following section contains examples about how to use this class:

示例：带实时端点的聊天完成​

from langchain_community.chat_models.azureml_endpoint import (
    AzureMLEndpointApiType,
    CustomOpenAIChatContentFormatter,
)
from langchain_core.messages import HumanMessage

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/score",
    endpoint_api_type=AzureMLEndpointApiType.dedicated,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter(),
)
response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")]
)
response
AIMessage(content='  The Collatz Conjecture is one of the most famous unsolved problems in mathematics, and it has been the subject of much study and research for many years. While it is impossible to predict with certainty whether the conjecture will ever be solved, there are several reasons why it is considered a challenging and important problem:\n\n1. Simple yet elusive: The Collatz Conjecture is a deceptively simple statement that has proven to be extraordinarily difficult to prove or disprove. Despite its simplicity, the conjecture has eluded some of the brightest minds in mathematics, and it remains one of the most famous open problems in the field.\n2. Wide-ranging implications: The Collatz Conjecture has far-reaching implications for many areas of mathematics, including number theory, algebra, and analysis. A solution to the conjecture could have significant impacts on these fields and potentially lead to new insights and discoveries.\n3. Computational evidence: While the conjecture remains unproven, extensive computational evidence supports its validity. In fact, no counterexample to the conjecture has been found for any starting value up to 2^64 (a number', additional_kwargs={}, example=False)

示例：聊天完成与按需部署（模型即服务）​

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter,
)
response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")]
)
response

若需要向模型传递额外参数，请使用model_kwargs作为参数：

chat = AzureMLChatOnlineEndpoint(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/v1/chat/completions",
    endpoint_api_type=AzureMLEndpointApiType.serverless,
    endpoint_api_key="my-api-key",
    content_formatter=CustomOpenAIChatContentFormatter,
    model_kwargs={"temperature": 0.8},
)

参数也可以在调用期间传递:

response = chat.invoke(
    [HumanMessage(content="Will the Collatz conjecture ever be solved?")],
    max_tokens=512,
)
response