IBM watsonx.ai

WatsonxLLM 是 IBM watsonx.ai 基础模型的包装器。

此示例说明如何与watsonx.ai使用LangChain.

概述

集成详细信息

类	包	本地化	序列 化	JS 支持	软件包下载	最新包装
WatsonxLLM	langchain-ibm	❌	❌	✅

设置

要访问 IBM watsonx.ai 模型，您需要创建一个 IBM watsonx.ai 帐户，获取 API 密钥，并安装langchain-ibm集成包。

凭据

下面的单元格定义了使用 watsonx Foundation 模型推理所需的凭证。

行动：提供 IBM Cloud 用户 API 密钥。有关详细信息，请参阅管理用户 API 密钥。

import os
from getpass import getpass

watsonx_api_key = getpass()
os.environ["WATSONX_APIKEY"] = watsonx_api_key

此外，您还可以将其他密钥作为环境变量传递。

import os

os.environ["WATSONX_URL"] = "your service instance url"
os.environ["WATSONX_TOKEN"] = "your token for accessing the CPD cluster"
os.environ["WATSONX_PASSWORD"] = "your password for accessing the CPD cluster"
os.environ["WATSONX_USERNAME"] = "your username for accessing the CPD cluster"
os.environ["WATSONX_INSTANCE_ID"] = "your instance_id for accessing the CPD cluster"

安装

LangChain IBM 集成位于langchain-ibm包：

!pip install -qU langchain-ibm

实例

您可能需要调整模型parameters针对不同的模型或任务。有关详细信息，请参阅文档。

parameters = {
    "decoding_method": "sample",
    "max_new_tokens": 100,
    "min_new_tokens": 1,
    "temperature": 0.5,
    "top_k": 50,
    "top_p": 1,
}

初始化WatsonxLLM类。

注意：

• 要为 API 调用提供上下文，您必须添加project_id或space_id.有关更多信息，请参阅文档。
• 根据预置服务实例的区域，请使用此处描述的 URL 之一。

在此示例中，我们将使用project_id和 Dallas url.

您需要指定model_id这将用于推理。您可以在文档中找到所有可用模型。

from langchain_ibm import WatsonxLLM

watsonx_llm = WatsonxLLM(
    model_id="ibm/granite-13b-instruct-v2",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="PASTE YOUR PROJECT_ID HERE",
    params=parameters,
)

API 参考：WatsonxLLM

或者，您可以使用 Cloud Pak for Data 凭证。有关详细信息，请参阅文档。

watsonx_llm = WatsonxLLM(
    model_id="ibm/granite-13b-instruct-v2",
    url="PASTE YOUR URL HERE",
    username="PASTE YOUR USERNAME HERE",
    password="PASTE YOUR PASSWORD HERE",
    instance_id="openshift",
    version="4.8",
    project_id="PASTE YOUR PROJECT_ID HERE",
    params=parameters,
)

而不是model_id，您还可以将deployment_id之前调整的模型。整个模型优化工作流程在使用 TuneExperiment 和 PromptTuning 中进行了介绍。

watsonx_llm = WatsonxLLM(
    deployment_id="PASTE YOUR DEPLOYMENT_ID HERE",
    url="https://us-south.ml.cloud.ibm.com",
    project_id="PASTE YOUR PROJECT_ID HERE",
    params=parameters,
)

对于某些要求，可以选择将 IBM 的APIClient对象复制到WatsonxLLM类。

from ibm_watsonx_ai import APIClient

api_client = APIClient(...)

watsonx_llm = WatsonxLLM(
    model_id="ibm/granite-13b-instruct-v2",
    watsonx_client=api_client,
)

您还可以将 IBM 的ModelInference对象复制到WatsonxLLM类。

from ibm_watsonx_ai.foundation_models import ModelInference

model = ModelInference(...)

watsonx_llm = WatsonxLLM(watsonx_model=model)

调用

要获得完成，您可以使用字符串提示直接调用模型。

# Calling a single prompt

watsonx_llm.invoke("Who is man's best friend?")

"Man's best friend is his dog. Dogs are man's best friend because they are always there for you, they never judge you, and they love you unconditionally. Dogs are also great companions and can help reduce stress levels. "

# Calling multiple prompts

watsonx_llm.generate(
    [
        "The fastest dog in the world?",
        "Describe your chosen dog breed",
    ]
)

LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound. Greyhounds can run up to 45 mph, which is about the same speed as a Usain Bolt.', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Labrador Retriever is a breed of retriever that was bred for hunting. They are a very smart breed and are very easy to train. They are also very loyal and will make great companions. ', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 82, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': None}, run=[RunInfo(run_id=UUID('750b8a0f-8846-456d-93d0-e039e95b1276')), RunInfo(run_id=UUID('aa4c2a1c-5b08-4fcf-87aa-50228de46db5'))], type='LLMResult')

流式处理 Model 输出

您可以流式传输模型输出。

for chunk in watsonx_llm.stream(
    "Describe your favorite breed of dog and why it is your favorite."
):
    print(chunk, end="")

My favorite breed of dog is a Labrador Retriever. They are my favorite breed because they are my favorite color, yellow. They are also very smart and easy to train.

链接

创造PromptTemplate对象，这些对象将负责创建一个随机问题。

from langchain_core.prompts import PromptTemplate

template = "Generate a random question about {topic}: Question: "

prompt = PromptTemplate.from_template(template)

API 参考：PromptTemplate

提供主题并运行链。

llm_chain = prompt | watsonx_llm

topic = "dog"

llm_chain.invoke(topic)

'What is the origin of the name "Pomeranian"?'

API 参考

有关所有WatsonxLLM功能和配置可参考 API 参考。

相关

• LLM 概念指南
• LLM 操作指南