上传包含跟踪记录的文件

推荐阅读

在深入研究此内容之前，阅读以下指南会有所帮助：

• 使用可追溯的装饰器或包装器使用 LangSmith 进行跟踪

最低 SDK 版本

以下 SDK 版本提供以下功能：

• Python 开发工具包：>=0.1.141
• JS/TS 开发工具包：>=0.2.5

LangSmith 支持上传包含跟踪的二进制文件（例如图像、音频、视频、PDF 和 CSV）。这在使用多模态输入或输出的 LLM 管道时特别有用。

在 Python 和 TypeScript SDK 中，都可以通过指定每个文件的 MIME 类型和二进制内容来将附件添加到您的跟踪中。 本指南介绍如何使用Attachment在 Python 中键入Uint8Array / ArrayBuffer在 TypeScript 中。

蟒

在 Python SDK 中，您可以使用Attachment键入以将文件添加到跟踪中。 每Attachment需要：

• mime_type（str）：文件的 MIME 类型（例如"image/png").
• data（字节 |Path）：文件的二进制内容或文件路径。

您还可以使用以下形式的元组元组定义附件(mime_type, data)为了方便。

只需用@traceable并包含您的Attachment实例作为参数。 请注意，要使用文件路径而不是原始字节，您需要设置dangerously_allow_filesystemflag 设置为True在你的可跟踪装饰器中。

from langsmith import traceable
from langsmith.schemas import Attachment
from pathlib import Path
import os


# Must set dangerously_allow_filesystem to True if you want to use file paths
@traceable(dangerously_allow_filesystem=True)
def trace_with_attachments(
  val: int,
  text: str,
  image: Attachment,
  audio: Attachment,
  video: Attachment,
  pdf: Attachment,
  csv: Attachment,
):
  return f"Processed: {val}, {text}, {len(image.data)}, {len(audio.data)}, {len(video.data)}, {len(pdf.data), {len(csv.data)}}"


# Helper function to load files as bytes
def load_file(file_path: str) -> bytes:
  with open(file_path, "rb") as f:
      return f.read()


# Load files and create attachments
image_data = load_file("my_image.png")
audio_data = load_file("my_mp3.mp3")
video_data = load_file("my_video.mp4")
pdf_data = load_file("my_document.pdf")
image_attachment = Attachment(mime_type="image/png", data=image_data)
audio_attachment = Attachment(mime_type="audio/mpeg", data=audio_data)
video_attachment = Attachment(mime_type="video/mp4", data=video_data)
pdf_attachment = ("application/pdf", pdf_data) # Can just define as tuple of (mime_type, data)
csv_attachment = Attachment(mime_type="text/csv", data=Path(os.getcwd()) / "my_csv.csv")

# Define other parameters

val = 42
text = "Hello, world!"

# Call the function with traced attachments

result = trace_with_attachments(
val=val,
text=text,
image=image_attachment,
audio=audio_attachment,
video=video_attachment,
pdf=pdf_attachment,
csv=csv_attachment,
)

TypeScript （类型脚本）

In the TypeScript SDK, you can add attachments to traces by using Uint8Array or ArrayBuffer as data types. Each attachment's MIME type is specified within extractAttachments:

• Uint8Array: Useful for handling binary data directly.
• ArrayBuffer: Represents fixed-length binary data, which can be converted to Uint8Array as needed.

Wrap your function with traceable and include your attachments within the extractAttachments option.

In the TypeScript SDK, the extractAttachments function is an optional parameter in the traceable configuration. When the traceable-wrapped function is invoked, it extracts binary data (e.g., images, audio files) from your inputs and logs them alongside other trace data, specifying their MIME types.

Note that you cannot directly pass in a file path in the TypeScript SDK, as accessing local files is not supported in all runtime environments.

type AttachmentData = Uint8Array | ArrayBuffer;
type Attachments = Record<string, [string, AttachmentData]>;

extractAttachments?: (
...args: Parameters<Func>
) => [Attachments | undefined, KVMap];

import { traceable } from "langsmith/traceable";

const traceableWithAttachments = traceable(
(
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer,
attachment5: Uint8Array,
) =>
`Processed: ${val}, ${text}, ${attachment.length}, ${attachment2.byteLength}, ${attachment3.length}, ${attachment4.byteLength}, ${attachment5.byteLength}`,
{
name: "traceWithAttachments",
extractAttachments: (
val: number,
text: string,
attachment: Uint8Array,
attachment2: ArrayBuffer,
attachment3: Uint8Array,
attachment4: ArrayBuffer,
attachment5: Uint8Array,
) => [
{
"image inputs": ["image/png", attachment],
"mp3 inputs": ["audio/mpeg", new Uint8Array(attachment2)],
"video inputs": ["video/mp4", attachment3],
"pdf inputs": ["application/pdf", new Uint8Array(attachment4)],
"csv inputs": ["text/csv", new Uint8Array(attachment5)]
},
{ val, text },
],
}
);

const fs = Deno // or Node.js fs module

const image = await fs.readFile("my_image.png"); // Uint8Array
const mp3Buffer = await fs.readFile("my_mp3.mp3");
const mp3ArrayBuffer = mp3Buffer.buffer; // Convert to ArrayBuffer

const video = await fs.readFile("my_video.mp4"); // Uint8Array
const pdfBuffer = await fs.readFile("my_document.pdf");
const pdfArrayBuffer = pdfBuffer.buffer; // Convert to ArrayBuffer
const csv = await fs.readFile("test-vals.csv"); // Uint8Array

// Define example parameters
const val = 42;
const text = "Hello, world!";

// Call traceableWithAttachments with the files
const result = await traceableWithAttachments(val, text, image, mp3ArrayBuffer, video, pdfArrayBuffer, csv);

以下是上述内容在 LangSmith UI 中的样子。您可以展开每个附件以查看其内容。