跟踪生成器函数

在大多数 LLM 应用程序中，您需要流式传输输出，以最大限度地缩短用户看到第一个令牌的时间。

LangSmith 的跟踪功能原生支持流式输出generator功能。下面是一个示例。

蟒

from langsmith import traceable

@traceable
def my_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk

# Stream to the user
for output in my_generator():
  print(output)

# It also works with async functions
import asyncio

@traceable
async def my_async_generator():
  hunk in ["Hello", "World", "!"]:
      yield chunk

# Stream to the user
async def main():
  async for output in my_async_generator():
      print(output)

asyncio.run(main())

TypeScript （类型脚本）

import { traceable } from "langsmith/traceable";

const myGenerator = traceable(function* () {
  for (const chunk of ["Hello", "World", "!"]) {
      yield chunk;
  }
});

for (const output of myGenerator()) {
  console.log(output);
}

聚合结果

默认情况下，outputs被跟踪的函数聚合到 LangSmith 中的单个数组中。 如果要自定义它的存储方式（例如，将输出连接成单个字符串），可以使用aggregate选项 （reduce_fn在 Python 中）。 这对于聚合流式 LLM 输出特别有用。

注意

聚合输出仅影响输出的跟踪表示。它不会更改函数返回的值。

蟒

from langsmith import traceable

def concatenate_strings(outputs: list):
  return "".join(outputs)

@traceable(reduce_fn=concatenate_strings)
def my_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk

# Stream to the user
for output in my_generator():
  print(output)

# It also works with async functions
import asyncio

@traceable(reduce_fn=concatenate_strings)
async def my_async_generator():
  for chunk in ["Hello", "World", "!"]:
      yield chunk

# Stream to the user
async def main():
  async for output in my_async_generator():
      print(output)

asyncio.run(main())

TypeScript （类型脚本）

import { traceable } from "langsmith/traceable";

const concatenateStrings = (outputs: string[]) => outputs.join("");

const myGenerator = traceable(function* () {
  for (const chunk of ["Hello", "World", "!"]) {
      yield chunk;
  }
}, { aggregator: concatenateStrings });

for (const output of await myGenerator()) {
  console.log(output);
}