Skip to main content
Open on GitHub

RWKV-4

此页面介绍了如何在LangChain中使用RWKV-4包装器。 它分为两部分：安装和设置，然后通过示例进行使用。

安装与设置

  • 使用 pip install rwkv 安装 Python 包
  • 使用pip install tokenizer安装tokenizer Python包
  • 下载一个RWKV模型并将其放置在您希望的目录中
  • 下载 令牌文件

用法​

RWKV​

要使用RWKV封装，您需要提供预训练模型文件的路径以及分词器的配置。

from langchain_community.llms import RWKV

# Test the model

```python

def generate_prompt(instruction, input=None):
    if input:
        return f"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

# Instruction:
{instruction}

# Input:
{input}

# Response:
"""
    else:
        return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.

# Instruction:
{instruction}

# Response:
"""


model = RWKV(model="./models/RWKV-4-Raven-3B-v7-Eng-20230404-ctx4096.pth", strategy="cpu fp32", tokens_path="./rwkv/20B_tokenizer.json")
response = model.invoke(generate_prompt("Once upon a time, "))
API 参考:RWKV

模型文件​

您可以在此处找到模型文件下载链接：RWKV-4-Raven 仓库。

Rwkv-4 模型 -> 推荐显存​

RWKV VRAM
Model | 8bit | bf16/fp16 | fp32
14B   | 16GB | 28GB      | >50GB
7B    | 8GB  | 14GB      | 28GB
3B    | 2.8GB| 6GB       | 12GB
1b5   | 1.3GB| 3GB       | 6GB

See the rwkv pip page for more information about strategies, including streaming and cuda support.