Skip to main content
Open In ColabOpen on GitHub

SambaStudio

SambaNova 的 Sambastudio 是一个平台，可让您训练、运行批量推理任务，并部署在线推理端点，以运行您自行微调的开源模型。

注意

您当前所在的页面介绍了 SambaStudio 模型作为文本补全模型的使用方法。我们建议您使用对话补全模型。

您可能正在寻找 SambaStudio 聊天模型 。

概览​

集成细节​

Class包本地序列化JS支持Package downloadsPackage 最新版本
SambaStudiolangchain_community❌beta❌PyPI - DownloadsPyPI - Version

本示例介绍如何使用 LangChain 与 SambaStudio 模型进行交互

设置​

Credentials​

部署模型需要 SambaStudio 环境。更多信息请访问 sambanova.ai/products/enterprise-ai-platform-sambanova-suite

您需要部署一个端点并设置SAMBASTUDIO_URL和SAMBASTUDIO_API_KEY环境变量：

import getpass
import os

if "SAMBASTUDIO_URL" not in os.environ:
    os.environ["SAMBASTUDIO_URL"] = getpass.getpass()
if "SAMBASTUDIO_API_KEY" not in os.environ:
    os.environ["SAMBASTUDIO_API_KEY"] = getpass.getpass()

安装​

该集成位于 langchain-community 包中。我们还需要安装 sseclient-py 包，这是运行流式预测所必需的。

%pip install --quiet -U langchain-community sseclient-py

Instantiation​

from langchain_community.llms.sambanova import SambaStudio

llm = SambaStudio(
    model_kwargs={
        "do_sample": True,
        "max_tokens": 1024,
        "temperature": 0.01,
        "process_prompt": True,  # set if using CoE endpoints
        "model": "Meta-Llama-3-70B-Instruct-4096",  # set if using CoE endpoints
        # "repetition_penalty":  1.0,
        # "top_k": 50,
        # "top_logprobs": 0,
        # "top_p": 1.0
    },
)
API 参考:SambaStudio

Invocation​

现在我们就可以实例化我们的模型对象并生成聊天完成内容:

input_text = "Why should I use open source models?"

completion = llm.invoke(input_text)
completion
"Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:\n\n1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.\n2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.\n3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.\n4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.\n5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.\n6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.\n7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.\n8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.\n9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.\n10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.\n11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.\n12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.\n13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.\n14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.\n15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.\n\nOverall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair."
# Streaming response
for chunk in llm.stream("Why should I use open source models?"):
    print(chunk, end="", flush=True)
Using open source models can have numerous benefits. Here are some reasons why you should consider using open source models:

1. **Cost-effective**: Open source models are often free to use, modify, and distribute, which can significantly reduce costs compared to proprietary models.
2. **Customizability**: Open source models can be modified to fit your specific needs, allowing you to tailor the model to your project's requirements.
3. **Transparency**: Open source models provide complete transparency into the model's architecture, training data, and algorithms, which can be essential for understanding how the model works and identifying potential biases.
4. **Community involvement**: Open source models are often maintained by a community of developers, researchers, and users, which can lead to faster bug fixes, new feature additions, and improved performance.
5. **Flexibility**: Open source models can be used in a variety of applications, from research to production, and can be easily integrated into different workflows and systems.
6. **Auditability**: With open source models, you can audit the model's performance, data, and algorithms, which is critical in regulated industries or when working with sensitive data.
7. **No vendor lock-in**: By using open source models, you're not tied to a specific vendor or proprietary technology, giving you more freedom to switch or modify your approach as needed.
8. **Improved security**: Open source models can be reviewed and audited by the community, which can help identify and fix security vulnerabilities more quickly.
9. **Access to cutting-edge research**: Open source models can provide access to the latest research and advancements in AI and machine learning, allowing you to leverage the work of experts in the field.
10. **Ethical considerations**: By using open source models, you can ensure that your AI systems are transparent, explainable, and fair, which is essential for building trust in AI applications.
11. **Reduced risk of bias**: Open source models can help reduce the risk of bias by providing transparency into the model's development, training data, and algorithms.
12. **Faster development**: Open source models can accelerate your development process by providing pre-trained models, datasets, and tools that can be easily integrated into your project.
13. **Improved collaboration**: Open source models can facilitate collaboration among researchers, developers, and organizations, leading to faster progress and innovation in AI and machine learning.
14. **Access to large datasets**: Open source models can provide access to large datasets, which can be essential for training and testing AI models.
15. **Compliance with regulations**: In some cases, using open source models can help ensure compliance with regulations, such as GDPR, HIPAA, or CCPA, which require transparency and explainability in AI systems.

Overall, using open source models can provide numerous benefits, from cost savings to improved transparency and customizability. By leveraging open source models, you can accelerate your AI and machine learning projects while ensuring that your systems are transparent, explainable, and fair.

链式调用​

from langchain_core.prompts import PromptTemplate

prompt = PromptTemplate.from_template("How to say {input} in {output_language}:\n")

chain = prompt | llm
chain.invoke(
    {
        "output_language": "German",
        "input": "I love programming.",
    }
)
API 参考:提示模板
'In German, you can say:\n\n"Ich liebe das Programmieren."\n\nHere\'s a breakdown of the sentence:\n\n* "Ich" means "I"\n* "liebe" is the verb "to love" in the present tense, first person singular (I love)\n* "das" is the definite article "the"\n* "Programmieren" is the noun "programming"\n\nSo, "Ich liebe das Programmieren" literally means "I love the programming".\n\nIf you want to make it sound more casual, you can say:\n\n"Ich liebe\'s Programmieren."\n\nThe apostrophe in "liebe\'s" is a contraction of "liebe es", which is a more informal way of saying "I love it".\n\nAlternatively, you can also say:\n\n"Programmieren ist meine Leidenschaft."\n\nThis sentence means "Programming is my passion". Here, "Programmieren" is the subject, "ist" is the verb "to be" in the present tense, and "meine Leidenschaft" means "my passion".'

API 参考​

所有 SambaStudio 个LLM功能和配置的详细文档请参阅API参考：https://python.langchain.com/api_reference/community/llms/langchain_community.llms.sambanova.SambaStudio.html