2344

🚀 Opik SDK: Develop LLM Applications with Python

Opik is a tool for tracking and evaluating large language model (LLM) applications. It supports multiple popular frameworks and offers detailed metric analysis.

🚀 Quick Start

Opik simplifies the process of developing and evaluating LLM applications. With its support for various frameworks and flexible tracking and evaluation features, you can quickly get started with your LLM projects.

✨ Features

🔧 Supported Frameworks and Integrations

Opik currently supports the following mainstream large language model frameworks:

⚠️ Important Note

If you are using a tool other than the above frameworks, feel free to submit an issue or add integration support via a Pull Request.

🕶️ Tracking Function Calls

If you are not using any of the above frameworks, Opik provides a flexible track decorator to record function calls:

import opik

opik.configure(use_local=True)  # Run locally

@opik.track
def my_llm_function(user_question: str) -> str:
    # LLM code

    return "Hello"

💡 Usage Tip

The track decorator can be used in combination with any integration and can also be used to track nested function calls.

🧑⚖️ Metrics with LLM as a Judge

Opik offers a series of metrics to help you evaluate the effectiveness of LLM applications. For more information, please refer to the Metrics Documentation.

Using these metrics is very simple. Just import the relevant metric and call the score method:

from opik.evaluation.metrics import Hallucination

metric = Hallucination()
score = metric.score(
    input="What is the capital of France?",
    output="Paris",
    context="France's capital has been Paris since 1590."
)
print(score)  # Output the evaluation result

📊 Evaluation and Analysis

Opik provides detailed performance reports and visualization tools to help you comprehensively understand the performance of LLMs.

from opik import evaluate

results = evaluate(
    model="gpt-4",
    inputs=inputs,
    outputs=outputs,
    metrics=["precision", "recall", "f1"]
)
print(results)  # Output the evaluation result

🛠️ Integration and Deployment

Opik supports seamless integration of LLMs into existing CI/CD pipelines:

from opik.deploy import deploy

model = ...  # Load your LLM
deploy(model, "my-model-endpoint", platform="aws")  # Deploy to AWS cloud service

🤝 Contribution and Support

We welcome contributions from the community. You can participate in the following ways:

• Submit an issue: GitHub Issues
• Create a Pull Request: GitHub Pull Requests
• Discussion forum: Discussions

📋 Other Resources

• Official Documentation
• Example Code Repository
• Developer Blog

Thank you for choosing Opik. Let's drive the development of LLM technology together!