Skip to main content

Comet Opik - Logging + Evals

Opik is an open source end-to-end LLM Evaluation Platform that helps developers track their LLM prompts and responses during both development and production. Users can define and run evaluations to test their LLMs apps before deployment to check for hallucinations, accuracy, context retrevial, and more!

info

We want to learn how we can make the callbacks better! Meet the LiteLLM founders or join our discord

Pre-Requisites​

Ensure you have run pip install opik for this integration

pip install opik litellm

Quick Start​

Use just 2 lines of code, to instantly log your responses across all providers with Opik

Get your Opik API Key by signing up here!

import litellm
litellm.success_callback = ["opik"]

Full examples:

# pip install opik
import litellm
import os

os.environ["OPIK_API_KEY"] = ""

# LLM provider API Keys:
os.environ["OPENAI_API_KEY"] = ""



# set "opik" as a callback, litellm will send the data to an Opik server (such as comet.com)
litellm.success_callback = ["opik"]

# openai call
response = litellm.completion(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Why is tracking and evaluation of LLMs important?"}
    ]
)

If you are using a streaming response, you need to surround the call with Opik's @track and provide current_span and current_trace:

from opik import track
from opik.opik_context import get_current_trace, get_current_span

litellm.success_callback = ["opik"]

@track()
def streaming_function(input):
    messages = [{"role": "user", "content": input}]
    response = litellm.completion(
        model="gpt-3.5-turbo",
        messages=messages,
        metadata = {
            "opik": {
                "current_span": get_current_span(),
                "current_trace": get_current_trace(),
                "tags": ["streaming-test"],
            },
        },
        stream=True,
    )
    return response

response = streaming_function("Why is tracking and evaluation of LLMs important?")
chunks = list(response)

Support & Talk to Founders​