Arize has first-class support for instrumenting OpenAI calls and seeing both input and output messages. We support role types such as system, user, and assistant messages, as well as function calling.

We follow a standardized format for how a trace data should be structured using openinference, which is our open source package based on OpenTelemetry.

Use our code block below to get started using our OpenAIInstrumentor.

import openai
import os

# Import open-telemetry dependencies
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from opentelemetry.sdk.resources import Resource

# Import the automatic instrumentor from OpenInference
from openinference.instrumentation.openai import OpenAIInstrumentor

# Set the Space and API keys as headers for authentication
headers = f"space_key={ARIZE_SPACE_KEY},api_key={ARIZE_API_KEY}"

# Set resource attributes for the name and version for your application
resource = Resource(
        "model_id":"openai-llm-tracing", # Set this to any name you'd like for your app
        "model_version":"1.0", # Set this to a version number string

# Define the span processor as an exporter to the desired endpoint
endpoint = ""
span_exporter = OTLPSpanExporter(endpoint=endpoint)
span_processor = SimpleSpanProcessor(span_exporter=span_exporter)

# Set the tracer provider
tracer_provider = trace_sdk.TracerProvider(resource=resource)

# Finish automatic instrumentation

Now start asking questions to your LLM app and watch the traces being collected by Arize. For more examples of instrumenting OpenAI applications, check our openinferenece-instrumentation-openai examples.

Last updated

Copyright ยฉ 2023 Arize AI, Inc