DSPy

DSPy is a framework for automatically prompting and fine-tuning language models. It provides composable and declarative APIs that allow developers to describe the architecture of their LLM application in the form of a "module" (inspired by PyTorch's nn.Module). It them compiles these modules using "teleprompters" that optimize the module for a particular task. The term "teleprompter" is meant to evoke "prompting at a distance," and could involve selecting few-shot examples, generating prompts, or fine-tuning language models.

Arize makes your DSPy applications observable by visualizing the underlying structure of each call to your compiled DSPy module.

Python

To trace your DSPy application, ensure that the following packages are installed in addition to DSPy:

pip install openinference-instrumentation-dspy opentelemetry-exporter-otlp

Configure your OpenTelemetry exporter, which will export spans and traces to Arize, and run the DSPy instrumentor to wrap calls to the relevant DSPy components.

import os
from openinference.instrumentation.dspy import DSPyInstrumentor
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# Set the Space and API keys as headers for authentication
headers = f"space_key={ARIZE_SPACE_KEY},api_key={ARIZE_API_KEY}"
os.environ['OTEL_EXPORTER_OTLP_TRACES_HEADERS'] = headers

# Set resource attributes for the name and version for your application
resource = Resource(
    attributes={
        "model_id":"dspy-llm-tracing", # Set this to any name you'd like for your app
        "model_version":"1.0", # Set this to a version number string
    }
)

# Setup your tracer with the URL to send traces + API keys
endpoint = "https://otlp.arize.com/v1"
tracer_provider = trace_sdk.TracerProvider(resource=resource)
tracer_provider.add_span_processor(SimpleSpanProcessor(OTLPSpanExporter(endpoint)))
tracer_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace_api.set_tracer_provider(tracer_provider=tracer_provider)
DSPyInstrumentor().instrument()

Now run invoke your compiled DSPy module. Your traces should appear inside of Phoenix.

Traces and spans from an instrumented DSPy custom module.

For a full working example, check out the Colab.