DSPy

DSPy is a framework for automatically prompting and fine-tuning language models. It provides composable and declarative APIs that allow developers to describe the architecture of their LLM application in the form of a "module" (inspired by PyTorch's nn.Module). It them compiles these modules using "teleprompters" that optimize the module for a particular task. The term "teleprompter" is meant to evoke "prompting at a distance," and could involve selecting few-shot examples, generating prompts, or fine-tuning language models.

Arize makes your DSPy applications observable by visualizing the underlying structure of each call to your compiled DSPy module.

Python

To trace your DSPy application, ensure that the following packages are installed in addition to DSPy:

pip install openinference-instrumentation-dspy opentelemetry-exporter-otlp

Configure your OpenTelemetry exporter, which will export spans and traces to Arize, and run the DSPy instrumentor to wrap calls to the relevant DSPy components.

# Import open-telemetry dependencies
from arize_otel import register_otel, Endpoints

# Setup OTEL via our convenience function
register_otel(
    endpoints = Endpoints.ARIZE,
    space_key = "your-space-key", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    model_id = "your-model-id", # name this to whatever you would like
)

# Import the automatic instrumentor from OpenInference
from openinference.instrumentation.dspy import DSPyInstrumentor

# Start the instrumentor for DSPy
DSPyInstrumentor().instrument()

Now run invoke your compiled DSPy module. Your traces should appear inside of Phoenix.

Traces and spans from an instrumented DSPy custom module.

For a full working example, check out the Colab.