LlamaIndex

Arize has first-class support for LlamaIndex applications. After instrumentation, you will have a full trace of every part of your LLM application, including input, embeddings, retrieval, functions, and output messages.

We follow a standardized format for how a trace data should be structured using openinference, which is our open source package based on OpenTelemetry.

Use our code block below to get started using our LlamaIndexInstrumentor.

Python

Phoenix supports LlamaIndex's latest instrumentation paradigm.

To get started, pip install the following.

pip install "llama-index-core>=0.10.43" "openinference-instrumentation-llama-index>=2" "opentelemetry-proto>=1.12.0" opentelemetry-exporter-otlp opentelemetry-sdk

The following code snippet showcases how to automatically instrument your LLM application.

# Import OpenTelemetry dependencies and OpenInference instrumentation
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.trace.export import BatchSpanProcessor

endpoint = "https://otlp.arize.com/v1" 

tracer_provider = trace_sdk.TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint)))

LlamaIndexInstrumentor().instrument(tracer_provider=tracer_provider)

Now start asking questions to your LLM app and watch the traces being collected by Arize. For more in-detail demonstration, check our Colab tutorial:

Google Colaboratory

Tutorial on instrumenting a LlamaIndex application and sending traces to Arize

Javascript (coming soon)