Langchain

Arize has first-class support for LangChain applications. After instrumentation, you will have a full trace of every part of your LLM application, including input, embeddings, retrieval, functions, and output messages.

We follow a standardized format for how a trace data should be structured using openinference, which is our open source package based on OpenTelemetry.

Use our code block below to get started using our LangChainInstrumentor.

Python

import os

# Import open-telemetry dependencies
from opentelemetry import trace as trace_api
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk import trace as trace_sdk
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

# Import the automatic instrumentor from OpenInference
from openinference.instrumentation.langchain import LangChainInstrumentor

# Set the Space and API keys as headers for authentication
headers = f"space_key={ARIZE_SPACE_KEY},api_key={ARIZE_API_KEY}"
os.environ['OTEL_EXPORTER_OTLP_TRACES_HEADERS'] = headers

# Set resource attributes for the name and version for your application
resource = Resource(
    attributes={
        "model_id":"langchain-llm-tracing", # Set this to any name you'd like for your app
        "model_version":"1.0", # Set this to a version number string
    }
)

# Define the span processor as an exporter to the desired endpoint
endpoint = "https://otlp.arize.com/v1"
span_exporter = OTLPSpanExporter(endpoint=endpoint)
span_processor = SimpleSpanProcessor(span_exporter=span_exporter)

# Set the tracer provider
tracer_provider = trace_sdk.TracerProvider(resource=resource)
tracer_provider.add_span_processor(span_processor=span_processor)
trace_api.set_tracer_provider(tracer_provider=tracer_provider)

# Finish automatic instrumentation
LangChainInstrumentor().instrument()

For more in-detail demonstration, check our Colab tutorial:

Google Colaboratory

Tutorial on instrumenting a LangChain application and sending traces to Arize

Javascript

The LangChain instrumentation package via npm.

npm install @arizeai/openinference-instrumentation-langchain

Library	Instrumentation	Version
LangChain	@arizeai/openinference-instrumentation-langchain

The example below utilizes the OpenInference JavaScript LangChain example.

openinference/js/examples/langchain-express at main · Arize-ai/openinferenceGitHub

Navigate to the backend folder.

In addition to the above package, sending traces to Arize requires the following package: @opentelemetry/exporter-trace-otlp-grpc. This package can be installed in your environment by running the following command in your shell.

npm install @opentelemetry/exporter-trace-otlp-grpc @grpc/grpc-js

instrumentation.ts should be implemented as below (you'll need to install all of the packages imported below in the same manner as above):

/*instrumentation.ts */
import { LangChainInstrumentation } from "@arizeai/openinference-instrumentation-langchain";
import { ConsoleSpanExporter } from "@opentelemetry/sdk-trace-base";
import {
  NodeTracerProvider,
  SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-node";
import { Resource } from "@opentelemetry/resources";
import { OTLPTraceExporter as GrpcOTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-grpc"; // Arize specific
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import { Metadata } from "@grpc/grpc-js";
import * as CallbackManagerModule from "@langchain/core/callbacks/manager";

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

// Arize specific - Create metadata and add your headers
const metadata = new Metadata();

// Your Arize Space and API Keys, which can be found in the UI
metadata.set('space_key', 'your-space-key');
metadata.set('api_key', 'your-api-key');

const provider = new NodeTracerProvider({
  resource: new Resource({
    // Arize specific - The name of a new or preexisting model you 
    // want to export spans to
    "model_id": "your-model-id",
    "model_version": "your-model-version"
  }),
});

provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
provider.addSpanProcessor(
  new SimpleSpanProcessor(
    new GrpcOTLPTraceExporter({
      url: "https://otlp.arize.com/v1",
      metadata,
    }),
  ),
);

const lcInstrumentation = new LangChainInstrumentation();
// LangChain must be manually instrumented as it doesn't have
// a traditional module structure
lcInstrumentation.manuallyInstrument(CallbackManagerModule);

provider.register();

If you simultaneously want to send spans to a Phoenix collector, you should also add the following code blocks, from the original instrumentation.ts files.

import { 
  OTLPTraceExporter as ProtoOTLPTraceExporter 
} from "@opentelemetry/exporter-trace-otlp-proto";

// add as another SpanProcessor below the previous SpanProcessor
provider.addSpanProcessor(
  new SimpleSpanProcessor(
    new ProtoOTLPTraceExporter({
      // This is the url where your phoenix server is running
      url: "http://localhost:6006/v1/traces",
    }),
  ),
);

Follow the steps from the backend and frontend readme. Or simply run:

docker compose up --build

to build run the frontend, backend, and Phoenix all at the same time. Navigate to localhost:3000 to begin sending messages to the chatbot and check out your traces in Arize at app.arize.com or Phoenix at localhost:6006.