Setup Tracing

We recommend starting with the auto-instrumentation first. If you require more control or want to further customize, you can leverage our OTEL compliant instrumentation API directly.

1. Use Tracing Integrations (Auto-Instrumentation)

Arize makes it easy to log traces with minimal changes by using our tracing integrations. These can then be further .

LLM Models

Orchestration Frameworks

Other

2. Manual Instrumentation

In some cases, you might want full control over what is being traced in your application. Arize supports OpenTelemetry (OTEL) which means that you can create and customize your spans using the OpenTelemetry Trace API.

Notebooks

LlamaIndex Tracing

Langchain Tracing

Logging data manually

Example code

Step 1: Install Prerequisities

Python 3.6 or higher
OpenTelemetry API and SDK1

pip install opentelemetry-api
pip install opentelemetry-sdk
pip install openinference-semantic-conventions

Step 2: Configuring a Tracer

Manually configuring an OTEL tracer involves some boilerplate code that Arize takes care of for you. We recommend using the register helper function below to configure a tracer.

import openai
import opentelemetry
import pandas as pd
from openai import OpenAI
from openinference.instrumentation.openai import OpenAIInstrumentor
from arize.otel import register

# Setup OTel via our convenience function
tracer_provider = register(
    space_id = "your-space-id", # in app space settings page
    api_key = "your-api-key", # in app space settings page
    project_name = "your-project-name", # name this to whatever you would like
)

Step 3: Creating Spans

# Because we are using Open AI, we will use this along with our custom instrumentation
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

If you want fully customize the spans, you can do that using manual instrumentation.

First we should set the tracer for our manual spans below

from opentelemetry import trace

trace.set_tracer_provider(tracer_provider)
tracer = trace.get_tracer(__name__)

Next we create spans by starting spans and defining our name and other attributes

def do_work():
    with tracer.start_as_current_span("span-name") as span:
        # do some work that 'span' will track
        print("doing some work...")
        # When the 'with' block goes out of scope, 'span' is closed for you

You can also use start_span to create a span without making it the current span. This is usually done to track concurrent or asynchronous operations.

Creating nested spans

If you have a distinct sub-operation you'd like to track as a part of another one, you can create span to represent the relationship:\

def do_work():
    with tracer.start_as_current_span("parent") as parent:
        # do some work that 'parent' tracks
        print("doing some work...")
        # Create a nested span to track nested work
        with tracer.start_as_current_span("child") as child:
            # do some work that 'child' tracks
            print("doing some nested work...")
            # the nested span is closed when it's out of scope

        # This span is also closed when it goes out of scope

When you view spans in a trace visualization tool, child will be tracked as a nested span under parent.

Creating spans with a decorator

It's common to have a single span track the execution of an entire function. In that scenario, there is a decorator you can use to reduce code:

@tracer.start_as_current_span("do_work")
def do_work():
    print("doing some work...")

Use of the decorator is equivalent to creating the span inside do_work() and ending it when do_work() is finished.

To use the decorator, you must have a tracer instance in scope for your function declaration.

The code below shows how to setup tracing to export spans to Arize. To see how to use our auto instrumentations see Tracing Integrations.

Step 1: Install dependencies

npm install @opentelemetry/exporter-trace-otlp-grpc @grpc/grpc-js @opentelemetry/sdk-trace-base @opentelemetry/sdk-trace-node @opentelemetry/resources @opentelemetry/api

Step 2: Create an instrumentation.ts file

import { registerInstrumentations } from "@opentelemetry/instrumentation";
import { ConsoleSpanExporter } from "@opentelemetry/sdk-trace-base";
import {
  NodeTracerProvider,
  SimpleSpanProcessor,
} from "@opentelemetry/sdk-trace-node";
import { Resource } from "@opentelemetry/resources";
import { 
  OTLPTraceExporter as GrpcOTLPTraceExporter 
} from "@opentelemetry/exporter-trace-otlp-grpc"; // Arize specific
import { diag, DiagConsoleLogger, DiagLogLevel } from "@opentelemetry/api";
import { Metadata } from "@grpc/grpc-js"

// For troubleshooting, set the log level to DiagLogLevel.DEBUG
diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG);

// Arize specific - Create metadata and add your headers
const metadata = new Metadata();

// Your Arize Space and API Keys, which can be found in the UI, see below
metadata.set('space_id', 'your-space-id');
metadata.set('api_key', 'your-api-key');

const provider = new NodeTracerProvider({
  resource: new Resource({
    // Arize specific - The name of a new or preexisting model you 
    // want to export spans to
    "model_id": "your-model-id",
    "model_version": "your-model-version"
  }),
});

provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));
provider.addSpanProcessor(
  new SimpleSpanProcessor(
    new GrpcOTLPTraceExporter({
      url: "https://otlp.arize.com/v1",
      metadata,
    }),
  ),
);

// Add auto instrumentatons here

provider.register();

Step 3: Make sure this file runs at server startup

This file needs to run at the entry point of your application, before any other code on your server runs. You can do this by importing it at the top of your entry point for example in something like server/app.ts:

import "./instrumentation"
// server startup

# require
npx ts-node --require ./instrumentation.ts app.ts
# import
npx ts-node --import ./instrumentation.ts app.ts
# Or without ts-node something like
node --import ./dist/instrumentation.cjs ./dist/index.cjs

Step 4: Acquiring a tracer Anywhere in your application where you write manual tracing code should call getTracer to acquire a tracer. For example:

import { trace } from '@opentelemetry/api';
//...

const tracer = trace.getTracer(
  'instrumentation-scope-name',
  'instrumentation-scope-version',
);

// You can now use a 'tracer' to create spans!

It’s generally recommended to call getTracer in your app when you need it rather than exporting the tracer instance to the rest of your app. This helps avoid trickier application load issues when other required dependencies are involved.

First, in the application file app.ts (or app.js):

/*app.ts*/
import { trace } from '@opentelemetry/api';
import express, { Express } from 'express';
import { OpenAI } from "openai";

const tracer = trace.getTracer('llm-server', '0.1.0');

const PORT: number = parseInt(process.env.PORT || '8080');
const app: Express = express();

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

app.get('/chat', (req, res) => {
  const message = req.query.message
  let chatCompletion = await openai.chat.completions.create({
    messages: [{ role: "user", content: message }],
    model: "gpt-3.5-turbo",
  });
  res.send(chatCompletion.choices[0].message);
});

app.listen(PORT, () => {
  console.log(`Listening for requests on http://localhost:${PORT}`);
});

The API of OpenTelemetry JavaScript exposes two methods that allow you to create spans:

In most cases you want to use the latter (tracer.startActiveSpan), as it takes care of setting the span and its context active.

The code below illustrates how to create an active span.

import { trace, Span } from "@opentelemetry/api";
import { SpanKind } from "@opentelemetry/api";
import {
    SemanticConventions,
    OpenInferenceSpanKind,
} from "@arizeai/openinference-semantic-conventions";

/** ... */

export function chat(message: string) {
    const tracer = trace.getTracer(
      'instrumentation-scope-name',
      'instrumentation-scope-version',
    );
    // Create a span. A span must be closed.
    return tracer.startActiveSpan(
        "chat",
        (span: Span) => {
            span.setAttributes({
                [SemanticConventions.OPENINFERENCE_SPAN_KIND]: OpenInferenceSpanKind.chain,
                [SemanticConventions.INPUT_VALUE]: message,
            });
            let chatCompletion = await openai.chat.completions.create({
                messages: [{ role: "user", content: message }],
                model: "gpt-3.5-turbo",
            });
            span.setAttributes({
                attributes: {
                    [SemanticConventions.OUTPUT_VALUE]: chatCompletion.choices[0].message,
                },
            });
            // Be sure to end the span!
            span.end();
            return result;
        }
    );
}

The above instrumented code can now be pasted in the /chat handler. You should now be able to see spans emitted from your app.

Start your app as follows, and then send it requests by visiting http://localhost:8080/chat?message="how long is a pencil" with your browser or curl.

ts-node --require ./instrumentation.ts app.ts

After a while, you should see the spans printed in the console by the ConsoleSpanExporter, something like this:

{
  "traceId": "6cc927a05e7f573e63f806a2e9bb7da8",
  "parentId": undefined,
  "name": "chat",
  "id": "117d98e8add5dc80",
  "kind": 0,
  "timestamp": 1688386291908349,
  "duration": 501,
  "attributes": {
    "openinference.span.kind": "chain"
    "input.value": "how long is a pencil"
  },
  "status": { "code": 0 },
  "events": [],
  "links": []
}

Step 1: Dependency management The OpenTelemetry API provides a set of interfaces for collecting telemetry, but the data is dropped without an implementation. The OpenTelemetry SDK is the implementation of the OpenTelemetry API provided by OpenTelemetry. To use it if you instrument a Java app, begin by installing dependencies.

Here is an example Maven-based pom.xml file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.arize.app</groupId>
  <artifactId>java-tracing-test</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>java-tracing-test</name>
  <url>http://maven.apache.org</url>
  <dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.opentelemetry</groupId>
            <artifactId>opentelemetry-bom</artifactId>
            <version>1.38.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
      </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-api</artifactId>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-sdk</artifactId>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-exporter-logging</artifactId>
    </dependency>
    <dependency>
      <!-- Not managed by opentelemetry-bom -->
      <groupId>io.opentelemetry.semconv</groupId>
      <artifactId>opentelemetry-semconv</artifactId>
      <version>1.25.0-alpha</version>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-exporter-otlp</artifactId>
      <version>1.38.0</version>
    </dependency>
  </dependencies>
</project>

Step 2: Initializing and Setting Up the Tracer The first step is to set up the OpenTelemetry SDK with the right headers, resource attributes and endpoint. For example:

import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.SpanProcessor;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import io.opentelemetry.sdk.trace.export.SpanExporter;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;

public static OpenTelemetrySdk initOpenTelemetry() {
         // Make sure SPACE and API Keys are passed in as env variables for headers
         System.out.println(System.getenv("OTEL_EXPORTER_OTLP_TRACES_HEADERS"));

         // Set up the OTLP exporter with the endpoint and additional headers
         SpanExporter spanExporter = OtlpGrpcSpanExporter.builder()
                 .setEndpoint("https://otlp.arize.com/v1")
                 .build();

         // Create a simple span processor or use a batch span processor for production
         SpanProcessor spanProcessor = SimpleSpanProcessor.create(spanExporter);

         // Set up resource attributes
         Resource resource =
                 Resource.getDefault()
                         .merge(Resource.builder().put("model_id", "java-examples").build());

        //  Set the tracer provider
         SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
                 .addSpanProcessor(spanProcessor)
                 .setResource(resource)
                 .build();

         // Set the OpenTelemetry instance with the tracer provider
         return OpenTelemetrySdk.builder().setTracerProvider(tracerProvider).build();

     }

Now, you can simply call the initOpenTelemetry() function in your Java app in order to initialize OTEL tracing.

OpenTelemetrySdk otelSdk = initOpenTelemetry();

To create a span, you only need to specify the name of the span and use the method below (make sure to call end() to end the span when you want it to end):

The code below illustrates how to create an active span.

// Initialize OTEL Tracer
Tracer tracer = otelSdk.getTracer("my-application", "1.0.0");
                
// Define your span
Span singleAttrSpan = tracer.spanBuilder("span-name").startSpan()

Shutdown thread and tracer Finally, since Opentelemetry's SDK processes spans asynchronously in a separate thread, you need to make sure anything that has been enqueued is processed before exiting the main thread. Thus, it's recommended to make sure you shutdown the application gracefully by adding a shutdown hook at the end. An example of this is below:

finally {
        // Shutdown the thread and tracer
        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
                otelSdk.getSdkTracerProvider().shutdown().join(10, java.util.concurrent.TimeUnit.SECONDS);
        }));

Last updated 1 month ago

Was this helpful?