Hybrid Instrumentation

Combining auto instrumentation with manual

If you're using JavaScript auto-instrumentation packages and need to track additional application details manually, consider integrating manually instrumented spans. This is particularly useful for actions outside standard frameworks or LLM clients. Below, we outline how to maintain cohesive traces using both auto-instrumented and manual spans.

Example

For instance, in the chat example from the previous section, we may want to create a span to capture some information about our request before we call out to OpenAI which is auto instrumented using the OpenInference OpenAI package.

/*app.ts*/
import { trace } from '@opentelemetry/api';
import express, { Express } from 'express';
import { OpenAI } from "openai";
import {
  MimeType,
  OpenInferenceSpanKind,
  SemanticConventions,
} from "@arizeai/openinference-semantic-conventions";

const tracer = trace.getTracer('llm-server', '0.1.0');

const PORT: number = parseInt(process.env.PORT || '8080');
const app: Express = express();

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

app.get('/chat', (req, res) => {
  const message = req.query.message
  // Start a chain span, this will be the parent of all the work done in this route
  // including the spans created by the OpenAI auto instrumentation package
  tracer.startActiveSpan("chat chain", async (span) => {
     span.setAttributes({
        [SemanticConventions.OPENINFERENCE_SPAN_KIND]:
          OpenInferenceSpanKind.CHAIN,
        [SemanticConventions.INPUT_VALUE]: message,
        [SemanticConventions.INPUT_MIME_TYPE]: MimeType.TEXT,
        // Metadata can be used to store user defined values
        [SemanticConventions.METADATA]: JSON.stringify({
          "userId": req.query.userId,
          "conversationId": req.query.conversationId 
        })
      });
      
    // Will be picked up by auto instrumentation
    const chatCompletion = await openai.chat.completions.create({
      messages: [{ role: "user", content: message }],
      model: "gpt-3.5-turbo",
    });
    
    const response = chatCompletion.choices[0].message;
    span.setAttributes({
        [SemanticConventions.OUTPUT_VALUE]: response,
        [SemanticConventions.OUTPUT_MIME_TYPE]: MimeType.TEXT,
        [`${SemanticConventions.LLM_OUTPUT_MESSAGES}.0.${SemanticConventions.MESSAGE_CONTENT}`]:
          streamedResponse,
        [`${SemanticConventions.LLM_OUTPUT_MESSAGES}.0.${SemanticConventions.MESSAGE_ROLE}`]:
          role,
      });
    span.setStatus({ code: SpanStatusCode.OK });
    // End the span
    span.end();
    res.send(response);
  })
});

app.listen(PORT, () => {
  console.log(`Listening for requests on http://localhost:${PORT}`);
});

This example demonstrates how to use a CHAIN span to wrap our LLM span, allowing additional application data to be tracked. This data can then be analyzed in Arize. For more complex applications, different strategies might be required. Refer to the previous section for detailed guidance on creating and nesting spans effectively.