Setup Tracing (TS)
Last updated
Was this helpful?
Last updated
Was this helpful?
Phoenix is written and maintained in Python to make it natively runnable in Python notebooks. However, it can be stood up as a trace collector so that your LLM traces from your NodeJS application (e.g., LlamaIndex.TS, Langchain.js) can be collected. The traces collected by Phoenix can then be downloaded to a Jupyter notebook and used to run evaluations (e.g., , Ragas).
Instrumentation is the act of adding observability code to an app yourself.
If you’re instrumenting an app, you need to use the OpenTelemetry SDK for your language. You’ll then use the SDK to initialize OpenTelemetry and the API to instrument your code. This will emit telemetry from your app, and any library you installed that also comes with instrumentation.
Now lets walk through instrumenting, and then tracing, a sample express application.
Install OpenTelemetry API packages:
Install OpenInference instrumentation packages. Below is an example of adding instrumentation for OpenAI as well as the semantic conventions for OpenInference.
If a TracerProvider
is not created, the OpenTelemetry APIs for tracing will use a no-op implementation and fail to generate data. As explained next, create an instrumentation.ts
(or instrumentation.js
) file to include all of the provider initialization code in Node.
Node.js
Create instrumentation.ts
(or instrumentation.js
) to contain all the provider initialization code:
This basic setup has will instrument chat completions via native calls to the OpenAI client.
Picking the right span processor
In our instrumentation.ts
file above, we use the BatchSpanProcessor
. The BatchSpanProcessor
processes spans in batches before they are exported. This is usually the right processor to use for an application.
In contrast, the SimpleSpanProcessor
processes spans as they are created. This means that if you create 5 spans, each will be processed and exported before the next span is created in code. This can be helpful in scenarios where you do not want to risk losing a batch, or if you’re experimenting with OpenTelemetry in development. However, it also comes with potentially significant overhead, especially if spans are being exported over a network - each time a call to create a span is made, it would be processed and sent over a network before your app’s execution could continue.
In most cases, stick with BatchSpanProcessor
over SimpleSpanProcessor
.
Tracing instrumented libraries
Now that you have configured a tracer provider, and instrumented the openai
package, lets see how we can generate traces for a sample application.
First, install the dependencies required for our sample app.
Next, create an app.ts
(or app.js
) file, that hosts a simple express server for executing OpenAI chat completions.
Then, we will start our application, loading the instrumentation.ts
file before app.ts
so that our instrumentation code can instrument openai
.
Finally, we can execute a request against our server
After a few moments, a new project openai-service
will appear in the Phoenix UI, along with the trace generated by our OpenAI chat completion!
Anywhere in your application where you write manual tracing code should call getTracer
to acquire a tracer. For example:
It’s generally recommended to call getTracer
in your app when you need it rather than exporting the tracer
instance to the rest of your app. This helps avoid trickier application load issues when other required dependencies are involved.
Below is an example of acquiring a tracer within application scope.
The API of OpenTelemetry JavaScript exposes two methods that allow you to create spans:
In most cases you want to use the latter (tracer.startActiveSpan
), as it takes care of setting the span and its context active.
The code below illustrates how to create an active span.
The above instrumented code can now be pasted in the /chat
handler. You should now be able to see spans emitted from your app.
Start your app as follows, and then send it requests by visiting http://localhost:8080/chat?message="how long is a pencil"
with your browser or curl
.
After a while, you should see the spans printed in the console by the ConsoleSpanExporter
, something like this:
You can also add attributes to a span as it’s created:
Semantic Attributes
First add both semantic conventions as a dependency to your application:
Add the following to the top of your application file:
Finally, you can update your file to include semantic attributes:
While Phoenix captures these, they are currently not displayed in the UI. Contact us if you would like to support!
The status can be set at any time before the span is finished.
sdk-trace-base
and manually propagating span contextIn some cases, you may not be able to use either the Node.js SDK nor the Web SDK. The biggest difference, aside from initialization code, is that you’ll have to manually set spans as active in the current context to be able to create nested spans.
Initializing tracing with sdk-trace-base
Initializing tracing is similar to how you’d do it with Node.js or the Web SDK.
Like the other examples in this document, this exports a tracer you can use throughout the app.
Creating nested spans with sdk-trace-base
To create nested spans, you need to set whatever the currently-created span is as the active span in the current context. Don’t bother using startActiveSpan
because it won’t do this for you.
All other APIs behave the same when you use sdk-trace-base
compared with the Node.js SDKs.
Phoenix natively supports automatic instrumentation provided by OpenInference. For more details on OpenInference, checkout the on GitHub.
To enable in your app, you’ll need to have an initialized .
As shown above with OpenAI, you can register additional instrumentation libraries with the OpenTelemetry provider in order to generate telemetry data for your dependencies. For more information, see .
The following code assumes you have Phoenix running locally, on its default port of 6006. See our documentation if you'd like to learn more about running Phoenix.
Learn more by visiting the Node.js documentation on and or see our documentation for an end to end example.
The values of instrumentation-scope-name
and instrumentation-scope-version
should uniquely identify the , such as the package, module or class name. While the name is required, the version is still recommended despite being optional.
Now that you have initialized, you can create .
: Starts a new span without setting it on context.
: Starts a new span and calls the given callback function passing it the created span as first argument. The new span gets set in context and this context is activated for the duration of the function call.
Sometimes it’s helpful to do something with the current/active at a particular point in program execution.
It can also be helpful to get the from a given context that isn’t necessarily the active span.
let you attach key/value pairs to a so it carries more information about the current operation that it’s tracking. For OpenInference related attributes, use the @arizeai/openinference-semantic-conventions
keys. However you are free to add any attributes you'd like!
There are semantic conventions for spans representing operations in well-known protocols like HTTP or database calls. OpenInference also publishes it's own set of semantic conventions related to LLM applications. Semantic conventions for these spans are defined in the specification under . In the simple example of this guide the source code attributes can be used.
A is a human-readable message on an that represents a discrete event with no duration that can be tracked by a single timestamp. You can think of it like a primitive log.
You can also create Span Events with additional
A can be set on a , typically used to specify that a Span has not completed successfully - Error
. By default, all spans are Unset
, which means a span completed without error. The Ok
status is reserved for when you need to explicitly mark a span as successful rather than stick with the default of Unset
(i.e., “without error”).
It can be a good idea to record exceptions when they happen. It’s recommended to do this in conjunction with setting .