Last updated
Copyright © 2023 Arize AI, Inc
Last updated
A session is a grouping of traces based on a session ID attribute. When building or debugging a chatbot application, being able to see groups of messages or traces belonging to a series of interactions between a human and the AI can be particularly helpful. By adding session.id and user.id as attributes to spans, you can:
Find exactly where a conversation "breaks" or goes off the rails. This can help identify if a user becomes progressively more frustrated or if a chatbot is not helpful.
Find groups of traces where your application is not performing well. Adding session.id and/or user.id from an application enables back-and-forth interactions to be grouped and filtered further.
Construct custom metrics based on evals using session.id or user.id to find best/worst performing sessions and users.
Session and user IDs can be added to a span using auto instrumentation or manual instrumentation of Open Inference. Any LLM call within the context (the with block in the example below) will contain corresponding session.id or user.id as a span attribute. session.id and user.id must be a non-empty string.
When defining your instrumentation, you can pass the sessionID attribute as shown below.
Requires pip install openinference-instrumentation-openai
Once you define your OpenAI client, any call inside our context managers will attach the corresponding attributes to the spans.
Alternatively, if you wrap your calls inside functions, you can use them as decorators:
To access an applications sessions in the platform, select "Sessions" from the left nav.
Context manager to add session ID to the current OpenTelemetry Context. OpenInference auto will read this Context and pass the session ID as a span attribute, following the OpenInference . Its input, the session ID, must be a non-empty string.
Context manager to add user ID to the current OpenTelemetry Context. OpenInference auto will read this Context and pass the user ID as a span attribute, following the OpenInference . Its input, the user ID, must be a non-empty string.
We provide a setSession function which allows you to set a sessionId on context. You can use this utility in conjunction with to set the active context. OpenInference will then pick up these attributes and add them to any spans created within the context.with callback.
We also provide a setUser function which allows you to set a userId on context. You can use this utility in conjunction with to set the active context. OpenInference will then pick up these attributes and add them to any spans created within the context.with callback.