Dashboards

Dashboards are an essential tool for teams working on LLM applications. By centralizing key metrics—such as token usage, response quality, latency, error counts, user insights, and experiment results—dashboards simplify trend monitoring and data-driven adjustments. Given the sensitivity of LLM applications to even subtle changes, dashboards provide transparency, enabling teams to track usage, understand patterns, assess performance, and align model outputs with strategic goals.

Token Tracking and Latency Template

This default dashboard template is designed for generative applications, offering essential insights into key usage and performance metrics. The template includes:

• Total Prompt and Completion Token Counts: Tracks the cumulative count of tokens used for prompts and completions over time.

• Average Prompt and Completion Token Counts: Provides insights into the average token usage, ensuring prompts and responses remain efficient.

• Number of Requests: Monitors the total volume of requests to gauge application demand.

• Average Latency: Measures average response time, helping teams optimize performance and user experience.

This dashboard is ideal for understanding usage patterns, tracking token efficiency, and maintaining optimal latency across LLM applications.

Custom Dashboards

Create a custom dashboard tailored to your needs by leveraging our diverse widget types, allowing for flexible and insightful visualizations:

• Timeseries Widget: Visualize trends in custom metrics or spanProperty quality metrics (e.g., average, count, median, cardinality) over time. Great for monitoring spikes in token usage, errors, or business-specific metrics.

• Distribution Widget: See the distribution of evaluation labels or spanProperty values. Ideal for analyzing usage by user, tool usage, or your evaluation results.

• Experiments Widget: Track experiment outcomes by selecting datasets and evaluators, making it easy to visualize and compare results over time.

• Statistic Widget: Get an aggregate view of a custom metric or spanProperty quality metric. Perfect for quickly seeing overall counts, averages, or sums.

• Text Widget: Add context or notes to your dashboard by embedding text, keeping everyone aligned on the purpose or focus of the displayed data.

Enhance your custom dashboard by applying filters to each widget, creating a refined, focused view of your most relevant metrics.

Using Dashboards

Dashboards offer a high-level overview of your application, ideal for daily check-ins to monitor performance and usage patterns. With global filters, date selectors, and interactive charts, you can dive deeper to view specific traces or experiment details. Dashboards not only provide a quick, comprehensive view but are also a powerful tool for gaining detailed insights, helping you stay on top of key metrics and optimize application performance effectively.