Arize AI
Search…
⌃K

12. Custom Metrics

Define metrics tailored to your ML needs
This feature is currently in Beta release. Get in touch with [email protected] for access.

Custom Metrics Overview

Custom metrics allow you to define a metric tailored to your ML needs. Define metrics such as business KPIs, custom performance calculations, weighted averages, etc. Use custom metrics across the platform in dashboards, monitors, and performance tracing.
Derive new metrics using a combination of existing model dimensions and functions using a SQL-like query language. This allows you to combine your model's features, tags, predictions, and actuals using aggregations, operators, and metric functions.
Use this page to learn:

Arize Query Language

Custom metrics are defined using a bespoke easy-to-use SQL-like syntax. While the Arize query language and SQL are very similar, the Arize query language deviates from SQL by handling time granularity and time ranges within the Arize platform. This allows you to easily switch time ranges and granularities without changing your query.

Custom Metric Example

This example will compare constructing a custom metric in SQL versus the Arize SQL-like query language.
Using a loan model, we'll calculate the average dollar loss as a percentage of total loan volume. Additionally, we want to filter for loan amounts greater than $1000, and where the state is 'CA'. We want to aggregate the result daily.
For more example use cases, please refer to our examples page here.
SQL:
SELECT
date_trunc('day', timestamp) AS ts,
SUM(loan_amount - repayment_amount) / SUM(loan_amount) AS avg_loss
FROM
Auto_loans_model
WHERE state = 'CA'
AND loan_amount > 1000
AND timestamp > '2022-01-01'
AND timestamp < now()
GROUP BY 1
Arize Query Language:
SELECT
SUM(loan_amount - repayment_amount) / SUM(loan_amount)
FROM model
WHERE state = 'CA'
AND loan_amount > 1000
Expressing the same example with the Arize query language looks similar to SQL but is simplified in comparison by removing time granularity and timestamps. The time parameters are handled automatically for you by the Arize date picker.
Learn more about our custom syntax here.
Navigate to the custom metric editor in the top navigation bar of any model. Click on 'Custom Metrics' to uncover the editor.

Custom Metric Editor

The custom metric editor allows you to write your queries with syntax validation and auto-completion of dimension names. You will be able to preview your query before saving it.
Name and describe your new metric to adequately convey your new metric's purpose when using Arize collaboratively.

Documentation

Similar to a SQL editor, use the documentation panel to familiarize yourself with the custom syntax via the syntax reference and model schema explorer.

Using A Custom Metric

Custom metrics have the same functionality as any other metric within the Arize platform. Use custom metrics in monitors, dashboards, and performance tracing to ensure your model works as expected.

Dashboards

Use custom metrics in a dashboard to focus your troubleshooting efforts in one place.
Custom Metrics are supported in the 'Time Series' widget. Add a custom metric to the plot by selecting the model for your custom metric. Under 'Metric', select 'Custom Metric', and select the metric you want to add.

Monitors (Coming Soon)

Monitor custom metrics to get alerted when something goes awry.
From the 'Monitors Listing Page' navigate to the 'New Monitor' button located on the top of the card. Click 'Create Custom Metric Monitor' and select your metric from the 'Custom Metric' dropdown menu. From there, follow the monitor setup workflow.