Choosing Your Metrics
Monitor Performance, Drift, Data Quality, and Custom Metrics
Monitors automatically detect drift, data quality issues, or anomalous performance degradations with highly configurable dimensions based on both common KPIs and custom metrics.
Type
Custom Metrics
Metrics
Model performance metrics measure how well your model performs in production. Monitor model performance with daily or hourly checks using an evaluation metric. Your model type determines your performance metric.
Metrics are batched into Metric Groups that align with model types and their variants.
Metric Group  Metrics 

Classification  Accuracy, Recall, Precision, FPR, FNR, F1, Sensitivity, Specificity 
Regression  MAPE, MAE, RMSE, MSE, RSquared, Mean Error 
Ranking  
Ranking Labels  [email protected], MRR 
AUC/LogLoss  AUC, PRAUC, Log Loss 
Computer Vision  MAP, IoU 
Model Type  Metric Group Combination 

Regression  Regression 
Binary Classification  Classification and/or Regression and/or AUC/LogLoss 
Ranking w/ label  Ranking and/or Ranking Labels 
Ranking w/ score  Ranking and/or AUC/LogLoss 
Metric  Metric Family 

auc/logloss  
auc/logloss  
Mean Error  classification
regression 
classification
regression  
regression  
regression  
regression  
regression  
MSE  regression 
rSquared  regression 
classification  
classification  
classification  
classification  
classification  
classification  
classification  
classification  
classification
ranking 
Drift monitors measure distribution drift, which is the difference between two statistical distributions.
Arize offers various distributional drift metrics to choose from when setting up a monitor. Each metric is tailored to a specific use case; refer to this guide to help choose the appropriate metric for various ML use cases.
Metric  Data Type  Description 

integer, floats, string 
 
Embedding Vectors  Euclidean distance check determines if the group of production data’s average centroid has moved away from the baseline group
For unstructured data types, learn more here  
integer, floats, string 
 
integer, floats, string 
 
integer, floats 

Model health depends on highquality data that powers model features. Data quality monitors help identify key data quality issues such as cardinality shifts, data type mismatch, missing data, and more.
Metric  Data Type  Description 

Percent Empty  integer, floats, string
(Embedding vectors coming soon)  The percent of nulls in your model features 
Cardinality (Count Distinct)  string  The cardinality of your categorical features 
string  Count of new unique values that appear in production but not in baseline
Note: this monitor requires a baseline to compare against  
string  Count of new unique values that appear in baseline but not in production
Note: this monitor requires a baseline to compare against  
integer, floats  p99.9, p99, p95, p50  
Sum  integer, floats  Sum of your numeric data over the evaluation window 
Count  integer, floats, string  Traffic count of predictions, features, etc. Can be used with filters 
Average  integer, floats  Average of your numeric data over the evaluation window 
Couldn't find your metric above? Arize supports the ability to monitor custom metrics using SQL. Here is an example of a custom metric for the percent of a loan that is outstanding:
SELECT
SUM(loan_amount  repayment_amount) / SUM(loan_amount)
FROM model
WHERE state = 'CA'
AND loan_amount > 1000